Scientific responsibility in counting the war dead - a response to Washington Post and HRDAG

US soldier lays an American flag on a statue of Saddam Hussein to signify the 'liberation' of Iraq in 2003

In April I released an exclusive investigation via my new crowdfunded journalism project, INSURGE INTELLIGENCE, into the well-known NGO, Iraq Body Count (IBC), and how its techniques and methods have been co-opted by the Western foreign policy establishment to systematically undercount civilian casualties from conflicts in Iraq, Afghanistan and Colombia - and potentially in other conflicts around the world involving US interests.

I was inspired to undertake the investigation after my Middle East Eye story, "Unworthy victims: Western wars have killed four million Muslims since 1990," elicited some critical responses on social media from various people associated with and/or sympathetic to Iraq Body Count. 

The more I looked into the matter, the more perturbed I became by IBC's odd relationships to US and European government agencies, questionable public statements, and pathological obsession with defending IBC methodologies without any regard for meaningful scientific or academic integrity. Central to the controversy is the work of Prof. Michael Spagat, an economist at Royal Holloway University affiliated to IBC, who has published a number of peer-reviewed papers purporting to show that a 2006 Lancet survey implying an Iraq death toll of more than a million people to date, was in fact false, fraudulent, and unreliable.

My investigation showed, for the first time, that Spagat's work itself is false, fraudulent, and unreliable, and that its publication in scientific journals appears to have been enabled through unethical and undeclared conflicts of interests.

You can read the full investigation, "How the Pentagon is hiding the dead," here. Here's the opening summary:

In the name of ‘counting every casualty,’ the Pentagon is systematically undercounting deaths from the ‘war on terror’ and the ‘war on drugs,’ in the Middle East, Central Asia, and Latin America. Complicit in this great deception are some of the world’s most respected anti-war activists. 

In this exclusive investigation, Insurge Intelligence reveals that a leading anti-war monitoring group, Iraq Body Count (IBC), is deeply embedded in the Western foreign policy establishment. IBC’s key advisers and researchers have received direct and indirect funding from US government propaganda agencies and Pentagon contractors. It is no surprise, then, that IBC-affiliated scholars promote narratives of conflict that serve violent US client-regimes and promote NATO counter-insurgency doctrines. 

IBC has not only systematically underrepresented the Iraqi death toll, it has done so on the basis of demonstrably fraudulent attacks on standard scientific procedures. IBC affiliated scholars are actively applying sophisticated techniques of statistical manipulation to whitewash US complicity in violence in Afghanistan and Colombia. 

Through dubious ideological alliances with US and British defense agencies, they are making misleading pseudoscience academically acceptable. Even leading medical journals are now proudly publishing their dubious statistical analyses that lend legitimacy to US militarism abroad.This subordination of academic conflict research to the interests of the Pentagon sets a dangerous precedent: it permits the US government to control who counts the dead across conflicts involving US interests — all in the name of science and peace.

The piece shortly received two responses. One from US statistician Prof. Andrew Gelman of Columbia University, who responded in the Washington Post here. Another from US statistician Dr. Patrick Ball of the Human Rights Data Analysis Group (HRDAG), who responded here. 

Last week, I updated and revised the original piece to take into account the issues raised by Gelman and Ball, and responds explicitly to them in the text - so the piece remains a comprehensive analysis of the issues, including their reservations and critical observations.

However, for ease of reference, I'm addressing their points here directly and specifically.

Response to Andrew Gelman

In my original article, I cited Gelman fairly extensively on his views on the critique by Michael Spagat of the 2006 Lancet Iraq death toll survey by Burnham et. al. The issues I cited via Gelman that were of relevance to the investigation concerned Spagat's use of a graph to claim that Burnham et. al had fraudulently extrapolated data from previous conflicts, and Spagat's criticisms of the Lancet team's explanation of their cluster-sampling methodology.

In those cases, Gelman argued that Spagat had not really proved his case of fraud. 

In his Washington Post missive, Andrew Gelman, took issue with my quotations of his observations on these issues, and complained that I hadn't acknowledged that he had been generally sceptical of Burnham et. al's findings despite his view that some of Spagat's claims were not proven:

I’m an expert on survey sampling but not anything of an expert on Iraq, so my statement that I found Spagat’s article convincing shouldn’t count for much. But it’s where I stood the last time I looked at the topic. 

So you can imagine my annoyance when I read Ahmed’s report, which selectively quotes me in his case against Spagat. Ahmed’s quotes are fine — they’re indeed what I said — but it would be better for him to have framed them in some way like, “Andrew Gelman, a statistician who was largely convinced by Spagat that the Burnham et al. study was untrustworthy, still departs from Spagat on certain points.” 

My point is not that I’m being misquoted — indeed I’m not, and in any case this is not about me. Rather, Ahmed is writing an article about a political controversy. To the extent that this article is news rather than propaganda, he should represent the participants’ views accurately. 

As Gelman clearly felt that his views weren't represented accurately in my piece, as soon as I had the opportunity to do so, I revised the piece accordingly and the story explicitly points out his views on the matter with a more precise version of the phrase he recommended here.

Of course, Gelman's response is incoherent. On the one hand, he concedes that as an expert on survey sampling, but "not anything of an expert on Iraq," his views on Spagat's article "shouldn't count for much." He is right. 

I quoted Gelman as an expert on survey sampling, specifically on his views about Spagat's inferences about the survey sampling by the 2006 Lancet team. I didn't quote his views about the 2006 Lancet study generally, firstly because his various writings on the matter did not seem to me to evince a clear view, and secondly because as he concedes, he is not an expert on Iraq, so why would his general uncritical acceptance of Spagat's absurd statistical sleights-of-hand be of any relevance to the story?

Also bizarre is the way Gelman selectively quotes himself in the Washington Post to show that he was unambiguously "largely convinced" by Spagat's critique. Here's my quote from the same blog we are referring to:

“The study looked reasonable to me… Burnham et al. provide lots of detail on the first stage of the sampling (the choice of provinces) but much less detail later on… Unfortunately, it is a common problem in research reports in general: to lack details on exact procedures; it’s surprisingly difficult for people to simply describe exactly what they did… This is a little bit frustrating but unfortunately is not unique to this study. Unfortunately, I’d still have to go with this general position: it’s common to not share data or methods (indeed, as anyone knows who’s ever tried to write a report on anything, it can be surprisingly effortful to write up exactly what you did), so that alone is not evidence of a serious flaw in the research.”

Here, he described the 2006 Lancet study as appearing "reasonable," and that despite Spagat's criticisms, they did not necessarily prove "a serious flaw in the research." He also confirms that I hadn't misquoted him. But since he wants it to be clear that he generally found Spagat's article convincing and demands me not to "misrepresent where I'm coming from" - despite admitting this doesn't really count for much - the story was amended to ensure Gelman was properly represented.

In any case, Gelman went on to concede that it is precisely his lack of familiarity with Iraq that may have allowed him to find Spagat's article convincing: 

Ahmed may be correct that the Burhnam study was performed well and that Spagat’s criticisms are baseless and that my acceptance of Spagat’s criticisms were misinformed — as I wrote above, I know nothing about Iraq. If Ahmed wants to say this, and to paint me as out of touch in my ivory tower, fine... Ahmed’s article discusses how some of the work of Spagat and colleagues was funded by a U.S. government organization with connections to the Defense Department. It’s legitimate to point this out, and in this spirit I will also say that I’ve received National Security Agency funding for some of my research. 

Gelman's recognition of the legitimacy of accounting for the ethical connotations of research funding in judging its validity and objectivity is welcome. This is well-known across all academic disciplines, which is why it is a routine element of publication that research funding sources and any other potential conflicts of interest relevant to the nature of the research undertaken are expected to be disclosed. 

Gelman's disclosure in the Post of his own receipt of NSA funding for some of his "more theoretical research" on statistical issues far outside my own sphere of knowledge, is also welcome and commendable. But it is in this instance unnecessary. The point of disclosure, as Gelman is no doubt aware, is to bring attention to those assessing the validity of a research project the potential for a conflict of interest to raise questions about the legitimacy of the research and its findings. In this case, the fact that Gelman at some time in his career received some NSA funding for some specific research is neither here nor there - it reveals little if nothing about the general validity of his research on statistics, and certainly the same applies to his views on the Iraq question.

The point here is that Gelman would be under obligation to have disclosed his funding to publications with respect to the specific research being funded and published, in order for readers and evaluators of the researchers to be aware of the relevant context. 

This is precisely what didn't happen in relation to all of the peer-reviewed publications put out by the IBC team and those associated with the IBC, Spagat included. In not a single one of those publications did they disclose that a significant portion of funding for their conflict research, including specifically on Iraq, came from US and European government agencies which happen to be closely linked with foreign policy. 

In the case of Spagat and IBC executives receiving funding, and having a close institutional relationship with, the US Institute for Peace (USIP), the matter is even more alarming, given USIP's deep involvement in Iraq War policymaking under the Bush administration. Not only is the lack of such disclosure unethical, it tends to confirm legitimate suspicions about deep-seated conflicts of interest behind the IBC's and Spagat's work, which in turn does raise legitimate questions about the integrity of the research methodologies.

In summary, Gelman's substantive response solely concerns the way in which my article represented his views on Spagat's work, and on the Burnham et. al Iraq death toll survey. This has been addressed in the piece. On the main issue - my critique of Spagat and defence of the Burnham Lancet study - Gelman concedes that I may well be correct, and that he may have been "misinformed" in taking Spagat's work seriously. On the related question of the role of Pentagon-linked research funding behind Spagat's work, Gelman also states clearly that this is a legitimate line of inquiry, particularly given questions about the reliability of Spagat's work.

Response to Patrick Ball

Like Prof. Gelman, on the substantive issues that I raise regarding the work of IBC and Spagat, Dr. Ball and I are in agreement.

Ball begins by noting: 

We welcome Dr Ahmed’s summary of various points of scientific debate about mortality due to violence, specifically in Iraq and Colombia. We think these are very important questions for the analysis of data about violent conflict, and indeed, about data analysis more generally. We appreciate his exploration of the technical nuances of this difficult field.

Unlike Gelman, though, Ball disagrees that I should raise questions about research funding. 

Unfortunately, parts of Dr Ahmed’s article focus on sources of funding that IBC and Professor Spagat have received, and on speculation about how such funding might affect their substantive conclusions. We find such criticism to distract from the important points of scientific debate. 

Policy arguments that draw on scientific findings will inevitably include scientists who have personal opinions and political associations. Those personal opinions often influence which areas we choose to research. The challenge for all scientists is to do rigorous, reviewable, transparent work that supports or rebuts existing theories — or advances a new theory — about how the world works. Such work should be done without the scientists working in fear of personal attacks.

Here Ball misses the point entirely in a manner I find quite extraordinary. He misunderstands the Insurge story in several ways. Firstly, the story does not engage in "speculation" about how research funding "might affect" the "substantive conclusions" of IBC and Spagat. The story does raise serious questions about the scientific integrity of those conclusions on the one hand, and on the other, raises serious questions about the role of undisclosed research funding and conflicts of interest in undermining the demonstrated lack of scientific integrity in their published research and methods. 

Does Ball not recognise the ethical significance of disclosing sources of research funding? The failure of IBC authors including Spagat to fully and clearly disclose their institutional affiliations and research funding sources connected to US and European government foreign policy agencies and departments, is an elementary, but fundamental, breach of ethics. The breach is committed with such systematic impunity throughout their academic publication record that it does, indeed, raise serious and fundamental and perfectly legitimate questions about the integrity of their research. 

Ball also suggests that the "personal opinions" and "political associations" of scientists should not be part of the debate, and that scientists should be able to work without "fear of personal attacks."

The insinuation here is that my journalistic investigation of the IBC constitutes some sort of personal attack on Spagat and IBC authors, by disclosing their personal opinions and political associations. Yet this is clearly untrue. 

Far from trying to find out what their private "personal opinions" are, the article draws exclusively on the authors' statements in the public record, in their own published papers and academic presentations, and then subjects these to critical analysis. 

If scientists do not want journalists or academics to scrutinise their statements in published papers and academic presentations, then they should not make them, or review what is wrong with them in the first place. 

In terms of "political associations," my investigation disclosed not "political associations," but specifically their sources of research funding in government agencies which have been deeply involved in the wars that IBC researchers claim to be studying scientifically from a position of independence - and not just independence, but from a position as anti-war activists who are opposed to war.

Those are not "personal opinions" beyond the realm of scrutiny, but publicly stated mandates explaining the motivation and nature of the work they claim to be undertaking. If the public is to be asked to not ask legitimate questions about such claims, perhaps those making the claims need to assess why they are making them.

If Ball feels that I have personally attacked any of the IBC authors, I would welcome his highlighting a specific quote where this is the case. 

I especially take issue with Ball's decision to issue a statement on this, as while offering unwarranted criticisms of my efforts to disclose IBC researchers' funding sources and conflicts of interests - surely matters of public interest and academic integrity - he failed to do the same in relation to Spagat himself. In his primary paper which criticises the 2006 Lancet survey, and which I have critiqued in the Insurge article, Spagat brings up spurious (and false) charges against Burnham et. al for failing to disclose their research funding from George Soros's Open Society Institute. Although this was in fact disproved before Spagat's piece was published, it inexplicably passed peer-review.

Nevertheless, I don't recall Ball having ever issued a statement highlighting the questionable nature of Spagat raising questions about Burnham's research funding. 

Ball continues:

Good scientific work does not depend on who did the science. Every scientific finding must be assessed by the quality and appropriateness of the data and methods, and their relevance to the theory being tested. Science depends on anonymous peer review precisely so that the reviewers are not biased by who the authors might be. The personalities and biographies of the scientists involved must be irrelevant to the quality and veracity of the scientific result. Similarly, science is tested by replication: given a finding, another scientist should be able to use the same data and method to reach the same result. Anonymous review and replication are fundamental to science precisely in order to distinguish scientific knowledge from the scientists who do the work.

All of this is largely correct, except the fundamental assertion that good scientific work "does not depend on who did the science."

Ball omits to mention something that natural and social scientists learn while they are still students. That all science depends on research funding, and that the integrity of scientific research depends on disclosure of research funding to ensure no conflicts of interest that might undermine the integrity of the research and the findings.

Here is a link to an online textbook (Boundless Sociology, 2015) about research funding ethics for social science research:

If the funding source for a research project has an interest in the outcome of the project, this can represent a conflict of interest and a potential ethical breach. In other words, when research is funded by the same agency that can be expected to gain from a favorable outcome, there is a potential for biased results. The existence of a conflict of interest, or a potential one at that, can call into question the integrity of a sociologist's research and findings.

So yes, who does the research - and how they are funded - is exceptionally important in ensuring good scientific work. 

Ball goes on, equally surprisingly, to offer a deeply misinformed defence of USIP:

In particular, we think the article’s criticism of the United States Institute of Peace (USIP) is excessive and inaccurate. Our colleague Professor Amelia Hoover Green, the lead author of our white paper criticizing the Dirty War Index (DWI) measure cited extensively in Dr Ahmed’s article, has been supported by USIP grant funding. Our sister project, Martus, has received funding from USIP. Many other academics whose work is deeply critical of U.S. policy, Colombian policy, and the academic proponents thereof have received USIP support. (For full disclosure, HRDAG’s current and past funders are listed here.)

It is, of course, welcome and commendable - and a matter of basic scientific integrity - to disclose one's research funders when undertaking academic and scientific research. But once again, Ball misses the point, and demonstrates an unfortunate lack of understanding of organisational power dynamics generally, and of USIP more specifically.

Due to USIP’s institutional and structural connections to the US government, foreign policy, and defense establishment, USIP will inevitably seek out types of project that fit its overarching ideological goals. It is due to this sort of widely recognised organisational dynamic that research funding needs to be disclosed.

USIP’s deep-seated structural ideological biases are in fact widely understood in the peace research community, and have been demonstrated in the peer-reviewed scientific literature. If Ball had taken the time to actually investigate USIP before jumping to the institution's defence, he might have avoided publishing such a confused paragraph.

Prof. Sreeram Chaulia, for instance, in his study in the International Journal of Peace Studies published via the School for Conflict Analysis and Resolution at George Mason University, finds that USIP “has been consistently used by US foreign policy elites as an instrument to counter the peace movement.” Research funded by USIP, Chaulia finds, functions:

“… by projecting the US as a benevolent hegemon that unswervingly pursues peace in various conflict zones… USIP is an ideational weapon that subjugates the knowledge of the American peace movement on behalf of the state… by validating certain ideological strands of peace research as ‘objective’ while disqualifying others as biased or ‘unscientific.’”

The goals of foreign policy departments are complex and do not preclude sponsorship of beneficial or critical projects, as long as they still fit the above ideological framework and goals. Ball’s dismissal of this critically important issue, as if who funds and frames research that purports to be ‘scientific’ is irrelevant to the integrity of the scientific process, is fundamentally problematic.

In all of their research outputs, neither Spagat nor his colleagues at IBC ever chose to declare to the journals in which they published their funding sources and funding affiliations to USIP, US military, and European foreign policy agencies related directly to the subjects on which they happen to be publishing. Dr. Ball offers no comment on this matter. Spagat’s and IBC’s data disclosures to Ball and his team are welcome but do not ameliorate these particular egregious ethical breaches.

The most disappointing thing here is that Ball ignores the central revelation of this part of my investigation, which is that the historical and empirical record demonstrates unambiguously that USIP’s involvement in Iraq has from the outset been deeply politicized, and deeply influenced by the foreign policy agenda of the US government.

While USIP funding on issues unrelated to Iraq may well be much less politicized, this is demonstrably not the case on Iraq.  As I document in my Insurge investigation, USIP was itself complicit in the illegal military occupation of Iraq, having established an office in Baghdad with Bush administration funding, through which senior USIP officers and representatives worked closely with the Coalition Provisional Authority, US military officials, and the US-installed Iraqi administration. 

That USIP has funded positive scientific peace research in areas unrelated to Iraq has no bearing on the fact that USIP’s entire relationship to Iraq is deeply and directly embedded in official US government goals - the idea, therefore, that USIP can be seen as a neutral research funder on Iraq is preposterous. 

Ball, for instance, ignores the very specific evidence I report from the USIP-convened Iraq Study Group project on the question of casualty counting, demonstrating that USIP’s priorities undermined and whitewashed the task of assessing US forces’ role in violence. 

This is clear empirical evidence of USIP’s ideological bias on the issue of casualty counting and responsibility for violence in Iraq. For USIP, attacks on US soldiers were being insufficiently counted, US violence in Iraq against Iraqi citizens did not meaningfully exist - and more focus needed to be placed on sectarian violence in Iraq as a cause of violent deaths. Ball’s untenable and unscientific suggestion is that this bias should simply be ignored in assessing the integrity of USIP’s selection of an IBC-affiliate for funding.** 

It should not.

The context of this sort of institutional funding bias provides important context in understanding the serious and egregious statistical misinformation that is replete throughout IBC's and Spagat's conflict work. Having established that Spagat's major critique of the 2006 Lancet survey, for instance, is almost entirely spurious and fraudulent - and based on utterly meaningless (and often demonstrably false) assumptions about society and behaviour in Iraq - this context of undisclosed research funding illuminates exactly how scientific research can be politicised and compromised by the institutions that incubate it. 

Given IBC's incubation by the very Western foreign policy agencies complicit in the Iraq War - complicit in the deaths that Spagat systematically underestimates - we are now able to understand the alarming link between their institutional backing and how IBC researchers have generated deeply questionable research over the last decade or more, much of which flies in the face of what most conflict studies experts already know about the nature of conflicts.

The problem with number-crunching

Ultimately, this raises important lessons that it appears many people who are trying to understand the world's conflicts have yet to grasp. 

Numbers, alone, do not reveal very much.

Numbers require contextualisation against the real-world. 

Statistical approaches alone as a means to understand conflict dynamics are for this reason deeply problematic. While much can be achieved using statistical analysis, there are  inherent limitations that one can only grasp when one begins to recognise that the real world factors driving conflict dynamics involve such a wide range of variables across so many different yet overlapping and interconnected sectors, that a statistical approach alone will never be able to offer a sufficient understanding of conflict dynamics. 

To understand a conflict properly, requires an interdisciplinary approach drawing on sociology, history, political economy, international relations, culture, security studies. Against this, statistical analysis can be a very powerful tool to increase insights into the nature of the conflict. But by itself, it can be potentially very misleading. 

Yes, statistical analysis can offer some important insights into patterns and trends, but one not only needs a continual input of highly precise data from the real world, but one also an-indepth theoretical and qualitative framework for understanding the real world against which statistical analysis can be calibrated, which is where disciplines like historical sociology come in.

The moment a framework of statistical analysis fails to grasp an element of the real world, the risk is that its entire findings can become increasingly flawed, the more the analysis is allowed to follow a pathway dictated by the wrong assumptions.

Ball and his excellent team at HRDAG know this, and it shows in most of their work - but even they are prone to forgetting the inherent limitations of their field. 

Ball for instance offers the following "correction" of one of my findings:

He [i.e. me] repeatedly asserts that IBC might be missing many large events (e.g., massacres). HRDAG’s analysis, and a 2008 RAND study cited by Dr Ahmed, leads HRDAG to conclude that IBC is likely missing many smaller events with fewer victims, while missing relatively few large events. 

This is a very revealing response. HRDAG, on the one hand, argue that the IBC database that it is "likely" to miss smaller incidents with fewer victims. In their separate critique of Spagat's 'Dirty War Index' (partly derived and applied to the IBC database), HRDAG also conclude that the index is "likely" to miss increasingly "dirty" forms of warfare if those "dirty" forms of warfare end up eliminating large numbers of the population, contributing to statistical artefacts where seemingly reduced incidents of "dirty" violence are in fact evidence of the 'success' of previous "dirty war" operations in killing people, because there are now less people to be killed. 

I have oversimplified their argument to make a point, but there is an obvious tension in HRDAG's work here. That doesn't prove that HRDAG's analyses are incorrect, but underscores the inherent limitations of these sorts of statistically-grounded analyses which fail to draw more directly on a historical, sociological and empirical context of understanding violence dynamics that goes beyond playing around with numbers of deaths.

In the context of such a historical, sociological and empirical evaluation, we can come to an explanation. Ball ignores, for example, that my conclusion about IBC's capacity to miss large massacres is not based on the sort of tenuous probability-based statistical inferences that both HRDAG and Spagat are making (I say this having reached the conclusion that HRDAG's statistical research is about as good as it gets, and is much more careful and empirically grounded in the real world than Spagat's could ever dream to be).

Rather, I draw that conclusion from credible eyewitness testimony from people on the ground in Iraq, who have experienced the conflict. As cited in my Insurge investigation, the vast bulk of the media in Iraq were embedded with coalition military forces, and therefore provided a very selective view of the conflict that sanitised the US-UK role. How, then, would Ball or Spagat expect embedded journalists to pick up on and report larger-scale war crimes and massacres committed by those forces in areas where journalists were not escorted by the occupying power? 

Numerous sources - non-embedded journalists in Iraq, Iraqi journalists, and former US and British soldiers who participated in the occupation - confirm that large-scale massacres were committed routinely by US and UK forces every single day in Iraq, which were never reported by the media. Ball, despite his fundamental disagreements with Spagat, simply ignores this rich source of empirical data, just as IBC does. 

This is because, Ball is concerned with statistics. This what he does, and rightly so. That is good, and important work, and it needs to be done, by principled scholars who abide by ethical procedures in their research, as HRDAG experts do. 

But the problem is that the inherently limited data on conflicts like Iraq mean that the numbers they lead to can only do two things - provide opportunities to conduct deeply misleading statistical manipulation that serves the interests of warmongers; or provide valid but inherently limited statistical insights, insights which nevertheless remain simply incapable of computing the litany of unacknowledged war crimes and massacres confirmed by the body of testimony made available by, for example, Iraq Veterans Against War in the US.

We are not going to grasp the scale of violence in Iraq by playing with the fundamentally limited, flawed and politicized numbers produced by the IBC, whether the researchers involved are IBC-affiliated and funded by pro-Iraq War agencies, or independent statisticians at HRDAG.

The belief and insistence that we can is not just unscientific, but does a deep disservice to the victims of conflict in Iraq, and elsewhere. 

Scientific responsibility

A final lesson that must be reflected on by all who really care about the violence escalating against innocents in so many parts of the world is as follows: simply because we are scientists, or academics, does not absolve us of responsibility for inhabiting the ivory towers of our preferred disciplines. If we are going to start commenting on conflicts, and publishing grand conclusions about them, however generalised, then we are involved, and we are no longer in the ivory tower. 

So we need to recognise our responsibility for venturing out of that ivory tower, and get familiar with those disciplines that we need to be familiar with, in order to really understand the conflicts and crises we've decided to comment on.

Those scholars and scientists who have taken Spagat's work seriously in the past due to their complete lack of familiarity with what has actually been happening in Iraq, and who have published statements about the conflict as a consequence, including Ball and Gelman, are not going to absolve themselves of responsibility by issuing frivolous statements in response to me, which completely miss the point, and only further illustrate their lack of engagement with relevant facts related to the conflict.

I would urge Ball and Gelman to take their own advice, and think long and hard about the way in which their own statements about Spagat's deeply questionable work have been abused by others and contributed to propaganda (and indeed personal attacks against Burnham et. al) designed to sanitise the truly devastating impact of Western operations in Iraq.

And if that's too much to ask, then please, stay in the ivory tower.

**This post was amended on 3rd June 2015 to correct a mistaken reference to USIP's selection of IBC for funding. USIP selected Every Casualty for funding. Every Casualty is run and set-up by the same directors as IBC, and is inspired by the same methodology.