Abstract

In the wake of 9/11 and the assessment of Iraq's WMD, several inquiries placed the blame primarily on the Intelligence Community. Part of the reform that followed was a codification of analytic tradecraft standards into Intelligence Community Directive (ICD) 203 and the appointment of an analytic ombudsman in the newly created Office of the Director of National Intelligence charged with monitoring the quality of analytic products from across the intelligence community. In this paper we identify three assumptions behind ICD203: (1) tradecraft standards can be employed consistently; (2) tradecraft standards sufficiently capture the key elements of good reasoning; (3) good reasoning leads to more accurate judgments. We then report on two controlled experiments that uncover operational constraints in the reliable application of the ICD203 criteria for the assessment of intelligence products. Despite criticisms of the post-9/11 and post-Iraq reform, our results highlight that ICD203, properly applied, holds potential to improve precision and accountability of intelligence processes and products.