DIFFERENCES

Even the most original assessment draws on findings from other studies. But even the simplest synthesis contains original elements-and an assessment of risk and development needs is always original to the agency concerned. The instruments surveyed fall into two broad categories: primary and secondary, with primary instruments undertaking original data collection and analysis and the secondary instruments relying mainly on the data, analysis, and findings of the primary sources.

The predominantly primary instruments are PERs, CFAAs, CPARs, and Fiscal ROSCs. The secondary instruments are DFID assessments and HIPC AAPs (though AAPs have included short field missions, sometimes combined with missions on PERs, CFAAs, and Fiscal ROSCs).3 The new approach proposed for EC assessments and compliance tests falls in between, relying on the findings of other instruments but also carrying out some original analysis-as with the ECs traditional ex post audits. As discussed, the instruments vary considerably in how they gather data and use questionnaires and diagnostic checklists. Coordination and integration issues mainly arise as part of the data collection and analysis performed for PERs, CFAAs, Fiscal ROSCs, and EC audits; they are less important for CPARs given their specialized focus.

The first main difference among the instruments is the level of uniformity in report approaches, scope, and coverage. Fiscal ROSCs are arguably the most uniform: they are applied using a focused, standardized questionnaire and supported by a detailed user manual and very specific format. HIPC AAPs share these characteristics, including the use of questionnaires, a single set of benchmarks and indicators, and a common format. Both are highly organized, carefully structured exercises carried out under tight schedules-another factor that contributes to their consistency. These features stem from one of the main objectives of these instruments: enabling cross-country comparisons of their results.

CPARs, while tailored to country characteristics and problems, also adopt a relatively standardized approach because of their focus on a single technical area. Their uniformity is also aided by the World Banks 15-20 years of experience in assessing procurement systems.4

CFAAs have evolved rapidly since early 2002, and the revised guidelines issued in May 2003 and recent applications have a more uniform scope and content than in earlier years, with a standard format and basic mandatory coverage. The European Commission and many bilateral agencies now

consider CFAAs a core product, particularly for fiduciary risk analysis. But the considerable overlap between CFAAs and other instruments-especially PERs and Fiscal ROSCs-requires clarification (for further discussion see the next section, on issues for further consideration).

Though all PERs contain certain core material (for example, on each countrys macroeconomic and fiscal environment), their range and coverage vary much more than do the other instruments-an outcome that is probably inevitable given the instruments wide scope. It is perhaps partly for this reason that the draft PER guidelines prepared in 2001 were not issued formally. PERs almost always discuss main fiscal issues and key expenditure issues in a countrys major sectors (health, education, transportation, energy, and so on), but the choice of sectors, identification of problems, and depth of analysis differ considerably. For example, most PERs refer to the cost of public sector wages and pensions as part of their fiscal analysis, but only a few devote much attention to civil service or pension reform. Because of their wide reach, PERs take longer to complete and are conducted less often than the other instruments.5

Finally, despite common rhetoric, government ownership varies widely by instrument and by country and is very weak in some cases. It bears repeating that strong government ownership makes it far more likely that the results of any analysis will lead to realistic, sustainable reforms. In countries dependent on aid and with limited capacity, such ownership can be fostered by a government-led PER, which involves governments and development agencies in a continuing dialogue on improving the budget process and increasing local capacity (see the discussion above on Tanzania). In more developed countries with substantial capacity, government leadership ensures genuine ownership- as in Turkey, where the government set the agenda and defined the scope for an integrated approach to the assessment and reform of public expenditure.

INSTITUTIONAL AND GOVERNANCE CONTENT

With the possible exception of the broad constitutional and legal descriptions of the state apparatus in PERs and Fiscal ROSCs, the instruments provide little guidance on defining and accounting for institutional factors in assessments. Nor do any of the guidelines recommend that assessment teams include staff with specific expertise in public management and governance. It seems to be assumed that in most cases economists and financial management and procurement staff can undertake any needed institutional analysis.

Not surprisingly, this approach has led to neglect of institutional considerations and lack of focus and coherence in preparing action plans and designing capacity building initiatives-and occasionally to major mistakes. A review of the guidelines for assessment instruments-and, to a lesser extent, current practices-points to numerous institutional and governance issues that require increased coverage, deeper analysis, or both.

The draft PER guidelines state that, as a first step toward improving budget preparation and execution, an institutional analysis should be undertaken of the key dimensions of public expenditure management, guided by three questions: What formal rules and institutions govern budget management? How effective are they? And how could they be improved or applied more effectively? Many PER reports also address the broad institutional and legal context: constitutional requirements for the budget and its management, the states legal and institutional structure, and intergovernmental relations.

The draft guidelines do not explain how to perform such institutional analysis except to say that capacity issues need to be addressed in a highly participatory way. But PER teams often do not have the skills and experience required to conduct such work. As a result some PERs have confused organizational and institutional issues-and so have made recommendations for changing organizational responsibilities but paid little attention to the institutional necessity of changing the rules of the game and the incentives that lead to organizational forms and behaviors requiring modification.

The recently updated CFAA guidelines make little mention of institutions, yet the content and nature of these reports imply some degree of institutional and governance coverage (World Bank 2002b). In addition, the guidelines refer to the legal framework and institutional arrangements for the countrys public financial management. Though the guidelines do not explicitly discuss corruption, evidence from corruption surveys, Transparency International assessments, World Bank Institutional and Governance Reviews (IGRs), and other reports can be used in CFAAs. The guidelines emphasize the importance of action plans that are clearly justified and prioritized, [and] differentiated in terms of their short, medium and longer term impact. Yet no guidance is given on the extent to which action plans should address the root causes-as opposed to the technical symp-toms-of poor expenditure management.

The CPAR guidelines state that all aspects of a countrys procurement system must be assessed, including its legal and organizational framework, procedures, tools, decisionmaking and control procedures, anticorruption initiatives, and contract administration (World Bank 2002c). Enforcement