IRELAND 24 February 2018
Value for Money Examination 39; Expenditure Review Initiative Summary
In May 1997, the Department of Finance established a process that was intended to result in the efficiency and effectiveness of all government expenditure being reviewed over a three-year period. This process ? referred to in this report as the Expenditure Review Initiative, or ERI ? was part of the more general Strategic Management Initiative (SMI), which aims to move public sector management away from the traditional focus on inputs, to concentrate more on the delivery of outputs and the achievement of intended results.
Reviewing existing public expenditure from a results perspective was intended to provide reliable information for decision makers when they considered whether or not there is a continuing public need for specific areas of public sector activity and, if so, how the delivery of services might be improved. The very rapid increases in government expenditure (up from almost ?12 billion in 1996 to ?21.5 billion in 2001) reinforce the need for comprehensive and regular review of government spending.
This examination was carried out to identify and evaluate the extent to which the ERI achieved its objectives in the period 1997-2000. Specifically, the examination addressed the following questions.
- Was public expenditure comprehensively reviewed over the period?
- Were the expenditure reviews carried out well?
- Were the arrangements put in place to manage the ERI successful?
Evaluation Effort before the ERI
At the time the ERI started, the capacity of government departments to carry out reviews varied considerably, both in terms of technical skills and availability of resources. In some departments, expenditure review systems were already relatively well developed. This was particularly the case in departments with a significant level of EU-funded programmes, since formal expenditure review and reporting procedures are a requirement for EU-funding. In other departments, little or no review of expenditure was undertaken.
The ERI was intended to reduce the degree of variability in the extent to which evaluation was carried out and to make expenditure review an integral part of the management process in all departments and across all programmes. To achieve this, departments needed to build up their capacity in evaluation skills and expertise.
Extent of Expenditure Reviews
The general aim at the outset of the ERI was to review all expenditure programmes every three years. In practice, it soon became clear that this objective would not be achieved.
Schedules of reviews to be carried out were agreed between the Department of Finance and line departments and approved by Government in 1997 and 1999. A total of 118 review subjects were included in these lists. Had all of these reviews been carried out, they would have covered around 50% of government spending, at most.
Many departments were slow to start work on their schedules of reviews and were also slow in carrying out those they did start. By the end of 2000, only 62 of the planned reviews had been completed, and a further 21 were still in progress. The reviews completed or underway related, at most, to 37% of government expenditure. While this body of review work in itself represents a substantial achievement, it falls far short of the level of coverage intended when the ERI started. Variation between departments in the level of coverage achieved was also very significant.
When the ERI was launched, the Department of Finance stated that the aim was to ensure that all spending programmes were reviewed.
Defining the scope of topics for review in terms of spending programmes is somewhat in conflict with the general principles of the SMI, which emphasise results, outputs and outcomes. The alternative is to define the scope of reviews in terms of the objectives and results sought in a particular social or economic area.
Operating on the principle of results-based reviews, the scope of the ERI should also be broadened to include tax ?expenditures? such as tax reliefs and exemptions. While such provisions do not involve direct government expenditure, they nevertheless impact on Exchequer resources and are directed towards the achievement of specific objectives.
Quality of Expenditure Reviews
If reviews are to contribute meaningfully to decision making about public sector spending, they must be carried out to a high professional standard. To assess the quality of the review work carried out in the period 1997-2000, a sample of thirteen review reports ? 20% of the reviews completed up to the end of 2001 ? was selected and examined by expert readers attached to the London School of Economics against agreed criteria.
The brief given to departments under the ERI specified four main elements which reviews were expected to include. These were
- a review of the objectives of the spending being reviewed
- a review of performance indicators relevant to the area being reviewed
- an evaluation of the effectiveness of spending
- an evaluation of efficiency in the area being reviewed.
Many of the individual review reports in the sample assessed achieved the required professional standard in relation to only part of the ERI brief. The evaluations generally were somewhat better at review of objectives and assessment of cost efficiency than in identifying and assessing performance indicators or evaluating effectiveness. Frequently, the existing information base was poor and formal performance indicators were mostly absent or underdeveloped.
Nonetheless, the review process clearly directed some useful attention to many policy areas not regularly or well analysed before. Overall, the expenditure review process provided some additional limited assurance that government expenditures were being spent to some purpose and in a well-directed manner. The review process also served to help the introduction and development of the concept of evaluation in areas of the civil service where previously it had either been non-existent or poorly understood.
The reviews included in the sample for assessment were all completed by July 1999 so that the impacts of the reviews since then could also be investigated. Reviews completed later may have benefited from lessons learned in the first studies and be of a generally higher standard. The Central Steering Committee (CSC) established by the Department of Finance to oversee the ERI considers that there was a quite perceptible improvement in the quality of the evaluations carried out over the period 1997-2000.
Other developments under the SMI may also make it easier in the future for departments to carry out evaluations to the required standard. For example, the production of comprehensive statements of strategy and detailed business plans should ensure that the objectives of spending are defined. Similarly, the development of management information frameworks in departments should help in the definition and recording of relevant performance indicators.
Developing the Review Process
Given the variation in evaluation capacity and experience across departments in 1997, they were unlikely to deliver equally in the period 1997-2000 on the objectives of the ERI or to be able to achieve a consistently high standard in carrying out reviews. This examination found that there is still wide variation in evaluation capacity. Many departments also still have a considerable distance to go to integrate evaluation into decision making. Building up stronger support mechanisms, central guidance and impact reviews should therefore be the priorities in developing the review process.
A comprehensive strategy for the effective development of the ERI over the next planning period should be put in place by the Department of Finance in consultation with the CSC and with line departments. The strategy should establish a clear set of principles to enable areas of expenditure to be prioritised and scheduled for review. This should take account of the need to review key strategic results areas and cross-cutting issues.
The most usual arrangement in departments for carrying out reviews under the ERI was to rely primarily on the line managers in the area under review to carry out the bulk of the work. Since line management may often be too close to programmes to achieve the required degree of objectivity, there was frequently something of a mis-match between the organisation of reviews and their objectives. This may, in part, explain why many reviews failed to address adequately the effectiveness of the spending they examined.
The CSC should put in place a systematic process of quality assessment of reviews, using agreed assessment criteria. Departments should be provided with the assessments in a timely fashion.
The CSC should report on the findings of each review and should, in conjunction with departments, institute a formal process for monitoring review impacts.
Review reports should, where possible, be published, together with the CSC assessment of the quality of the review.