Public Service Agreements - Measuring Performance

Since 1998, all Northern Ireland government departments have been required to publish Public Service Agreements (PSAs) setting out performance targets for each three-year government spending cycle. These specify the targets to be used to measure performance against key departmental objectives. Progress against the targets is reported and published annually in an NICS-wide publication by the Office of the First Minister and Deputy First Minister (OFMDFM).

It is important that departments have sound data systems for the purposes of monitoring and reporting progress against their PSA targets and to support better performance management. A report published today by the Comptroller and Auditor General, John Dowdall CB, sets out the results of an examination of a number of PSA data systems in place during the 2006-07 financial year.

The report noted that:

On the design of data systems

• A strong corporate control environment is important to ensure robust PSA data systems. We found that senior management tended to be primarily concerned with ensuring targets were in place and that progress against targets was monitored. There was less evidence that they took an active interest in underlying data systems or issues relating to quality control and data accuracy.
• PSA data systems need to be clearly and comprehensively documented and departments are required to publish Technical Notes. The general quality of the Technical Notes we reviewed was inadequate and, in some cases, baselines were not being set and key terms not being defined.
• Technical specialists, such as statisticians, can play an important quality control role in overseeing PSA data systems. Overall, we found that use of specialists across departments was variable. We welcome the undertaking given that statisticians and other professionals are being more formally integrated into the design and monitoring process for newer PSA targets.
• The data systems developed by departments to report progress against PSA targets must be fit for purpose. We found that departments tended to rely on existing data sources, such as administrative databases or established surveys, and there were a number of examples where these were not fully adequate for PSA purposes. Where it is not feasible to develop bespoke PSA data systems, the correct balance should be struck between maximising the use of existing resources and tailoring these for PSA purposes.
• It is also essential that departments disclose details of known weaknesses in their PSA data systems. However we found a general failure to fully disclose such limitations.

On the operation of data systems and reporting of performance

• Methods for the collection, processing and analysis of data must be appropriate to produce valid estimates of target achievement. We identified a number of PSA targets which suffered from methodological weaknesses such as the data system not measuring all aspects of the stated target.
• Where data have been obtained from external sources, it is important that the risks associated with the data are appropriately assessed. We recommend more proactive management, monitoring and control of external data sources.
• Departments should aim to ensure that their PSA data systems can produce performance results in a regular and timely manner. For some PSA data systems, we identified substantial time lags which have prevented timely reporting.
• PSA targets should be consistently stated and should not be subject to unnecessary change. We found a number of cases where departments had made changes without any documented explanation of the need for, and nature of, the changes.
• The introduction of PSAs has formalised the process for reporting performance to the Assembly and the public, and OFMDFM co-ordinates and produces an annual progress report on achievement against PSA targets. We found that the format of this report did not allow for the adequate reporting of target performance.

In his report, the Comptroller and Auditor General acknowledged assurances from departments that, since devolution in May 2007, steps have been taken to ensure that the monitoring and reporting of performance against PSA targets is robust. However these new targets were introduced after the Comptroller and Auditor General’s study commenced and were therefore not examined as part of this report.