Continuous Improvement in Policing 2021

Executive Summary

Background

1. The Northern Ireland Policing Board (the Board) was established on 4 November 2001 by the Police (Northern Ireland) Act 2000, which put the recommendations of the Patten Report on policing into practice. At the same time, the Police Service of Northern Ireland (the PSNI) came into being, replacing the Royal Ulster Constabulary.

2. Section 28 of the Police (Northern Ireland) Act 2000 requires the Board to make arrangements to secure continuous improvement in the way in which it functions, and those of the Chief Constable, are exercised, having regard to economy, efficiency and effectiveness. The Board is required to prepare and publish a Performance Plan for each financial year. The Board also has to prepare and publish a performance summary in respect of the previous year.

Basis and scope of the audit by the Comptroller and Auditor General

3. As the Comptroller and Auditor General (C&AG) for Northern Ireland, I am required under Section 29 of the Police (Northern Ireland) Act 2000 to audit the Policing Plan and performance summary and to send a report to the Board, the Chief Constable and the Department of Justice for Northern Ireland (the Department).

The C&AG’s certificate and audit opinion to the Assembly on the Northern Ireland Policing Board’s Performance Summary and Performance Plan

4. In accordance with section 29 of the Police (Northern Ireland) Act 2000 as amended, I certify that I have audited the Board’s and the PSNI’s:

Performance summary for the year ended 31 March 2021; and
Annual Performance Plan for the year ended 31 March 2022.

Basis of my opinion

5. I planned and performed my work to obtain all the information and explanations that I considered necessary in order to provide an opinion on whether:

(i) for the 2020-21 Performance Summary – the Board has prepared and published a summary assessment of its own and the Chief Constable’s performance in 2020-21, measured by performance actions and performance measures against the Board’s Business Plan and the Annual Performance Plan 2020-21 respectively. My work included examination, on a test basis, of the evidence supporting both the objectives and actions, and the performance indicators and measures as set out in the above documents; and,

(ii) for the 2021-22 Annual Performance Plan – arrangements are in place to secure continuous improvements; the Performance Plan includes those matters prescribed in legislation; the arrangements for publishing the Performance Plan complied with those requirements; and that the performance indicators and measures are reasonable.

Main findings of my review

6. I have given an unqualified audit opinion on the 2020-21 Annual Assessment and the 2021-22 Annual Performance Plan (Appendix 1). I have raised some issues and recommendations for the attention of the Board and the PSNI, details of which are outlined in the following paragraphs.

On 2020-21 performance – The Policing Board and the PSNI

7. Performance actions and measures are published for both the Board and the PSNI in the Board’s Business Plan and the Annual Performance Plan respectively.

8. The Board’s Business Plan for 2020-21 included 13 performance actions underpinning four corporate objectives. The 2020-21 Annual Performance Plan set out nine indicators with 11 performance measures. The same three overarching outcomes align both the Board’s Business Plan and the Annual Performance Plan as well as the draft Programme for Government and the Department of Justice (DoJ) Corporate Plan. These are:

We have a safe Community;
We have confidence in Policing; and,
We have engaged and supportive communities.

9. The Board has reported its own performance in its Annual Report for 2020-21. Overall the Board reported that in 2020-21, 11 of its 13 actions (85 percent) (2019-20: 73 percent) had been achieved and the remaining two actions were partially achieved. The Board’s Annual Report for 2020-21 also provided a narrative summary of PSNI’s performance against the Annual Performance Plan outcomes but it did not conclude the status of each indicator at the year-end as having been achieved or otherwise.

10. The PSNI has reported its own performance against the Annual Performance Plan 2020-21 in its Annual Report for the same year. The reporting against the outcomes, indicators and measures encompasses a large volume of both narrative and numerical information, with detail of the numerous underlying operational activities.

11. In my previous report, I recommended the PSNI should include an assessment of each measure at the end of the reporting period. I am pleased to see this year that PSNI have included its assessment of the status of each of the performance indicators at 31 March 2021; reporting that, in its opinion, of the total nine indicators, three indicators (33 per cent) had been fully achieved, five (56 per cent) indicators were partially achieved and the remaining one (11 per cent) indicator was not achieved.

On the 2021-22 Annual Performance Plan

12. The Annual Performance Plan for 2021-22 supports the second year of The Northern Ireland Policing Plan 2020-25 and comprises nine indicators and 13 measures (2020-21: nine indicators and 11 measures) across the same three overarching outcomes noted above. A Policing Plan Review Working Group (PPRWG), comprising both Board members and PSNI representatives, was established in November 2020 to take forward the statutory requirement to review the Annual Performance Plan 2020-21 and propose any changes to the new Performance Plan for 2021-22. As a result, there are two additional measures included in the 2021-22 Performance Plan relating to Neighbourhood Policing Team initiatives and Partnership Working with local communities. The other 11 measures are carried forward from the previous 2020-21 Annual Performance Plan and remain unchanged.

13. Whilst described as measures, there is still limited data included against which some of the measures can be quantified, for example, Indicator 1.2: ‘Fewer repeat offenders of crime’ as baselines have yet to be developed and Indicator 2.1: ‘The level of public confidence in policing’ where the Policing Plan survey remains in development.

14. If this data is not available then the value of these indicators could be reduced as it may prove to be more difficult for PSNI to demonstrate achievement or improvement of individual measures. It will also be difficult for the Board to prepare an assessment of PSNI’s annual performance measured by reference to performance indicators and the extent to which these were met as required by Section 28 (5) of the Act.

15. The extent that PSNI’s annual performance cannot be quantified against the measures and indicators stated in the Annual Performance Plan may make it more difficult for the Board to prove in future that it has met its obligation to make arrangements to secure continuous improvement having regard to economy, efficiency and effectiveness as per Section 28 (4) of the Act. Part 4 of this report will look at this issue in more detail.

16. I had recommended in my previous year’s reports that the Annual Performance Plan would benefit from the inclusion of baseline data as it becomes available. Given this is the second Annual Performance Plan under the Northern Ireland Policing Plan 2020-25, I would have envisaged that year one data, as a minimum, could have been included as a baseline upon which to compare performance. It is disappointing that this has not been the case.

On the continuous improvement arrangements

17. In my previous year’s report, I noted that continuous improvement arrangements were not included as an annex within the Annual Performance Plan 2020-21 and as such, I would consider the implications of this going forward. This remains the Board’s position again for the 2021-22 Annual Performance Plan.

18. Section 28 (4) of the Police (Northern Ireland) Act 2000 requires the Board’s Annual Performance Plan to contain details of its arrangements to secure continuous improvement in the way in which it functions and those of the Chief Constable regarding economy, efficiency and effectiveness.

19. To satisfy this requirement, the Board has stated that both the Northern Ireland Policing Plan 2020-25 and the subsequent Annual Performance Plans have been developed using an Outcomes Based Accountability Framework (OBA). This new approach is in line with the draft Programme for Government (PfG).

20. The key features of the OBA approach include working in reverse from a set of desired outcomes, the use of indicators and performance measurement – ‘How much did we do?’, ‘How well did we do it?’ and ‘Is anyone better off?’. The validity of the approach requires good quality and relevant datasets; and baseline information is fundamental. However, my initial observations are, that the ‘evidence based monitoring, analysis and evaluation’ which the Board has stated as ‘valued’ in the introduction of the Northern Ireland Policing Plan 2020-25, appears incomplete for a number of the performance indicators.

21. This may be at odds with the OBA methodology and raises concern how the Board can continue to make a considered performance assessment and, how it will fulfil its continuous improvement obligations under Section 28 of the Act. This is discussed further in Part 4 of this report.

Update on the legislation supporting the continuous improvement arrangements

22. As I have reported in a number of previous reports, in my opinion, it is important that the DoJ considers changes to the legislation underpinning the audit of continuous improvement, which has been in place since 2000. The role of the C&AG defined in the 2000 Act in respect of continuous improvement here now appears to be unique in the United Kingdom (UK) with the Department finding that many of the corresponding provisions to the Police (Northern Ireland) Act 2000 in the UK have been repealed and, for the most part, ‘best value’ arrangements in England and Wales no longer apply to the police. Any such proposal would be subject to Ministerial approval.

Summary of Recommendations

The Performance Plan requires the inclusion of up to date and relevant data and baseline information for all indicators and measures. This is essential in order to monitor performance, to establish targets and for the evaluation of success against the measures, indicators and outcomes stated in the Annual Performance Plan.

The Board should undertake a review of existing arrangements to ensure that it is content with the new approach under the OBA Framework; that it is able to fully support the requirement to assess PSNI performance and as such make arrangements to secure Continuous Improvement within the PSNI as required by Section 28 of the Act

Part One: Introduction and Background

Responsibilities of the Northern Ireland Policing Board

1.1 Under Section 28 of the Police (Northern Ireland) Act 2000, the Northern Ireland Policing Board (the Board) is required to make arrangements to secure continuous improvement in the way in which its functions, and those of the Chief Constable of the Police Service of Northern Ireland (the PSNI), are exercised, with regard to economy, efficiency and effectiveness.

1.2 The Board must prepare and publish a performance plan for each financial year, containing details of how these continuous improvement arrangements are to be implemented. In particular, the Performance Plan must:

identify performance indicators, by reference to which performance in exercising functions can be measured;
set performance outcomes to be met in relation to those performance indicators; and
include a summary of the Board’s assessment of:
- it’s, and the Chief Constable’s, performance in the previous financial year, measured by reference to performance indicators and measures; and
- the extent to which any performance standard, which applied at any time during that year was met.

1.3 In practice, the Board works in partnership with the PSNI to develop the Performance Plan and monitor and review progress in its implementation, within an overall context of continuous improvement.

Responsibilities of the Comptroller and Auditor General

1.4 Under Section 29 of the Police (Northern Ireland) Act 2000, I am required to audit the Performance Plan (including the assessment of the previous financial year’s performance) to establish whether it was prepared and published in accordance with the requirements of Section 28 of the Act. Accordingly, I must issue a report (Appendix 1 and 2):

certifying that I have audited the Performance Plan;
stating whether I believe the Performance Plan was prepared and published in accordance with the requirements of Section 28;
stating whether I believe the performance indicators and measures are reasonable and, if appropriate, recommending changes to them;
if appropriate, recommending how the Performance Plan should be amended so as to accord with the requirements of Section 28; and
recommending whether the DoJ should give a ‘direction’ to the Board, under Section 31 of the Police (Northern Ireland) Act 2000. Such a direction would require the Board to take appropriate corrective action to ensure compliance with the Act.

1.5 Under Section 30 of the Police (Northern Ireland) Act 2000, I may carry out an examination of the Board’s compliance with the requirements of Section 28 of the Act.

1.6 In October 2021, I published a similar report summarising my audit of the previous (2019-20) Annual Performance Plan. My report at that time contained a number of recommendations that are presented at Appendix 2, along with an update from the Board on progress of such.

Scope of the review

1.7 During the course of the review, my staff liaised closely with the Policing Board and the PSNI. My findings are set out as follows:

Part Two: Review of Performance: summarising the Board’s and the Chief Constable’s performance in 2020-21 and the extent to which performance standards were met;
Part Three: Review of the Annual Performance Plan: considering whether the Performance Plan meets the Board’s statutory obligations and whether proposed performance indicators and measures are reasonable; and
Part Four: Continuous Improvement: reviewing the arrangements made by the Board to secure continuous improvement and, examining how such arrangements are being implemented.

Part Two: Review of Performance

Introduction

2.1 The Board is required to report, each year, a summary of its assessment of:

it’s, and the Chief Constable’s, performance in the previous financial year, measured by reference to performance indicators; and
the extent to which any performance standard which applied at any time during that year was met.

2.2 In this part of the report, I consider whether the performance summary published in the Annual Reports for 2020-21 meet the statutory obligations of the Board and PSNI.

Policing Board performance

2.3 The Board’s Corporate Plan for the period 2020-2023 sets out the Board’s Purpose, Vision and Values and outlines the key challenges envisaged during the period. The Board has set four key objectives based on the three overarching outcomes in line with the outcomes in the Northern Ireland Policing Plan 2020-25. See Figure 1.

2.4 The detail of how the Board will deliver these four key objectives is set out in the Corporate Plan and is published within the Annual Business Plan for 2020-21. For each of the four objectives, a number of associated actions have been identified (13 in total) which will be used by the Board to gauge its progress in achieving the overall objective.

2.5 In accordance with the legislation, the Board’s performance was summarised in its 2020-21 Annual Report. Figure 1 below presents a summary of the Board’s performance against the key objectives and associated actions as set out in the Annual Business Plan 2020-21. Of the 13 actions identified, 11 are reported as fully achieved (85 per cent) and the remaining 2 reported as partially achieved (15 per cent).

2.6 The two actions that were reported as partially achieved at 31 March relate to delays in publication of the ‘Human Rights Monitoring Framework’ and guidance to ‘Selected Medical Practitioners and Independent Medical Referees’.

Figure 1: Summary of progress against Objectives and Actions

Objectives	Actions	Achieved / Partially Achieved / Not Achieved
A. To monitor resourcing plans for the PSNI; advocating on issues which support policing, including transformational change, and delivery of a representative service.	i. Assess and approve PSNI senior leadership staffing requirements and progress appointments to fill vacancies to agreed timescales.	Achieved
ii. Assess police service resourcing requirements, key transformational projects and legacy issues; advocating for PSNI when appropriate.	Achieved
iii. Assess and monitor police service composition levels advocating for the PSNI on initiatives which increase community representativeness.	Achieved
B. To monitor, oversee and assess the performance of the PSNI through the Board and its Committees and ensure the delivery of Human Rights based, community focussed policing.	i. Deliver the 2020-21 Performance Plan in line with the 2020-2025 Northern Ireland Policing Plan	Achieved
ii. Develop and implement a framework to assess PSNI’s compliance with the Human Rights Act (1998)	Partially Achieved
iii. Implement the Board’s action from the Local Policing Review	Achieved
iv. Deliver and report on the work of the Board and its Committees	Achieved
C. To work collaboratively with the community, PCSPs and partners, to deliver the outcomes for policing and allow them to be informed and engaged with the Board’s work.	i. Support the delivery of PCSP Action Plans which contribute to increased community safety and confidence in policing	Achieved
ii. Deliver a Communications Action Plan	Achieved
iii. Develop and deliver an Engagement Strategy to support the Board’s outcomes	Achieved
iv. Deliver a funding programme to support and increase community engagement with policing	Achieved
D. To deliver independent, fair and transparent processes for former and serving officers in line with Police Pension, Injury Benefit and Appeal legislation.	i. Develop and implement revised guidance to our Selected Medical Practitioners and Independent Medical Referees	Partially Achieved
ii. Progress any recommendations made by the Northern Ireland Audit Office from their reporting on the Northern Ireland Injury Benefit Scheme	Achieved

Source: NIAO summary based on the NIPB Annual Report 2020-21.

2.7 The performance information in the Board’s Annual Report reproduces the detailed information from the Board’s Business Plan and is consistent with it, supporting the higher-level Corporate Plan outcomes. The Annual Report also contains a progress summary and status of each of the Business Plan actions as at 31 March 2021.

PSNI performance 2020-21

2.8 The Northern Ireland Policing Plan 2020-25 sets out three overarching outcomes that the Board have set for policing to deliver over the five-year period. The Annual Performance Plan for 2020-21 includes nine indicators and 11 measures, which are used to measure progress in the delivery of the first year of the Policing Plan 2020-25. A summary of PSNI’s progress against the Annual Performance Plan 2020-21 is set out at Figure 2.

Figure 2: Summary of PSNI’s progress against Annual Performance Plan 2020-21

Outcome	Indicator	Measures	Achieved / Partially Achieved / Not Achieved
1. We have a safe community	1.1 Fewer repeat victims of crime	1.1.1 Repeat victimisation rate and report on initiatives to support repeat victims with a focus on victims of (i) Domestic Abuse, (ii) Child Sexual Abuse and Exploitation (CSAE) and (iii) Hate Crime in 2020-21.	Partially Achieved
1.2 Fewer repeat offenders of crime	1.2.1 Repeat offending rate and report on initiatives to reduce repeat offenders with a focus on Domestic Abuse in 2020-21. 1.2.2 Repeat offending of Organised Crime Groups (OCGs) and paramilitary organisations.	Partially Achieved
1.3 People in all communities feel safe	1.3.1 Number of people in Northern Ireland who feel safe in their; local area, local high street or town centre and own home. 1.3.2 Rate of places repeatedly victimised.	Partially Achieved
1.4 Crime rates and trends showcase an effective police response	1.4.1 Benchmark PSNI crime rates against previous PSNI levels and other most similar police services	Achieved
2. We have confidence in policing	2.1 The level of public confidence in policing	2.1.1 Number of people in Northern Ireland who are confident that PSNI is accessible, visible, responsive and victim focused.	Not Achieved
2.2 The level of satisfaction with the service received	2.2.1 Number of victims who are satisfied with the service they have received.	Achieved
2.3 The representativeness of the police service	2.3.1 Improve representativeness of the service across ranks, grades and departments by gender and community and socio-economic background.	Partially Achieved
2.4 Delivery of effective crime outcomes	2.4.1 Levels of crime outcomes to identify and respond to areas of concern in outcomes statistics, with a particular focus on domestic abuse in 2020-21.	Achieved
3. We have engaged and supportive communities	3.1 Police in partnership with local communities, including PCSPs, identify and deliver local solutions to local problems.	3.1.1 In collaboration with the community deliver the commitments outlined in the Local Policing Review.	Partially Achieved

Source: NIAO summary based on Annual Performance Plan 2020-21and PSNI Annual Report and Accounts 2020-21

2.9 Both the Board and the PSNI have reported on the PSNI’s performance against the 2020-21 Annual Performance Plan. The Board’s assessment of PSNI’s performance should be read in conjunction with that of the PSNI’s assessment for a full understanding.

2.10 The Board published its assessment of the PSNI’s performance against the 2020-21 Performance Plan in its Annual Report 2020-21. The narrative summarises how the PSNI reported how it met each performance indicator through its OBA Report Cards that were presented to the Board via one of its Performance, Partnership or Resources Committees. However this assessment does not make any reference to baseline data or established targets, nor does it include any detail as to the status of each indicator at the year-end as having been achieved or otherwise.

2.11 The PSNI has reported its own performance against the Annual Performance Plan 2020-21 in its Annual Report for the same year. The reporting against the outcomes, indicators and measures encompasses a large volume of both narrative and numerical information, with detail of the underlying operational activities.

2.12 In my previous report, I recommended for the PSNI to include an assessment of each measure at the end of the reporting period. I am pleased to see that PSNI have included a table summarising the designated status of each of the performance indicators at 31 March 2021 as ‘achieved’, ‘partially achieved’ or ‘not achieved’. PSNI reported that of the nine indicators within the Annual Performance Plan for 2020-21, three were ‘achieved’, five were ‘partially achieved’ and one was ‘not achieved’. Additionally, for each indicator, PSNI have included a summary identifying the position in respect of data development, baselines and surveys; this is also a welcome addition to this year’s assessment.

2.13 However, I note that the survey data or baselines are summarised as ’in development’ or ‘to be developed’ for four of the six indicators which are categorised as ‘partially’ or ‘not achieved’. Such information is essential to enable PSNI to demonstrate progress against the indicators and outcomes of the Annual Performance Plan.

2.14 Now that the Board is progressing the OBA Framework to develop both the Northern Ireland Policing Plan 2020-25 and the subsequent Annual Performance Plans it has become more important to ensure that there is a current data set and relevant baselines against which to judge progress. Without these it is difficult to understand how the three performance measures synonymous with OBA - “How much did we do? How well did we do it? Is anyone better off?” can be measured.

Conclusion

2.15 The Board has consistently disclosed its own performance against the Corporate Plan and the Business Plan in its Annual Report and Accounts. The Board and the PSNI, in their own respective Annual Reports have reflected the outcomes and measures included in the 2020-21 Annual Performance Plan, albeit not all indicators can be assessed due to the lack of data and baseline information. Presenting an overall conclusion as to the annual progress against the Annual Performance Plan for 2020-21 as a whole would be beneficial for the reader once all baselines and data trends are available. Whilst it is early in the OBA process, without the baselines and other data for comparison, it will become increasingly difficult for the Board to demonstrate that it can fully comply with its requirements under Section 28 with regards to continuous improvement.

Recommendation

Part Three: Review of the Performance Plan 2021-22

Introduction

3.1 In this part of the report, I consider;

whether the Annual Performance Plan for 2021-22 meets the Board’s statutory obligations; and
whether the proposed performance measures are reasonable.

Development of the Annual Performance Plan 2021-22

3.2 Section 25 (1) of the Police (Northern Ireland) Act 2000 provides for the Board to determine the objectives for the policing of Northern Ireland. In 2020 I reported that the Board had developed in partnership with the PSNI, an updated five year plan covering the period 2020 to 2025 known as ‘The Northern Ireland Policing Plan 2020-2025’.

3.3 The Northern Ireland Policing Plan 2020-2025, sets out three overarching outcomes for policing to deliver over the next five years. It will be accompanied by Annual Performance Plans which will contain indicators and measures used to evaluate the PSNI’s progress on a twelve month basis against the outcomes.

3.4 In November 2020, a Policing Plan Review Working Group (PPRWG) was established to take forward the statutory requirement to review the Northern Ireland Policing Plan 2020-25, the Annual Performance Plan 2020-21 and propose a new Performance Plan for 2021-22. This working group comprises both Board members and PSNI representatives.

Review of Performance measures and indicators

3.5 The Annual Performance Plan for 2021-22 includes nine indicators and 13 measures (Figure 3) covering the three outcomes and as such meets the Board’s statutory obligations for this.

Figure 3: Annual Performance Plan 2021-22

Outcome	Indicator	Measures
1. We have a safe community	1.1 Fewer repeat victims of crime	1.1.1 Repeat victimisation rate and report on initiatives to support repeat victims with a focus in 2021-22 on victims of (i) Domestic Abuse, (ii) Child Sexual Abuse and Exploitation (CSAE) and (iii) Hate Crime.
1.2 Fewer repeat offenders of crime	1.2.1 Repeat offending rate and report on initiatives to reduce repeat offenders with a focus on Domestic Abuse in 2021-22.
1.2.2 Through Frustrated, Disrupted and Dismantled activity, reduce the capacity and capability of Organised Crime Groups (OCGs) and paramilitary organisations to engage in criminal activity.
1.3 People in all communities feel safe	1.3.1 Number of people in Northern Ireland who feel safe in their; local area, local high street or town centre and own home.
1.3.2 Rate of places repeatedly victimised.
1.4 Crime rates and trends showcase an effective police response	1.4.1 Benchmark PSNI crime rates against previous PSNI levels and other most similar police services.
2. We have confidence in policing	2.1 The level of public confidence in policing	2.1.1 Number of people in Northern Ireland who are confident that PSNI is accessible, visible, responsive and victim focused.
2.2 The level of satisfaction with the service received	2.2.1 Number of victims who are satisfied with the service they have received.
2.3 The representativeness of the police service	2.3.1 Improve representativeness of the service across ranks, grades and departments by gender and community and socio-economic background.
2.4 Delivery of effective crime outcomes	2.4.1 Levels of crime outcomes to identify and respond to areas of concern in outcomes statistics, with a particular focus on domestic abuse in 2021-22.
3. We have engaged and supportive communities	3.1 Police in partnership with local communities, including PCSPs, identify and deliver local solutions to local problems.	3.1.1 In collaboration with the community deliver the commitments outlined in the Local Policing Review.
3.1.2 Identify and report on the Neighbourhood Policing Team initiatives to address local problems and tackle local issues, including co-designed solutions, in line with Neighbourhood Policing Guidelines.
3.1.3 Assess and evaluate the impact of partnership working with local communities, including but not exclusively, in areas of high deprivation and areas that have been repeatedly victimised.

Source: NIAO Summary based on Annual Performance Plan 2021-22.

3.6 The two additional measures (3.1.2 and 3.1.3) approved by the Board for inclusion in the Annual Performance Plan for 2021-22 support indicator 3.1 under the overarching outcome ‘We have engaged and supported communities’. These indicators will measure initiatives undertaken by the Neighbourhood Policing Teams and the impact of partnership working with local communities.

3.7 In previous reports, I have recommended the need for clear and appropriate outcomes, linked indicators and measures and the fundamental inclusion of baseline information. Together these can demonstrate positive impacts for citizens, communities, and secure the continuous improvement of the organisation as a whole.

3.8 While the link between outcomes, indicators and measures is clear within the Annual Performance Plan 2021-22; the lack of baseline data and established targets is particularly concerning given this is the end of the second year of the five year Policing Plan 2020-25.

3.9 Without baseline data and established targets it may be difficult for the Board to fully satisfy Section 28 of the Act, and make an assessment of the PSNI’s annual performance ‘measured by reference to performance indicators’. I refer to paragraph 2.13 of this report where I note that the PSNI recorded in its Annual Report 2020-21 that four of the six indicators reported for the Performance Plan of the same year were ‘partially’ or ‘not achieved’ due to the unavailability of current data and baselines. Without inclusion of such information going forward, it may raise the same questions of future assessment with regards to the OBA framework methodology and the Board’s requirements under Section 28 of the Act.

Conclusion

3.10 The Board and PSNI should continue to develop the OBA approach through clear defined outcomes linked to indicators and quantifiable measures. The inclusion of up to date and relevant data and baselines that can be used to clearly demonstrate performance is fundamental to this.

Recommendation

The recommendation is the same as that at the end of Part 2.

Part Four: Operation of the Continuous Improvement programme

Introduction

4.1 Section 28 (4) of the Police (Northern Ireland) Act 2000 requires the Board to make arrangements to secure continuous improvement within it’s and the PSNI’s functions, having regard to a combination of economy, efficiency and effectiveness. The Board is also required to carry out reviews of the way in which its functions are exercised.

Continuous improvement within the Policing Board

4.2 The Board’s previous Continuous Improvement programme 2017-20, supported the action plan for the Board to obtain the Investors in People (IIP) reaccreditation. Due to be completed in May 2020, I reported in my previous report that COVID-19 had affected the Board’s ability to meet this initial date; and while a Strategic Review was conducted in December 2020, a rescheduled accreditation date planned for October 2021 has also been missed.

4.3 The Board has cited the continued impact of the pandemic alongside the resignation of the CEO in September 2021 as the main factors which have prevented the Board from progressing this matter and undertaking the necessary steps for a full reaccreditation. The Board has not provided a planned future date for completion of this reaccreditation and no other continuous improvement projects are reported as planned within the Board at present.

Continuous improvement within the PSNI

4.4 In my previous report, I concluded on the completion of the PSNI Continuous Improvement Programme 2016-20, and noted that the 2020-21 Annual Performance Plan made no reference to future Continuous improvement projects. As such, this is the first year I will be reporting on Continuous Improvement within the PSNI under the Board’s newly adopted OBA framework methodology.

4.5 The development of the Policing Plan 2020-25 follows the OBA framework and aims to embed continuous improvement throughout the design and monitoring of the outcomes of the Annual Performance Plans. To facilitate this new methodology, the PPRWG reconvened to support the Board in its review of PSNI past performance; and to challenge future Annual Performance Plans, ensuring that the measures within remain appropriate and relevant. The monitoring of the individual measures and indicators is delegated to the Board’s Performance, Partnership or Resources Committees. PSNI provide monthly information to the Board using the OBA methodology in ‘report card’ format, setting out the three OBA performance measures “How much did we do? > How well did we do it? > Is anyone better off?”

4.6 The Board’s new approach to securing continuous improvement using the OBA framework methodology relies on four key documents:

Performance Plan Timetable;
OBA Report Card (produced by PSNI) How much did we do? > How well did we do it? > Is anyone better off?;
Board Analysis (scrutiny of OBA report card - identifying issues or opportunities); and
PSNI Update (presentation to the Board).

4.7 To obtain assurance that the Board’s new approach is fully compliant with Section 28 (4) of the Act and, in particular, the Board’s requirement to make arrangements to secure continuous improvement within the PSNI we selected and tested the following two performance indicators:

1.3 ‘People in all communities feel safe’; and
2.1 ‘The level of public confidence in policing’ for further examination.

4.8 These indicators were designated as ‘partially achieved’ and ‘not achieved’ respectively within the PSNI Annual Report for year-end 31 March 2021. The audit work therefore focused on reviewing the underlying supporting documentation, in particular the four key documents listed above.

Figure 4: Summary of NIAO findings from key documentation review

Indicator 1.3 - People in all communities feel safe

Document	Findings
Performance Plan Timetable 20-21	Partnership Committee minutes were dated 17 December 2020.
OBA Report Card	Measure 1.3.1: Number of people in Northern Ireland who feel safe in their: local area, local high street or town centre and own home. We noted the latest published findings available to assess this measure are based on Northern Ireland Safe Community Survey 2018-19. Measure 1.3.2: Rate of places repeatedly victimised We noted recorded crime figures are available for financial year to date.
Board Analysis	We noted that the Partnership Committee identified the following: ‘impacts provided within the PSNI report card fall short in addressing measures with regard to the number of people who feel safe and the rate of places repeatedly victimised’. and the opportunity ‘to develop survey work which could allow for greater understanding on policing issues and develop robust baseline data to support the effective delivery of the Policing Plan 2020-2025’.
PSNI Update	We noted that a PSNI Performance officer highlighted ‘that whilst there is good information in the reports, there remains a lack of impact information to fully evaluate progress in the areas’.

NIAO View: Overall we conclude there is inadequate data and baseline information for indicator 1.3.

Indicator 2.1 - The level of public confidence in policing

Document	Findings
Performance Plan Timetable 20-21	Partnership Committee minutes were dated 17 September 2020.
OBA Report Card	Measure 2.1.1 – Number of people in Northern Ireland who are confident that PSNI is accessible, visible, responsive and victim focused. We noted the latest published findings available are based on the Northern Ireland Crime Survey (replaced by the Northern Ireland Safe Community Survey) 2017-18. The 2018-19 findings were due to be published in October 2020 but have been delayed due to COVID19. We noted the 2019-20 findings are due to be published May 2021.
Board Analysis	We noted that the Partnership Committee identified the following: ‘there is a lack of data available to measure confidence in policing, and the data available is somewhat dated. These factors subsequently mean a significant lack of data provided by PSNI to evidence their activity on this Measure.’ and that there is a ‘lack of data, including dated data, and the cessation of the Omnibus Survey, there is an opportunity to establish a new and bespoke survey to inform multiple areas of policing, including perceptions and confidence in policing‘.
PSNI Update	We noted that a PSNI representative discussed ‘the difficulties around the absence of the Omnibus Survey as a key issue’.

NIAO View: Overall we would conclude there is inadequate data and baseline information for indicator 2.1.

Source: NIAO Summary of review of 4 key documents – Performance Timetable, OBA Report Card, Board Analysis and PSNI update to the Board

4.9 Our review of the key documents identified the same issue across both indicators and measures selected; that there was a lack of up to date and relevant data on which to adequately assess the current performance and establish baseline information. Figure 4 presents a summary of findings from the key documents which support our view of this.

4.10 While I understand scoping work has commenced for a new Northern Ireland Policing Plan Survey, the completion of this should be the highest priority for the Board given that four indicators in the 2020-21 Performance Plan were assessed as ‘partially’ or ‘not achieved’ due to the unavailability of current data and baselines. In Part 3 of this report, I had documented my concern regarding the need for baseline data and established targets to enable the PSNI to demonstrate improvement and for the Board to assess the PSNI’s annual performance ‘measured by reference to performance indicators’ as required by Section 28 of the Act.

4.11 The lack of current data and baselines makes it difficult to reconcile the rationale stated in the Board’s Annual Report 2020-21 in adopting the OBA methodology. The Board values ‘evidence-based monitoring, analysis and evaluation’ and that the OBA approach ‘enables the Plans to focus on outcomes which clearly demonstrate continuous improvement in police performance and a positive impact within the community ’.

4.12 The three performance measures synonymous with OBA “How much did we do? > How well did we do it? > Is anyone better off?” can only be facilitated with up to date data and relevant baselines with success defined as ‘doing better than the baseline’ and ‘turning the curve’. Whilst I have given an unqualified opinion this year and acknowledge that the Board has made some progress through its implementation of the new OBA approach, there remains a significant gap because of the lack of data and baseline information currently available.

4.13 An urgent and concerted effort is therefore needed to remedy what is required to ensure that going forward the Board can demonstrate that it has in place adequate arrangements to secure continuous improvement as required by Section 28 of the Act in its current adoption of the OBA methodology.

Conclusion

4.14 At this point of the five year Policing Plan 2020-25 only five of the nine indicators are supported by data and baseline information enabling an assessment of performance to be made. I intend to revisit this area in more detail in my future reports as the new OBA methodology becomes fully embedded.

Recommendation

Appendix 1: The Comptroller and Auditor General’s certificate and opinion to the Assembly on the Northern Ireland Policing Board’s Performance Plan and Performance Summary

1. In accordance with Section 29 of the Police (Northern Ireland) Act 2000 as amended, I certify that I have audited the Northern Ireland Policing Board and Police Service of Northern Ireland’s:

performance summary for the year ended 31 March 2021; and
annual performance plan for the year ended 31 March 2022.

Basis of my opinion

Audit of the Performance Summary

2. I planned and performed my work to obtain all the information and explanations that I considered necessary in order to provide an opinion on whether the Northern Ireland Policing Board has prepared and published a summary of the Board’s assessment of its own, and the Chief Constable’s, performance in 2020-21, measured by reference to performance objectives and performance indicators.

3. My work comprised a review and assessment and, where appropriate, examination on a test basis of the evidence supporting performance against the indicators and measures as prescribed in the 2020-21 Annual Performance Plan. I obtained sufficient evidence to satisfy myself that the summary provided includes those matters prescribed in legislation, and that the arrangements for publishing the summary complied with those requirements.

Audit of the Performance Plan

4. I planned and performed my work to obtain all the information and explanations that I considered necessary in order to provide an opinion on whether:

the plan has been prepared and published in accordance with statutory requirements; and
the performance indicators and performance measures for 2021-22 are reasonable.

5. My work comprised a review and assessment of the plan and, where appropriate, examination on a test basis of relevant evidence sufficient to satisfy me that arrangements to secure continuous improvements are in place, that the plan includes those matters prescribed in legislation, and that the arrangements for publishing the plan complied with those requirements.

Opinion

6. In my opinion:

the Northern Ireland Policing Board has prepared and published its and the Police Service of Northern Ireland’s performance summary for the year ended 31 March 2021 in accordance with the requirements of Section 28 of the Police (Northern Ireland) Act 2000;
the Northern Ireland Policing Board has prepared and published its Performance Plan for the year ended 31 March 2022, as required by the Police (Northern Ireland) Act 2000; and
the performance indicators included within the Performance Plan 2021-22 are reasonable.

Recommendation to the Department of Justice

7. Under Section 29 of the Police (Northern Ireland) Act 2000, I am required to make a recommendation as to whether the Department of Justice should issue a direction to the Policing Board under Section 31 of the Act.

8. On the basis of my work, I do not recommend that the Department of Justice issues a direction under Section 31 of the Police (Northern Ireland) Act 2000.

Kieran Donnelly
   Comptroller and Auditor General
   1 Bradford Court
   Galwally
   Belfast
   BT8 1EU

30 June 2022

Appendix 2: Progress on Recommendations in the Comptroller and Auditor General’s Report

NIAO Continuous Improvement Arrangements in Policing Report 2021

NIAO Recommendations

The PSNI’s performance summary should include an assessment of each measure at the end of the reporting period. A summary of progress against the Annual Plan would be beneficial.

NIPB Management Response

Accepted - The Board, in partnership with the PSNI, is continuing to develop the reporting of performance in line with the Outcomes Based Accountability principles. This includes improving how impact is recorded which will support an overall assessment of progress against each Measure within the Policing Plan. The Policing Plan Review Working Group (PPRWG) is taking the reporting of impacts forward as a key priority.

Progress

In line with the full update provided within the next recommendation progress, the PPRWG examined this as a key issue during September 2021 to January 2022.
The next Annual Reporting of PSNI Performance is due to take place from 1 April 2022 and in line with last year’s and this year’s Annual Assessment, an evidence based assessment of each Measure will be undertaken against the identified impacts.
In addition, within this year’s Board’s Annual Assessment of the PSNI Performance a summary of progress will be included, in line with the NIAO recommendation, by way of a RAG status.

NIAO Recommendations

The Performance Plan would benefit from the inclusion of baseline data as this becomes available. This would allow the progress of individual measures to be monitored; and the establishment of data trends would support the evaluation of success against outcomes and the resulting positive impact for citizens.

NIPB Management Response

Accepted - The need to establish robust baseline data was identified as a key priority by the Board in its 2020-21 reporting year. As such work is being taken forward at pace between the PSNI and the Board, through the PPRWG, to identify baseline data from which to measure PSNI performance for the duration of the Policing Plan period (2020-2025).

In addition, the cessation of the Northern Ireland Omnibus Survey in 2018, resulted in a gap in terms of available data to measure public satisfaction in policing and PCSPs and for the Board to monitor confidence. In order to fill this gap, the Board has worked closely with NISRA to develop a Policing Plan Survey, aimed to ensure the Board meets its legislative requirements and effectively re-ports against the Outcomes, Indicators and Measures identified within the Policing Plan 2020-2025 and Annual Performance Plans. The fieldwork for the survey is due to commence winter 2021, with results available by the end of the 2021-22 financial year. This will provide additional baseline data to inform the Board’s oversight for 2022-23 reporting year of the Policing Plan.

Progress

A PPRWG was established by the Board in September 2021 to take the statutory requirement to review the Northern Ireland Policing Plan 2020-2025 and Annual Performance Plan 2021-22. The Board approved the Terms of Reference for the review and identified the Board Members who would be represented; with the Board’s Vice Chair, Tom Frawley, appointed as PPRWG Chair.
Subsequently the PSNI nominated their three representatives to sit on the PPRWG, T/ACC Melanie Jones and PSNI staff Lindsey Jeapes and Sam Hagan.
In addition an invitation was extended to DOJ officials to attend the meetings of the PPRWG. This invitation was accepted and Maura Campbell, Deputy Director of Policing Policy and Strategy Division, attended the 25 November PPRWG meeting.
The first meeting of the PPRWG was held on 30 September 2021, with five meetings in total until the last meeting of 13 January 2022. Within this period the PPRWG was supported by Board officials.
At the first meeting it was agreed by PPRWG Members that, in line with the Terms of Reference (TOR) the group would undertake its review of the Annual Performance Plan centred on four areas, three of these link directly to the NIAO Continuous Improvement Arrangements in Policing, as follows:
- Review the reporting of Impact(s)
- Consider the Recommendations detailed within the Draft NIAO Continuous Improvement Arrangements in Policing;
- Consider the progress made in relation to the development of surveys related to the meaningful evaluation of the Policing Plan.
Throughout the previous and current reporting year, Board Members had raised concerns in relation to the reporting of impacts within the PSNI’s Report Cards. It was acknowledged that reporting on the Policing Plan was still at a relatively early stage, at that stage in the second year of reporting on a five year plan, and there was a raised expectation for a significant improvement in this area within the third year of reporting.
Throughout the review of the Performance Plan the PPRWG considered the recommendations contained within the annual NIAO report on “The Continuous Improvement Arrangements in Policing” and their implications for the Policing Plan. It was recognised that the focus on Continuous Improvement in policing arrangements continued to be embedded within the Policing Plan and Annual Performance Plan and is a key aspect of the reporting arrangements. As demonstrated by the decision to adopt the Outcome Based Accountability (OBA) approach and the regular attendance by senior PSNI officers at all Committees of the Board when Policing Plan Measures are being examined by Members.
It was recognised that the NIAO recommendations aligned with previous feedback from Members that more focus needs to be brought by the PSNI, through the report cards, to the particular area of policing covered by the indicator and its related Outcome. There was also a recognition amongst PPRWG Members that the Board had potentially not clearly articulated what impact they wish to see the PSNI achieve against each of the Measures. To address this issue the PPRWG undertook a review of the reporting of impacts.
The PPRWG considered the reporting of impact for each of the three Policing Plan Outcomes in turn. This included the identification of draft impacts and accompanying baseline data (provided by the PSNI) which may be used to evidence performance. The draft impact framework was developed to ensure that the strategic links between the PSNI activity, initiatives and programmes undertaken, and the impact they have on people and communities, are clearly identified in order to inform Board Member’s scrutiny of police performance and contribute to tracking continuous improvements in the PSNI’s performance.
At the Board meeting of 3 February 2022 the PPRWG recommended that the Board approve the baseline and draft impact framework and that it was published as an Annex within the Annual Performance Plan for 2022-23.
In line with the PPRWG TORs regular updates were also received on the progress to develop survey activity to support the Performance Plan, again in line with the NIAO Recommendations and towards additional baseline data development. The focus of the updates centred on the:
- The Northern Ireland Policing Plan Survey;
- Measure 3.1.2 - the PSNI’s Self-assessment Neighbourhood Policing Team (NPT) Survey; and
- Measure 3.1.3 - the online Community Stakeholder Survey and Community Stakeholder Focus Groups
At the 3 February Board meeting, updates of a significant nature were provided to evidence the PPRWG TOR delivery in respect of considering the progress made in relation to the development of surveys related to the meaningful evaluation of the Policing Plan.
Northern Ireland Policing Plan Survey - the Board has invested significant human and financial resources in taking forward work to deliver a Northern Ireland Policing Plan Survey, for delivery by 31 March 2022. The survey will provide data to support analysis of the PSNI’s performance against a number of Measures in the Northern Ireland Policing Plan and includes questions designed to gather data on the three Policing Plan Outcomes, as well as additional questions designed to meet the Board’s legislative duties.

The next two survey updates are directly linked to the establishment of data for the two new Measures within the 2021-22 Performance and support the NIAO recommendation.

Measure 3.1.2 - PSNI Self-assessment NPT Survey - a NPT Self-assessment and Continuous Improvement Survey was developed by the PSNI with input from the Board. Currently 71 out of 75 NPT areas have completed the self-assessment, with Board officials being advised that the four outstanding NPTs will submit their responses in due course. Analysis is currently being undertaken on the findings and it is anticipated that the Partnership Committee will be updated on the findings at their March meeting.
Measure 3.1.3 – Board’s Online Community Stakeholder Survey and Community Stakeholder Focus Groups - to support oversight of Measure 3.1.3, the Board developed an Online Community Stakeholder Survey to gather views from the community on the impact of partnership working by the PSNI. It was agreed by Board Members that the Community Stakeholder survey should focus on eight pilot NPT areas.
The online community survey data response was analysed by Board officials who subsequently produced a final report for Members which will be provided at the March Partnership Committee meeting in respect of Measure 3.1.3 of the Policing Plan. In summary the report provides an analysis broken down for Members by the type of respondent, the most common issues identified (including a breakdown across each NPT area) and proposed solutions. It also included quantitative data in respect of (i) how involved respondents were in seeking solutions, (ii) their rating for shared priorities being resolved with a mutually satisfactory solution and (iii) a rating for the status of the issue.
A series of stakeholder focus groups specific to their NPT area is currently being delivered in order to establish qualitative feedback from community individuals and organisations.

All three datasets (the PSNI’s NPT self-assessment survey; the Community Stakeholder Online Survey and; the Community Stakeholder Focus Groups) will form part of the independent evaluation, undertaken by BCS.

NIAO Recommendations

The Department of Justice should consider changes to the legislation underpinning the audit of continuous improvement to bring the arrangements in Northern Ireland back into line with the rest of the UK.

NIPB Management Response

The Board is supportive of NIAO’s position in relation to this area and I would note that it was raised by the Board with regards to the Minister’s current Stocktake of Policing Arrangements.

Progress

This issue was raised by the Board directly with the DOJ Minister at its meeting of 30 June 2021 in relation to the Minister’s Stocktake of Policing Arrangements. Subsequently, the issue is currently featured as a question in the Minister’s consultation on the Stocktake of Policing Oversight and Accountability Arrangements.