Foreword from the Comptroller and Auditor General (C&AG) – Dorinnia Carville
I was appointed as C&AG of the Northern Ireland Audit Office in August 2022, and the 2023-24 year was, therefore, my first full year as C&AG.
2023-24 was a challenging but rewarding year for NIAO with many important achievements.
One of the most significant events was the return of the NI Executive in February 2024. Since then, I have been working closely with our MLAs, in particular, the Public Accounts Committee (PAC) to promote good governance and accountability across the Northern Ireland Civil Service. A number of evidence sessions have been scheduled for early 2024-25 and will result in the publication of PAC reports on important topics such as Mental Health, Public Procurement and
Child Poverty. My staff will provide briefing to PAC on these topics and assist the PAC as required.
At a time when public bodies are under huge budgetary pressures audit is increasingly important. I also engaged extensively, therefore, with the NI Assembly’s Audit Committee to ensure that I secure a budget which will enable me to deliver our strategic priorities and provide quality support to the NI Assembly going forward. My ambition is to continue to grow our organisation to deliver our vision of independence and excellence in audit to improve public services.
Another important development during the year was the introduction of our new ISA 315 audit methodology which we developed in partnership with Audit Wales. The first year of implementation was challenging and this is reflected in our quality control review scores for the year. We have responded by undertaking a robust post implementation review of the methodology and developing detailed action plans to address the issues raised by our Quality Control Reviewers. I welcome and value this objective external rigour on our work and the recommendations for improvement arising from their review process.
Our data analytics team also made an important contribution to our audit approach during the year through the roll out of an internally developed analytics tool (adapt) which allows our audit teams to assess the impact of key risks on their audit approach and to select samples accordingly. This will further enhance the quality and robustness of our audit evidence. Further developments of this nature are planned for the coming year.
During 2023-24 we developed a new Corporate Plan 2024-2029 and a Public Reporting Programme. The process involved extensive engagement with both internal and external stakeholders and for the first time, we opened these corporate documents up to public consultation. Both documents published in early 2024-25.
Our people remain a key priority for me as C&AG and I want to offer all staff opportunities for development and growth. The quality report provides details of the significant investment we made in providing technical training for our staff in addition to the procurement of an online resource which provides access to technical auditing and accounting resources and advice. In addition to the technical training, we have also rolled out an aspiring leaders’ programme to develop talent within the organisation and have established a Corporate Management Team to provide our managers with greater empowerment and more autonomy and input into strategic decision making.
I also welcome the continued support of our non-executive members and the contribution that our advisory board and audit and risk committee make in relation to the quality of our work. Throughout the year the members of our audit and risk committee were kept appraised of all matters relating to quality and, in particular, our response to the quality control review process. I continue to be hugely impressed with the level of commitment, professionalism and resilience shown by our staff during a particularly challenging year. Whilst I do not underestimate the challenges that lie ahead, I think we have some exciting opportunities to further grow and develop our organisation and continue to deliver our work in line with the priorities in our new Corporate Plan and I look forward to reporting on progress in next year’s quality report.
Introduction
NIAO is committed to delivering high quality audits and we seek continuous improvement in the work that we do. We design and implement our quality arrangements to ensure compliance with auditing standards.
Two new quality standards, ISQM1 & 2, came into effect in December 2022, replacing ISQC1. In response, NIAO has adopted a new leadership-led, risk-based approach to quality management.
ISQM 1 requires firms to design, implement and operate a system of quality management (SOQM) to manage their engagement quality. ISQM 2 builds upon ISQM 1 by including specific requirements for engagement quality review, which forms part of the system of quality management.
NIAO’s Technical and Quality Team have developed a new Quality Manual and in addition an Evaluation of Quality Management was undertaken and reported to the Senior Leadership Team in December 2023 for the first time, in line with ISQM1. It concluded that the NIAO system of quality management provides the Office with reasonable assurance that the quality objectives are being achieved.
While ISQM 1 requirements are primarily relevant to the Office’s financial audit work, many of them are applied to our public reporting work, where it is relevant and practical to do so. During 2023-24 we undertook a review and refreshed our Public Reporting Guidance. The revised guidance includes clarification around activities at the identification and planning stages and also includes an additional Quality Assurance meeting to discuss emerging issues and findings during the development of the report.
Dorinnia Carville – Comptroller and Auditor General
Ultimate responsibility and accountability for the NIAO's system of quality management rests with the Comptroller and Auditor General.
Rodney Allen – Chief Operating Officer
As Quality Management Director is responsible for policies and procedures in respect of integrity, objectivity, independence and compliance with the Financial Reporting Council’s Ethical Standard.
ISQM1 requires us to evaluate the system of quality management at least annually. Our first evaluation under ISQM1 was undertaken in December 2023.
This report covers our audit quality arrangements and monitoring activity under ISQM1 for the year ended 31 March 2024.
Key facts for 2023-24
Our system of quality management is underpinned by a suite of policies and procedures including:
- Quality Control Manual
- NIAO Code of Conduct
- Financial audit manual
- NIAO financial audit
- Audit policy circulars
- Personnel policy circulars
- Public Reporting Guidance
In 2023-24 key changes were the development of the Quality Manual, which fully documents our new system of quality management in one place.
The Office recruited thirteen staff consisting of one Director, one Senior Auditor, three Auditors, three Data Scientists, three Trainee Accountants and two Higher Level Apprentices. We are a registered training organisation and support the Graduate Trainees in completing their qualification with Chartered Accountants Ireland. In addition, we provide professional training for our Apprentices undertaking the Institute of Accounting Technicians Ireland qualifications.
During 2023-24 a total of ten staff left the office.
During 2023-24 we undertook a post implementation review of the new financial audit approach, which had been developed to ensure compliance with ISA 315. The review highlighted a number of areas where additional training and guidance was required. Audit documentation and forms were updated and refined to reflect feedback received. In response two training videos were produced and released to staff. Training was facilitated by director led teams to discuss the application of the audit approach to each team’s portfolio of audits. Additional guidance has also been issued in the form of audit policy circulars and amendments were made to audit templates as required.
Further technical training was organised for staff on a range of topics including:
- IT audit – ISA 315 post implementation review implications for IT controls work;
- IT audit – overview of IT audit work;
- Refresher training on ISA 315;
- Local Government update;
- Financial Accounting Update; and
- Auditing Standards Update.
The Technical Team will continue to provide support to audit teams as they continue to apply the revised audit methodology and there are plans to develop an audit manual to provide guidance and further support audit teams.
During 2023-24 we certified 110 central government accounts and 16 local government accounts. In the period we also certified 20 prior year accounts. In total, 20 audits with significant matters of judgement were referred to the Technical Director. The C&AG’s opinions on 15 accounts were qualified (some accounts received more than one qualification) and a further 2 opinions were modified (but not qualified).
Six financial audits were subject to an internal Engagement Quality Review by an independent director. In each case, the engagement quality reviewer upheld the conclusions reached by the engagement team.
Three financial audits were subject to external quality control review (QCR) by the Institute of Chartered Accountants in England and Wales (ICAEW). This is the fourth year that ICAEW has independently reviewed our audits. Our contract with ICAEW expired in April 2024 and a procurement exercise to obtain an external quality control reviewer for 2024 is currently underway. One of the audit files reviewed was rated as ‘generally acceptable’, two of the audit files were rated as ‘significant improvements required’. ICAEW identified areas of good practice on all three files.
During 2023-24 we published:
- 18 Public Reports
- 2 Good Practice Guides
- 11 Local Government Improvement Reports
- 1 Local Government Report
Eleven public reports published in 2023-24 were assessed by an external review panel, with an average rating of 6.75 on a scale of 1 to 10, with 10 representing outstanding (2022-23: Four reports, average score of 6.75).
In addition, two public reports published in 2023-24 were peer reviewed by the Office of the Comptroller and Auditor General and Audit Scotland. These reviews continue to strengthen our public reports, through the provision of constructive feedback and sharing of better practices.
- 96% indicating that NIAO audit staff provided a high quality and professional service;
- 98% consider that the NIAO’s work leads to improvement in the provision of public services;
- 100% considered NIAO good practice guides as a useful resource.
Policies and procedures
The quality management policies, procedures and practices of the NIAO are currently documented in the ‘Quality Control in the Northern Ireland Audit Office’ (Quality Control guide), supplemented by, and cross referenced to, a variety of documentation and procedures including:
- Quality Control Manual
- NIAO Code of Conduct
- Financial Audit Manual
- NIAO financial audit methodology
- Audit policy circulars
- Personnel policy circulars
- Public Reporting Guidance
In 2023-24 we developed a Quality Manual which comprises all our quality control policies and procedures. In line with ISQM1 we also undertook an evaluation of quality management, which identified and assessed NIAO quality objectives and associated risks. It concluded that the NIAO system of quality management provides the Office with reasonable assurance that the quality objectives are being achieved. We will be assessing the system of quality management on an annual basis, with the next review due to take place in early 2024-25.
Operating a system of quality management is an iterative process which will develop and evolve over time. Work remains for us to fully embed quality risk management into our business processes, which includes establishing Audit Quality Indicators.
People
Dorinnia Carville was appointed to the role of C&AG in August 2022..
The C&AG is the head of NIAO and recognises that, “People are key to delivering high quality work and to our success.”
Recruitment, retention and development of staff is a key priority, in order for NIAO to maintain and develop as an organisation. During the year we put in place an aspiring leaders’ programme to develop talent within the organisation and we established a Corporate Management Team to provide managers with more autonomy and input into strategic decision making.
Independence
The Chief Operating Officer Rodney Allen, as Quality Management Director is responsible for policies and procedures in respect of integrity, objectivity, independence and compliance with the Financial Reporting Council’s Ethical Standard.
The NIAO Code of Conduct (the Code) outlines the ethical requirements to which staff must adhere. The requirements encompass the five fundamental principles of professional ethics:
- Integrity
- Objectivity
- Professional behaviour
- Confidentiality
- Professional Competence and Due Care
The Code restates the established policy that staff need to be independent of audited bodies or other interested groups and have an unbiased attitude to the issues and topics under review. It requires staff to complete an annual return setting out in writing any potential conflicts of interest, including personal or domestic relationships with employees of bodies they audit. All completed declarations are recorded and held centrally in the eHR system. The Code makes it clear that staff are expected to notify their line director immediately of any changes in circumstances affecting their previous declarations. In addition, all staff record a declaration of independence on the audit file of each engagement they work on.
The Ethical Standard requires the engagement team to consider threats to independence, objectivity and integrity in respect of all covered persons. The definition of a covered person is "a person in a position to influence the conduct or outcome of the engagement". On an annual basis, the Quality Director identifies all covered persons and for each obtains details of interests notified in their Code of Conduct return. Where the Quality Director considers these individuals have potential conflicts with audit engagements, he will notify the engagement directors and audit managers affected to ensure that appropriate safeguards are put in place to mitigate risk. For the purpose of evidence on audit files, confirmation is provided to all staff when this exercise is complete.
Staff rotation
Rotation requirements have been complied with in 2023-24.
To safeguard against conflicts of interest arising as a result of over familiarity, we have a policy of rotating staff. It is our policy for engagement directors and engagement managers to continue with a specific client or engagement to the fifth year of association (inclusive) unless there are any identified threats to their or the C&AG’s objectivity or perceived loss of independence that cannot be properly mitigated.
In years six and seven the presumption is that engagement directors and engagement managers will be rotated due to length of association alone unless there are overriding operational reasons for them to remain in place. No engagement director or engagement manager will act as part of an engagement team for a period of more than seven years in any twelve year period. Once rotated, relevant individuals should have no further involvement in work relating to the client for a further five years. Where an engagement quality reviewer involved in a financial audit becomes the audit engagement director, the combined period of service in these positions shall not exceed seven years. We also have a cooling off period of two years before the engagement director can assume the role of engagement quality reviewer.
Where the role of engagement director is delegated, the same rotation policy applies to the individual undertaking the role. All other staff will be rotated regularly to ensure that they have experience across audit clients. No member of staff should work on a particular engagement for a period of more than seven years within any twelve-year period.
Hospitality
We maintain a register of all hospitality offered to NIAO staff. For transparency, disclosures of hospitality and gifts accepted, declined and provided by each NIAO Non-Executive Member and member of the NIAO Directorate are published on our website annually.
Recruitment
We continue to seek to recruit the best quality candidates to meet our current and future needs. Recruitment is competency and values based, and in compliance with equal opportunities requirements. In 2023-24 we focussed on attracting different skills and abilities to add to the diversity to our work and keep pace with digital developments.
During 2023-24 a total of ten staff left the Office. We recruited thirteen staff during this time:
- Director: 1
- Senior Auditor: 1
- Auditor: 3
- Data Scientist: 3
- Trainee Accountant : 3
- Higher Level Apprentice: 2
An online resourcing tool, Time and Space, is updated and monitored regularly and management routinely considers resource allocations to ensure that the Office has sufficient personnel with the capabilities, competence, commitment and ethical principles necessary to perform its engagements.
One of the challenges we continue to face is increased market demand for qualified auditors. In response to that we continue to look at ways to raise awareness and attract more applicants whilst also enhancing our ability to develop and retain staff.
Competence and capabilities of staff
We have established policies and procedures designed to provide the Office with assurance that it has sufficient personnel with capabilities, competence and commitment to ethical principles necessary to perform its engagements in accordance with professional standards, and regulatory and legal requirements. These policies and procedures are recorded in the Office’s Personnel Policy Circulars (PPCs) supported by the Northern Ireland Civil Service (NICS) Staff handbook. To ensure compliance with ethical and professional requirements, the competence of financial audit teams is considered and documented at the outset of each audit engagement as part of risk management procedures. All staff also have a performance objective specifically relating to audit quality.
Learning and Development
The Office acknowledges the importance of and encourages and supports the training and development of all staff to maintain and develop the required capabilities and competence necessary. The Office develops the capabilities and competence of its staff through a combination of structured and unstructured training, work experience and coaching.
The performance management framework includes a consideration of training undertaken during the period under review and a review of current developmental needs.
We are a registered training organisation and support the Graduate Trainees in completing their qualification with Chartered Accountants Ireland. In addition, we provide professional training for our Higher Level Apprentices undertaking the Institute of Accounting Technicians Ireland qualifications.
In order to make a positive impact in delivering our priorities, it is essential that our staff are experienced and skilled in the work that they undertake. Approximately 64 per cent of staff have professional accountancy qualifications, allowing us to meet the professional standards required of all financial auditors. This base is supplemented by other staff with relevant professional and research skills, IT audit and data analytics skills and by contractual agreements with private firms.
To assist in the achievement of the above, our Technical Team develops a detailed annual training programme to address technical training skills. This includes a number of mandatory training courses each year, to ensure audit quality and non-attendance at these courses is monitored and followed up on. Staff can apply for a variety of training courses to meet Continuing Professional Development (CPD) and the competency requirements of their work. Booking takes place via the eHR system which facilitates accurate recording of training and development. In addition to booking training, eHR allows staff to centrally record their CPD activities, including those undertaken outside core work. Qualified staff have a responsibility to ensure that they are attending sufficient courses and updating their knowledge in order to comply with the requirements of their respective professional institutes.
The training needs for staff are identified from a variety of sources including:
- strategic workforce planning;
- Corporate/Business plans;
- changes in working practices e.g. new technology, legislation and systems;
- technical requirements;
- professional requirements; and
- performance management, including personal development plans.
During 2023-24 we undertook a post implementation review of the new financial audit approach which we developed to ensure compliance with ISA 315 (revised). In response to the review, we developed additional bite size training for staff to remind them of key changes within the methodology. In addition, we amended a number of our audit templates, to further enhance the templates.
The Technical Team will continue to provide support to audit teams as they continue to apply the new approach.
Further technical training was organised for staff on a range of topics including:
- Data Analytics – use of adapt;
- IT audit – ISA 315 post implementation review;
- IT audit – overview of IT audit work;
- Refresher training on ISA 315;
- Local Government update;
- Financial Accounting Update;
- Auditing Standards Update; and
- Training on public reporting guidance.
In addition to formal training provided, we issued guidance on several technical matters, such as review of financial processes (RoFP); leases disclosures due to the change in IFRS 16; the impact of irregularities on the true & fair audit opinion; and material uncertainties in valuation reports.
During the year we procured licenses to provide all of our audit staff with access to croner-i. Croner-i is an online reference service which allows users to access auditing and accounting standards as well as providing online training and practical interpretation/application of the standards.
New staff, at all staff levels are assigned a ‘buddy’ when they join and as an additional quality measure for leadership, an EQR director is assigned for all high-risk audits of first year directors.
On-boarding procedures include training on NIAO methodology provided to all new staff. Junior members of staff at trainee grades are also provided with in-house introductory accounting and audit training prior to commencing their studies.
In addition to our technical and financial audit training we offer additional training and development opportunities on other areas such as public reporting and leadership and management.
A new Development and Talent Management strategy for 2024-25 was published, which aims to develop capabilities, drive innovation, create a proactive culture, support continuous development and improvement and succession planning. This included an internally developed Aspiring Leaders Programme which was launched with the aim of developing our internal capabilities and support internal progression.
During 2023-24 we completed the assessment process for IIP accreditation and were informed in May that we have maintained standard accreditation. The IIP report identifies many strengths for example, a shared sense of purpose, a understanding of the link between our work and our strategic aims and objectives and our physical working environment. However, there are areas where we can improve. For example, creating a culture of engagement, continuing to develop reward and recognition strategies and learning and development.
Performance management
The current performance management system provides a framework, including process, documents and tools, to clarify work priorities, discuss expectations, review performance, and plan and support development to continue to build capabilities. The aim is to promote a culture of continuous dialogue about practice (behaviours) and performance (delivery) based on clear expectations. It is based on a partnership between the individual and the organisation. The process is about recognising and supporting the ongoing development needs of staff, so that they can contribute towards organisational performance to the best of their ability and develop as individuals.
Performance reporting is recorded using the Office’s eHR application. This assists the Office in monitoring compliance with its performance management policy.
Use of external resources
In all cases of contracted-out audit arrangements, the C&AG retains overall responsibility for the audit of the financial statements and will sign the audit certificate/opinion and report.
In 2023-24 we undertook a tendering exercise to appoint contractors for the next few years. We noted changes in the marketplace in terms of availability of firms and associated costs.
A decision was taken to build internal capacity and bring all IT audit work in-house. To achieve this, we recruited additional staff with specialist IT audit skills, supported an auditor to undertake their ISACA Certified Information Systems Auditor (CISA) qualification and offered IT audit training to all staff.
Acceptance and continuance procedures
The vast majority of audits undertaken by the Office are by statutory appointment and in these circumstances, we cannot withdraw or decline the appointment. However, Practice Note 10 Audit of Financial Statements and Regularity of Public Sector Bodies in the UK (Revised 2022) (PN 10) indicates that there are other avenues open to the Office; for example, we can report to the NI Assembly on matters that might otherwise have caused us to withdraw from the engagement. During the reporting period there were no instances which might have caused us to withdraw from the engagement.
In light of the statutory appointment of the C&AG as auditor, acceptance procedures are not always relevant to the Office. However, we have procedures in place to provide assurance that we only undertake or continue relationships and engagements where we have:
- considered the integrity of the client;
- the competency to perform the engagement;
- the capabilities, time and resources to do so; and
- where it has confirmed that it can comply with ethical requirements.
Acceptance and continuance procedures are embedded within the Quality Control guide and the electronic audit template for all audits regardless of whether these are statutory appointments. All new client and engagement requests must be submitted to the C&AG for approval. Staff have been reminded to be alert to the requirement for public bodies to consult with the Office in cases where the Department of Finance has agreed that the accounts of a body should be subject to examination and certification by the C&AG, or that the C&AG should have rights of inspection. There were no new engagements accepted during 2023-24.
Engagement performance
Overall responsibility for engagement performance and quality rests with the engagement director assigned to each audit.
Technical and specialist advice
We have a designated Technical Director whose responsibilities include:
- Establishing policies and procedures to provide reasonable assurance that engagements are performed in accordance with professional standards, and regulatory and legal requirements.
- Liaising with other public sector audit agencies on technical matters of common interest.
- Providing informed advice on technical matters and reviewing certificate modifications.
The Technical Director, supported by the Technical Team, is available to advise all staff on complex or judgemental accounting and auditing issues. All qualifications or proposed qualifications must be referred to the Technical Director. Should a conflict situation arise (e.g. the Technical Director is involved in an audit requiring technical review), the matter will be allocated to an independent director or handled by the Quality Management Director.
Differences of opinion within the engagement team or between the engagement team and the engagement quality reviewer should be referred to the Technical Director for resolution.
During the year a wide range of technical guidance and advice was provided on key audit or accounting issues. This covered areas such as using the work of experts; the impact of irregularities on the true & fair audit opinion; budgeting influencing accounting; and material uncertainties in valuation reports. In total, twenty audits with significant matters of judgement were referred to the technical director during 2023-24. Arising from these, the C&AG’s opinions on fifteen accounts were qualified (some accounts received more than one qualification) and a further two opinions were modified (but not qualified).
Review of engagement performance
During the period, the vast majority of audits performed were subject to a two-stage review process:
First stage review:
- a detailed review of all working papers and audit procedures by another team member who is senior to the preparer
- detailed review is the responsibility of the engagement manager but the task may be delegated to another team member
- this ensures that the audit team is complying with ISA (UK) 220 - Quality Management for an Audit of Financial Statements
Second stage review
- an overall review undertaken by the engagement director to confirm that sufficient and appropriate audit evidence has been obtained to satisfy the objectives of the engagegment procedures and to support the recommended audit opinion.
- this review will consider the major issues of judgement and other key areas of the audit process
Audits that are assessed as being lower risk and lower complexity have only one stage of review and are delegated to another member of the audit team who will act as the engagement director. The rationale for adopting this approach will be clearly documented and approved by the portfolio director.
Engagement Quality Review (EQR)
A further level of review is performed on the audits of those accounts deemed to be high risk or high interest. This independent review is carried out by an engagement quality reviewer whose role is clearly defined.
The independent review process is designed to provide an objective evaluation of the significant judgements made by the engagement team and the conclusions reached.
As part of the financial audit planning process, the engagement director is responsible for determining whether the audit requires an EQR based on the following considerations:
- whether the entity is a Public Interest Entity (PIE) or whether any relevant laws or regulations require an engagement quality review; or
- the nature of the engagement, including the extent to which the entity’s financial statements are of high Assembly or public interest; or
- the identification of unusually complex circumstances or technical risks and/ or judgements in an engagement or class of engagements; or
- whether it is likely that there may be a significant modification to the audit opinion; and/or
- if an engagement quality review is an appropriate response to address one or more quality risk(s); and/or
- where senior engagement staff are new to their role.
An engagement quality reviewer is assigned for all high-risk audits of first year directors to ensure audit quality.
The Quality Management Director determines who performs the engagement quality reviewer function for engagements where it is determined that an EQR is needed. The decision takes into consideration an individual’s independence from the body being audited, their specific skill set and their experience at the director grade. The engagement quality reviewer will not be a member of the engagement team and an engagement director may not act as engagement quality reviewer until a period of at least two years has passed after previously serving as engagement director.
The engagement quality reviewer notifies the engagement director if they have concerns that the significant judgements made by the engagement team, or the conclusions reached thereon, are not appropriate. If such concerns are not resolved to the engagement quality reviewer’s satisfaction, the engagement quality reviewer notifies the Technical Director or the Quality Management Director that the engagement quality review cannot be completed.
Six financial audits were subject to EQR in 2023-24. In each case the independent reviewer upheld the judgements and conclusions made by the engagement team.
[Placeholder for flow chart]
Quality control reviews
As part of quality control monitoring procedures we undertake an annual programme of quality control reviews. This is led by the Technical Director and is an important tool in promoting audit quality and continuous improvement. The objective of the reviews is to consider whether each audit was properly planned and conducted in accordance with our methodology and professional standards, and whether the documented audit work supports the audit opinion provided.
The review programme was externally provided by the Institute of Chartered Accountants in England and Wales (ICAEW). In line with good practice, this ensures that the monitoring arrangements are completely independent. The current process for selecting audits for review ensures that:
- engagement directors with ten or more financial audit engagements are reviewed every two years. Otherwise, engagement directors will be reviewed every three years;
- one delegated audit engagement is reviewed every three years;
- where possible, one high risk audit is reviewed each year.
The process excludes contracted out audit engagements, which have separate quality monitoring arrangements in place.
Where a quality control review identifies that significant improvements are required, the engagement director’s portfolio of engagements will be included in the following year’s selection and an additional review will be undertaken. Until the grade improves, the engagement director will be selected on an annual basis.
The 2023 sample period covered all in house financial audit engagements certified between November 2022 and September 2023, and the cold review process commenced in November 2023. Three files were subject to review by ICAEW.
ICAEW rated two of our financial audit files reviewed as ‘significant improvements required’ and one financial audit file as ‘generally acceptable’, they also noted areas of good practice on all three files. The main issue arising from the QCRs was a misinterpretation and application of the new methodology which meant that the risk assessments did not adequately direct the fieldwork testing leading to gaps in audit testing.
In response to the issues raised by ICAEW, the Technical Team facilitated the completion of root cause analysis exercises at the conclusion of each quality control review. Considerable time and effort went into completing the root cause analysis work which entailed liaising closely with the engagement teams to understand the reason for any issues identified during the reviews and to develop detailed action plans, the implementation of which will be monitored by the Technical Team and reported to SLT.
Further ISA 315 training was provided to all audit staff and audit templates were enhanced in response to the quality control reviews. The results of the QCRs were communicated to all staff with the aim of ensuring the quality of our work continues to improve. Staff were asked to consider findings and take recommendations into consideration when undertaking all future work.
The outcome of the QCR process was considered by the Corporate Risk Register Working Group and a proposal to reflect this in the Corporate Risk Register with suggested actions were forwarded to SLT. SLT subsequently agreed the proposal made and the actions required.
The results of the QCR process were reported to the Office’s Audit and Risk Assurance Committee (ARAC) and Advisory Board. Assurance was provided to ARAC that the following actions had been progressed to ensure a timely response to the issues raised by ICAEW:
- Amendments were made to audit documentation forms to help teams understand the approach and to encourage them to record the right level of detail in the documentation;
- An email summarising the key points raised by the QCR reviews (relevant to the ISA 315 approach) was issued to all staff;
- An Audit Policy Circular (APC) was issued to all staff sharing the ICAEW report and drawing key matters to the attention of all audit teams. Managers have been tasked with discussing the findings with their teams and how they apply to their audits at their next team meeting;
- Mandatory training on the ISA 315 approach was delivered to all staff through director led teams; and
- The technical team liaised with the other PAF audit agencies to discuss QCR findings and responses.
There were no audit files rated as ‘improvements required’ as part of the 2022 quality control review programme, therefore no follow up reviews were required. The 2022 quality control reviews were also undertaken by ICAEW. The contract with ICAEW has now expired and we will be running a procurement exercise in July 2024 to obtain a quality reviewer for our 2024 audits.
Review of contracted out audits
NIAO contracts with private sector audit firms to undertake NIAO audit engagements. While contractor firms provide shadow audit certificates, the C&AG retains responsibility for certification and reporting.
In 2023-24 we assessed contractor performance by:
- regularly monitoring Key Performance Indicator scores and holding quarterly performance review meetings with contractors;
- carrying out a programme of pre-certification file reviews, based on experience, risk and prior performance;
- undertaking quality control reviews of a sample of contracted out audits. This is normally undertaken by engagement directors and managers; and
- requiring contractors to undertake cold reviews of the work they are contracted to do (one audit per contract).
Overall results of the KPI monitoring were generally satisfactory. One issue of concern arose regarding a contractor’s poor communication with the relevant NIAO teams. This issue has now been addressed. Four financial audits were reviewed by independent teams from NIAO, two have been graded as ‘good’ and two were graded as ‘generally acceptable’. One of these files received a split rating whereby it also received a rating of ‘improvements required’ for the work completed by NIAO.
During our quality control reviews in 2022, ICAEW identified an issue with our audit approach in relation to contracted out audits. In response, we revised our contracted-out approach in 2023-24, to demonstrate an enhanced level of direct involvement from the NIAO engagement team and to ensure compliance with ISA 220 (Revised) Quality Management for an Audit of Financial Statements.
Assembly of certified audit files
We are required to establish a quality objective to assemble and close our audit files on a timely basis. Where there is no time limit prescribed in law or regulation, ISQM1 recommend that an appropriate time limit is ordinarily not more than 60 days from the date of the auditor’s report. We have adopted this as our time limit for audit closure and it is incorporated within both our financial audit methodology and electronic audit file system.
During 2023-24, 67 per cent of all audits certified were closed within 60 days (54 per cent in 2022-23; 33 per cent in 2021-22). At the start of 2022-23 we issued guidance to define the procedures for audit closure and remind staff of the requirement to close the audit file within the 60-day time limit. Unfortunately, there continues to be limited compliance although it has improved. We will continue to monitor the completion of post certification audit procedures and the administrative close of audit files.
Complaints
We are required to have policies and procedures to ensure that we deal appropriately with complaints and allegations that the work performed by auditors does not comply with professional standards and regulatory and legal requirements. The NIAO Code of Conduct requires that staff discuss any such matters that come to their attention with their line manager, director or, if appropriate, the Chief Operating Officer.
Obtaining feedback on the quality of our financial audit is important in ensuring that we fully understand the needs and expectations of our stakeholders and continue to provide the quality of the audit they expect. Audited bodies are provided with information on making a complaint in the Report to those Charged with Governance prepared at the completion of an audit. In addition, our website provides further information and contact details regarding complaints about the work of the Office.
No complaints were received by the office during 2023-24.
Auditee Survey
We undertook a survey of audited bodies during the year and feedback was very positive, with overall impressions being:
- 96% indicating that NIAO audit staff provided a high quality and professional service;
- 98% consider that the NIAO’s work leads to improvement in the provision of public services;
- 100% considered NIAO good practice guides as a useful resource.
Public reporting Quality Assurance arrangements
Public reporting processes are subject to a range of formal internal quality assurance checks in order to maintain and improve the quality of published reports.
Most published reports are reviewed by an external review panel (its members selected to provide a range of experience) who rate the report’s presentation, technical content and quality of recommendations. Reports may also be subjected to an external peer review (through reciprocal arrangement) where a sample of reports are circulated to other UK and Ireland public audit agencies for their assessment, using agreed assessment criteria.
Eleven public reports that were published in 2023-24 were assessed by an external review panel, with an average rating of 6.75 on a scale of 1 to 10, with 10 representing outstanding (2022-23: Four reports, average score of 6.75).
In addition, two public reports published in 2023-24 were peer reviewed by the Office of the Comptroller and Auditor General and Audit Scotland. These reviews continue to strengthen our public reports, through the provision of constructive feedback and sharing of better practices.