5.2 Steps in the Audit Process

5.2.1  Assignment of auditors for conduct of the audit

Normally, no fewer than three auditors, selected by the Executive Director of the Quality Council, conduct an institutional audit. These auditors will be at arm’s length from the institution undergoing the audit. The Executive Director and a member of the Secretariat normally accompany the auditors on their site visit.

5.2.2  Selection of the sample of programs for audit

Auditors independently select programs for audit, typically four undergraduate and four graduate cyclical program reviews. At least one of the undergraduate programs and one of the graduate programs will be a New Program or Major Modifications to an Existing Program approved within the period since the previous audit. The Executive Director authorizes the proposed selection, assuring, for example, a reasonable program mix.

Specific programs may be added to the sample when an immediately previous audit has documented causes for concern, and when so directed in accordance with Framework Section 5.2.5 (b). When the institution itself so requests, specific programs may also be audited.

The auditors may consider, in addition to the required documentation, any additional elements and related documentation stipulated by the institution in its IQAP.

5.2.3  Desk audit[1] of the institutional quality assurance practices

Once every eight years, and in preparation for a scheduled on-site visit, the auditors participate in a desk audit of the institution’s quality assurance practices. Using the institution’s records of the sampled cyclical program reviews, together with associated documents, this audit tests whether the institution’s practice conforms to its own IQAP, as ratified by the Quality Council.[2]

It is essential that the auditors have access to all relevant documents and information to ensure they have a clear understanding of the institution’s practices. The desk audit serves to raise specific issues and questions to be pursued during the on-site visit and to facilitate the conduct of an effective and efficient on-site visit.

The documentation to be submitted for the programs selected for audit will include:
a)    All the documents and other information associated with each step of the institution’s IQAP, as ratified by the Quality Council.
b)    The record of any revisions of the institution’s IQAP, as ratified by the Quality Council.

Institutions may provide any additional documents at their discretion.

During the desk audit, the auditors will also determine whether or not the institution’s web-based publication of the Executive Summaries, and subsequent reports on the implementation of the review recommendations for the programs included in the current audit, meet the requirements of Framework Section 4.2.6.

The auditors undertake to preserve the confidentiality required for all documentation and communications and meet all applicable requirements of the Freedom of Information and Protection of Privacy Act (FIPPA).

5.2.4  On-site interaction with the institution

After the desk audit, auditors normally visit the institution over two or three days. The principal purpose of the on-site visit is to answer questions and address information gaps that arose during the desk audit. Ultimately, the purpose of the on-site visit is for the auditors to get a sufficiently complete and accurate understanding of the institution’s application of its IQAP so that they can meet their audit responsibilities.

In the course of the site visit, the auditors will speak with those identified by the IQAP as participants and in particular those accountable for various steps, responsibilities, and obligations in the process. The institution, in consultation with the auditors, will establish the program and schedule for these interviews prior to the site visit.

5.2.5  Audit report

a)    Following the conduct of an institutional audit, the auditors prepare a report, which:

  1. Describes the audit methodology and the verification steps used;
  2. Provides a status report on the program reviews carried out by the institution;
  3. On the basis of the programs audited, describes the institution’s compliance with its IQAP as ratified by the Quality Council;
  4. Identifies and records any notably effective policies or practices revealed in the course of the audit of the sampled programs; and
  5. Where appropriate, makes suggestions and recommendations and identifies causes for concern.

Suggestions will be forward-looking, and are made by auditors when they identify opportunities for the institution to strengthen its quality assurance practices. Suggestions do not convey any mandatory obligations and sometimes are the means for conveying the auditors’ province-wide experience in identifying good, and even on occasion, best practices. Institutions are under no obligation to implement or otherwise respond to the auditors’ suggestions, though they are encouraged to do so.

Recommendations are recorded in the auditors’ report when they have identified failures to comply with the IQAP and/or there is misalignment between the IQAP and the Quality Assurance Framework. The institution must address these recommendations.

Causes for concern In some cases the auditors may identify that there is cause for concern. These may be potential structural weaknesses in quality assurance practices (for example, when, in two or more instances, the auditors identify inadequate follow-up monitoring (as called for in Framework Section 4.2.5[c]); a failure to make the relevant implementation reports to the appropriate statutory authorities (as called for in Framework Section 4.2.6), or the absence of the Manual (as called for in Framework Section 4.2.8).

b)    When the auditors have identified, with supporting reason and evidence, cause for concern, it will be reported to the Audit Committee and the institution. Following deliberation, including possible discussion with the institution, the Committee may then recommend that the Quality Council investigate by taking one of the following steps:

  1. Directing specific attention by the auditors to the issue within the subsequent audit as provided for in Framework Section 5.2.2;
  2. Scheduling a larger selection of programs for the institution’s next audit; and/or
  3. Requiring an immediate and expanded institutional audit (further sample) of the respective process(es).

The decision of the Quality Council will be reported to the institution by the Executive Director.

5.2.6  Disposition of the audit report and summary

The auditors prepare a draft report, together with a summary of the principal findings suitable for subsequent publication. The Secretariat provides a copy of these to the institution’s “authoritative contact” identified in Framework Section 4.2.1(b), for comment. This consultation is intended to ensure that the report and associated summary do not contain errors or omissions of fact.

That authority submits a response to the draft report and summary within sixty days. This response becomes part of the official record, and the auditors may use it to revise their report and/or associated summary prior to their submission to the Audit Committee.

The Executive Director submits the final audit report and associated summary, together with the institutional response, to the Audit Committee for consideration and, when necessary, for consultation with the auditors. When satisfied that the auditors followed the required audit procedures correctly and that the university had an appropriate opportunity to respond, the Audit Committee recommends to the Quality Council approval of the report and associated summary. When a report or associated summary is rejected, the Council determines the actions to be taken.

5.2.7  Submission of the audit report to the institution

The Secretariat sends the approved report and associated summary to the institution and to the Ontario Council of Academic Vice-Presidents (OCAV), the Council of Ontario Universities (COU) and the Ministry of Advanced Education and Skills Development (MAESD, formerly the Ministry of Training, Colleges and Universities) for information.

5.2.8  Publication of main audit findings

The Secretariat publishes the approved summary of the overall findings, together with a record of the recommendations on the Quality Council’s website, and sends a copy of both to the institution for publication on its website.

5.2.9  Institutional one-year follow-up

Within a year of the publication of the final audit report, the institution will inform the auditors, through the Secretariat, of the steps it has taken to address the recommendations. The auditors will draft an accompanying commentary on the scope and adequacy of the institution’s response, together with a draft summary of their commentary, suitable for publication. The auditors’ response and summary are then submitted to the Audit Committee for consideration. The Audit Committee will submit a recommendation to the Quality Council on whether or not to accept the institutional one-year follow-up response. When the Audit Committee is not satisfied with the reported institutional response, it recommends to the Quality Council the course of action to be taken.

5.2.10  Web publication of one-year follow-up report

The Secretariat publishes the auditors’ summary of the scope and adequacy of the institution’s response on the Quality Council website and sends a copy to the institution for publication on its website and to OCAV, COU and MAESD for information.

Footnotes

  1. A desk audit is a limited-scope, off-site examination of the relevant documents and records by the auditors.
  2. Changes to the institution’s process and practices within the eight-year cycle are to be expected. The test of the conformity of practice with process will always be made against the ratified Institutional Quality Assurance Process applying at the time of the conduct of the review.