As stated in Principle 14, “expert independent peer review is foundational to quality assurance.” External reviewers should have a strong track record as academic scholars in the discipline and ideally should also have had academic administrative experience in such roles as undergraduate or graduate program coordinators, department chair, dean, graduate dean or associated positions. This combination of experience allows a reviewer to provide the most value to reviews of new program proposals and existing programs.

It is also important that the external reviewers have an appreciation of pedagogy. Further, there should be at least one person within the membership of the Review Committee who understands and appreciates the role that program-level learning outcomes and the methods for assessing student achievement of these outcomes plays within the Ontario context. For example, including a Chair of Curriculum, Teaching and Learning (or equivalent) as a member of the Review Team can provide critical external perspective and expertise.

For Cyclical Program Reviews, additional discretionary members may be assigned to the Review Committee as long as they are deemed by the program to be appropriately qualified and experienced individuals selected from industry or the profession under review. If the IQAP allows, students may also be added to the Review Committee.

Advice for Choosing External Reviewers

Best practice in quality assurance ensures that reviewers are at arm’s length from the program under review. This means that reviewers are not close friends, current or recent collaborators, former supervisors, advisors or colleagues.

Arm’s length does not mean that the reviewer must never have met or even heard of a single member of the program. It does mean that reviewers should not be chosen who are likely, or perceived to be likely, to be predisposed, positively or negatively, about the program. It may be helpful to provide some examples of what does and does not constitute a close connection that would violate the arm’s length requirement.

Suggestions and recommendations made during the first cycle of audits have shown that introducing the following would align with best practice:

• That guidance be provided to units undergoing review to provide detail surrounding the nomination process of external reviewers for Cyclical Program Reviews and New Program Proposals.
• That the unit be required to provide a minimum number of potential external reviewers’ names.
• That an external reviewer nomination form be developed, which includes space for disclosure of potential conflicts of interest.
• That the IQAP clarifies who may contact potential external reviewers to seek their willingness and availability to serve as reviewers, and stipulates who is responsible for inviting the Review Team. To avoid any perception that the independence of the review has been compromised, initial contact with potential External Reviewers should be made by the Dean’s/Provost’s/Quality Assurance office and not by the unit itself;
• That the selected external reviewers also be asked to confirm that there is no conflict of interest at the time of being invited to conduct the review.
• That a standardized method for indicating how external reviewers were chosen and how each reviewer satisfies the requirements for an “arm’s length” relationship to the program under review be developed.

Examples of what may not violate the arm’s length requirement:

• Appeared on a panel at a conference with a member of the program
• Served on a granting council selection panel with a member of the program
• Author of an article in a journal edited by a member of the program, or of a chapter in a book edited by a member of the program
• External examiner of a dissertation by a doctoral student in the program
• Presented a paper at a conference held at the university where the program is located
• Invited a member of the program to present a paper at a conference organized by the reviewer, or to write a chapter in a book edited by the reviewer
• Received a bachelor’s degree from the university (especially if in another program)
• Co-author or research collaborator with a member of the program more than seven years ago
• Presented a guest lecture at the university

• Reviewed for publication a manuscript written by a member of the program

Examples of what may violate the arm’s length requirement:

• A previous member of the program or department under review (including being a visiting professor)
• Received a graduate degree from the program under review
• A regular co-author and research collaborator with a member of the program, within the past seven years, and especially if that collaboration is ongoing
• Close family/friend relationship with a member of the program
• A regular or repeated external examiner of dissertations by doctoral students in the program
• A recent doctoral supervisor (past several years) of one or more members of the program
• A previous external reviewer for a Cyclical Program Review or a New Program Proposal in the department/unit in question. Whilst this is preferable, in cases where it is not ideal, at least one of the external reviewers must not have previously reviewed a program in the department/unit.