The means of assessing the quality of education offered by an institution, faculty, or department has recently become a hot issue in many countries. The three approaches to quality assessment are accreditation, evaluation and academic audit.

Accreditation

Accreditation asks whether academic credits can be trusted and, in the United States, whether an institution’s students should be eligible for federal financial aid. America’s accreditation system originally addressed the threshold standards needed to assess credit transfer and financial aid eligibility. Does an institution have the resources needed to provide threshold quality levels (e.g. inputs like the quantity and quality of faculty, libraries, physical plant)? Does it comply with accepted criteria for diversity, academic freedom and the like?

U.S. accreditors are redesigning their processes to evaluate and improve quality at all levels as well as to certify threshold standards. For example, the Western Association of Schools and Colleges (WASC) now defines itself as a ‘capacity building organization with regulatory functions’ where ‘capacity’ includes the ability to improve and assure education quality. In addition to WASC’s use of audit principles, the North Central Association’s new system uses criteria from the Malcolm Baldrige National Quality Award and other regional and disciplinary accreditors are experimenting along similar lines.

Evaluation

Evaluation of delivered quality at the subject level by teams of outside experts has been the mainstay in the UK and a number of other European countries. European oversight agencies tend toward direct evaluations of quality by outside experts. Britain calls its evaluations ‘subject-level assessments’ to emphasise that the focus of attention is the department rather than the institution. The programme began with the polytechnic universities in the late 1980s and was extended to all of the country’s 150 or so universities a few years later. (The UK’s new programme uses evaluation only on an exception basis, if an institution fails an audit). The Netherlands introduced evaluation at about the same time, Denmark started in 1992, and a number of other countries have implemented or are considering similar programmes.

Education quality evaluation requires that a team of experts in the discipline visit the department and gauge the quality of education actually delivered to students. Self-studies help focus the visit, but team members also review syllabi and examination papers, interview faculty and students, and visit classrooms. The team forms it own conclusions rather than relying mainly on the self-evaluation.

Evaluation provides incentives for improvement, but this is offset by its intrusiveness. One cannot evaluate delivered quality without ‘getting inside’ the department. Once inside, the outside experts may second-guess professors on what are essentially matters of judgment. This problem is particularly severe when the experts have less research prestige than local faculty. Research prowess doesn’t necessarily confer good pedagogical judgment, but experience in the UK, particularly, demonstrates how the lack of a strong research record can undermine an evaluator’s credibility.

Opinions differ on whether the advantages of external quality evaluation outweigh its disadvantages. On the positive side, evaluation provides credible information to further the sponsoring entity’s accountability, public reassurance, and market information goals. On the negative side, its intrusiveness undermines the improvement agenda. Direct evaluation of individual subjects also is very expensive, and many commentators question whether site visit teams can produce meaningful evaluations in the time available to them.

Academic Audit

Academic audit, originally developed in the UK, is a central feature of quality oversight in Hong Kong, Sweden, New Zealand, Australia, and (again, since mid-2002) the UK. In the United States, audit is being tested by WASC (the accreditor for west-coast universities) and the University of Missouri.

Academic audit assesses education quality processes rather than education quality itself. It asks whether a unit is performing the activities necessary to produce, assure, and regularly improve quality. Therefore, audit is less intrusive and less expensive than quality evaluation.

By not disempowering professors, audit spurs improvement as well as generates information. It elicits structured conversations about education quality processes: first within the unit and then with the site visit team. Conversations are important because that is how actions get launched in academe. Structure is important because it focuses attention on the following five process domains that are important for quality.

1. Determination of Desired Learning Outcomes
   
   What should a student who successfully completes the course or program know and be able to do? How will the course or program build on the student’s prior knowledge and capability? How will it contribute to the student’s future employment opportunities, capacity to make social contributions, and quality of life?
   
   
2. Design of Curricula
   
   What will be taught, in what order, and from what perspective? How will the teaching contribute to desired learning outcomes? What course materials will be used? How will these materials relate to other parts of the student’s programme?
   
   
3. Design of Teaching and Learning Processes
   
   How will teaching and learning be organised? What methods will be used for first exposure to material, for answering questions and providing interpretation, for stimulating involvement, and for providing feedback on student work? What roles and responsibilities will the faculty need to assume? What other resources will be required and how will they be used?
   
   
4. Student Learning Assessment
   
   What measures and indicators will be used to assess student learning? Will they compare performance at the beginning and end of the term, or simply look at the end result? How will the long-term outcomes of the educational experience be determined? Will baseline and trend information be available? Who will be responsible for assessment? How will the assessment results be used?
   
   
5. Implementation Quality Assurance
   
   How will faculty assure themselves and others that content is delivered as intended, teaching and learning processes are being implemented consistently, and assessments are performed as planned and their results used effectively?

Deans, provosts, and external oversight bodies can conduct academic audits for units within their jurisdiction. The process is flexible and inexpensive, and it can spur improvement as well as produce oversight. Professors can participate as auditors outside their disciplines, which helps spread exemplary practices. Those who do so are pleased with the process—a feeling that is shared by professors in the units being audited.

Prof Massy is a specialist in the economic and leadership issues confronting today’s colleges and universities. In the 1970s and 1980s, he held senior administrative positions at Stanford University, and in the 1990s, he was a Senior Scholar at Stanford’s National Center for Postsecondary Improvement. His research focuses on academic quality assurance and improvement, performance indicators, faculty roles and responsibilities, cost analysis, processes for resource allocation, and mathematical modelling of higher education institutions—including a full-scale computer simulation of university behaviour recently released under the title, “Virtual U”. His latest book, Honoring the Trust: Quality and Cost Containment in Higher Education was published in February 2003 (Anker Publishing Company, Inc).

Footnote:

* Source of material: Massy, William, F. (2003). Honoring the Trust: Quality and Cost Containment in Higher Education. Bolton, MA: Anker Publishing Company, Inc. © 2002 by Massy, William F. Permission for use by the National University of Singapore is granted.