The means of assessing the quality of education offered by
an institution, faculty, or department has recently become
a hot issue in many countries. The three approaches to quality
assessment are accreditation, evaluation and academic audit.
Accreditation asks whether academic credits can be trusted
and, in the United States, whether an institution’s
students should be eligible for federal financial aid. America’s
accreditation system originally addressed the threshold standards
needed to assess credit transfer and financial aid eligibility.
Does an institution have the resources needed to provide threshold
quality levels (e.g. inputs like the quantity and quality
of faculty, libraries, physical plant)? Does it comply with
accepted criteria for diversity, academic freedom and the
U.S. accreditors are redesigning their processes to evaluate
and improve quality at all levels as well as to certify threshold
standards. For example, the Western Association of Schools
and Colleges (WASC) now defines itself as a ‘capacity
building organization with regulatory functions’ where
‘capacity’ includes the ability to improve and
assure education quality. In addition to WASC’s use
of audit principles, the North Central Association’s
new system uses criteria from the Malcolm Baldrige National
Quality Award and other regional and disciplinary accreditors
are experimenting along similar lines.
Evaluation of delivered quality at the subject level by
teams of outside experts has been the mainstay in the UK and
a number of other European countries. European oversight agencies
tend toward direct evaluations of quality by outside experts.
Britain calls its evaluations ‘subject-level assessments’
to emphasise that the focus of attention is the department
rather than the institution. The programme began with the
polytechnic universities in the late 1980s and was extended
to all of the country’s 150 or so universities a few
years later. (The UK’s new programme uses evaluation
only on an exception basis, if an institution fails an audit).
The Netherlands introduced evaluation at about the same time,
Denmark started in 1992, and a number of other countries have
implemented or are considering similar programmes.
Education quality evaluation requires that a team of experts
in the discipline visit the department and gauge the quality
of education actually delivered to students. Self-studies
help focus the visit, but team members also review syllabi
and examination papers, interview faculty and students, and
visit classrooms. The team forms it own conclusions rather
than relying mainly on the self-evaluation.
Evaluation provides incentives for improvement, but this
is offset by its intrusiveness. One cannot evaluate delivered
quality without ‘getting inside’ the department.
Once inside, the outside experts may second-guess professors
on what are essentially matters of judgment. This problem
is particularly severe when the experts have less research
prestige than local faculty. Research prowess doesn’t
necessarily confer good pedagogical judgment, but experience
in the UK, particularly, demonstrates how the lack of a strong
research record can undermine an evaluator’s credibility.
Opinions differ on whether the advantages of external quality
evaluation outweigh its disadvantages. On the positive side,
evaluation provides credible information to further the sponsoring
entity’s accountability, public reassurance, and market
information goals. On the negative side, its intrusiveness
undermines the improvement agenda. Direct evaluation of individual
subjects also is very expensive, and many commentators question
whether site visit teams can produce meaningful evaluations
in the time available to them.
Academic audit, originally developed in the UK, is a central
feature of quality oversight in Hong Kong, Sweden, New Zealand,
Australia, and (again, since mid-2002) the UK. In the United
States, audit is being tested by WASC (the accreditor for
west-coast universities) and the University of Missouri.
Academic audit assesses education quality processes rather
than education quality itself. It asks whether a unit is performing
the activities necessary to produce, assure, and regularly
improve quality. Therefore, audit is less intrusive and less
expensive than quality evaluation.
By not disempowering professors, audit spurs improvement
as well as generates information. It elicits structured conversations
about education quality processes: first within the unit and
then with the site visit team. Conversations are important
because that is how actions get launched in academe. Structure
is important because it focuses attention on the following
five process domains that are important for quality.
- Determination of Desired Learning Outcomes
What should a student who successfully completes the course
or program know and be able to do? How will the course or
program build on the student’s prior knowledge and
capability? How will it contribute to the student’s
future employment opportunities, capacity to make social
contributions, and quality of life?
- Design of Curricula
What will be taught, in what order, and from what perspective?
How will the teaching contribute to desired learning outcomes?
What course materials will be used? How will these materials
relate to other parts of the student’s programme?
- Design of Teaching and Learning Processes
How will teaching and learning be organised? What methods
will be used for first exposure to material, for answering
questions and providing interpretation, for stimulating
involvement, and for providing feedback on student work?
What roles and responsibilities will the faculty need to
assume? What other resources will be required and how will
they be used?
- Student Learning Assessment
What measures and indicators will be used to assess student
learning? Will they compare performance at the beginning
and end of the term, or simply look at the end result? How
will the long-term outcomes of the educational experience
be determined? Will baseline and trend information be available?
Who will be responsible for assessment? How will the assessment
results be used?
- Implementation Quality Assurance
How will faculty assure themselves and others that content
is delivered as intended, teaching and learning processes
are being implemented consistently, and assessments are
performed as planned and their results used effectively?
Deans, provosts, and external oversight bodies can conduct
academic audits for units within their jurisdiction. The process
is flexible and inexpensive, and it can spur improvement as
well as produce oversight. Professors can participate as auditors
outside their disciplines, which helps spread exemplary practices.
Those who do so are pleased with the process—a feeling
that is shared by professors in the units being audited.
Prof Massy is a specialist in the economic
and leadership issues confronting today’s colleges and
universities. In the 1970s and 1980s, he held senior administrative
positions at Stanford University, and in the 1990s, he was
a Senior Scholar at Stanford’s National Center for Postsecondary
Improvement. His research focuses on academic quality
assurance and improvement, performance indicators, faculty
roles and responsibilities, cost analysis, processes for resource
allocation, and mathematical modelling of higher education
institutions—including a full-scale computer simulation
of university behaviour recently released under the title,
“Virtual U”. His latest book, Honoring the Trust: Quality and Cost Containment in Higher Education was published in February 2003 (Anker Publishing Company,
* Source of material: Massy, William, F.
(2003). Honoring the Trust: Quality and Cost Containment
in Higher Education. Bolton, MA: Anker Publishing Company,
Inc. © 2002 by Massy, William F. Permission for use by
the National University of Singapore is granted.