|
Abstract
Quality Assurance for Learning and Teaching (QALT) in universities is
commonly relatively piecemeal in terms of strategy, organisation and
implementation. In overview form, this paper and the associated presentation
argues that more systemic and effective approaches to QALT are possible and
desirable. An indicative systemic framework is provided and discussed as are
some practical examples of quality assurance (QA) components (of such a
framework) developed in an Australian university.
Keywords
Higher education, quality assurance, learning and teaching, quality in
teaching
Introduction
Historically, internal approaches to Quality Assurance for Learning and
Teaching (QALT) in universities have typically been based on an overly
simplistic notion of quality assurance and/or a disorganised and unrelated set
of elements and practices.
Viewing QALT as little more than the measurement of student and graduate
outcomes is extremely limiting, if not unethical. Outcomes are clearly
extremely important, but the student experience (of several years) at
university is a major part of their lives and shapes their approach to work,
learning and personal life beyond graduation. In a very real sense, the
legacy of the total student experience (e.g. approaches to learning that are
developed) is an important outcome. In addition, many students these days
are paying for their educational experience, not just a particular outcome.
There is also an analogy with sports training which is appropriate for a
consideration of universities’ internal approaches to QALT. Elite sports
coaches consistently express the importance of athletes focusing on getting
the practices, experiences and processes right first, and the good results
(outcomes) will follow. So paying attention to the enablers (e.g. support
processes) of high quality student outcomes is important in QALT.
Often, the components in place to provide QALT are not comprehensive or
well organised. For example, there is always a process for the approval of
new programmes. Course, if not programme coordination roles usually exist,
and student evaluation of teaching occurs, often with no clear expectations
or processes for action based on collected data. While such arrangements
commonly exist in universities, they are seldom linked in ways that provide
synergies for QALT.
The overall theme of our presentation and this paper is that there is a
need to take a more holistic and systemic view of QALT in universities.
This requires the identification and sensible linking of QA components in
a conceptual and practical framework. The ideas presented are based on
review of research and practice internationally and our own experience
in universities over many years, including the last five years together in a
respected Australian university.
The Need for More Systemic QALT
The development and successful implementation of comprehensive and
systemic approaches for QALT is still an ideal and a challenge. While
support for moving in this direction can be found in the literature (e.g.
Bowden & Marton, 1998; Boyle & Bowden, 1997; Burden-Leahy, 2005;
Dill, 1992; Massy, 1997), more powerful drivers can be seen in the emerging
expectations of national and international quality assurance agencies and
programme accreditation bodies (e.g. Association to Advance Collegiate
Schools of Business [AACSB]; Australian Medical Council; Australian
Universities Quality Agency [AUQA]). The trend in both of these kinds
of agencies is towards greater expectations of universities to demonstrate
multi-faceted, relatively sophisticated and coherent approaches to assuring
quality in learning and teaching. Increasingly, expectations cover a wider
range of quality assurance arrangements, including those related not just
to programme structure and content, but to outcome standards, evaluation
and improvement processes, student experiences and services, learningteaching
strategies and continuing development for teachers.
We believe that in the future, commended approaches to QALT will be
identified essentially on the basis that they involve multiple and aligned
components that are systemically linked to achieve real synergies for the
continual enhancement of learning and teaching.
A Systemic Framework for QALT
So what might a systemic approach to QALT look like? First, it has to
identify the key components of what a university does to achieve high
quality learning by its students, which must be the ultimate goal. Second, it
must seek to arrange these components so they interact to achieve synergies.
This might appear obvious, but it seems that in universities we rarely sit
back and consider these interactions. As a result, much effort is wasted
and the ultimate goal is not achieved. For example, a sophisticated course
evaluation system might be developed, but if teaching is not valued highly
by the institution’s senior executive, as indicated by academic promotion
processes and decisions, then typically only the already excellent teachers
take evaluation results seriously.
Figure 1 provides an indicative framework to illustrate how QALT can be
conceived and organised in a systemic way. In the remainder of the paper,
we briefly discuss key components in this framework and provide some
examples of how alignment and synergies between them are necessary
to achieve genuinely systemic and comprehensive quality assurance for
learning and teaching.
Key Elements in a Systemic QALT Framework
In the brief discussion that follows, we start with what we believe are
the two critical elements in the QA framework shown in Figure 1. High
Quality Student Learning is at the top of our diagram and Leadership in
Learning and Teaching is at the bottom. Both need to be seen as the most
important drivers for QA, with High Quality Student Learning being the
primary focus and Leadership the critical enabler.
High Quality Student Learning as the Primary Focus for QA
Few would challenge the view that the achievement of high quality
student learning is what we aspire to in universities (along with excellent
research). Some would argue that this is all that matters. It follows
logically that QALT should focus on student learning outcomes, along
with the operational elements of universities that we are confident have
the most effect on the quality of those outcomes (e.g. planned student
experiences).
Student and graduate outcomes are clearly important and need to be
measured and evaluated as accurately as possible. This is important for
summative student assessment and the accountability and improvement
aspects of quality assurance. However, the precise evaluation of learning
outcomes is often not simple in higher education, given the complex
capabilities being aspired to and the numerous variables that influence
student achievement and graduates’ success. Many involved in the
fundamental changes to the design of medical curricula over the past
thirty years, starting with the pioneering work at McMaster University
in Canada, are convinced that some of the new curriculum designs
and approaches to learning are better, but it is difficult to prove this
conclusively based purely on outcome measures. The most effective
approaches to QALT will be those that sensibly integrate elements
that we are confident of, based on research, experience and logic, have
significant influences on the quality of student learning outcomes.
Leadership in Learning and Teaching
It is not possible to provide a detailed discussion of this topic here. However,
we do wish to make a couple of general points very strongly.
The beginning and the end of success in achieving high quality student
learning for the majority of students depend on the quality and commitment
of leadership. If the higher executive makes it clear to staff that research is
predominantly valued and pay only lip service to the importance of teaching, with little hard evidence of the voiced support (e.g. clear recognition and
reward for merit in teaching; allocation of funding for innovation in learning
and teaching), then the institution will fail to achieve the ultimate goal of
maximising the quality of student learning. While the best students will
succeed in any case, and there will be pockets of staff that teach brilliantly
and inspire their students for life, the overall culture and the occurrence of
negative experiences for many students will not change. We will return to
this later in the paper.
Policy and Values on Learning and Teaching
Policies and value statements relating to teaching, and increasingly
learning, often remain the province of academic boards or senates and their
committees. This is appropriate but university leadership needs to ensure
that mechanisms exist to support these policies. Four examples of important
areas of policy and values, and strategies used to ensure they become
operational are described briefly below.
Promotion on the Basis of Merit in Teaching
Staff confidence that promotion processes provide due weight to merit
in teaching is critical to ensure that greater proportions of staff put
effort into enhancing their teaching and the experience they provide for
their students. In most university promotion schemes, performance in
research is based on peer assessment, because of the assumption of their
expertise. Peers, for this purpose, typically include respected scholars
(e.g. editors of journals and grant assessors). Accordingly, the peer
review evidence on research merit used by promotion panels includes
publications and competitive grant funding, both of which are based
on external assessment. However, when it comes to assessing merit in
teaching, historically this has been left to the internal promotion panels
made up of people who do not necessarily have expertise in teaching and
learning or its evaluation. Regrettably, the appraisal of merit in teaching
is still often based almost entirely on the unsystematic examination of
student feedback data. Recently a new academic promotion policy and
schema was introduced at the University of New South Wales (UNSW)
in Australia. This enabled applicants to indicate the weighting they
wanted to give their teaching, research and administrative performance.
For those staff who weighted teaching as 50% or above, there was a
requirement that their teaching portfolio be assessed by a separate panel
of peers with special expertise in learning and teaching. The peer review
reports provide the evidence on which promotion panels make their
judgments.
Guidelines on Learning
Much has been written and said about universities moving from a
perspective of educational quality based on good teaching (skills) to one
which places high quality student learning in the foreground (e.g. Biggs,
1991; Barr & Tagg, 1995). But is much happening? Conceptually, many
staff in universities have difficulty shifting their central focus from teaching
to student learning. The Guidelines on Learning that Inform Teaching, with
a supporting Toolkit, is an innovative resource that has been developed and
put into practice at UNSW with good early effects. Other universities (e.g.
MIT; Victoria University) have expressed interest in using the Guidelines
as a device to increase their focus on student learning. Readers may like
to review the relevant website (http://www.guidelinesonlearning.unsw.edu.
au/) to see if it could be useful for them.
Linking of Budget Decisions to Performance in Learning and Teaching
Making a component of budget allocation to faculties or departments based
on performance in learning and teaching can send a very clear message to
academic leaders (e.g. department heads). Importantly, it shows that senior
management is serious about valuing quality in teaching. Such a scheme,
which drew its inspiration from a similar process at the University of
Queensland (see http://www.uq.edu.au/hupp/index.html?policy=3.10.11),
has been implemented at UNSW for the past two years. In brief, senior
management defined a set of indicators (LTPIs) in consultation with
academic leaders and on the basis of performance against these indicators,
5% of faculties’ budgets were allocated. Although there is vigorous debate
on the nature of these indicators, there is no doubt the strategy has great
potential for supporting behaviours (changes) that are likely to result in
enhancement of the student experience. Some examples of the LTPIs will
be provided in the presentation (see http://www.ltu.unsw.edu.au/content/userDocs/2006_07PerformanceIndicators_Summary.pdf).
Serious Teaching-related KPIs for University Managers
Further evidence that a university places high value on the quality of
the educational experience provided to students is to have clear key
performance indicators (KPIs) written into the performance appraisal and
reward arrangements for academic leaders and managers, particularly
deans and heads of department. Such indicators should be explicit about
expectations and responsibilities in relation to the implementation of
important policies, processes and practices in the area of teaching and
learning.
Programme Design
Just as we tend to compartmentalise the different components that
contribute to quality assurance in education without looking at the whole,
curriculum development (and evaluation) is often focused on the subject
or unit level rather than at whole-of-programme level. Often, insufficient
thought and planning are invested in the design of the whole programme
in terms of how the planned student learning activities in particular
subjects facilitate valued student learning, and more importantly, how the
learning activities in particular subjects link with those in other subjects
to build capabilities over time. While space does not permit a discussion
of the many important aspects of programme design relevant to QALT,
the specification and mapping of graduate attributes is one key aspect we
will comment on briefly.
Currently, it is trendy in higher education to define our institution’s
‘Graduate Attributes’—the knowledge, capabilities and qualities that
students are meant to acquire while in their programme. In practice,
individual subject conveners may or may not be doing their best to
ensure that students are getting effective experiences to develop these
attributes in their subject. Taking ‘information literacy’ as an example,
some academics work hard to incorporate discipline relevant learning
activities that ensure gradual development of this attribute in students.
Their efforts might include using authentic or real world student projects,
engaging students with workforce practitioners, or working closely with
library staff to make more diverse and effective experiences available to
students. Some academic staff invest less effort. Thus, whether a student
develops required competence in information literacy is based on the luck
they have in selecting subjects where effective learning experiences are
provided.
While mapping of graduate attributes across existing curricula is happening
increasingly, there is a need to focus much more on the programme/course
level. This is very important at programme design, approval and review
stages so that student learning activities for different attributes can be
planned and incorporated, and their effectiveness (or likely effectiveness)
assessed.
Supports and Resources
A comprehensive and systemic QALT needs to include appropriate supports
and resources to enable quality assurance activities to occur. QA mechanisms
such as policies and processes for assessment and evaluation, even if effective, are not sufficient. The critically important continual improvement
aspect of QALT will only be achieved if appropriate supports for staff
and students are provided. Strategic staff development, programmes to
support innovations in learning and teaching, recognition and reward
mechanisms for staff, and student support services are all particularly
important. In the time available in our presentation, we will elaborate
briefly on one particular strategy for staff development, the development
of communities of practice. This includes the training and participation
of experienced academic staff in staff development activities in learning
and teaching (see http://www.ltu.unsw.edu.au/content/teaching_support/unilt.cfm?ss=0).
Teaching and Learning Practices
There is a vast and rich literature on how students learn and the quality
of their learning. Brief summary reviews of some of this literature are
provided elsewhere (e.g. Biggs, 1989; Boyle & Trevitt, 1997; Ramsden,
2003; Trigwell & Shale, 2004). To achieve the ultimate goal of optimal
student learning, teaching and learning practices should be based on this
literature.
The Guidelines on Learning initiative referred to above was founded on
the literature on student learning. In brief, UNSW wanted to produce a
resource that was likely to be useful to staff in improving the student
experience, particularly their learning. A document was drafted,
based on this principle as well as on the findings of a survey of other
Australian universities, and a review of relevant literature on learning
and undergraduate education. This draft document was presented to the
UNSW Committee on Education (CoE) in mid-2003. After a consultation
process, the CoE agreed on a set of 16 statements based on the educational
research literature and acknowledged good practices in teaching. These
statements provide a basis for the development of teaching and learning
practices that are likely to enhance student learning (i.e. they are guidelines
that inform teaching). These statements were endorsed at the November
2003 meeting of the Academic Board as the Guidelines on Learning
that Inform Teaching at UNSW. The website was developed not only to
list the guidelines but also to provide explanations and links to relevant
literature. As the goal was to produce something that was useful and used,
the website also includes a so-called Toolkit. The Toolkit, based on the
Guidelines, was designed to assist staff in reflecting on the effectiveness
of their practices. They are invited to use it to review their classes, course
or programme, and as a reflective tool. An example is the uncontested but often ignored Guideline 1 which states “Effective learning is supported
when students are actively engaged in the learning process”. Asking staff
to provide examples in their teaching and learning practice that are taking
note of this guideline is a powerful stimulus for reviewing their practice
and encourages more active engagement in their teaching with the likely
result of improved student learning (see http://www.guidelinesonlearning.unsw.edu.au/).
The Student Experience and Student Engagement
Internationally, research into, evaluation of and actions to enhance the
overall student experience at university are becoming more widespread
and sophisticated. There are many aspects of the student experience, and
several are fundamentally important for shaping the ultimate learning
outcomes that students achieve.
Student engagement, while a subset of student experience, is itself a
complex and powerful shaper of the quality of student learning. Research
and evaluation studies over time support this argument and provide
valuable guidance for the improvement of levels of student engagement
(Fraser & Walberg, 1991; Kuh, Douglas, Lund, & Ramin-Gyurnek, 1994;
Coates, 2005). To stress the importance of ‘student engagement’, Coates
(2006) asserts that: “The best facilities, courses and teachers are of no
consequence if students don’t capitalise on them” (p. 28).
Evaluation of student engagement, along with the degree to which
we are providing the conditions that foster engagement in different
ways (in our individual universities), is very important. This is a
critical first step towards being able to strengthen practices that
optimise student engagement. We advocate strongly that evaluation
and improvement of levels of student engagement needs to be at the
heart of thinking and practice within a systemic QALT framework.
While this is not yet the case in Australia, the opposite is the case
in the United States. One indication of the importance being placed
on student engagement there is the fact that in 2005 more than
550 colleges and universities used the National Survey of Student
Engagement (NSSE) (see http://webdb.iu.edu/Nsse/?view=tools/ using_nsse_data). At UNSW, research into and evaluation of student
engagement has commenced recently, including the use of a number
of modified NSSE items in evaluation instruments. The potential of
resulting data from such activities for improvement in learning and
teaching can be seen already.
Evaluation and Continual Improvement Processes
Effective evaluation and follow-up improvement and communication
processes are essential for any comprehensive approach to QALT. On
its own, the topic of the design and implementation of evaluation and
improvement processes is a complex one, and we make only two points
here, one general and one more specific. First, evaluation and improvement
processes need to exist not just for operational aspects such as classroom
teaching. They need to be established to enable continual monitoring and
improvement of all of the major components within the broader QALT
framework (e.g. student experience; student and graduate outcomes;
teaching practices). At UNSW, ground-breaking work is being carried out
in the Faculty of Medicine to develop a comprehensive and integrated set
of evaluation and improvement processes for all aspects of learning and
teaching (see http://www.peig.med.unsw.edu.au).
On the more specific issue of the evaluation of the student experience, so
much has been written that we make only two points briefly here. First, we
are confident that most people here today would acknowledge that welldesigned
processes for obtaining evaluative feedback on teaching from
students provide one valid kind of evidence on the quality of education
in an institution. This is overwhelmingly supported by the literature.
Unfortunately, despite much advocacy, the power of student feedback
data is still seriously under-utilised, particularly for improvement
purposes. The general but important notion of closing the loop, moving
from data to action and follow-up monitoring of effects, is still not being
taken seriously enough. One good example of progress on this front can
be found at Monash University, where comprehensive processes exist
to help senior managers use student feedback data as a first screen to
shape decisions on where improvements can be made (see http://www.adm.monash.edu.au/cheq/evaluations/other-surveys/meq/index.html).
At UNSW, the CATEI Process design (see http://www.ltu.unsw.edu.au/content/course_prog_support/ catei.cfm?ss=0) stresses the importance of
explicit responsibilities for follow-up reporting and actions to facilitate
improvements in courses and teaching (see Boyle & Lee, 2005).
Concluding Thoughts
This paper and our presentation focus on a systemic framework for QALT,
of the kind represented in Figure 1. Our overarching message is that QA
for learning and teaching needs to be driven primarily by a concern for
high quality student learning and be based on a multi-faceted and systemic approach. This overall approach is necessary if the achievement of substantial
and sustainable quality improvement results in the teaching and learning
aspect of university enterprise is a serious aspiration. Ideally, everything
we do in trying to improve student learning, including their experience,
should be interconnected in ways that enable synergies.
At UNSW several examples of such synergies can be cited. The potential of
the Guidelines on Learning for longer-term improvement of teaching was
multiplied significantly when evidence of their use became a requirement
for the teaching portfolio submitted by staff seeking promotion. Careful
definition of performance indicators for teaching at faculty level and
linking budget allocation to these has also mobilised improvements.
Evidence of the constructive use of data from the CATEI Process has been
built into these indicators.
There has been a growing conviction at UNSW that this holistic and
systemic approach is the best way to provide serious QA for learning and
teaching. Progress has been gradual but after six years focused attention
there is clear evidence of success. Staff have begun to feel, for the first time,
that teaching is being valued. Staff and student support and development
mechanisms have been enhanced. External grants, awards and recognition
for excellence in teaching have increased considerably.
As we write, the critical influence of senior management support for
valuing teaching highly and developing comprehensive approaches
to QALT is again in the foreground. There are clear pressures on
universities from some quarters in Australia to define their visions in
terms of being a world-class university on the basis of excellence in
research only, rather than excellence in teaching and research. In our
view, world-class in research and world-class in teaching should not
be seen in either/or terms. In other words, a competitive perspective on ‘teaching’ and ‘research’ is not desirable. In the longer term, aspiring
to excellence in both will enhance the status of an institution. Most
students are not attracted to a university because of its reputation in
research alone, and students’ enthusiasm to become researchers is
shaped significantly by their experiences of teaching. We will continue
to be hopeful that university-leadership wisdom in the future includes
insistence that a high value be placed on excellence in learning and
teaching. For this value to be demonstrable, clear support is needed for
the continual evolution of policies and practices that ensure systematic
approaches to QALT.
References
Barr, R.B. & Tagg, J. (1995). From teaching to learning: A new paradigm for
undergraduate education. Change, 27(6), 13–26.
Biggs, J. (1989). Approaches to the enhancement of university teaching.
Higher Education Research and Development, 8(1), 7–25.
Biggs, J. (1991). Teaching for better learning. Legal Education Review, 2,
133–148.
Bowden, J. & Marton, F. (1998). The university of learning: Beyond quality
and competence. London: Kogan Page.
Boyle, P. & Bowden, J. (1997). Educational quality assurance in universities:
an enhanced model. Assessment and Evaluation in Higher Education,
22(2), 111–121.
Boyle, P. & Lee, A. (2005). Closing loops and fitness for purposes: enhancing
evaluation process design. In Proceedings of the 2005 Evaluations and
Assessment Conference, Sydney, November 2005.
Boyle, P. & Trevitt, C. (1997). Enhancing the quality of student learning
through the use of subject learning plans. Higher Education Research and
Development, 16(3), 293–308.
Burden-Leahy, S. (2005). Introducing graduate outcomes into the program
quality assurance system: an overview of the first phase of implementation
in a vocational higher education institution in the Middle-East. Quality in
Higher Education, 11(2), 129–136.
Coates, H. (2005). The value of student engagement for higher education
quality assurance. Quality in Higher Education, 11(1), 25–36.
Coates, H. (2006, July 26). Engage the entire experience. The Australian, The
Higher Education Supplement.
Dill, D. (1992). Quality by design: toward a framework for academic quality
management. Higher Education: Handbook of Theory and Research. New
York: Agathon.
Fraser, B. & Walberg, H.J. (1991). Educational environments: Evaluation,
antecedents and consequences. Oxford: Pergamon.
Kuh, G.D.; Douglas, K.B.; Lund, J.P. & Ramin-Gyurnek, J. (1994). Student
learning outside the classroom: transcending artificial boundaries.
Washington: ERIC Clearinghouse on Higher education.
Massy, W.F. (1997). Teaching and learning quality-process review: the Hong
Kong programme. Quality in Higher Education, 3(3), 249–262.
Ramsden, P. (2003). Learning to teach in higher education. London:
Routledge.
Trigwell, K. & Shale, S. (2004). Student learning and the scholarship of
teaching. Studies in Higher Education, 29(4), 523–536.
*Selected paper from TLHE 2006: Quality in Higher Education
|
