6
 
Volume 6
October 2008
Quality Assurance for Learning and Teaching: A Systemic Perspective*
Professor Adrian Lee Patrick Boyle
University of New South Wales, Australia University of New South Wales and Q Associates, Australia
 

Abstract
Quality Assurance for Learning and Teaching (QALT) in universities is commonly relatively piecemeal in terms of strategy, organisation and implementation. In overview form, this paper and the associated presentation argues that more systemic and effective approaches to QALT are possible and desirable. An indicative systemic framework is provided and discussed as are some practical examples of quality assurance (QA) components (of such a framework) developed in an Australian university.

Keywords
Higher education, quality assurance, learning and teaching, quality in teaching

Introduction
Historically, internal approaches to Quality Assurance for Learning and Teaching (QALT) in universities have typically been based on an overly simplistic notion of quality assurance and/or a disorganised and unrelated set of elements and practices.

Viewing QALT as little more than the measurement of student and graduate outcomes is extremely limiting, if not unethical. Outcomes are clearly extremely important, but the student experience (of several years) at university is a major part of their lives and shapes their approach to work, learning and personal life beyond graduation. In a very real sense, the
legacy of the total student experience (e.g. approaches to learning that are developed) is an important outcome. In addition, many students these days are paying for their educational experience, not just a particular outcome.

There is also an analogy with sports training which is appropriate for a consideration of universities’ internal approaches to QALT. Elite sports coaches consistently express the importance of athletes focusing on getting the practices, experiences and processes right first, and the good results (outcomes) will follow. So paying attention to the enablers (e.g. support
processes) of high quality student outcomes is important in QALT.

Often, the components in place to provide QALT are not comprehensive or well organised. For example, there is always a process for the approval of new programmes. Course, if not programme coordination roles usually exist, and student evaluation of teaching occurs, often with no clear expectations or processes for action based on collected data. While such arrangements commonly exist in universities, they are seldom linked in ways that provide synergies for QALT.

The overall theme of our presentation and this paper is that there is a need to take a more holistic and systemic view of QALT in universities. This requires the identification and sensible linking of QA components in a conceptual and practical framework. The ideas presented are based on review of research and practice internationally and our own experience
in universities over many years, including the last five years together in a respected Australian university.

The Need for More Systemic QALT
The development and successful implementation of comprehensive and systemic approaches for QALT is still an ideal and a challenge. While support for moving in this direction can be found in the literature (e.g. Bowden & Marton, 1998; Boyle & Bowden, 1997; Burden-Leahy, 2005; Dill, 1992; Massy, 1997), more powerful drivers can be seen in the emerging
expectations of national and international quality assurance agencies and programme accreditation bodies (e.g. Association to Advance Collegiate Schools of Business [AACSB]; Australian Medical Council; Australian Universities Quality Agency [AUQA]). The trend in both of these kinds of agencies is towards greater expectations of universities to demonstrate
multi-faceted, relatively sophisticated and coherent approaches to assuring quality in learning and teaching. Increasingly, expectations cover a wider range of quality assurance arrangements, including those related not just to programme structure and content, but to outcome standards, evaluation and improvement processes, student experiences and services, learningteaching strategies and continuing development for teachers.

We believe that in the future, commended approaches to QALT will be identified essentially on the basis that they involve multiple and aligned components that are systemically linked to achieve real synergies for the continual enhancement of learning and teaching.

A Systemic Framework for QALT
So what might a systemic approach to QALT look like? First, it has to identify the key components of what a university does to achieve high quality learning by its students, which must be the ultimate goal. Second, it must seek to arrange these components so they interact to achieve synergies. This might appear obvious, but it seems that in universities we rarely sit
back and consider these interactions. As a result, much effort is wasted and the ultimate goal is not achieved. For example, a sophisticated course evaluation system might be developed, but if teaching is not valued highly by the institution’s senior executive, as indicated by academic promotion processes and decisions, then typically only the already excellent teachers
take evaluation results seriously.

Figure 1 provides an indicative framework to illustrate how QALT can be conceived and organised in a systemic way. In the remainder of the paper, we briefly discuss key components in this framework and provide some examples of how alignment and synergies between them are necessary to achieve genuinely systemic and comprehensive quality assurance for learning and teaching.

figure1

Key Elements in a Systemic QALT Framework
In the brief discussion that follows, we start with what we believe are the two critical elements in the QA framework shown in Figure 1. High Quality Student Learning is at the top of our diagram and Leadership in Learning and Teaching is at the bottom. Both need to be seen as the most important drivers for QA, with High Quality Student Learning being the
primary focus and Leadership the critical enabler.

High Quality Student Learning as the Primary Focus for QA
Few would challenge the view that the achievement of high quality student learning is what we aspire to in universities (along with excellent research). Some would argue that this is all that matters. It follows logically that QALT should focus on student learning outcomes, along with the operational elements of universities that we are confident have the most effect on the quality of those outcomes (e.g. planned student experiences).

Student and graduate outcomes are clearly important and need to be measured and evaluated as accurately as possible. This is important for summative student assessment and the accountability and improvement aspects of quality assurance. However, the precise evaluation of learning outcomes is often not simple in higher education, given the complex
capabilities being aspired to and the numerous variables that influence student achievement and graduates’ success. Many involved in the fundamental changes to the design of medical curricula over the past thirty years, starting with the pioneering work at McMaster University in Canada, are convinced that some of the new curriculum designs and approaches to learning are better, but it is difficult to prove this conclusively based purely on outcome measures. The most effective
approaches to QALT will be those that sensibly integrate elements that we are confident of, based on research, experience and logic, have significant influences on the quality of student learning outcomes.

Leadership in Learning and Teaching
It is not possible to provide a detailed discussion of this topic here. However, we do wish to make a couple of general points very strongly.

The beginning and the end of success in achieving high quality student learning for the majority of students depend on the quality and commitment of leadership. If the higher executive makes it clear to staff that research is predominantly valued and pay only lip service to the importance of teaching, with little hard evidence of the voiced support (e.g. clear recognition and reward for merit in teaching; allocation of funding for innovation in learning and teaching), then the institution will fail to achieve the ultimate goal of maximising the quality of student learning. While the best students will succeed in any case, and there will be pockets of staff that teach brilliantly and inspire their students for life, the overall culture and the occurrence of negative experiences for many students will not change. We will return to this later in the paper.

Policy and Values on Learning and Teaching
Policies and value statements relating to teaching, and increasingly learning, often remain the province of academic boards or senates and their committees. This is appropriate but university leadership needs to ensure that mechanisms exist to support these policies. Four examples of important areas of policy and values, and strategies used to ensure they become
operational are described briefly below.

Promotion on the Basis of Merit in Teaching
Staff confidence that promotion processes provide due weight to merit in teaching is critical to ensure that greater proportions of staff put effort into enhancing their teaching and the experience they provide for their students. In most university promotion schemes, performance in research is based on peer assessment, because of the assumption of their
expertise. Peers, for this purpose, typically include respected scholars (e.g. editors of journals and grant assessors). Accordingly, the peer review evidence on research merit used by promotion panels includes publications and competitive grant funding, both of which are based on external assessment. However, when it comes to assessing merit in teaching, historically this has been left to the internal promotion panels made up of people who do not necessarily have expertise in teaching and learning or its evaluation. Regrettably, the appraisal of merit in teaching is still often based almost entirely on the unsystematic examination of student feedback data. Recently a new academic promotion policy and schema was introduced at the University of New South Wales (UNSW) in Australia. This enabled applicants to indicate the weighting they
wanted to give their teaching, research and administrative performance. For those staff who weighted teaching as 50% or above, there was a requirement that their teaching portfolio be assessed by a separate panel of peers with special expertise in learning and teaching. The peer review reports provide the evidence on which promotion panels make their
judgments.

Guidelines on Learning
Much has been written and said about universities moving from a perspective of educational quality based on good teaching (skills) to one which places high quality student learning in the foreground (e.g. Biggs, 1991; Barr & Tagg, 1995). But is much happening? Conceptually, many staff in universities have difficulty shifting their central focus from teaching to student learning. The Guidelines on Learning that Inform Teaching, with a supporting Toolkit, is an innovative resource that has been developed and put into practice at UNSW with good early effects. Other universities (e.g. MIT; Victoria University) have expressed interest in using the Guidelines as a device to increase their focus on student learning. Readers may like
to review the relevant website (http://www.guidelinesonlearning.unsw.edu. au/) to see if it could be useful for them.

Linking of Budget Decisions to Performance in Learning and Teaching
Making a component of budget allocation to faculties or departments based on performance in learning and teaching can send a very clear message to academic leaders (e.g. department heads). Importantly, it shows that senior management is serious about valuing quality in teaching. Such a scheme, which drew its inspiration from a similar process at the University of Queensland (see http://www.uq.edu.au/hupp/index.html?policy=3.10.11), has been implemented at UNSW for the past two years. In brief, senior management defined a set of indicators (LTPIs) in consultation with academic leaders and on the basis of performance against these indicators, 5% of faculties’ budgets were allocated. Although there is vigorous debate
on the nature of these indicators, there is no doubt the strategy has great potential for supporting behaviours (changes) that are likely to result in enhancement of the student experience. Some examples of the LTPIs will be provided in the presentation (see http://www.ltu.unsw.edu.au/content/userDocs/2006_07PerformanceIndicators_Summary.pdf).

Serious Teaching-related KPIs for University Managers
Further evidence that a university places high value on the quality of the educational experience provided to students is to have clear key performance indicators (KPIs) written into the performance appraisal and reward arrangements for academic leaders and managers, particularly deans and heads of department. Such indicators should be explicit about
expectations and responsibilities in relation to the implementation of important policies, processes and practices in the area of teaching and learning.

Programme Design
Just as we tend to compartmentalise the different components that contribute to quality assurance in education without looking at the whole, curriculum development (and evaluation) is often focused on the subject or unit level rather than at whole-of-programme level. Often, insufficient thought and planning are invested in the design of the whole programme
in terms of how the planned student learning activities in particular subjects facilitate valued student learning, and more importantly, how the learning activities in particular subjects link with those in other subjects to build capabilities over time. While space does not permit a discussion of the many important aspects of programme design relevant to QALT, the specification and mapping of graduate attributes is one key aspect we will comment on briefly.

Currently, it is trendy in higher education to define our institution’s ‘Graduate Attributes’—the knowledge, capabilities and qualities that students are meant to acquire while in their programme. In practice, individual subject conveners may or may not be doing their best to ensure that students are getting effective experiences to develop these attributes in their subject. Taking ‘information literacy’ as an example, some academics work hard to incorporate discipline relevant learning
activities that ensure gradual development of this attribute in students. Their efforts might include using authentic or real world student projects, engaging students with workforce practitioners, or working closely with library staff to make more diverse and effective experiences available to students. Some academic staff invest less effort. Thus, whether a student
develops required competence in information literacy is based on the luck they have in selecting subjects where effective learning experiences are provided.

While mapping of graduate attributes across existing curricula is happening increasingly, there is a need to focus much more on the programme/course level. This is very important at programme design, approval and review stages so that student learning activities for different attributes can be planned and incorporated, and their effectiveness (or likely effectiveness) assessed.

Supports and Resources
A comprehensive and systemic QALT needs to include appropriate supports and resources to enable quality assurance activities to occur. QA mechanisms such as policies and processes for assessment and evaluation, even if effective, are not sufficient. The critically important continual improvement aspect of QALT will only be achieved if appropriate supports for staff and students are provided. Strategic staff development, programmes to support innovations in learning and teaching, recognition and reward mechanisms for staff, and student support services are all particularly important. In the time available in our presentation, we will elaborate briefly on one particular strategy for staff development, the development
of communities of practice. This includes the training and participation of experienced academic staff in staff development activities in learning and teaching (see http://www.ltu.unsw.edu.au/content/teaching_support/unilt.cfm?ss=0).

Teaching and Learning Practices
There is a vast and rich literature on how students learn and the quality of their learning. Brief summary reviews of some of this literature are provided elsewhere (e.g. Biggs, 1989; Boyle & Trevitt, 1997; Ramsden, 2003; Trigwell & Shale, 2004). To achieve the ultimate goal of optimal student learning, teaching and learning practices should be based on this
literature.

The Guidelines on Learning initiative referred to above was founded on the literature on student learning. In brief, UNSW wanted to produce a resource that was likely to be useful to staff in improving the student experience, particularly their learning. A document was drafted, based on this principle as well as on the findings of a survey of other Australian universities, and a review of relevant literature on learning and undergraduate education. This draft document was presented to the UNSW Committee on Education (CoE) in mid-2003. After a consultation process, the CoE agreed on a set of 16 statements based on the educational research literature and acknowledged good practices in teaching. These
statements provide a basis for the development of teaching and learning practices that are likely to enhance student learning (i.e. they are guidelines that inform teaching). These statements were endorsed at the November 2003 meeting of the Academic Board as the Guidelines on Learning that Inform Teaching at UNSW. The website was developed not only to
list the guidelines but also to provide explanations and links to relevant literature. As the goal was to produce something that was useful and used, the website also includes a so-called Toolkit. The Toolkit, based on the Guidelines, was designed to assist staff in reflecting on the effectiveness of their practices. They are invited to use it to review their classes, course
or programme, and as a reflective tool. An example is the uncontested but often ignored Guideline 1 which states “Effective learning is supported when students are actively engaged in the learning process”. Asking staff to provide examples in their teaching and learning practice that are taking note of this guideline is a powerful stimulus for reviewing their practice
and encourages more active engagement in their teaching with the likely result of improved student learning (see http://www.guidelinesonlearning.unsw.edu.au/).

The Student Experience and Student Engagement
Internationally, research into, evaluation of and actions to enhance the overall student experience at university are becoming more widespread and sophisticated. There are many aspects of the student experience, and several are fundamentally important for shaping the ultimate learning outcomes that students achieve.

Student engagement, while a subset of student experience, is itself a complex and powerful shaper of the quality of student learning. Research and evaluation studies over time support this argument and provide valuable guidance for the improvement of levels of student engagement (Fraser & Walberg, 1991; Kuh, Douglas, Lund, & Ramin-Gyurnek, 1994;
Coates, 2005). To stress the importance of ‘student engagement’, Coates (2006) asserts that: “The best facilities, courses and teachers are of no consequence if students don’t capitalise on them” (p. 28).

Evaluation of student engagement, along with the degree to which we are providing the conditions that foster engagement in different ways (in our individual universities), is very important. This is a critical first step towards being able to strengthen practices that optimise student engagement. We advocate strongly that evaluation and improvement of levels of student engagement needs to be at the heart of thinking and practice within a systemic QALT framework. While this is not yet the case in Australia, the opposite is the case in the United States. One indication of the importance being placed
on student engagement there is the fact that in 2005 more than 550 colleges and universities used the National Survey of Student Engagement (NSSE) (see http://webdb.iu.edu/Nsse/?view=tools/ using_nsse_data). At UNSW, research into and evaluation of student engagement has commenced recently, including the use of a number of modified NSSE items in evaluation instruments. The potential of resulting data from such activities for improvement in learning and teaching can be seen already.

Evaluation and Continual Improvement Processes
Effective evaluation and follow-up improvement and communication processes are essential for any comprehensive approach to QALT. On its own, the topic of the design and implementation of evaluation and improvement processes is a complex one, and we make only two points here, one general and one more specific. First, evaluation and improvement
processes need to exist not just for operational aspects such as classroom teaching. They need to be established to enable continual monitoring and improvement of all of the major components within the broader QALT framework (e.g. student experience; student and graduate outcomes; teaching practices). At UNSW, ground-breaking work is being carried out
in the Faculty of Medicine to develop a comprehensive and integrated set of evaluation and improvement processes for all aspects of learning and teaching (see http://www.peig.med.unsw.edu.au).

On the more specific issue of the evaluation of the student experience, so much has been written that we make only two points briefly here. First, we are confident that most people here today would acknowledge that welldesigned processes for obtaining evaluative feedback on teaching from students provide one valid kind of evidence on the quality of education
in an institution. This is overwhelmingly supported by the literature. Unfortunately, despite much advocacy, the power of student feedback data is still seriously under-utilised, particularly for improvement purposes. The general but important notion of closing the loop, moving from data to action and follow-up monitoring of effects, is still not being taken seriously enough. One good example of progress on this front can be found at Monash University, where comprehensive processes exist to help senior managers use student feedback data as a first screen to shape decisions on where improvements can be made (see http://www.adm.monash.edu.au/cheq/evaluations/other-surveys/meq/index.html). At UNSW, the CATEI Process design (see http://www.ltu.unsw.edu.au/content/course_prog_support/ catei.cfm?ss=0) stresses the importance of
explicit responsibilities for follow-up reporting and actions to facilitate improvements in courses and teaching (see Boyle & Lee, 2005).

Concluding Thoughts
This paper and our presentation focus on a systemic framework for QALT, of the kind represented in Figure 1. Our overarching message is that QA for learning and teaching needs to be driven primarily by a concern for high quality student learning and be based on a multi-faceted and systemic approach. This overall approach is necessary if the achievement of substantial and sustainable quality improvement results in the teaching and learning aspect of university enterprise is a serious aspiration. Ideally, everything we do in trying to improve student learning, including their experience, should be interconnected in ways that enable synergies.

At UNSW several examples of such synergies can be cited. The potential of the Guidelines on Learning for longer-term improvement of teaching was multiplied significantly when evidence of their use became a requirement for the teaching portfolio submitted by staff seeking promotion. Careful definition of performance indicators for teaching at faculty level and
linking budget allocation to these has also mobilised improvements. Evidence of the constructive use of data from the CATEI Process has been built into these indicators.

There has been a growing conviction at UNSW that this holistic and systemic approach is the best way to provide serious QA for learning and teaching. Progress has been gradual but after six years focused attention there is clear evidence of success. Staff have begun to feel, for the first time, that teaching is being valued. Staff and student support and development mechanisms have been enhanced. External grants, awards and recognition for excellence in teaching have increased considerably.

As we write, the critical influence of senior management support for valuing teaching highly and developing comprehensive approaches to QALT is again in the foreground. There are clear pressures on universities from some quarters in Australia to define their visions in terms of being a world-class university on the basis of excellence in research only, rather than excellence in teaching and research. In our view, world-class in research and world-class in teaching should not
be seen in either/or terms. In other words, a competitive perspective on ‘teaching’ and ‘research’ is not desirable. In the longer term, aspiring to excellence in both will enhance the status of an institution. Most students are not attracted to a university because of its reputation in research alone, and students’ enthusiasm to become researchers is shaped significantly by their experiences of teaching. We will continue to be hopeful that university-leadership wisdom in the future includes insistence that a high value be placed on excellence in learning and teaching. For this value to be demonstrable, clear support is needed for the continual evolution of policies and practices that ensure systematic approaches to QALT.

References
Barr, R.B. & Tagg, J. (1995). From teaching to learning: A new paradigm for undergraduate education. Change, 27(6), 13–26.

Biggs, J. (1989). Approaches to the enhancement of university teaching.
Higher Education Research and Development, 8(1), 7–25.

Biggs, J. (1991). Teaching for better learning. Legal Education Review, 2, 133–148.

Bowden, J. & Marton, F. (1998). The university of learning: Beyond quality
and competence. London: Kogan Page.

Boyle, P. & Bowden, J. (1997). Educational quality assurance in universities:
an enhanced model. Assessment and Evaluation in Higher Education,
22(2), 111–121.

Boyle, P. & Lee, A. (2005). Closing loops and fitness for purposes: enhancing evaluation process design. In Proceedings of the 2005 Evaluations and Assessment Conference, Sydney, November 2005.

Boyle, P. & Trevitt, C. (1997). Enhancing the quality of student learning through the use of subject learning plans. Higher Education Research and Development, 16(3), 293–308.

Burden-Leahy, S. (2005). Introducing graduate outcomes into the program quality assurance system: an overview of the first phase of implementation in a vocational higher education institution in the Middle-East. Quality in Higher Education, 11(2), 129–136.

Coates, H. (2005). The value of student engagement for higher education quality assurance. Quality in Higher Education, 11(1), 25–36.

Coates, H. (2006, July 26). Engage the entire experience. The Australian, The Higher Education Supplement.

Dill, D. (1992). Quality by design: toward a framework for academic quality management. Higher Education: Handbook of Theory and Research. New York: Agathon.

Fraser, B. & Walberg, H.J. (1991). Educational environments: Evaluation, antecedents and consequences. Oxford: Pergamon.

Kuh, G.D.; Douglas, K.B.; Lund, J.P. & Ramin-Gyurnek, J. (1994). Student learning outside the classroom: transcending artificial boundaries. Washington: ERIC Clearinghouse on Higher education.

Massy, W.F. (1997). Teaching and learning quality-process review: the Hong Kong programme. Quality in Higher Education, 3(3), 249–262.

Ramsden, P. (2003). Learning to teach in higher education. London: Routledge.

Trigwell, K. & Shale, S. (2004). Student learning and the scholarship of teaching. Studies in Higher Education, 29(4), 523–536.


*Selected paper from TLHE 2006: Quality in Higher Education

 

published by
Centre for Development of Teaching and Learning (CDTL)
National University of Singapore
© CDTL 2000 - 2008