Purpose

The purpose of assessment is very important. Is it to assess our students or is it to assess the effectiveness of our teaching? And is it correct to have the same criteria for all kinds of assessment? We have to think deeply about these, for it is part and parcel of our professional duties and responsibilities.

Change

As recent as two years ago, we still had the British external examiner system in which we had to send our exam papers for vetting way ahead of time, even before teaching began. Hence, we had always been constrained by assessment. But most faculties have now changed the system. The examination questions are now vetted by the external examiner only after the event. As far as the registrar’s office is concerned, the deadline for submission of exam papers is one month ahead for printing and processing. However, if the deans and the heads are “kiasu” and bring this deadline forward by one or two months, we would be stuck with the old system again. We should also aim to decentralize our examination process. We have decided to gain some valuable experience by first decentralizing the postgraduate examination process.

It is always painful to have to change. But it will be ridiculous if we do not. We are now experimenting with more empowerment, I would really like to see a total empowerment like the system which I had experienced as a visiting professor at Yale University in 1983. I went to the head of department and asked him when I would have to submit my exam paper. His reply was: “I don’t have time to read your exam paper. What I had time to read in detail was your C.V. and when the universitycouncil approved of your appointment as visiting professor, that was it. You decide how to set your exam questions; you decide the number of questions to set and the format of the paper. You decide the grades.”

That was full empowerment. I liked that as a teacher and I think we should set it as our target. If you think about it, one day when you become a head of department, would you really have time to read all those exam questions? Are you an expert in that particular area to critique one of your experts and say that the question is too difficult or too easy? And when a student has done only half the question and the professor decides to give him 80% of the mark, can you challenge him?

So what we have been doing is not very effective. Time is better spent in a session like this when we discuss what we are trying to achieve through assessment. We have to think about it. If we do not, we are not fair to ourselves, the department and the system. The university depends a lot on our teachers to provide feedback to us on the effectiveness of their teaching, on how they find it in the field. Are we wasting our teachers’ time? Are we wasting our students’ time? Students spend three or four years with us as undergraduates—very precious time. Are we fair to them?

Assessment can help the students to learn, particularly continuous assessment, class test, assignment; less so for the major exam. Perhaps the major exam could be used to tell the society whether a particular student deserves a scholarship or whether he is a fairly weak student. But if we really think about it, thefundamental purpose of assessment is to help a student know whether he has really understood the subject.

Active Teaching

I have circulated a paper on active teaching. Here is an example of active teaching/learning:

A Harvard professor thought that he had been doing a good job teaching first year physics because he had done it for more than ten years. One day, he read a paper about the ineffectiveness of the conventional teaching method and the advantage of active learning. He did not believe and decided to test it. He was teaching Newton’s third law. This is High School physics—Action Equals Reaction —common sense. Normally, he would set the tutorial questions and 90% would get them right. But this time he decided to test whether the students really understood. So he devised a simple problem: One big truck and one small truck collided. Is the big truck exerting a bigger force on the small truck? Half the class said: “Yes”, and half said: “No”. Yet they were just taught the same principle.

To implement active learning, he then formed four small groups for students to discuss and argue among themselves. Ten minutes later, 90% got it right. This raises a question over the conventional way of teaching and assessment. With the conventional system, weaker students would have a superficial understanding of the subject and only the very bright ones would get into the deeper level of understanding. It would help tremendously if the students could discuss and help each other understand better.

We have to ask ourselves whether the way we assess, teach and engage students in discussion is effective. Do we compromise on our teaching because we have to complete the syllabus? Or is it because the exam questions have already been set? Think about it.

There is a real case which someone wrote to me about. To qualify for engineering, our students usually need a minimum of two A’s and one B. Most of these students going to the U.K. will end up with a minimum of a second upper, if not a first class honours. And every year, we are dealing with eight or nine hundred of them, yet only thirty or so get a first class honours, a hundred or so will get a second upper, while the majority will be second lower, and some will go into third class and so on. We thought this bell curve was very good.

One of our second lower students went to do the GRE and the TOEFL. He had an almost perfect TOEFL score and 2100 points for the GRE. When we have overseas applicants with such scores, we would offer them a postgraduate scholarship. But this is our second lower student. Are we too strict in our assessment? This student who is labelled second lower may one day be the managing director of a large engineering company and try to fund our research. Are we caught by the bell curve? Not every student will be a professor and this is certainly not our goal. Are we assessing them based on who has the potential to be a professor?

Creativity

I will touch on two more topics very briefly; first, about creativity. Increasingly, we are setting our purpose to educate in such a way that our students will not only have sufficient knowledge to work in the society and continue to learn, but they must also be creative. Our country has reached an advanced stage of development where the conventional solutions may not work anymore. Some of our students are engaged by top companies and they have to produce world-class products. Some will become entrepreneurs competing with the best in the world. They have to come up with creative ideas and solutions. But during their years with us, have we engaged them in this kind of learning and given them the opportunity to think of creative solutions? Teaching creativity is a challenge. Assessing creativity is another.

The first thing to do is to read more about creativity. We should then ask ourselves some questions: Have we read the top three books in this area? Have we read the top ten papers about creative teaching? Better still, have we been practising creative teaching and have we been confident enough to share our experience with colleagues?

Open-Book Exams

The second and associated topic is open-book exams. There is a lot of resistance against open-book tests and exams. They are difficult to carry out. Unlike the conventional way of assessing, there are few examples available for reference.

If we really think about it, open-book exams is real life. Today I am standing here; it is an open-book exam for me. I can bring books and notes but they will not really help me when I do not know the topic that I am talking about. There is nothing wrong if I forget some details and refer to my notes and a few sheets of data. So in open-book exams the books are not really very helpful although they could help psychologically. By the time one reads them to seek new understanding, two hours would have gone.

So, what is holding us back? If we say that it is unfair because students can refer to chapter five and then copy the first two paragraphs and score an A, then our exam question is set wrongly in the first place because we are testing memory recall. We should assume that if a student is training to be a medical doctor, he must know the fundamentals. An engineer designing a bridge must also know the basics. So we do not test this type of knowledge anymore. But they may not remember 100% of the material, they may remember 90%; so they could refer to a book for the details they may have forgotten. But if they do not understand such knowledge, even if they refer to books, it is useless. We should be testing how they apply the concepts and whether they know which concepts to apply. So by that understanding, every paper can be open-book.

Looking at the statistics, we are hardly reaching 20% in open-book exams. I would argue that postgraduate should be 100%; final year should be 100%; and we can aim at 50-50 for first year. But we are far from these figures. We are nervous because it is new to us. We should set ourselves a five-year target. It will be embarrassing to be caught in a situation where we are said to be a worldclass institution, yet there are still major practices that are not world-class. We are preparing our students for the 21st century, but more than 80% of our exams are still closed-book.

I hope I have achieved what I have set out to achieve— to stimulate you to think about the subject of assessment. Please ask questions and continue to challenge me by sending me e-mails. You have to be sporting, knowing that I am still experimenting with new ideas in order to pioneer changes. But I am single-handed unless you give me more input and feedback so I could use these to strengthen my case when I discuss the issue with the deans and the heads. Hopefully together, we will move in the right direction.