Students are expected to study in order to master as best they can the material covered in each module they take. To obtain a fair gauge of this achievement, how then should examinations be set?

For most students, “study” is just what needs to be done to do well at the examinations. Memorisation is evidently easier than serious study (i.e. understanding concepts and learning application strategies). If students can score well at examinations by spewing out material learnt by heart, most would do so. So an examination that inadvertently or otherwise allows success by memorisation may not be adequate in measuring the level of mastery. When marking examination scripts, it is also not easy to tell a regurgitated answer from one composed through the use of appropriate concepts and application strategies. The difficulty arises when the student needs to make use of what is supposed to have already been learnt. Thus, it makes good sense for examiners to provide more meaningful assessments that prevent regurgitation.

Some believe traditional close-book examinations tend to permit success by regurgitation. In contrast, the open-book format is thought to be a better evaluation format. However, does providing access to information during examinations by itself solves the problem of identifying those who have simply regurgitated memorised material? The answer, which may also vary from discipline to discipline, can be found indirectly by examining the functional needs of graduates, as these needs will expose any mismatch between current performance and past grades.

Turning to the practice of engineering, there are things that engineers need to know anytime, anywhere. A civil engineer should not have to look up a textbook to find out why reinforcements are normally needed in a concrete structure or what factors should be considered when designing a building. In general, engineers should remember formulae of fundamental significance in the subject area, design parameters commonly used by engineers and configurations that are basic in practice. For obvious reasons, an evaluation that attempts to find out how much students “know” in this respect can best be measured by close-book examinations.

There are charts, tables, equations and other reference material engineers must know how to use. To evaluate one’s ability to determine when and how to use these resources, appropriate reference materials should be available to the student at assessment time. Open-book examinations or project-based assessments can be adopted for measuring the level of such application skills. Project assignments also enable development of teamwork and other practical skills in students.

Most valuable to a person when practising the art of engineering is the level of understanding of engineering concepts and ability to use them creatively. In evaluating a student’s achievement in this regard, how information and basic skills are adopted and composed to respond to a question or an assigned task is more important than the information that is accessed and presented by the student. Both close- and open-book formats as well as project assignments can be adopted here for assessment.

The need for adopting a variety of examination formats in engineering disciplines is clear. However, preventing regurgitation remains a separate problem irrespective of whether access to external and unspecified information is allowed or not during examinations. Strategies adopted by students when preparing for examinations reveal a clue: they rely on familiarity in order to memorise and regurgitate. This is apparent when students provide the occasional incorrect but elegantly packaged solutions to a slightly twisted question that they mistake for a familiar one.

Setting unfamiliar questions is therefore at least one way of minimising the possibility of success by regurgitation. But this concept should be introduced carefully. Only the questions asked or tasks assigned in the assessment process need to be unfamiliar to the extent that needs to break the potential of regurgitation. Overall assessment format may remain the same year after year, and the scope of the assessment will have to strictly remain within that of the module.

There are several hurdles to clear if the academic community wants to use unfamiliar questions in examinations. Those who think familiarity as an absolute primary requirement of examinations will need to change their mindset. Students are also likely to consider unfamiliar questions as scary. Nevertheless, what is fundamental is that examiners should assess a student’s achievement on a “regular as-is” basis rather than under temporary “exam-primed” conditions. Consequently, what can be achieved by thoughtfully introduced unfamiliarity at examinations should not be underestimated.