Writing your own assessment tools

In this course, you'll learn how to write three simple kinds of assessment tools: Tasks, Observation checklists, and Interview questionnaires.

What are assessment tools?	Assessment tools are simply ways of asking students to produce specific evidence in concrete situations that can be used for assessment. They include: • Instructions to the student on what to do. • Instructions to the assessor on what to do, if it is not obvious in the instructions to the student, and, • A form to record the assessment.
When might you need to write your own?	In some unusual cases, you might be unable to use the existing assessment tools. For example, a written journal was not very helpful for a group of deaf students, so local staff devised a system of interviews that worked much better and still addressed all the same competencies.

Other different kinds of simple assessment tools include:

Descriptions of acceptable performance
Form templates and proformas
Checklists for evaluating work samples
Recognition portfolios
Student self-assessment materials

What are assessment modes?

Assessment modes are ways of gathering evidence of the student's skills. Exams aren't the only way, and sometimes they're totally inappropriate. In most cases, you can call them assessment tasks, that is, tasks that students will do to be assessed on.

The assessment mode is sometimes decided and written down beforehand; this often happens if the assessment mode is integral to the skill. Here are two examples:

The element statement might require students to be able to present their thoughts in a written report. In this case, report writing (the assessment mode) is part of the skill to be learnt.
The only real way to assess some practical skills is for the student to demonstrate them in a real or simulated workplace.

In many cases, all you will be given is the standards and criteria. You must take responsibility to choose a suitable assessment mode when you plan the assessment. By being more creative, you might be able to make your assessments much more valid, and you may be able to give the student a choice of assessment modes.

Remember:

The skill to be learned must match
The way it is taught,
The way it is practiced,
and
The way it is assessed

Tasks

A task tells students to do something. It should contain enough instructions for students to know exactly what you expect of them, and should be the natural thing to do when performing the skill. For example, if the skill is to replace a flat tyre, the task should be to replace a flat tyre.

Tasks can cover performance criteria in three ways:

The task instructions can contain the assessment criteria, so that students who do the whole task well according to the instructions can be assessed as performing satisfactorily.
In other cases, the textbook explains the details of what is required.
You can list the requirements on the same page.

The advantages

First, this approach is useful for any activity that can be expressed as a task.

Second, you can easily ask for more than one kind of evidence. For example, you can:

ask for the actual performance of the task,
tell the students what kinds of questions the assessor will ask, and
ask for any related paperwork.

Third, the instructions to the student also tell the instructor what is required, so you don’t usually need to write a separate set of instructions to the assessor.

Fourth, if the task (or a series of tasks) collects all required evidence for a unit, you can go straight to recording the assessment result for that unit.

Fifth, reporting is easy because most of the documentation is already done in the task description. You can use a fairly standard form to report the assessment. (Reporting is even easier if the form is in software.) Besides, it can be quite brief, containing at least:

The name of the student
The name of the assessor
A list of tasks, each of which has a tick-box
The name of the unit(s)
The name of the task(s)
A place to record the assessment results.
A place for general comments on the student’s performance, which usually doubles as feedback to the student
A date.
A place for the assessor’s signature and date.

Observation checklists

Many assessors use use a checklist to check off all essential aspects based on their observations.

An observation checklist is a form that contains at least:

The name of the student
The name of the assessor
The name of the unit
A list of criteria, each of which has a tick-box
A place for general comments on the student’s performance, which usually doubles as feedback to the student
A place for an assessment conclusion (e.g. Satisfactory/ Not satisfactory)
A date.
A place for the assessor’s signature.

Many checklists only ask for one kind of evidence, so they are often inadequate to gather all evidence for a unit. However, you can add instructions about seeing relevant paperwork and asking questions.

Put the instructions to the assessor at the top of the form where people will see them. This is usually much better than a separate piece of paper, which people will easily lose.
As much as possible, try to get the whole form on onto one piece of paper.
Add more columns if you have to do make multiple observations. This is usually better than using more forms.
You can sometimes copy performance criteria from the units into your form, but they normally need to be edited into plain language.

Interview questionnaire

An interview questionnaire is quite similar to an observation checklist. An interview questionnaire is a form that contains at least:

A set of instructions for the assessor. (Again, put them put them at the top of the form where people will see them. This is usually much better than a separate piece of paper, which people will easily lose.)
The name of the student
The name of the assessor
The name of the unit
A list of questions, each of which has a tick-box.
A place for general comments on the student’s performance, which usually doubles as feedback to the student
A place for an assessment conclusion (e.g. Satisfactory/ Not satisfactory)
A date.
A place for the assessor's signature.

Most of the questions should be open-ended questions, that is, they cannot be answered with Yes or No. They need to examine more than jusst remembered information. Even when students give an answer, you usually need to ask the reasons why.

💡Tips
1. The simplest way to write an interview questionnaire is to go though the list of requirements and turn them into questions. The trick is to make them clear and concrete, not Explain everything in the universe. You can split up complicated questions into simple questions, ask people to explain the steps, or ask them how they would do or respond to something.
2. When you write a set of questions to test required knowledge, consider offering it as either an interview or an assignment.
3. When the list of knowledge items is quite detailed, you can replace one item with one question. But when a knowledge item only specifies a broad area of knowledge, you will normally need a series of questions to cover it.
4. Knowledge is sometimes adequately demonstrated by doing the assessment task.
5. when you use the questionnaire:
• You do not need to ask questions twice. When students answer the first question, they will probably answer many questions that are intended to come up later on. You don't need to ask a question again if the student has already answered it satisfactorily.
• You may probe with any necessary follow-up questions.
• If students get an answer wrong, it might be that their nervousness makes them forgetful. Talk the student around so that they have another attempt.
• Your interview should corroborate other evidence that the student has submitted. For example, they should be able to explain the work in their portfolio.

Some other kinds of assessment modes

To create an assessment, the assessor normally needs to define a kind of activity. The kind of activity is called a mode of assessment. There are many, many other different kinds of assessment modes:

Major projects
Students define their own projects and (usually) write a formal report. Projects often cover a cluster of related units.
Tasks
These are smaller tasks that reflect unit requirements.
Demonstration of skills in a classroom environment
This usually means that student shows the skill to the instructor (who is probably also the assessor) as he/she learns it. However, it can also be an umbrella term for many other kinds of assessment done in the classroom.
Written essays
Essays are appropriate as part of the assessment requirements for some knowledge-driven units, especially if the elements are to evaluate, compare or identify etc. In some cases, staff might set the topic. In other cases, students must be able to choose their own topics to identify, refine and explore issues.
Reading programs
These are most appropriate for knowledge-driven units, and deal more with mastering a body of knowledge than workplace competency. They are usually tutorials, in which students present and discuss their papers in the group, book reviews, or annotated bibliographies.
Private journals
These show how the student is thinking and maturing and why, and often involve emotional or attitudinal factors of personal development.
Written assessments under "exam conditions."
Students don't get told the questions beforehand, may not communicate with each other, and cannot use textbooks. A person responsible to the assessor supervises the assessment. However, questions and topics are based directly on the element statements, and assessment follows the given assessment criteria. Answers may be explanations of factual matter, essays where the student presents and supports a viewpoint, or multiple choice. Students may also be free to select only some of the questions, for example, "three of the following five questions."
Electronic examinations
are the same as written assessments, but the computer terminal and its software are also under exam conditions. In this way, multiple choice assessments may be graded immediately through the software or anywhere in the world over the Internet, and other kinds of essays may be sent immediately to the assessor for grading.
Open book assessment
is much like a written assessment, but students may use books. These are not an easy option, because the student must have a good grasp of the assessed material to be able to use the books efficiently. It generally produces more informed answers.
Written examinations that students may research in the library beforehand
These are not an easy option either, for the same reason as open book assessments. It also produces more informed answers.
Written examinations that students may take home or do by email
These are almost the same as essays. Authenticity may be a problem in some cases.
Oral examination
It might involve asking the student each question in a list of questions. Oral language proficiency can be appraised through a face-to-face conversation with a proficient speaker of the language, who may be the assessor.
Oral questioning
This may be more like a discussion or a personal interview to establish the authenticity of the student's work, required knowledge and/or to establish competence. The trend is toward using a fixed set of questions that can be mapped to the requirements. You can also ask spontaneous oral questions about something the student is demonstrating, with no list of questions.)
Oral presentation or debate
Students might be required to present their learning in a prepared speech or lecture, perhaps with an adversary against whom they must defend their ideas.
On-job assessment
Some skills require assessment in an on-job situation against a checklist of relevant competencies. The assessor can do a simple observation on the job, or assess a practicum involving reports from other credible people (usually the workplace supervisor).
Simulations that are observed by the assessor
These are most useful when a real on-the-job situation is not feasible, but the skill must be demonstrated in as near-to-real situation as practical.
Scenario analysis
Respond to a particular scenario, described in writing, on video, or in simulation. The student must identify important features, what is happening, and the correct response. The student might be required to explain the rationale behind his/her approach.
Critical incident analysis
When most of a difficult job is routine and predictable, competence can be assessed by the way student reacts in particularly difficult or unpredicted circumstances. Assessments typically look at the context, preceding events, the incident, the factors determining response, other possible responses, and consequences of the response taken.
Assessment by portfolio of documentary evidence
Portfolios normally include written work done by the student (reports, manuals, essays, data printouts, letters, brochures, self-evaluations, resume, log books, designs, drawings, etc.) and supporting documentary verification (e.g. professional licenses and memberships, reports of supervisors, instructors, and responsible authorities, references, job descriptions.)
Standardized tests
Program audit
Logbooks
A diary of satisfactory work done by the student and verified by the supervisor. In some cases, it can also act as a reference if the supervisor makes a recommendation of competence.
Supervisor reference
The supervisor states that the student has performed satisfactorily. These usually work better if the supervisor addresses specific competencies; a general reference saying that the student is a nice person and has done well does little to establish competency.
Authenticated list of achievements
Check the claims made in the students CV with credible authorities.
Lessons or workbooks that are filled in
Distance educators often use this kind of material.
360° appraisal
The assessor gathers a self-assessment from the student and detailed appraisals from the student's supervisor, a peer, a subordinate, and a client.
Exhibition of an object made by the student
Ability is assessed through products such as manufactured goods, artworks (e.g. painting, drawing, crafts, sculpture), etc.
Public performance
Examples are public speaking, leading meetings, drama, singing, music, and dance.

💡 Tips and recommendations

When you plan a professional qualification, start by writing a job description of what you want your graduates to be able to do. It will make the course look much more practical to your students.
We strongly recommend tools that can be re-used without change. (Consider the traditional alternative; many lecturers need to write completely new examination questions every time they assess students.) You will easily be able to write reusable tools if you:

Focus on asking students to do something in their own context.
Don’t predetermine the exact answers you expect. For example, you might ask students to give a presentation but not determine the actual topic and outline they must use.

If the task is an ongoing responsibility with no beginning and end, you can specify a long enough period of time for workplace assessment.
If the task has a start and end, you can specify how many times the student must perform the task:
- Major, longer-term tasks normally need to be done only once.
- Students usually need to do smaller, short-term tasks at least twice, and you may need to specify different contexts.
Check that tools are clear enough for the student and assessor to know what they are expected to do.
Use simple language that is only as complex as necessary.
Lay them out on the page so that they are easy to read.
Check that tools match the competency standards.
Use workplace forms as assessment tools.
When your assessment tools are designed for a specific context, you might find that you need another set to assess students in another different context.
When your assessment tools are designed for a specific context, they often don't work for RPL students coming from very different contexts. Big hint: Go back to the original competency statements.
Integrate a cluster of competencies into a meaningful project. Students learn better and find the whole process more satisfying. (This is called project-based learning.)
Practitioners often don’t like unnecessary writing and generally do better in oral interviews. Besides, if they don’t get a point the first time, you can talk them round in a circle and let them have another go. Some very capable people don’t always get it right the first time around.
You can sometimes use discussion groups for assessments, as long as the group is small enough for you to observe what each student says.
In a professional qualifications, align your assessment goals with the business goals of the student’s employer. Everybody wins.
I like to write each unit’s set of assessments to be a standalone statement.
You can sometimes write an assessment activity that is equally relevant to multiple units. So students only have to do it once.

Trial your assessment tools

You cannot anticipate all your student's interpretations of your assessment tools. Trial your draft assessment tools to check that the content is right and they suit your group of students. Record outcomes and make any necessary corrections and improvements.

Decide how you will trial your assessment tools. You can simply get a colleague to check your assessment tools, such as another staff member or an industry person. You might need to trial the assessment tools with a sample group of people similar to your students to know how well they work.
Analyse feedback. In the feedback, people might say:

What do you mean by this sentence?
Could be more relevant to what we actually do here.
Didn't work on campus. But that's where we assess it.
Oops, you got this fact wrong.
We don't do it that way anymore. They brought out a new rule.
Difficult to use.
Good test, but it'd cost a fortune to use.
Took a long time to do. Could you do it some way that takes less time?
The jargon confused lots of people.
Do we really need all that paperwork?
The deaf student just gave up in frustration.

Fix anything wrong before use. If you have to make significant changes, then you should test it again.
File them properly. When you've finished, file a copy and make backups of the soft copies.