Today, at the TEASIG / CRELLA conference in Luton, I had the pleasure of hearing two talks by Dr John Field. He was focussing on creating L2 listening tests, but a lot of what he said will be useful for those of us writing ELT listening materials for teaching, too! Based on what John talked about, then, here are some common problems to bear in mind when designing listening tasks.
Multiple Choice tasks
Here, the key is finding the magic balance between not making one of the answer options too obviously wrong (or right) and not having possible answers which are too close in meaning or overlap. We often see multiple choice questions where some of the answer options are not mutually exclusive, rely on very fine lexical/semantic distinctions (therefore testing vocabulary but not listening comprehension!), can be answered from general knowledge of the world (therefore not requiring learners to understand the listening text!), or use language more sophisticated than that in the listening text we’re trying to check comprehension of! In listening tasks for lower levels, another danger is that learners may be encouraged to listen out for specific words but may not understand the overall message of the audio input, especially if the possible answers consist of only one word each. And sometimes it seems attempts to make listening tasks harder/higher level just result in longer answer options to choose from – but this simply increases the load on learners’ working memory whilst they’re listening, therefore making it more of a memory test than one of listening comprehension.
Gap Fill Tasks
What makes this task type hard is that it makes multiple simultaneous demands on the learners, as they have to read, listen and write at the same time. Not having the right answer(s) may then be a result of this mixed demands processing, rather than a lack of comprehension. Likewise, having the right answer does not necessarily signify true comprehension of the lisening text, as learners could simply fill in the word they (think they) hear, even without knowing what it means in this context.
Multiple Matching Tasks
This kind of matching task, e.g. identify the speaker who…., can be difficult for learners to cope with when used with listening texts. This is mainly because they usually involve audio texts of conversations with several speakers, whose voices and ways of speaking the learners then have to distinguish/recognise and adjust to – in the worst case, we may end up testing voice recognition skills rather than listening comprehension! Also, if there are the same number of items in both lists to be matched up, getting one pair wrong automatically means another pair is wrong. John also warned against getting too creative with names of speakers, as lower level learners might not recognise these words as names, but mistake them for unfamiliar vocabulary items!
True / False Tasks
The most obvious argument against using this type of task is that learners have a 50% chance of answering correctly, even without understanding a word of the listening text! Some task designers add in a “not mentioned” option to decrease these odds, but listening for something that is not there is quite a tricky task! Also, there is sometimes a pretty fine line between ‘false’ and ‘not mentioned’. Interestingly, research has shown that people are more likely to tick ‘true’ than ‘false’ when given this choice (something about human nature?), so to try to ensure a learner’s good result on such a task is due to actual comprehension, it might be better to include more items where the correct answer is ‘false’.
Overall, John recommends designing listening tasks based on the actual audio, not just the transcript. This may be one way to help make sure our tasks actually test listening, and not other skills or aspects of language. Also, the focus should remain on listening, and so we need to avoid tasks which cloud the water by requiring complex reading skills at the same time, for example. Of course, the task types desribed above can be effective in checking listening comprehension and providing ‘diagnoses’ of learners’ listening difficulties, but we need to bear these potential pitfalls and problems in mind when designing them and take care to keep our tasks aligned with the learning goals of the lesson.