Month: Oct 2017

Common problems with common listening tasks

Common problems with common listening tasks

Today, at the TEASIG / CRELLA conference in Luton, I had the pleasure of hearing two talks by Dr John Field. He was focussing on creating L2 listening tests, but a lot of what he said will be useful for those of us writing ELT listening materials for teaching, too! Based on what John talked about, then, here are some common problems to bear in mind when designing listening tasks.

Multiple Choice tasks

Here, the key is finding the magic balance between not making one of the answer options too obviously wrong (or right) and not having possible answers which are too close in meaning or overlap. We often see multiple choice questions where some of the answer options are not mutually exclusive, rely on very fine lexical/semantic distinctions (therefore testing vocabulary but not listening comprehension!), can be answered from general knowledge of the world (therefore not requiring learners to understand the listening text!), or use language more sophisticated than that in the listening text we’re trying to check comprehension of! In listening tasks for lower levels, another danger is that learners may be encouraged to listen out for specific words but may not understand the overall message of the audio input, especially if the possible answers consist of only one word each.  And sometimes it seems attempts to make listening tasks harder/higher level just result in longer answer options to choose from – but this simply increases the load on learners’ working memory whilst they’re listening, therefore making it more of a memory test than one of listening comprehension.

Gap Fill Tasks

What makes this task type hard is that it makes multiple simultaneous demands on the learners, as they have to read, listen and write at the same time. Not having the right answer(s) may then be a result of this mixed demands processing, rather than a lack of comprehension. Likewise, having the right answer does not necessarily signify true comprehension of the lisening text, as learners could simply fill in the word they (think they) hear, even without knowing what it means in this context.

Multiple Matching Tasks

This kind of matching task, e.g. identify the speaker who…., can be difficult for learners to cope with when used with listening texts. This is mainly because they usually involve audio texts of conversations with several speakers, whose voices and ways of speaking the learners then have to distinguish/recognise and adjust to – in the worst case, we may end up testing voice recognition skills rather than listening comprehension! Also, if there are the same number of items in both lists to be matched up, getting one pair wrong automatically means another pair is wrong. John also warned against getting too creative with names of speakers, as lower level learners might not recognise these words as names, but mistake them for unfamiliar vocabulary items!

True / False Tasks

The most obvious argument against using this type of task is that learners have a 50% chance of answering correctly, even without understanding a word of the listening text! Some task designers add in a “not mentioned” option to decrease these odds, but listening for something that is not there is quite a tricky task! Also, there is sometimes a pretty fine line between ‘false’ and ‘not mentioned’. Interestingly, research has shown that people are more likely to tick ‘true’ than ‘false’ when given this choice (something about human nature?), so to try to ensure a learner’s good result on such a task is due to actual comprehension, it might be better to include more items where the correct answer is ‘false’.

General comments

Overall, John recommends designing listening tasks based on the actual audio, not just the transcript. This may be one way to help make sure our tasks actually test listening, and not other skills or aspects of language. Also, the focus should remain on listening, and so we need to avoid tasks which cloud the water by requiring complex reading skills at the same time, for example. Of course, the task types desribed above can be effective in checking listening comprehension and providing ‘diagnoses’ of learners’ listening difficulties, but we need to bear these potential pitfalls and problems in mind when designing them and take care to keep our tasks aligned with the learning goals of the lesson.

Getting the (Conference Speaker) Balance Right

Getting the (Conference Speaker) Balance Right

There’s been a lot of talk about this recently – getting the balance right. The balance between men and women, between native and non-native speakers at ELT events and conferences. 

I’ve fairly recently joined the committee of an IATEFL SIG. I’m on the events team. So these kinds of ‘balance’ topics are more pertinent to me now than ever. 

This post is not really a ‘How to’: In fact, it’s me just kind of getting my thoughts in order, my pondering on the subject. There might be some tips, but this is definitely a request for more ideas!

So, let’s say we’re going to organise an ELT event. We put out a call for proposals. Various things could happen, and it’s how to deal with these that I want to talk about in this post. 

Scenario 1. We want someone to host a workshop. We review the submissions ‘blind’, i.e. without any information about the potential speaker who has submitted them. Proposal A fits the theme of our event, has a good balance in favour of practical ideas, includes interesting workshop activities, and sounds like it would be a good fit for our event. Proposal B is only loosely connected to the event’s theme, sounds too theoretical for a workshop, and the activities don’t sound like they would fit in the workshop time-slot. I’m guessing we want to accept Proposal A. Right?

And then we find out that the speaker who submitted Proposal A happens to be a white, male native speaker. Proposal B came, let’s just say, from someone who didn’t fit all of those labels. We are keen to avoid all-male or all-native-speaker presenters at our events. Should we accept Proposal B in order to fulfil this aim, and risk providing a less good workshop for our participants? I’m not really in favour of ‘positive discrimination’ in this case if it may endanger the quality of the event.

So what else could we do?

– Cancel the event. (This is probably the least favourable outcome for everyone involved.)

– Contact the person who submitted Proposal B and explain its weaknesses, asking for a re-submission. (This might take time we don’t have. And is it fair?)

– Find another way to include Proposal B, such as a poster presentation, so that the speaker can gain experience, get their voice heard, and hopefully submit a more fitting proposal next time. (If possible…)

– Accept it this time, and keep the person who submitted Proposal B in mind for a future event.

– Extend the deadline in the hope of receiving more proposals. (This might take time we don’t have.)

– In future, provide more specific guidelines for speaker proposals. (This doesn’t solve our immediate problem.)

– [What else could we do? Answers on a postcard, please!]

 

Scenario 2: We are looking for 6 speakers for a conference. We receive 5 proposals. All of them are from male native speakers. We could arrange the day to include 5 talks and a panel discussion with those speakers. We are keen to avoid all-male or all-native-speaker presenters at our event, but if we don’t accept all of the 5 proposals, we won’t be able to fill the day.

So what else could we do?

– Cancel the event. (This is probably the least favourable outcome for everyone involved.)

– Extend the deadline in the hope of receiving more proposals. (This might take time we don’t have.)

– Invite late proposals from other (female / non-native) speakers and re-evaluate the selection. (This poses a new set of questions:  Does this seem unfair? Who do you choose to invite a proposal from?)

– Invite other (female / non-native) speakers to take part in the panel discussion. (This poses a new set of questions: Who do you choose to invite? Should it then be an all-female panel – is that ‘positive discrimination’?)

– [What else could we do? Answers on a postcard, please!]

 

Scenario 3: We are co-organising an event with a sponsor, e.g. a publishing company. We agree that we will select 5 speakers from the proposals we receive, and they will send 5 speakers (maybe editors, authors, sales reps, etc.). We choose 3 female and 2 male speakers, of whom 3 are native and 2 are non-native speakers. We think we’ve got a pretty good balance. But the sponsoring company decides to send 5 male native speakers to hold talks at the event.

So what else could we do?

– Cancel the event. (This is probably the least favourable outcome for everyone involved.)

– Express our concerns and ask them to send alternative (female / non-native) speakers. (Not sure how well this would go down?)

– Change our speakers so they are all female non-native speakers. (How fair is this on the others we wanted to accept?)

– [What else could we do? Answers on a postcard, please!]

From all of this pondering, what have I / we learnt? OK, so I invented the scenarios and plucked the numbers out of thin air, just to make the point. But I think you get what I mean! But, well, sometimes we might just be in a bind and not be able to change he situation. We might end up with a line-up which seems to proliferate the male native-speaker presenter bias among conference speakers /workshop hosts that we want to discourage. People will complain – but maybe they don’t understand the difficult situation we are in. Still, at the very least, we can change how we approach our event organisation in the future. And if we’re planning an event in good time, which most of the time I’d guess we are, we might (should) be able to make that extra effort to move towards a better gender and non-/native speaker balance.

It seems to me, though, that some of the roots of the problem do not lie within the powers of events organisers. For example in Scenario 2 – why do we have so few proposals? Why are none of them from non-native speakers/ women? Perhaps the call for proposals was poorly advertised, not targeted at a wide range of potential speakers? That we could fix. But if lots of people (including women and non-native speakers) saw the call, then why did they not submit a proposal? I’m not the first one to say this, and I surely won’t be the last, but I think there must be reasons why these groups sem to put themselves forward for talks less often than others. Maybe it’s a confidence thing, maybe time or money concerns, or maybe extra-professional issues. Whatever it is, probably one of the most effective ways to avoid scenarios like the ones I invented here would be to somehow help these potential speakers  see themselves as potential speakers. But the ‘How to’ on that topic will have to be another post!