My LTSIG Talk: Using Multimodal Learner-Driven Feedback to Provide Sustainable Feedback on L2 Writing

Time for a little advertising! 😉

On October Thursday 5th October at 4.25pm UK time, I’ll be giving an online talk as part of the LTSIG /OllREN online conference and would be delighted to see you there!

Exploring efficient ways to give sustainable feedback on L2 writing is important because providing meticulous correction of language errors and hand-written summaries can be time-consuming and often seems less effective than desired. For feedback to be sustainable (i.e. effective long-term), it should be formative, interactive and impact on students’ future work (Carless et al 2011). Thus traditional, hand-written feedback practices may be inefficient at effecting sustainability. Integrating technology into feedback delivery has been shown to have potential in alleviating the situation, by stimulating students to engage with feedback they receive and enabling dialogues about their work.

Combining work into feedback on L2 writing with ideas promoted in higher education, I devised the Learner-Driven Feedback (LDF) procedure, where feedback is given by the teacher, but learners ‘drive’ how and on what they receive feedback: they can choose between various digital delivery modes and are required to pose questions about their work to which the tutor responds.

In this talk, I will summarise some recent literature which supports both the use of technologies such as email, audio recording, and text-editing software features, and responses to students’ individual queries in feedback procedures, before practically demonstrating LDF. I will refer to my own recently published article on LDF in EAP, and discuss my evaluation of its application in my teaching, providing compelling reasons and practical suggestions for its employment in various language teaching contexts. These discussions will also explore potential mechanisms underpinning the efficacy of multimodal approaches to making feedback more sustainable, in order to further aid teachers in making informed choices pertaining to their specific class groups. This includes topics such as learner autonomy, motivation, receptivity, learner-centredness and individualisation.

The talk is thus a combination of practical demonstration and theoretical background, of interest and relevance to a broad audience.

 

Reference: Carless, D., Salter, D., Yang, M., Lam, J., 2011. Developing sustainable
feedback practices. Studies in Higher Education, 36, 395–407.

Advertisement

Writing an ebook with students

My students have written an ebook!

You can read it for free here.

From an ELT perspective, this ebook is the result of a semester-long CLIL class, with project-based learning and a real and motivating outcome! If you want to find out how we did it, this post is for you!

Context

Our class was on British cultural studies, aimed at master’s level students of English Studies. This class aims to promote language learning and learning about content, in this case a particular British cultural topic. Usually, students are expected to do one oral presentation and one piece of written work as the assessment for this class. Only the other class members see the presentations, and the individual teacher is the only one who reads the essays, in order to grade them.  I’d say this is a pretty standard set up.

Background

Last summer, a colleague and I revamped our British cultural studies classes to move towards project-based learning. In 2016, our students hosted an exhibition open to staff and students a the University, which you can read about here. It was pretty successful, though the students involved found it a shame that all their hard work was only seen by a limited audience. Of course, the audience was a lot less limited than usual, but that’s what they said anyway…!

And so I came up with the idea of producing an ebook this year, which could then be made available publicly. I had seen other organisations use smashwords, and read about how easy it could be to publish a book through that site, so that’s what I thought we should do. I chose the umbrella topic of Britain in the Nineties for our focus, and 23 students signed up. I provided an outline for the class, which included a general module description, assessment requirements for the module, a provisional schedule for the ebook (to be sent to publish in the last week of semester!), and a selected bibliography of recommended reading on the topic.

Our semester is 14 weeks long, with one 90-minute lesson of this class each week. So how did we manage to produce an ebook in this time?

Weeks 1-3

In the first three lessons, I provided a video documentary, an academic article and a film for students to watch/read as a broad introduction to the topic. In lessons, we collected the main themes from this input (key words here: politics, music, social change), and discussed how they were interlinked. Each week, a different student was responsible for taking notes on our discussions and sharing these on our VLP for future reference. In week three, we rephrased our notes into potential research questions on key topic areas.

At the end of each lesson, we spent some time talking about the ebook in general. As the semester progressed, the time we spent on this increased and resembled business-like meetings.

Week 4

By this point, students had chosen topics / research questions to write about and discussed their choices in plenary to ensure that the ebook would present a wide-spread selection of topics on Britain in the Nineties. The students decided (with my guidance!) to write chapters for the ebook in pairs, and that each chapter should be around 2000 words, to fulfil the written assessment criteria of the class. Writing in pairs meant that they automatically had someone to peer review their work. To fulfil the oral assessment criteria, I required each writing team to hold a ‘work in progress’ presentation on the specific topic of their chapter. I had wanted to include these presentations to make sure I could tick the ‘oral assessment’ box, and because having to present on what they were writing would hopefully mean they got on with their research and writing sooner rather than later!

Weeks 5 & 7

The lessons in these two weeks were dedicated to writing workshops and peer review. We started both lessons by discussing what makes for good peer review, and I gave them some strategies for using colours for comments on different aspects of a text, as well as tables they could use to structure their feedback comments. These tables are available here. Regarding language, these are post-grad students at C1 level, so they’re in a pretty good position to help each other with language accuracy. I told them to underline in pencil anything that sounded odd or wrong to them, whether they were sure or not. If they were sure, they could pencil in a suggestion to improve the sentence/phrase, and if not then the underlining could later serve the authors as a note to check their language at that point.

In week 5, we looked at different genres of essay (cause/effect, compare/contrast, argument, etc), and how to formulate effective thesis statements for each of them. This focussed practice was followed by peer review on the introductions students had drafted so far. By this point, the students had decided that their chapters could be grouped thematically into sections within the ebook, and so did peer review on the work of the students whose chapters were going to be in the same section as their own.

In week 7, we reviewed summaries and conclusions, and also hedging language. Again, this was followed by peer review in their ‘section’ groupings, this time on students’ closing paragraphs.

Weeks 6 – 11

Almost half way through the semester, all writing teams were working on their chapters. In the lessons, we had a couple of ‘work in progress’ presentations each week. Further to my expectations, the presentations did an excellent job at promoting discussion, and particularly prompted students to find connections between their specific topics – so much so, that they decided to use hyperlinks within the ebook to show the readers these connections. Some students also used their presentations to ask for advice with specific problems they had encountered while researching/writing (e.g. lack of resources, overlaps with other chapters), and these were discussed in plenary to help each writing team as best we could. The discussions after the presentations were used to make any decisions that affected the whole book, for example which citation style we should use or whether to include images.

Week 12

In week 12, all writing teams submitted their texts to me. This was mainly because I needed to give them a grade for their work, but I also took the opportunity to give detailed feedback on their text and the content so they could edit it before it was published. I was also able to give some pointers on potential links to other chapters, since I had read them all. I felt much more like an editor, I have to say, than a teacher!

In the lesson, we had a discussion about pricing our ebook and marketing it. To avoid tax issues, we decided to make the ebook available for free. One student suggested asking for donations to charity instead of charging people to buy the book.

This idea was energetically approved, and students set about looking into charities we could support. In the end, SHINE education charity won the vote (organised by the students themselves!) I dutifully set up a page for us on justgiving.com: If you’d like to donate, it can be found here.

 

At this point, we also discussed a cover for the book. One student suggested writing ‘the Nineties’ in the Beatles’ style, to emphasise the links to the 1960s that some chapters mentioned. We also thought about including pencil sketches of some of the key people mentioned in the book, but were unable to source any that all students approved of. Instead, students used the advanced settings on the google image search to find images that were copyright free. A small group of students volunteered to finalise the cover design, and I have to say, I think they did a great job!

Week 13

During the lesson in this week, the ebook really came together. Some of the students were receiving more credit points than others for the class, based on their degree programme, and so it was decided that those students should be in charge of formatting the text according to smashwords’ guidelines, and also for collating an annotated bibliography. I organised a document on google docs, where all students noted some bullet points appraising one source they had used for their chapter, and the few who were getting extra points wrote this up and formatted it into a bibliography.

Formatting the text for publication on smashwords.com was apparently not too difficult, as the smashwords’ guidelines explain everything step-by-step, and you do not need to be a computer whizz to follow their explanations!

Week 14 and beyond

This week was the deadline I had set for sending the ebook for publication. After the formatting team had finished, I read through the ebook as a full document for the first time! I corrected any langauge errors that hadn’t been caught previously, and wrote the introduction for the book.  This took me about 2 evenings.

Then I set myself up a (free) account at smashwords.com and uploaded the ebook text and cover design. Luckily, the students had done a great job following the formatting rules, and the book was immediately accepted for the premium catalogue! (*very proud*)

 

Another small group of students volunteered to draw up some posters for advertising, and to share these with all class members so we could publicise the ebook on social media, on the Department’s webpage, and in the University’s newsletter.

Et voila! We had successfully published our ebook in just 14 weeks!

Evaluation

I’m so glad that I ran this project with my students! It honestly did not take more of my time than teaching the class as ‘usual’ – though usually the marking falls after the end of term, and it was quite pressured getting it done so we could publish in the last week! In future, I might move the publication date to later after the end of semester to ease some of the stress, though I do worry that students’ might lose momentum once we’re not meeting each week.

The students involved were very motivated by the idea that the general public would be able to read their work! I really felt that they made an extra effort to write the best texts they could (rather than perhaps just aiming to pass the class). This project was something entirely new for them, and they were pleased about their involvement for many reasons, ranging from being able to put it on their CV, to seeing themselves as ‘real’ writers. They have even nominated me for a teaching prize for doing this project with them!

Sadly, one student plagiarised. Knowingly. She said that she was so worried her writing wouldn’t be good enough, so she ‘borrowed’ large chunks of texts from an MA dissertation which is available online. Her writing partner didn’t catch it, and was very upset that their chapter would (discreetly!) not be included in the ebook. He was very apologetic to me; and probably also quite angry at her. If the reason she gave was true, it obviously rings alarm bells that I was expecting too much from the students or didn’t support them enough. I will aim to remedy this in future. It could, of course, just have been an excuse.

Also, some other students reported feeling that this project demanded more work from them than they would normally have to put into a class where the grade doesn’t count. Maybe this is because writing in a pair can take more time and negotiation, or maybe they also felt stressed by having to write their text during term time, rather than in the semester break when they would normally do their written assessments. Overall, though, the complaints were limited and often seemed to be clearly outweighed by the pride and enjoyment of being involved in such a great project!

I’m really pleased with how this project panned out, and would recommend other teachers give it a go! I’m very happy to answer any questions in the comments below, and for now, I wish you inspiration and happy ebook-project-planning! 🙂

 

 

What Postgraduates Appreciate in Online Feedback on Academic Writing #researchbites

In this article, Northcott, Gillies and Coutlon explore their students’ perceptions of how effective online formative feedback was for improving their postgraduate academic writing, and aim to highlight best practices for online writing feedback.

Northcott, J., P. Gillies & D. Caulton (2016), ‘What Postgraduates Appreciate in Online Tutor Feedback on Academic Writing’, Journal of Academic Writing, Vol. 6/1 , pp. 145-161.

Background

The focus of the study was on helping international master’s-level students at a UK university, for whom English is not their first/main language. The study’s central aim was investigating these students’ satisfaction with the formative feedback provided online by language tutors on short-term, non-credit-bearing ESAP writing courses. These courses, run in collaboration with subject departments, are a new provision at the university, in response to previous surveys showing dissatisfaction among students with feedback provided on written coursework for master’s-level courses. Participation is encouraged, but voluntary.  The courses consist of five self-study units (with tasks and answer keys), as well as weekly essay assignments marked by a tutor.

The  essays are submitted electronically, and feedback is provided using either Grademark (part of Turnitin) or ‘track changes’ in Microsoft Word . The feedback covers both  language correction and feedback on aspects of academic writing. These assignments are effectively draft versions of sections of coursework assignments students are required to write for the master’s programmes.

Research

The EAP tutors involved marked a total of 458 assignments, written by students in the first month of the master’s degrees in either Medicine or Politics. Only 53 students completed all five units of the writing course; though 94 Medicine and 81 Politics students completed the first unit’s assignment.

Alongside the writing samples, data was also collected by surveying students at three points during the writing course, plus an end-of-course evaluation form. Focussing on students who had completed the whole writing course, students’ survey responses were matched with their writing samples which had received feedback, as well as the final coursework assignment they submitted for credit in their master’s programme, for detailed analysis.

Findings

Analysing the feedback given by tutors, the researchers found both direct and indirect corrective feedback on language, as well as on subject-specific or genre-specific writing conventions and the academic skills related to writing. Tutors’ comments mostly refered to specific text passages, rather than being unfocused or general feedback.

Student engagement with feedback was evidenced by analysing writing samples and final coursework: only one case was found where ‘there was clear evidence that a student had not acted on the feedback provided’ (p. 155). However, the researchers admit that, as participation in the course is voluntary, the students who complete it are likely to be those who are in general appreciative of feedback, thus this finding may not be generalisable to other contexts.

In the surveys, most students’ reported feeling that the feedback had helped them to improve their writing. They acknowledged how useful the corrections provided were, and how the feedback could be applied in future. Moreover, comments demonstrated an appreciation of the motivational character of the feedback provided.

Summing up these findings, the researchers report:

It appeared to be the combination of principled corrective feedback with a focus on developing confidence by providing positive, personalised feedback on academic conventions and practices as well as language which obtained the most positive response from the students we investigated. (p. 154)

The students’ comments generally show that they responded well to this electronic mode of feedback delivery, and also felt a connection to their tutor, despite not meeting in person to discuss their work. As the researchers put it, students came to see ‘written feedback as a response to the person writing the text, not simply a response to a writing task’ (p. 156).

Take Away

The findings from this study highlight that simply using electronic modes of feedback delivery does not alone increase student satisfaction and engagement with feedback on their written work. Instead, the content and manner of the feedback given is key.

From the article, then, we can take away some tips for what kind of feedback to give, and how, to make electronic feedback most effective, at least for postgraduate students.

• Start with a friendly greeting and refer to the student by name.
• Establish an online persona as a sympathetic critical friend, ready to engage in dialogue.
• Don’t only focus on corrective feedback, but aim to guide the student to be able to edit and correct their work autonomously, e.g. provide links to further helpful resources.
• Be specific about the text passage the feedback refers to.
• Tailor the feedback to the student’s needs, in terms of subject area, etc.
• Give praise to develop the student’s confidence.
• Take account of the student’s L1 and background.
• Eencourage the student to respond to the feedback; especially if anything is unclear or they find it difficult to apply.

This post is part of ELT Research Bites 2017 Summer of Research (Bites) Blog Carnival! Join in here.

What library research skills training do EAP / undergrad students really need?

Colleagues and I have long since been aware of the lack of proper research and appropriate source use in our students’ EAP and academic essays. We decided to offer a one-hour workshop on researching in the library at our university, and enlisted the help of an expert – the library representative for our subject area.

We thought it was a great idea!

But students were not so impressed. And their work didn’t improve much, either.

After the session, students’ feedback centred on the following points:

•  the session was quite like a lecture, but not very interactive or with any opportunities for them to try things out for themselves
•  they were bored and confused by the explanation of possible search filters they could implement with Boolean search strings
• the MLA Bibliography they were introduced to “only gave them references but not the actual articles”
• that “there weren’t (m)any books on their topic”
• the searches of databases etc. don’t work “properly” at home.

So what do students new to library research really need to know?

• What an academic text is. (I recently asked MA-level students to bring in an academic journal article on our overall topic, and many of them turned up with texts from news sources like BBC, or from magazines like Time!)
• The fact that searching a database of academic texts is not like an internet search; i.e. you shouldn’t ask the catalogue your question (“Ok google, what are the differences between British and American spelling?”), but search for keywords or tags related to the topic.
• The fact that their specific topic may only be dealt with in a chapter within a book, which may not be searchable (unless the library has digitalised contents pages) and so they may need to search for more general terms.
• The difference between bibliographies and databases.
• The fact that many published sources are not available for free on the internet and so you can only access the full texts if your institution subscribes to that publication and you access it through their server. (Yes, this might mean, dear students, that you will actually have to physically go into the library!)
• The difference between reports of original (empirical) research and meta-studies or other summaries, and the importance of reading the primary work.
• The importance of using up-to-date sources, especially in areas where research and understandings have developed significantly in recent years.
• How to use keywords  / tags, and articles’ abstracts, or skim-reading, to judge a source’s relevance and appropriateness for their work.
• That something is not a fact just because it has been published – most academic work is about stance!

And so, I’ve come to the conclusion that a one-hour session might not be the best way to introduce students to the academic research community. A quick introduction to the specific institution’s library is a good idea, but that this clearly needs to be further supported within our teaching.

Over to you!

What kinds of tasks and activities do you get your students to do to help them to develop and train their researching skills? Please leave your ideas and tips in the comments below!

Reading Support Worksheets for EAP

Much is said in published literature about the necessity of EAP students reading authentic academic texts, and also about providing scaffolding and support for them to do so. I believe lecturers and academic tutors teaching their subject content in English and/or on a CLIL-based approach will also need to help students digest the readings for their classes.

Still, I often hear complaints from teachers that they set preparatory reading, but then found in the lesson that students were unable to discuss or work with the ideas from the reading, despite their claims that they did actually read the text. 

One way I’ve found to help students engage with the texts they are asked to read, then, is what I call ‘Reading Support Worksheets’. 

Reading Support Worksheets can help students to focus on the parts of a text or the ideas and concepts mentioned, so that they are better prepared to discuss or work with these in their lessons. Also, directing students’ attention to what the tutor deems the key concepts, the things they want to focus on in their lessons, the reasons they chose this reading text, can ease the load on students to comprehend every detail in a text and perhaps ease their frustration at the time and effort needed to do so. 

So how do I set up a Reading Support Worksheet?

I divide the text into manageable, logical sections, and pose questions or set quick tasks to guide students in the notes they should make whilst reading each section. Here are some of the question and task types I’ve used so far:

• What is the central claim presented in the introduction?
• What are the guiding questions and approach that this article is working with? How are these justified?
• Paraphrase the quote by xyz.
• Summarise the overall argument / point of paragraph xyz.
• What do these abbreviations stand for: x, y, z ?
• Give examples of xyz’s categories.
• Copy the diagram/table on page x and add two more examples of your own.
• Define xyz’s concept of xyz in your own words.
• What are the key terms used by xyz?
• On page x the example “xyz” is used to illustrate xyz. Explain the claim/theory/concept in your own words and add an example.
• Note the break-down into 5 steps/categories here. 
• Contrast xy’s idea/claim/theory with yz’s.
• What is an xyz? Why is this important to understand?
• Draw a diagram to illustrate xyz.
• How to xy’s categories/ideas/key terms relate/compare to yz’s?
• Make a time-line in note form, charting the development of xyz.
• Name and describe in your own words two views on xyz.
• What is special about xyz’s model?
• Outline some of the measures taken to address xyz.
• What are the reasons stated to support the claim that xyz.
• Draw a flow-chart illustrating the structure of this section of the article.
• How is the data presented in this section? What central claim is the data used to support?
• What data analysis method was used in this study, and why?
• For each graph in this section, write down I) what it plots (i.e. what the x-axis and y-axis show) and II) what trends are illustrated by the data presented.
• What do you know about the “xyz” mentioned here? (If not much – find out more!)
• Extension: Choose one source from the bibliography of this article to read as your next source on input on our topic xyz.

I believe that this type of scaffolding helps the students to get to grips with the content of a text at a mainly descriptive level, leaving activities which require higher-order thinking skills such as analysis and evaluation for the lesson time.

Of course, the number of questions or tasks should be suitable for the length of text – remember, students should have the feeling that the worksheet is helping them to digest the text, and not adding extra work!

In EAP, questions or tasks can be added to get students to focus on the langauge or other academic skills as they are demonstarted in the text. For example:

• Write the bibliography entry for this text.
• Why do you think the title of this section is pluralised?
• Find transition words/phrases in this section that show xyz. Note their position within the sentence.
• Find synonyms in this section which mean x, y, z.

Why not try it yourself? You can share your questions/tasks in the comments below, and let me know how it works out with your students!

Introducing #tleap

TLEAP: Teaching & Learning in EAP

Issues in EAP Discussion Group

#tleap is an active online community of EAP professionals who discuss issues and share ideas regarding English for academic purposes. The members are EAP teachers and others who are interested in this area of language teaching, from adjunct tutors to full-time lecturers, and even materials writers and policy makers. The purpose of the #tleap community is to discuss relevant pedagogical, logistical, and research-based issues with others, and to give those involved in EAP a voice that may otherwise go unheard.

#tleap evolved from the #EAPchat Twitter hashtag set up by Tyson Seburn, Adam Simpson, and Sharon Turner, and has now spread across a variety of social media platforms, also thanks to Kate Finegan, to enable and encourage wider participation. You can join in for free here:

Twitter: #tleap

Facebook: https://www.facebook.com/groups/tleap/

Google+: https://plus.google.com/communities/114679086713772400315

#tleap hosts biweekly discussion on Facebook: A focussed discussion point is posted on the 1st and 15th of every month. Please feel free to add your ideas to it and share widely. If there’s something you’d like to discuss, please add to this list: http://bit.ly/1OnYoWM.

#tleap also hosts bimonthly discussion chats on Twitter- look out for the next one!

The chat and discussion archievs are freely available, along with more information on the #tleap community, here http://tiny.cc/tleap

#tleap thrives on the contributions of members! You can start a new post on any of the paltforms anytime you have a question or wish to share something relevant for the group. Comments are always welcome on all posts.  With any blog, research article, or question, you can also always add the #tleap hashtag to your tweets to get everyone in our community to notice and engage.

We would love to welcome new members to the #tleap community, so please join in and share #tleap with your colleagues!

We look forward to hearing from you!

Competency-based planning and assessing

Earlier this week, I attended a workshop on competency-based (or competency-oriented) planning and assessing held by Dr Stefan Brall at Trier University, and would like to share some of the insights here.

The workshop was aimed at university-level teachers from various subject areas, and so concentrated generally on Competency-Based Education (CBE). According to Richards and Rogers (2001), the principles of CBE can be applied to the teaching of foreign languages (-> CBLT: Competency-Based Language Teaching), making the topic of interest to ELT professionals.

What is a competency?

In everyday language, we talk of people being ‘competent’ when they have the knowledge, qualification(s), or capacity to fulfil the expectations of a particular situation. They have the ability to apply the relevant skills appropriately and effectively. In the area of education, then, these skills are the individual competencies that students need to acquire and develop. Another important distinction here is between declarative knowledge, the theoretical understanding of something, and procedural knowledge, the ability to actually do it. In language teaching, I would argue, our focus is necessarily on the procedural side of things, on getting students to be able to actually communicate in the target langauge. The overarching goal of  CBLT is for learners to be able to apply and transfer this procedural knowledge in various settings, appropriately and effectively.

Literature on CBE explains how the approach can enhance learning, by

• Focusing on the key competencies needed for success in the field
• Providing standards for measuring performance and capabilities
• Providing frameworks for identifying learners’ needs
• Providing standards for measuring what learning has occurred

What are key competencies?

In the realm of tertiary education, a useful study to look at here is the Tuning Project. This is an EU-wide study which explored the most important competencies that students should develop at university. Although the specific ranking of the competencies may be debated, some of the capabilities that came out as very important include: the application of theory, problem solving, the adaptation of procedural knowledge to new situations, analytical thinking, synthesising information, and creativity (Gonzalez & Wagenaar, 2003). These kinds of skills are those often found at the top ends of taxonomies of learning. Compare, for example, with Bloom’s taxonomy:

Other taxonomies of learning use comparable sequential units to describe cognitive learning. For example, the SOLO model (Structure of Observed Learning Outcome, see Biggs & Tang, 2007) includes a quantitative phase of uni-structural and multi-strucutal learning (e.g. identyfing, describing, combining), and then a quantitative phase of relational (e.g. comparing, analysing causes, applying) and extended abstract learning (e.g. generalising, hypothesising). Seeing these important skills in a hierarchically organised scheme highlights how they build upon each other, and are themselves the products of mastering many sub-skills or competencies.

In language teaching, people have long since spoken of “the four skills”, i.e. skills covering the oral, aural, reading and writing domains. To this we might also add learning competencies. In CBLT, language is taught as a function of communicating about concrete tasks; learners are taught the langauge forms/skills they will need to use in various situations in which they will need to function. Scales such as the Common European Reference Framework for Languages help to break down these skills into distinct competences, whereby learners move up through the levels of mastery in each skill area, from elementary performance in a competency to proficient performance.

Competency-based Learning Outcomes

If we take scales of learning as the foundation for our planning, then, formulating statements of learning outcomes becomes quite a straightforward process. We will of course need to know the current level and needs of our students, especially in terms of competencies still to be learnt and competencies requiring further development. Associated with such learning taxonomies, we can easily find lists of action verbs which denote the skills associated with each developmental level of thinking skills. Based on the SOLO model, for example, we might find the following verbs:

Level	Verbs
Uni-structural learning (knowledge of one aspect)	count, define, find, identify, imitate, name, recognize, repeat, replicate
Multi-structural learning  (knowledge of several, unconnected aspects)	calculate, classify, describe, illustrate, order, outline, summarise, translate
Relational learning (knowledge of aspects is integrated and connected)	analyse, apply, compare, contrast, discuss, evaluate, examine, explain, integrate, organise, paraphrase, predict
Extended abstract learning (knowledge transferred to new situations)	argue, compose, construct, create, deduce, design, generalize, hypothesise, imagine, invent, produce, prove, reflect, synthesise

Based on our understanding of students’ current learning levels, students’ needs, and the general framework within which our lessons/courses are taking place (in terms of contact time, resources, etc), and with these action verbs, we can then formulate realistic learning goals. In most cases, there will be a primary learning outcome we hope to reach, which may consist of several sub-goals – this should be made clear.

For example, an academic writing course aimed at C1-level students (on the CEFR) might set the main learning outcome as:

By the end of this course, students should be able to produce a coherent analytical essay following the Anglo-American conventions for the genre.

A couple of the sub-goals might include:

• Students should be familiar with Anglo-American essay-writing conventions and able to apply these to their own compositions.
• Students should understand various cohesive devices and employ these appropriately within their writing.
• Students should understand the functions of Topic Sentences and Thesis Statements and be able to formulate these suitably in their own writing. 

Formulating clear learning outcomes in this way, and making them public, helps students to reflect on their own progress and may be motivating for them, and helps teachers to choose activities and materials with a clear focus, as well as helping to devise assessment tasks and grading rubrics.

Competency-based Assessment

Of course, most teachers will need to aim for economical assessment, in terms of time and resources. As far as possible, CBE advocates on-going assessment, so that students continue to work on the competency until they achieve the desired level of mastery. Competency-based assessment may thus require more effort and organisation on the part of the assessor – but it is able to provide a more accurate picture of students’ current stage of learning and performance.

Take multiple-choice tasks, for example; they can be marked very economically, but in reality they tend only to test the lower-level thinking skills, which may not have been the desired learning outcome. To test competency-based learning, we need to base our assessment tasks on the learning outcomes we have set, perhaps using the same action verbs in the task instructions. The focus is shifted to learners’ ability to demonstrate, not simply talk theoretically about, the behaviours noted in the learning outcomes. Still, especially in the realm of langauge teaching, there are some tasks we can easily set in written assignments which will also allow us to assess the higher levels of competencies more economically than oral presentations or practical assignments. If our learning outcome is the ability to apply a theory, for example, we could set a question such as ‘Describe a situation that illustrates the principles of xyz‘. Or, if we want to assess whether learners can discuss and evaluate, we might set a task like ‘Explain whether and why you agree or disagree with the following statement.‘ These kinds of tasks require learners to apply their acquired or developed competencies on a more qualitative level.

To enable objective assessments of students’ learning, we will need to devise a matrix based on the various levels of mastery of the competencies detailed in the learning outcomes. As a basis, we might start with something like this:

Grade	Description
A	An outstanding performance.
B	A performance considerably better than the average standard.
C	A performance that reaches the average standard.
D	Despite short-comings, the performance just about reaches the minimum standard required.
E	Because of considerable short-comings, the performance does not reach the minimum standard required.

For each sub-skill of the competencies we are aiming for students to achieve, we will need to state specifically, for instance, which ‘short-comings’ are ‘considerable’, e.g. if the students cannot demonstrate the desired level of mastery even with the tutor’s assistance. Also, it is important in CBE and CBLT that students’ performance is measured against their peers, especially to ascertain the ‘average standard,’ and not against the mastery of the tutor.

To return to the essay writing, example, a student’s composition might receive a B grade on the sub-competence of using cohesive devices if they employ several techniques to create cohesion in their work, but occasionally use one technique where another might be more effective. A student’s essay might receive a D grade on this competency if they repeatedly use the same cohesive device, or employ the techniques indiscriminately and inappropriately. An E grade might mean that the student has not tried to employ any cohesive devices. In this manner, the primary learning outcome is broken down into sub-skills, on which students’ performance can be objectively measured using a detailed grading matrix.

In a nutshell, then, CBE and CBLT aim for ‘Yes we can!’ rather than ‘We know’. Competency-based teaching and learning have become a staple in further education and language instruction in many places around the world. If you would like to implement the approach in your own classrooms, I hope this post has given you some useful insights on how to do so!

References

Biggs, J. & C. Tang, Teaching for Quality Learning at University (Maidenhead: Open University, 2007).

Brall, S., “Kompetenzorientiert planen und prüfen”, Workshop at Trier University, 21.2.17.

Gonzalez, J. & R. Wagenaar, Tuning Educational Structures in Europe: Final Report Phase One (Bilbao, 2003)

Richards, J.C. & T.S. Rodgers, Approaches and Methods in Language Teaching (Cambridge: CUP, 2001).

“What is the CEFR?”, English Profile, Cambridge University Press, http://www.englishprofile.org/the-cefr, accessed 24.2.17

ELT Research Bites

Followers of my blog will know that I believe we, as language teachers, all need to understand the pedagogical underpinnings of what we do in our language classrooms. That’s why I aim in my blog posts to provide information on theoretical backgrounds and lesson materials which apply them practically. I would also love for more teachers to read the research and background articles for themselves. But I know that teachers are all busy people, who may not have access to or time to access publications on the latest developments and findings from language education research.

ELT Research Bites is here to help!

As the founder, Anthony Schmidt, explains: ELT Research Bites is a collaborative, multi-author website that publishes summaries of published, peer-reviewed research in a short, accessible and informative way. 

The core contributors are Anthony Schmidt, Mura Nuva, Stephen Bruce, and me!

 

Anthony describes the problem that inpsired ELT Research Bites: There’s a lot of great research out there: It ranges from empirically tested teaching activities to experiments that seek to understand the underlying mechanics of learning. The problem is, though, that this research doesn’t stand out like the latest headlines – you have to know where to look and what to look for as well as sift through a number of other articles. In addition, many of these articles are behind extremely expensive pay walls that only universities can afford. If you don’t have access to a university database, you are effectively cut off from a great deal of research. Even if you do find the research you want to read, you have to pour through pages and pages of what can be dense prose just to get to the most useful parts. Reading the abstract and jumping to the conclusion is often not enough. You have to look at the background information, the study design, the data, and the discussion, too. In other words, reading research takes precious resources and time, things teachers and students often lack.

And so ELT Research Bites was born!  

The purpose of ELT Research Bites is to present interesting and relevant language and education research in an easily digestible format.

Anthony again:  By creating a site on which multiple authors are reading and writing about a range of articles, we hope to create for the teaching community a resource in which we share practical, peer-reviewed ideas in a way that fits their needs.

ELT Research Bites provides readers with the content and context of research articles, at a readable at the length, and with some ideas for practical implications. We hope, with these bite-size summaries of applied linguistics and pedagogy research, to allow all (language) teachers access to the insights gained through empirical published work, which teachers can adapt and apply in their own practice, whilst not taking too much of their time away from where it is needed most – the classroom.

CHECK OUT ELT Research Bites here: http://www.eltresearchbites.com/ 

FOLLOW ON TWITTER: @ResearchBites

 

Peer Presentation Feedback

I teach an EAP module which focusses on language and study skills. It’s aimed at first-semester students starting an English Studies degree where English is a foreign language for almost all students. They’re at the B2+ level.

In a 15-week semester, we spend the first five weeks or so looking at what makes a good academic presentation in English. We cover topics such as narrowing down a topic to make a point, logically building up an argument, linking pieces of information, maintaining the audience’s attention, formal langauge and appropriate use of register, body language and eye contact, volume and pacing, using sources effectively, and lots of sub-skills and langauge features that are relevant for presentations. In the second 2/3 of the semester, students give presentations (in groups of 3) on a topic of their choice related to the English-speaking world, and we discuss feedback altogether so that the others can learn from what was good or could be improved in the presentation they have watched.

This blog post describes my journey through trialling different ways of getting the best feedback to fulfil our overall learning aim. 

(Note: Don’t worry, we also use class time to practise other study skills pertaining to listening and speaking!)

1. ‘Who would like to give some feedback?’

I have experimented with various ways of getting audience members to give feedback. When I first started teaching on this module, I used to ask after the presentation ‘Who would like to give some feedback?’, which was usually qualified by saying something like ‘Remember the points we’ve covered on what makes a presentation good.’ Usually, only a few people commented, and they focussed mainly on the good things. Don’t get me wrong, I think it is important to highlight what students have done well! But the overall goal of having students give presentations was that we could constructively critique all aspects of these presentations. I had hoped that we could use these ‘real’ examples to review what we had learnt about good academic presentations. So this approach wasn’t as effective as I had hoped.

2. Feedback questions

It seemed that requiring students to keep in mind all of the features of a good academic presentation was asking a bit too much. And so, together with a colleague, I drew up a list of questions students could ask themselves about the presentation. Example questions include: Was all of the information relevant? Was the speech loud and clear, and easy to understand? Students were given the list before the first presentation and instructed to bring it each week to help them to give presentation feedback. Most people brought them most of the time. Still, students were pretty selective about which questions they wanted to answer, and (tactfully?) avoided the points where it was clear that the presentation group needed to improve. So we still weren’t getting the full range of constructive feedback that I was hoping for.

3. Feedback sandwich

It was clear to me that students wanted to be nice to each other. We were giving feedback in plenum, and no one wanted to be the ‘bad guy’. This is a good thing per se, but it meant that they were slightly hindered in giving constructive criticism and thus achieving the learning aims I had set for the course. So, before the first presentation, I set up an activity looking at how to give feedback politely and without offending the individual presenters. We explored the psychological and linguistic concepts behind ‘face saving’ and how people may become defensive if they feel their ‘face’ is attacked, and then psychologically ‘block out’ any criticism – so the feedback doesn’t help them improve their presentation; nor does it make for good student-student relationships! I explained the idea of a ‘feedback sandwich’ in which the positive comments form the bread, and the negative comments are the filling. This idea is said to ease any feelings of ‘attack’, thus making the feedback more effective. Students embraced this idea, and did their best to ‘sandwich’ their feedback. Overall, this was a helpful step in moving the class feedback towards waht I thought would be most effective for the learning aims.

4. Feedback tickets

Since I noticed we still weren’t always getting feedback on all aspects of the presentation, a colleague and I decided to make ‘feedback tickets’, each with one question from the list we had previously prepared. The tickets were handed out before a presentation, and each student was then responsible for giving feedback on that point. Combined with the ‘sandwich’ approach, this overall worked pretty well. The minor drawbacks were that sometimes the presenters had really done a good job on a certain aspect and there wasn’t much ‘filling’ to go with the ‘bread’; however, sometimes the ‘filling’ was important, but students seemed to counteract their constructive criticisms by emphasizing their lack of importance, especially compared to the positive comments. For me, though, the major downside to using these tickets was the time factor. Running through a set of ~15 feedback tickets (and feedback sandwiches!) after each presentation was productive for students’ presentation skills, but ate into the time in class that should have been used for practising other oral/aural skills. In extreme cases, with two 30-minute presentations plus Q&A in a 90-minute lesson, we simply ran out of time for feedback! Those poor presenters got no feedback on their presentations, and we as class were not able to learn anything from the example they had delivered.

5. Google forms

Actually, I first used Google Forms to collect feedback after one of these lessons where our time was up before we’d got through the plenary feedback round. I copied all of the feedback questions into a Google form (using the ‘quiz’ template) and emailed the link to the students. I was positively surprised by the results! Perhaps aided by the anonymity of the form, students used the ‘sandwich’ idea very effectively – suitably praising good aspects of the presentation, and taking time to explain their criticisms carefully and specifically. Wow – helpful feedback! I printed out the feedback to give to the presenters, along with my own written feedback, and also picked out a couple of poignant comments to discuss in plenum in the next lesson. Right from the off, this way of collecting and giving feedback seemed very effective, both in terms of time taken and achieving learning aims. It seemed presenters had some time to reflect on their own performance and were able to join in the feedback discussions more openly, and focussing on just a couple of key aspects meant it was time-eficient, too. I immediately decided to use the Google form for the next couple of weeks, and have continued to find it extremely useful. Sadly, we’re at the end of our semester now, so these are just very short-term observations. Still, I’m encouraged to use the online form in future semesters.

Just goes to show how important reflecting on our classroom practices can be!

I wonder if anyone else has had similar experiences, or can share other inspirational ways of collecting feedback on presentations? I’d love to hear from you!

MaWSIG Meetup – Questions from teachers/writers

Are you an ELT teacher looking to move into materials writing? This post is for you!

On Saturday 14th January, I hosted a Meetup for the Materials Writing Special Interest Group of IATEFL. The idea was to enable some informal networking for anyone in the area who is involved in writing ELT materials.

One of the activities we did involved editors/publishers and teachers/writers posing questions for each other on posters, and then discussing their answers to the “other side’s” poster. 

To share some of the insights beyond our cosy meetup in Germany, I also posed the questions from teachers/writers questions to some editors and other people who work in ELT publishing, and here are their answers:

• Is there any interest in / a market for writing smaller-scale projects? (e.g. topic worksheets / individual lessons)?

Yes. Generally when we commission these sorts of projects, they’re supplementary materials supporting a book, and there are specific things we need, generally things we feel the target group needs but the book has not provided. So if you regularly use a book and notice a gap, you should definitely let the publisher know, and perhaps send examples of supplementary worksheets you have created.

Yes, definitely, but I suggest a system of crowdsourcing. Writers can produce modules or collections of individual lessons (they need to be substantial lessons) and these can be sold as individual modules (after there are ten or so they can be made into a book).

There is a market, but not really so much for individual worksheets. Find something that links your materials together, a thread that flows though several worksheets or lesson plans. Sets of lesson materials which form a coherent unit are probably of more interest to potential publishers.

• Is it possible to have more access to writers to discuss objectives etc?

We can’t give out contact information, but we’re not secretive about who writes for us; just look in the copyright pages.

The teacher’s books often give more detail on the overall approach and aims of the activities than the student’s books, so maybe have a look there.

• How can we get into proof-reading / copy-editing work?
  

A good way to get a foot in the door is to offer to write readers’ reports on first drafts of material. Publishers are always happy to have readers, and I have personally seen examples of readers then getting writing work because they’ve made an impression.

About proofreading work: the best place to start is with your own materials, then offer to check worksheets or materials that are being written for your school/shared bank of materials. Create a style sheet that will give you consistency across the whole collection of materials. If you find you enjoy this kind of work, you could contact your local publishers’ office and express an interest, or you could consider doing a recognised proofreading course somewhere like The Publishing Training Centre or joining an organisation like the Society for Editors and Proofreaders (SfEP).

Definitely get some kind of recognised training if you want to do copy editing and/or proofreading professionally. It not only gives you credibility, it will teach you a lot. I did my training in Canada, and I went into it thinking ‘I teach grammar… this is just a formality.’ Not so. I learned so much about the process of editing a text that I would never have got by myself.

For anyone interested in ELT editing, follow the White Ink FB page for tips, tricks and work opportunities. Facebook.com/WhiteInkLimited.com

• How can (potential) writers make themselves known to you and/or find out about upcoming projects?
  

Send editors/publishers your CV and a couple of sample materials you’ve made.

Submit your work to materials writing competitions – most publishers and lesson sharing websites host competitions.

Most publishers have an email address or contact form for potential writers. It’s really important to make clear what kind of materials you can write – whoever processes the emails will want to forward it quickly to the relevant editorial department – so put ‘English’ and ‘Business / Primary / EAP / etc.’ in a prominent place in the email.

You can always email our editorial teams to discuss any potential opportunities.

To make yourself ‘findable’, make sure you join ELT Teacher 2 Writer. All of the publishers listed on the homepage use the database to find writers.

Most people suggest starting a blog where you share materials you have made for your classes more widely. Likewise, if you create something innovative then share it by presenting at conferences etc. This can get your name known, and if you do contact a publisher then you have a portfolio to show them.

If you can commit to piloting and reviewing material, you can impress editors that way and may then be offered writing work.

• What can teachers do if we notice a gap in the market?
  

Yes, if you spot a gap in the market that is innovative, get in touch  and most editors will send you a proposal document, or check the publisher’s website for an electronic proposal from. Make sure you approach the right kind of publisher, though.

Get in touch via the website of a relevant publisher – There’s ususally a list of details you should include on there.

Do your research! If the gap you find is very niche, publishers might be less interested, so you’ll need to ‘prove’ that your gap is relevant to a wider audience than just one of your classes.

• Is experience/expertise in digital materials writing essential nowadays?

I would say no, not yet, but a willingness and an interest is helpful.

Not essential – the vast majority of educational material sold is still print. However, it’s becoming increasingly relevant and we’re always on the lookout for people who can write this kind of content.

Depends what you want to write and who for. If you specifically want to write digital materials, then some experience will clearly help, but training will probably be provided if you’re new to the area – especially as different publishers use different digital platforms anyway.

• And if so, is there capacity for advice/training to produce this type of material?

I think it’s a case of learning by doing. Let your publisher know you’re interested. Probably the best “training” you can do is to get yourself familiar with the apps and things that are on the market, and try to imagine what had to be taken into consideration when the content was created.

I would say this is out there if you look hard enough. Nellie Deutsch runs courses on MOODLE for Teachers. And some organisations run Writers Retreats which might be relevant.

I think most producers of educational material are still learning what makes good digital content in our industry. In my opinion, the best thing to do is to learn and work with everything yourself (particularly the apps and websites that are successful, like Duolingo, Babbel, The Day, PlayPosit, etc. – or even brain training apps like Elevate) to get a better idea of what kind of content works well on a smartphone, tablet or PC.

For digital training, you could have a look at what ELTjam offer.

Are you involved in ELT materials writing? Do you have more questions from the teacher’s/writer’s perspective? Or answers to these questions from an editor’s/publisher’s perspective? Add your thoughts in the comments below!