Tag: EAP Writing

Analysing my Feedback Language

Analysing my Feedback Language

TL:DR SUMMARY

I ran a feedback text I’d written on a student’s work through some online text analysis tools to check the CEFR levels of my language. I was surprised that I was using some vocabulary above my students’ level. After considering whether I can nonetheless expect them to understand my comments, I propose the following tips:

  • Check the language of feedback comments before returning work and modify vocabulary necessary.
  • Check the vocabulary frequently used in feedback comments, and plan to teach these explicitly.
  • Get students to reflect on and respond to feedback to check understanding.

A couple of colleagues I follow on blogs and social media have recently posted about online text analysis tools such as Text Inspector, Lex Tutor and so on (see, for example Julie Moore’s post here and Pete Clements’ post here). That prompted me to explore uses of those tools in more detail for my own work – both using them to judge the input in my teaching materials or assessments, and also using them with students to review their academic essay writing.

Once I got into playing around with different online tools (beyond my go-to Vocab Kitchen), I wanted to try some out on my own texts. The thing I’ve been writing most recently, though, is feedback on my students’ essays and summaries. But, I’m a bit of a feedback nerd so I was quite excited when the idea struck me: I could use these tools to analyse my language in the feedback I write to help my students improve their texts. A little action research, if you will. 

Now I obviously can’t share the students work here for privacy and copyright reasons, but one recent assessment task was to write a 200-250 word compare/contrast paragraph to answer this question:

How similar are the two main characters in the last film you watched?

(Don’t focus on their appearance).

These students are at B2+ level (CEFR) working towards C1 in my essay writing class. They need to demonstrate C1-level language in order to pass the class assessments. One student did not pass this assessment because her text included too many language mistakes that impeded comprehension, because overall the language level did not reach C1, and because she didn’t employ the structural elements we had trained in class.

Here’s the feedback I gave on the piece of work and which I ran through a couple of text checkers. (Note: I usually only write this much if there are a lot of points that need improving!)

The language of this text demonstrates a B2 level of competence. Some of the phrasing is rather too colloquial for written academic language, e.g. starting sentences with ‘but’, and including contracted forms. You need to aim for more sophisticated vocabulary and more lexical diversity. More connectors, signposting and transitions are needed to highlight the genre and the comp/cont relationships between the pieces of information. The language slips lead to meaning not always being emphasised or even made clear (especially towards the end). Aim to write more concisely and precisely, otherwise your text sounds too much like a superficial, subjective summary.

Apart from the personal phrase at the beginning, the TS does an OK job at answering the question of ‘how similar’, and naming the features to be discussed. However, you need to make sure you name the items – i.e. the characters – and the film. In fact, the characters are not named anywhere in the text! The paragraph body does include some points that seem relevant, but the ordering would be more logical if you used signposting and the MEEE technique. For example, you first mention their goals but don’t yet explain what they are, instead first mentioning a difference between them– but not in enough detail to make sense to a reader who maybe doesn’t know the series. Also, you need to discuss the features/points in the order you introduce them in the TS – ‘ambition’ is not discussed here. The information in the last couple o sentences is not really relevant to this question, and does not function as a conclusion to summarise your overall message (i.e. that they are more similar than they think). In future, aim for more detailed explanations of content and use the MEEE technique within one of the structures we covered in class. And remember: do not start new lines within one paragraph – it should be one chunk of text.

I was quite surprised by this ‘scorecard’ summarising the analysis of the lexis in my feedback on Text Inspector – C2 CEFR level, 14% of words on the AWL, and an overall score of 72% “with 100% indicating a high level native speaker academic text.” (Text Inspector). Oops! I didn’t think I was using that high a level of academic lexis. The student can clearly be forgiven if she’s not able to improve further based on this feedback that might be over her head! 

(From Text Inspector)

In their analyses, both Text Inspector and Vocab Kitchen categorise words in the text by CEFR level. In my case, there were some ‘off list’ words, too. These include abbreviations, most of which I expect my students to know, such as e.g., and acronyms we’ve been using in class, such as MEEE (=Message, Explanation, Examples, Evaluation). Some other words are ‘off list’ because of my British English spelling with -ise (emphasise, summarise – B2 and C1 respectively). And some words aren’t included on the word lists used by these tools, presumably due to being highly infrequent and thus categorised as ‘beyond’ C2 level. I did check the CEFR levels that the other ‘off list’ words are listed as in learners’ dictionaries but only found rankings for these words: 

Chunk – C1

Genre – B2

Signposting – C1

(From Vocab Kitchen)

Logically, the question I asked myself at this point is whether I can reasonably expect my students to understand the vocabulary which is above their current language level when I use it in feedback comments. This particularly applies to the words that are typically categorised as C2, which on both platforms were contracted, superficial and transitions, and perhaps also to competence, diversity and subjective which are marked as C1 level. And, of course, to the other ‘off list’ words: colloquial, concisely, connectors, lexical, and phrasing.

Now competence, diversity, lexical and subjective shouldn’t pose too much of a problem for my students, as those words are very similar in German (Kompetenz, Diversität, lexikalisch, subjektiv) which all of my students speak, most of them as an L1. We have also already discussed contracted forms, signposting and transitions on the course, so I have to assume my students understand those. Thus, I’m left with colloquial, concisely, connectors, phrasing and superficial as potentially non-understandable words in my feedback. 

Of course, this feedback is given in written form, so you could argue that students will be able to look up any unknown vocabulary in order to understand my comments and know what to maybe do differently in future.  But I worry that not all students would actually bother to do so –  so they would continue to not fully understand my feedback, making it rather a waste of my time having written it for them.

Overall, I’d say that formulations of helpful feedback comments for my EAP students need to strike a balance. They should mainly use level-appropriate language in terms of vocabulary and phrasing so that the students can comprehend what they need to keep doing or work on improving. Also, they should probably use some academic terms to model them for the students and make matching the feedback to the grading matrices more explicit. Perhaps the potentially non-understandable words in my feedback can be classified as working towards the second of these aims. 

Indeed, writing in a formal register to avoid colloquialisms, and aiming for depth and detail to avoid superficiality are key considerations in academic writing. As are writing in concise phrases and connecting them logically. Thus, I’m fairly sure I have used these potentially non-understandable words in my teaching on this course.But so far we haven’t done any vocabulary training specifically focused on these terms. If I need to use them in my feedback though, then, the students do need to understand them in some way. 

So, what can I do? I think there are a couple of options for me going forward which can help me to provide constructive feedback in a manner which models academic language but is nonetheless accessible to the students at the level they are working at. These are ideas that I can apply to my own practice,  but that other teachers might also like to try out:

  • Check the language of feedback comments before returning work (with feedback) to students; modify vocabulary if necessary.
  • Check the vocabulary items and metalanguage I want/need to use in feedback comments, and in grading matrices (if provided to students), and plan to teach these words if they’re beyond students’ general level.
  • Use the same kinds of vocabulary in feedback comments as in oral explanations of models and in teaching, to increase students’ familiarity with it. 
  • Give examples (or highlight them in the student’s work) of what exactly I mean with certain words.
  • Get students to reflect on the feedback they receive and make an ‘action plan’ or list of points to keep in mind in future – which will show they have understood and been able to digest the feedback.

If you have further suggestions, please do share them in the comments section below!

As a brief closing comment, I just want to  point out here that it is of course not only the vocabulary of any text or feedback comment that determines how understandable it is at which levels. It’s a start, perhaps, but other readability scores need to be taken into account, too. I’ll aim to explore these in a separate blog post.

Advertisement
Academic Writing Skills & “Just in Time” Teaching

Academic Writing Skills & “Just in Time” Teaching

I’ve been looking back over my notes from IATEFL 2017 to find inspiration for another blog post. I’m a bit late now to just summarise talks, but I’d like to come back to one of the questions that was posed at a talk I attended. It was “Building bridges: the disciplines, the normative and the transformative” by Catherine Mitsaki. 

Catherine’s talk looked at the EAP/Genre-based and Academic Literacies models of academic writing instruction and assessed the pedagogical potential of the different approaches, whilst sharing her experience from teaching for UK and international students. As I said, I don’t want to summarise her whole talk here, just one key question she raised. Students from her classes gave feedback suggesting they would prefer to have been taught the specific academic writing skills required for their assignments (within subject classes) right at the time they were working on those assignments. Catherine calls it “just in time” teaching, and she asked us what we thought.

I have to say, I can’t embrace the “just in time” teaching concept fully when it comes to academic writing. There is just too much that students need to know. It might be more appropriate if students entered university programmes with a strong foundation of writing skills, which could then be honed by focussing on the relevant points and skills for each assignment. But this is usually not the case, at least not where I work. I always feel I’m squashing in a huge amount of input and practice into our essay-writing modules, and they run for 14 weeks! With all of the competencies that are involved in producing good academic writing, I find it is much better to give students the chance to digest the input and practice applying the skills to their work over a period of time so that everything can really ‘sink in’. They need time to practise actually transferring the transferable skills we’re teaching them, especially at undergraduate level!

Also, as Catherine pointed out, “just in time” teaching would seem to contradict Academic Literacies models which aim to promote criticality towards established norms as a productive way of growing academically. As she puts it in personal correspondence, “There is no room for questioning well established models if one is struggling to deal with the norms as they are.”

So I’m not convinced that doing what students want or think is best (easiest?) for them is the best approach here. Perhaps a better option is explaining the rationale for our writing courses to the students, in an attempt to increase the receptivity to the classes we teach.

What do you think? Could “just in time” teaching work where you are?

British Council Teaching for Success – My Webinar

British Council Teaching for Success – My Webinar

Here are the slides (inc. references) from my talk yesterday as part of the British Council’s “Teaching for Success” online conference. This talk takes research into feedback practices & translates it into practical ideas for classroom application!

Click here for Slides.

Link to the recorded talk: http://britishcouncil.adobeconnect.com/p424b8xlubb/

Abstract: Providing meticulous correction of errors and hand-written summaries on each student’s text can be time-consuming, and often seems less effective than desired. However, many teachers cannot access relevant publications discussing alternative feedback strategies, and remain unsure about which more time-efficient procedures might be applicable in their context. For this reason, this talk aims to discuss various strategies for assessing and giving feedback on EFL learners’ written work, which I have collected from recent publications, have applied and evaluated in my own teaching, and would like to share with fellow ELT practitioners.

This talk will demonstrate practicable strategies including ways of marking learners’ errors (underlining, correction codes, margin comments), as well as conducting successful peer review, delivering feedback with technology, and making the student-teacher feedback dialogue more constructive and efficient. For each strategy demonstrated, I will summarise recently published relevant research on its employment in various contexts, and briefly present discussions from the literature on the mechanisms underpinning its efficacy, with the main aim of aiding teachers in making informed choices pertaining to their specific learners and contexts. These factors include learner autonomy, motivation, learning styles, receptivity, learner-centredness and individualism.

The talk therefore encourages CPD within the British Council’s professional practices rubric of ‘Assessing Learning’, a topic of interest and relevance to a broad audience, provide practical ideas which can be immediately trialled in a wide range of teaching contexts, and will encourage open discussion on feedback practices among participants.

screen-shot-2016-10-08-at-6-18-21-am

Review: Writing – Learn to write better academic essays

Review: Writing – Learn to write better academic essays

I teach a lot of EAP and particularly essay writing classes, but have as yet not found a textbook to work with that I’m entirely happy with. Ever on the look-out, I came across the Collins EAP series, winner of the ELTon 2014 Innovation in Learner Resources Award, and more specifically:

Van Geyte, E., Writing: Learn to write better academic essays (Collins, 2013)

As we can see from the title, this is actually a self-study guide (with an answer key), but it seemed suitable for my students as it’s aimed at those on pre-sessional EAP courses or in their first year of undergraduate study. The word ‘better’ in the title also highlights that this is a book aimed at building students’ writing skills, for example moving from tests like IELTS or TOEFL to ‘proper’ academic papers.

The book is set up so that students, or classes, can work through it from start to finish, or dip in to the chapters most relevant for their current learning goals. In total, there are 11 chapters, which cover various study skills related to academic writing and language points relevant for written expression. The chapters are consistently structured, which makes for easy orientation, each including brief aims, a self-evaluation quiz, information on the chapter’s writing focus, practical exercises, and key reminders. Throughout the chapters, there are also ‘Tips’, and some more advanced vocabulary is explained in glossary boxes. The author estimates that it would take about 3-4 hours to complete one chapter. There are also lists of useful phrases and annotated sample essays at the end of the book, which are authentic examples of students’ work from a variety of subject areas.

Overall, this book takes a process approach to composition, though it also includes sections on evaluating and improving essays as a product. There is a nice emphasis on the development of learning and growing as an academic, fitting writing in to the students’ progression through their degree. It’s also good that the author highlights the importance of students informing themselves about the requirements and expectations at their specific institution / within their specific department, and not merely relying on this book for reference. I find this particularly important with regard to certain conventions. For example, the ‘Thesis Statements’ shown in the book are more statements of intention and outline, which may not be in-keeping with some disciplines and stand in contrast to most American published writing textbooks.

Within the chapters, the practical exercises, e.g. re-capping key terms, analysing example texts, are neatly spread throughout the sections, so they’re not only at the ends of the complete chapters. Nonetheless, these exercises are sometime rather short and perhaps slightly too specific, so they don’t always seem to be checking understanding of the whole section. Many of the example texts included are students’ answers to IELTS/TEOFL-type exams, and are authentic student-written texts, though I worry that these are not necessarily the best models for the ‘real’ academic papers students will have to write at university.

From the very first chapters, the focus is on writing essays, thus mimicking the process students will likely follow when dealing with coursework writing tasks. However, paragraph structure is touched on only briefly, and elements such as Topic Sentences and ‘one main idea per paragraph’ are somewhat lacking emphasis for my liking. Likewise, the ‘narrowing down the topic’ in an essay’s introduction, or the functions of a good conclusion (vs. summary) are not really emphasized. I think this is where we can see that the book is really aimed at those students who have some initial academic writing experience, perhaps in their main language, and need to expand on this to be successful at university. For an introductory EAP course there may be too little emphasis on these aspects of writing, though this might be less problematic if the book is used to supplement other teaching materials, as it could then function as homework preparation or a summary of the points covered in class.

A definite advantage of this book over composition textbooks aimed at native-speaker undergraduates is the good level of focus on the language of academic writing. I particularly like the ‘modesty’ (=cautious language) section and the ‘Authority’ chapter, though students will need some understanding of the metalanguage used to talk about language in this way. I have to say, I’m not entirely convinced that the general ‘Accuracy’ chapter is necessary in a book on writing, since these are language points that perhaps belong more in general EFL textbooks or other reference works. Instead, the critical thinking section may warrant more attention, and it also is important to note that the ‘Reading Comprehension’ chapter focuses mainly on sentence structures. Still, the ‘Research’ chapter does a very good job at clearly explaining and demonstrating note-taking from sources and making decisions about what information to include in an essay. Similarly commendable is the chapter on ‘Integrity’, which takes a more positive approach to using sources effectively for one’s writing, rather than simply avoiding plagiarism. It frames citing, quoting, and paraphrasing as one of many academic conventions to follow, thus removing students’ fear of plagiarism.

At the end of the book, chapter 11  – ‘Essay process and presentation’ – came as a slight surprise, as the entire book leads the students through the process of writing, though it does include some further information on drafting (which could come earlier?) and using tutors’ feedback – though this last point is also covered in Appendix 2. Appendix 1 presents full sample essays annotated with positive and negative comments, which are undoubtedly useful models for students. In Appendix 3, we find a list of ‘Useful Phrases’. I’m not generally a fan of such lists, as I prefer to encourage students to notice useful language from the source texts they read in their field/subject area, though these phrases may provide comforting scaffolding for students writing their first academic papers.

In general, then, this book provides a clearly-written and practical guide through the process of writing a university essay. I’m not convinced that it would be most effectively employed as the main text of an essay writing class, though it definitely includes elements very beneficial helping students to develop into academic writers, and I would absolutely recommend it as a supplementary resource for EAP learners.

Learner-Driven Feedback in Essay Writing

Learner-Driven Feedback in Essay Writing

A recent focus of work into feedback in ELT looks at ways of increasing students’ openness to teachers’ feedback and how students can be stimulated to engage more thoroughly with the feedback they receive. Learner-Driven Feedback (LDF) seems to be a promising practice here, and below is a summary of some research done in this area.

LDF is usually taken to mean responding to learners’ individual queries to make the feedback process more dialogic in nature, particularly in English for Academic Purposes (EAP) settings. For example, Bloxham and Campbell (2010)’s study of ‘Interactive Coversheets’, which require students to pose questions about their work when submitting essay drafts. Overall, they report good levels of uptake of the feedback provided, demonstrating that these Interactive Coversheets prompted their students to evaluate their writing in more detail, and that students responded positively to receiving this individualised feedback. Tutors in their study also found it quicker to give feedback based on the Interactive Coversheets, as students’ individual questions helped them focus their thoughts. However, Bloxham and Campbell noted that, if students had not put much effort into the draft or Interactive Coversheet, they were less able to make use of the formative feedback, for example if they only submitted an outline, or scribbled paragraph, instead of a properly formulated draft. This leads to the idea that the better organised or more autonomous students may be more likely to receive and engage with formative feedback, and Bloxham and Campbell thus note the limitation that this feedback procedure may merely help better students to perform even better.

Working within a similar framework, Campbell and Schumm-Fauster (2013) devised ‘Learner-Centred Feedback’, which also required their students to pose questions to direct tutors to give feedback on certain aspects of their writing when reading drafts, here in footnotes or as comments in the margins to their essays. They were interested in how students react to being asked/allowed to ‘drive’ the feedback they receive. Their survey showed that students were open to the dialogical feedback and reported finding it motivating and personal, and particularly helpful in working on their individual essay-writing weaknesses.

Studies on feedback on essay writing have also begun to explore the use of various delivery modes for feedback and has shown that this, too, may deepen students’ engagement with the feedback and may increase uptake. Technology-based modes can be used to deliver feedback on essays digitally, for example as in-text changes, as comments added to a document, as a feedback email, or as an audio recording. The focus here is on computer-mediated teacher feedback, i.e. not automated feedback.

In the field of language teaching, Cloete (2014) investigated the new multifaceted options for feedback which are afforded by EAP students submitting their work through online platforms. His study focused on the Turnitin platform, which his team of tutors used to give feedback by inserting comments into text’s margins or in separate columns, highlighting text in different colours, and recording audio feedback. Based on teachers’ evaluations of using Turnitin in this way, he notes that the time-efficiency of delivering feedback in electronic modes depends on tutors’ typing speed and general comfort with using the feedback functions of the software, but that the added value of such electronic modes stems from the scope and amount of multi-m feedback that can be given, and the option to provide feedback in various modes simultaneously. Students in his study also showed heightened engagement with the feedback they received.

My own, very recent, study (Fielder, 2016) focused on an LDF procedure I devised which combines and adapts these previously published ideas and allows learners to determine the feedback they receive. In my LDF, the feedback is given by the teacher, but learners ‘drive’  how and on what they receive feedback: they can choose between various formats (e.g. hand-written, email, audio recording), and are required to pose questions about their work to which the teacher responds (e.g. on grammar, vocabulary/register, referencing, organisation). The study is an initial exploration of students’ receptivity towards Learner-Driven Feedback in EAP. The findings from the detailed survey data highlight a high level of student receptivity towards the procedure, and that students perceive it as a useful tool for improving their general language accuracy and study skills related to essay writing. However, it seems from the survey responses that the specific skills which can be significantly improved by my LDF may depend on which skills have already been trained by students’ previous academic experience.  Nonetheless, this and the studies described above demonstrate compelling reasons for piloting LDF on EAP writing courses; many of which may also justify trialling the approach in other ELT classrooms.

 

References

  • Bloxham, S. & L. Campbell, “Generating dialogue in assessment feedback: exploring the use of interactive cover sheets”, Assessment and Evaluation in Higher Education, Vol 35 (2010), 291–300.
  • Campbell, N. and J. Schumm-Fauster (2013). Learner-centred Feedback on Writing: Feedback as Dialogue. In M. Reitbauer, N. Campbell, S. Mercer, J. Schumm and R. Vaupetitsch (Eds) Feedback Matters (pp. 55–68). Frankfurt: Peter Lang.
  • Cloete, R., “Blending offline and online feedback on EAP writing”, The Journal of Teaching English for Specific and Academic Purposes, Vol. 2, No. 4 (2014), pp. 559-
  • Fielder, C., “Receptivity to Learner-Driven Feedback in EAP”, ELT Journal [Advanced access 2016 – print issue Maas, C. in 2017]

Feedback on Demand: Learner-Directed Feedback on EAP Writing

IMAG0245My presentation this week at IATEFL 2014 in Harrogate was entitled “Learner-Directed Feedback: A useful tool for developing EAP writing and academic skills?”. It was a report on a small action research study I recently conducted on what I’m (for the moment) calling ‘Learner-Directed Feedback’. I’m aware that this may be slightly misleading to some, particularly when it’s confused with peer feedback or peer review – any ideas for a better term are welcome! For now, I’ve adopted a fun term suggested by my colleague as the title of this blog post: Feedback on Demand (but don’t worry, there’s no subscription fee!)

I define Learner-Directed Feedback as follows: Learners ask to receive feedback in a certain format and on specific aspects of their written work. The feedback is given by the teacher, but the learners ‘direct’ how and on what they receive feedback comments. In order to ‘direct’ the feedback, learners can often choose between various modes of delivery (e.g. email, electronic document, audio recording, face-to-face consultation), and are usually required to pose specific questions about their language and text to which the teacher responds. More details on instructions given to students working with this method of feedback can be found here: LDF Instructions

Here are the slides – I’ve edited them to include a little more detail which was part of my speech, in case you weren’t able to be there, or didn’t take very good notes! 🙂

IATEFL 2014 Learner Directed Feedback

I would welcome any comments or questions on what I have ‘said’ – please post them below.

 

Other Useful Links

The conference programme: http://iatefl.britishcouncil.org/2014/sites/iatefl/files/pages/harrogate-2014-conference-programme.pdf

 

See also: http://eltcattheuniversityofsheffield.blogspot.co.uk/2014/05/new-ideas-on-feedback-from-iatefl-2014.html