Tag: Assessment

Competency-based planning and assessing

Competency-based planning and assessing

Earlier this week, I attended a workshop on competency-based (or competency-oriented) planning and assessing held by Dr Stefan Brall at Trier University, and would like to share some of the insights here.

The workshop was aimed at university-level teachers from various subject areas, and so concentrated generally on Competency-Based Education (CBE). According to Richards and Rogers (2001), the principles of CBE can be applied to the teaching of foreign languages (-> CBLT: Competency-Based Language Teaching), making the topic of interest to ELT professionals.

What is a competency?

In everyday language, we talk of people being ‘competent’ when they have the knowledge, qualification(s), or capacity to fulfil the expectations of a particular situation. They have the ability to apply the relevant skills appropriately and effectively. In the area of education, then, these skills are the individual competencies that students need to acquire and develop. Another important distinction here is between declarative knowledge, the theoretical understanding of something, and procedural knowledge, the ability to actually do it. In language teaching, I would argue, our focus is necessarily on the procedural side of things, on getting students to be able to actually communicate in the target langauge. The overarching goal of  CBLT is for learners to be able to apply and transfer this procedural knowledge in various settings, appropriately and effectively.

Literature on CBE explains how the approach can enhance learning, by

  • Focusing on the key competencies needed for success in the field
  • Providing standards for measuring performance and capabilities
  • Providing frameworks for identifying learners’ needs
  • Providing standards for measuring what learning has occurred

What are key competencies?

In the realm of tertiary education, a useful study to look at here is the Tuning Project. This is an EU-wide study which explored the most important competencies that students should develop at university. Although the specific ranking of the competencies may be debated, some of the capabilities that came out as very important include: the application of theory, problem solving, the adaptation of procedural knowledge to new situations, analytical thinking, synthesising information, and creativity (Gonzalez & Wagenaar, 2003). These kinds of skills are those often found at the top ends of taxonomies of learning. Compare, for example, with Bloom’s taxonomy:

bloom

Other taxonomies of learning use comparable sequential units to describe cognitive learning. For example, the SOLO model (Structure of Observed Learning Outcome, see Biggs & Tang, 2007) includes a quantitative phase of uni-structural and multi-strucutal learning (e.g. identyfing, describing, combining), and then a quantitative phase of relational (e.g. comparing, analysing causes, applying) and extended abstract learning (e.g. generalising, hypothesising). Seeing these important skills in a hierarchically organised scheme highlights how they build upon each other, and are themselves the products of mastering many sub-skills or competencies.

In language teaching, people have long since spoken of “the four skills”, i.e. skills covering the oral, aural, reading and writing domains. To this we might also add learning competencies. In CBLT, language is taught as a function of communicating about concrete tasks; learners are taught the langauge forms/skills they will need to use in various situations in which they will need to function. Scales such as the Common European Reference Framework for Languages help to break down these skills into distinct competences, whereby learners move up through the levels of mastery in each skill area, from elementary performance in a competency to proficient performance.

cefr

Competency-based Learning Outcomes

If we take scales of learning as the foundation for our planning, then, formulating statements of learning outcomes becomes quite a straightforward process. We will of course need to know the current level and needs of our students, especially in terms of competencies still to be learnt and competencies requiring further development. Associated with such learning taxonomies, we can easily find lists of action verbs which denote the skills associated with each developmental level of thinking skills. Based on the SOLO model, for example, we might find the following verbs:

Level Verbs
Uni-structural learning (knowledge of one aspect) count, define, find, identify, imitate, name, recognize, repeat, replicate
Multi-structural learning  (knowledge of several, unconnected aspects) calculate, classify, describe, illustrate, order, outline, summarise, translate
Relational learning (knowledge of aspects is integrated and connected) analyse, apply, compare, contrast, discuss, evaluate, examine, explain, integrate, organise, paraphrase, predict
Extended abstract learning (knowledge transferred to new situations) argue, compose, construct, create, deduce, design, generalize, hypothesise, imagine, invent, produce, prove, reflect, synthesise

Based on our understanding of students’ current learning levels, students’ needs, and the general framework within which our lessons/courses are taking place (in terms of contact time, resources, etc), and with these action verbs, we can then formulate realistic learning goals. In most cases, there will be a primary learning outcome we hope to reach, which may consist of several sub-goals – this should be made clear.

For example, an academic writing course aimed at C1-level students (on the CEFR) might set the main learning outcome as:

By the end of this course, students should be able to produce a coherent analytical essay following the Anglo-American conventions for the genre.

A couple of the sub-goals might include:

  • Students should be familiar with Anglo-American essay-writing conventions and able to apply these to their own compositions.
  • Students should understand various cohesive devices and employ these appropriately within their writing.
  • Students should understand the functions of Topic Sentences and Thesis Statements and be able to formulate these suitably in their own writing. 

Formulating clear learning outcomes in this way, and making them public, helps students to reflect on their own progress and may be motivating for them, and helps teachers to choose activities and materials with a clear focus, as well as helping to devise assessment tasks and grading rubrics.

Competency-based Assessment

Of course, most teachers will need to aim for economical assessment, in terms of time and resources. As far as possible, CBE advocates on-going assessment, so that students continue to work on the competency until they achieve the desired level of mastery. Competency-based assessment may thus require more effort and organisation on the part of the assessor – but it is able to provide a more accurate picture of students’ current stage of learning and performance.

Take multiple-choice tasks, for example; they can be marked very economically, but in reality they tend only to test the lower-level thinking skills, which may not have been the desired learning outcome. To test competency-based learning, we need to base our assessment tasks on the learning outcomes we have set, perhaps using the same action verbs in the task instructions. The focus is shifted to learners’ ability to demonstrate, not simply talk theoretically about, the behaviours noted in the learning outcomes. Still, especially in the realm of langauge teaching, there are some tasks we can easily set in written assignments which will also allow us to assess the higher levels of competencies more economically than oral presentations or practical assignments. If our learning outcome is the ability to apply a theory, for example, we could set a question such as ‘Describe a situation that illustrates the principles of xyz‘. Or, if we want to assess whether learners can discuss and evaluate, we might set a task like ‘Explain whether and why you agree or disagree with the following statement.‘ These kinds of tasks require learners to apply their acquired or developed competencies on a more qualitative level.

To enable objective assessments of students’ learning, we will need to devise a matrix based on the various levels of mastery of the competencies detailed in the learning outcomes. As a basis, we might start with something like this:

Grade Description
A An outstanding performance.
B A performance considerably better than the average standard.
C A performance that reaches the average standard.
D Despite short-comings, the performance just about reaches the minimum standard required.
E Because of considerable short-comings, the performance does not reach the minimum standard required.

For each sub-skill of the competencies we are aiming for students to achieve, we will need to state specifically, for instance, which ‘short-comings’ are ‘considerable’, e.g. if the students cannot demonstrate the desired level of mastery even with the tutor’s assistance. Also, it is important in CBE and CBLT that students’ performance is measured against their peers, especially to ascertain the ‘average standard,’ and not against the mastery of the tutor.

To return to the essay writing, example, a student’s composition might receive a B grade on the sub-competence of using cohesive devices if they employ several techniques to create cohesion in their work, but occasionally use one technique where another might be more effective. A student’s essay might receive a D grade on this competency if they repeatedly use the same cohesive device, or employ the techniques indiscriminately and inappropriately. An E grade might mean that the student has not tried to employ any cohesive devices. In this manner, the primary learning outcome is broken down into sub-skills, on which students’ performance can be objectively measured using a detailed grading matrix.

In a nutshell, then, CBE and CBLT aim for ‘Yes we can!’ rather than ‘We know’. Competency-based teaching and learning have become a staple in further education and language instruction in many places around the world. If you would like to implement the approach in your own classrooms, I hope this post has given you some useful insights on how to do so!

References

Biggs, J. & C. Tang, Teaching for Quality Learning at University (Maidenhead: Open University, 2007).

Brall, S., “Kompetenzorientiert planen und prüfen”, Workshop at Trier University, 21.2.17.

Gonzalez, J. & R. Wagenaar, Tuning Educational Structures in Europe: Final Report Phase One (Bilbao, 2003)

Richards, J.C. & T.S. Rodgers, Approaches and Methods in Language Teaching (Cambridge: CUP, 2001).

“What is the CEFR?”, English Profile, Cambridge University Press, http://www.englishprofile.org/the-cefr, accessed 24.2.17

Advertisement

IATEFL ReSIG PCE Day

Today I was part of a really energising event: the Pre-Conference Event held by the Research SIG of IATEFL at the annual conference in Harrogate.  The day was especially focused on teachers’s research and set up to be “for teachers by teachers”. We heard some really interesting poster presentations, and had some interesting discussions in plenary. Here are my personal highlights.

 

Good ideas I learnt from poster presentations & the discussions that followed

–          “Process-folios”: Instead of assessing students on a portfolio consisting of the work they have chosen, their best pieces of work, get students to produce a portfolio which demonstrates the process through which they arrived at their final product and how they improved the product and their language/skills throughout the project. For example, if they are working on an essay, the process-folio could include reading notes, essay plan(s)/outline(s), drafts, feedback, peer review, reflections, etc, as well as the final essay. Since it’s hard to give a grade for this kind of documentation, the assessment could be a list of ‘can do’ statements, so the students have to prove through the documents included in their process-folio that they have the skills to complete the tasks listed in the statements, for example ‘I can narrow down a topic appropriately’, ‘I can read sources critically’, ‘I can incorporate feedback into my work’. These process-folios remove the pressure from students of producing one very good essay (for example) and re-focus their attention on the process of improving themselves and their work.  (Thanks to Jayne Pearson for this idea!)

 

–          Group peer review: Instead of having students bring copies of their written work to class and working in pairs or small groups on peer reviewing tasks, put the documents online (e.g. in a Google Group) and invite everyone in the class/group to leave comments on all the other pieces of work. Students can sign up to the group using pseudonyms, and the anonymity can help to make the feedback more honest. It will probably also mean that students receive feedback from peers who are not in their immediate friendship group (who they would probably work with if given the chance in class) and are thus likely to receive a broad range of comments and ideas. They may be exposed to new approaches to improving their writing, and receive more abundant feedback than just from one partner within the lesson time. (Thanks to Elena Oncevska for this idea!)

 

–          Online/Smart phone apps for improving oral fluency: Students can be encouraged to improve their own oral fluency if they are aware of where their weaknesses lie. There are a number of tools available on the internet or as smart phone apps which can make this work more motivating. For example, the IDEA (International Dialects of English Archive) provides recordings of English speakers from around the globe, who are speaking freely and naturally (not like the staged recordings often found in textbooks!). These recordings can be used to encourage students to notice what exactly it is they listen to when someone is speaking, what aspects of their speech make them sound ‘fluent’, etc. And then they can record their own speech (using smart phones, usually no special app is required) to ‘anaylse’ and compare with the IDEA samples. Other apps, such as The Ahcounter or The Hitcounter can be used by students (alone, in groups, or with/by the teacher) to measure things such as words per minute, the number of disfluent interruptions (e.g. ah, err, um) in a concrete nominal way, from which the students can set themselves targets and continue to monitor improvements in their fluency. Being allowed to use their mobiles in class will probably bring additional motivation to the task! (Thanks to Becky Steven and Jessica Cobley for this idea)

 

Interesting points of discussion

We didn’t really find clear answers to any of these, but I think the questions are important ones for teachers to be asking themselves and perhaps discussing with colleagues – and it would be great to read your ideas if you’d like to post them as comments below.

 

–          Is teacher/action research simply part of good teaching practice, or is there something more to it?

–          What is the relationship between action research and professional development?

–          Where is the boundary between teacher research and “proper” academic research?

–          How can we share and disseminate the findings of teacher research projects to reach the people who are actually in the classroom?

 

As you can see, an interesting and productive day … can’t wait to see what the rest of the conference holds in store!

 

By the way, if you’re interested in seeing/hearing more from the ReSIG PCE, here are the videos from all the poster presentations: http://resig.weebly.com/teachers-research-1-april-2014.html