I’ve been watching Civilisations, a lush BBC series about how art has shaped the human experience. In an episode on Japan, the viewer is introduced to Maruyama Okyo’s masterpiece from the late 18th century – Cracked Ice – a painted two-fold screen ostensibly intended for tea ceremonies. Its format and minimalism seem characteristically Japanese and yet elements such as the use of perspective and a vanishing point show the influence of the West. I reflected on this in the context of my teaching. I get very excited by ideas I encounter. I want to try everything but this sometimes results in overstuffed lessons and more critically, a strange pastiche that doesn’t really give learners a cohesive learning experience. As I acquire and adapt ideas and tools, I need to learn to pare back like Okyo and focus on what’s really important and let innovation emerge from what I haven’t articulated in my lesson plan, those blank spaces I rush to fill.
Image attribution: Maruyama Okyo, Cracked ice, a 2-fold screen painting | British Museum | CC BY-NC-SA 4.0
I’ve been trying to make a concerted effort to connect with my mother’s L1 – Tamil, a language that’s native to southern India and Sri Lanka. I’ve taught myself to read but I can’t manage its extreme diglossia so forget literature, even newspapers are out of reach. I am building my knowledge of the language in other ways such as David Shulman’s sumptuous and erudite ‘Tamil a Biography‘. In it, he discusses a concept called ‘vāymŏḻi’ (the ḻ happens to be a retroflex approximant) or true speech which I’ve been mulling over this past week.
“One might think that truth is a universal concept not in need of further, local characterization. There is truth and there is untruth, and the difference between them is, we could imagine, clear in every culture. But in fact the notion of truth or truthfulness is always culturally determined. The Greeks called truth aletheia, a “nonforgetting” or “noninattention,” and linked it with unveiling, penetrating past the shimmering surface. Tamil conceptions of truth are quite different. They are, above all, dependent on ideas about the autonomy and integrity of the spoken, audible (musical) word that, once uttered, will always live out its life in the world independent of the speaker’s will. Thus truth is connected to sound—specifically, to the phonemes of the Tamil language—and what sound can do in, or to, a world that is itself made up of sonic forces, inaudible quivers, subtle buzzes.”
That’s of course fascinating, beautiful and lyrical but what got me thinking was this idea of speech becoming truth because it is spoken in a certain context. When you’re training teachers, particularly trainee teachers, I get the sense that your word as the trainer is accepted as the truth regardless of whether it is or not outside that room or platform. And when these ideas go out into the real world, they continue to evolve because they’re being implemented in some form or the other. I certainly experienced this on my initial training and acquired ideas that continue to influence my practice even today. These notions once uttered, live out their life in the world independent of the trainer’s will. That’s aproposition I find deeply unsettling.
* A big thank you to Matt Noble for prompting me to start paragraph blogging with his frigging paragraph blogging fecundity this month!
Image attribution: Jaffna, Sri Lanka by arileu | Flickr | CC BY-NC 2.0
Earlier this year, I attended a course organised by the Hornby Regional School on designing communicative language assessments in Bangladesh. The course was taught by Dr. Rita Green, from Lancaster University, who is a research leader in the field of language assessments. My biggest take-away from the course was an alternative approach to designing listening tests called text mapping. Text mapping is a technique that Dr. Green conceived as a way of addressing some of the issues test designers experience when they select items from a listening text for a test. In January, when I was at the course, the technique was literally hot off the press and her new book Designing Listening Tests had just been published.
Text mapping questions prevailing practices for selecting items in a sound file for a test. Here’s what I normally do and perhaps you do something similar. I usually skim the transcript to get a sense of the text and maybe write a gist listening question and then read it again to come up with some listening for specific information questions. I might then listen to the clip to ensure that the accent or speed isn’t too challenging for the target learners.
Dr Green challenged this practice and these two quotes she cited drove the point home:
A transcript and the speech it represents are not the same thing, the original is a richer, contextualized, communicative event.
Life doesn’t come with a tapescript.
Text mapping attempts to address this gap in how we deal with listening texts. But, before we get on to the actual process, it’s important to distinguish between Specific Information & Important Details (SIID) and Main Ideas and Supporting Details (MISD). I think in teacher training, when we refer to these two listening strategies using the oft-used terms, listening for specific information and listening for detailed understanding, we inadvertently obfuscate what they really are. Dr Green differentiated the two in a way that was very easy to understand.
SIID requires selective listening. We listen for information such as dates, times, places, names, prices, percentages, numbers, measurements, acronyms, addresses, URLs, adjectives and nouns.
MISD requires careful listening. We listen for ideas, examples, reasons, clauses (nouns + verbs), descriptions, explanations, causes, evidence, opinions, conclusions, recommendations and points.
Text mapping can be used for gist, SIID and MISD but I’m going to describe the process for SIID which is what I experienced at the course and subsequently tried out on some unsuspecting colleagues.
Choose a level appropriate audio clip and organise a quiet room with good quality speakers. The text mappers you assemble should not have heard the clip before. The clip should be short (approximately 30 seconds)
You will need to prepare an Excel sheet with SIID from the clip along with the time stamps of individual items which means you will need to text map the sound file yourself.
You need at least three text mappers to ensure validity. A larger pool will increase validity. Explain to the text mappers that they are going to be listening for Specific Information and Important Details. You may need to ICQ this to ensure that all the text mappers are on the same page about what constitutes SIID. SIID is usually not more than one or two words.
3. Listening to the sound file
Play the clip only once and ask the text mapper to listen for SIID. They must not make any notes during this time. When the clip finishes, ask the text mappers to write down SIID. The clip is played only once because Dr. Green suggested that over exposure could lead to too many items being identified.
4. Text mapping
Ask the text mappers to tell you the SIID they wrote down. Enter these into an Excel worksheet. Poll the group to see who else got this SIID and maintain a tally. If you have variations in the response because they only heard a part of it or misheard it, record these as separate entries. After you’ve finished eliciting these responses, copy paste the time stamps that you’d prepared earlier. You’re likely to get items that are not SIID. A simple test is to check if the information being offered has a noun and verb in which case it is MISD and not SIID.
There may be variations with numerals because in real life we tend to write down numbers immediately or ask for them to be repeated. The test designer will need to keep this mind when selecting an item which has achieved consensus through a number of variations such as Room No. 4045, 4045, 4054, 4055 etc.
The text mappers might not give you items chronologically which is alright. You’ll just need to reorder them so that they appear sequentially in the worksheet. You’ll also need to be strict about disallowing any responses that were not written down. I experienced this with my colleagues when several said “Oh I remembered that but I didn’t write it down.”
Look at the SIID that a majority of the text mappers were able to identify. These are the items you ought to be testing. However, there are some things to bear in mind. Items at the very beginning of the clip should be disregarded even if you reach consensus (consensus means at least two thirds of the text mappers have identified it) with it because a test candidate may miss it merely because she is orienting herself to the clip in the first few seconds. Additionally, if two items appear within four to six seconds of each others, we ought to test one but not the other. Items should be evenly distributed through the sound file. It’s also important that all items test the same kind of listening behaviour – in this case selective listening for SIID.
6. Writing the test
The next step is to design questions using the items that were identified.
Reflections on text mapping
Here’s one that a colleague and I worked on with a sound file on making a hotel reservation. By text mapping a sound file, you have a systematic approach for identifying what you ought to test as we did with this file. The fact that you are listening to the file as opposed to reading a transcript facilitates the selection of more authentic items i.e., that reflect how we receive and process information in real situations. Selecting items from a transcript (and this often happens with me) may result in the testing of obscure items which we may not even register in a real life context.
When we ran this exercise with a group of our colleagues, we faced some resistance to the concept. The main bone of contention was that we were testing memory instead of listening skills. I think the clip we selected (at 2 min 10 seconds) was far too long. I recall Dr. Green using really short clips with us (around 20-30 seconds). In a Google Preview of her book, I also recall seeing something about chunking the clip for MISD and allowing text mappers to make notes while listening for SIID with longer clips. Unfortunately, those chapters are no longer available online.
However, our colleagues came around when they saw the extent to which there was consensus for the items outlined in yellow in the preceding table and interestingly this coincided with an earlier round of text mapping with another group of text mappers.
I’m still a little uncertain about the relationship between the text mappers who are selected and the items that are identified through consensus. Text mapping as a process is designed not just for test designers but also to empower teachers to work collaboratively to design meaningful tests. Therefore, wouldn’t the items selected depend on the language proficiency level of the text mappers? I suspect that in a monolingual English-speaking environment, the results of text mapping may be different than one where English is not the L1 like I experienced in Bangladesh. Further, what kind of impact does this have on item selection from the learner’s perspective, taking into consideration their own language proficiency. While theoretically, a sound file at B1 should have all of its items at B1 but in reality, this may not be the case.
These unanswered questions not withstanding, text mapping is a useful alternative to the somewhat random way in which listening tests are currently constructed. If you try out text mapping, do let me know about your experiences in the comments section.
Green, S. Designing Listening Tests: A Practical Approach. Palgrave Macmillan: 2017.
Helgesen in Wilson, J. J. How to teach listening. Pearson: 2008.
Lynch, D. Teaching Second Language Listening. OUP: 2010.
Many thanks to Azania Thomas for creating the text mapping sheet that I’ve used in this blog.
Dr. Green’s book is unfortunately really expensive (as interesting ELT books tend to be). You can read a preview here. It includes some relevant chapters on text mapping for gist and issues with listening texts and working with authentic sound files.
The full title of this talk is ‘Empowering teachers through continued professional development: frameworks, practices and promises’. The speaker, Gabriel Diaz Maggioli is from Uruguay and delivered this year’s opening plenary.
Maggioli kicked off his talk with a quote from Dennis Sparks about professional development lacking focus/effectiveness which I suspect comes from Designing Powerful Professional Development for Teachers and Principals. He stated that he wanted to probe what had been going on in CPD in the last 20 years. He felt professional development was oscillating between individual development and the institutional development which is more about advancing the community. He perceived the former as piecemeal and the latter as more systemic. He went on to suggest that top down reforms usually don’t involve the teachers who need to be ‘fixed’ (he used this odd word often, perhaps on purpose) and as a result are not successful and that a better way to think about development and change is a learning community which is a group of individuals who come together because they have some mutual interests.
Maggioli described two diametrically different roles for teachers within professional development programmes: the teacher as a technician who just implements policy and the teacher as a transformative intellectual who propels the development of learning communities. He argued that professional development that was effective drew on targeted professional expertise (based on teacher needs), adopted structured peer support, and provided opportunities for reflecting on why something worked or didn’t work. For these reasons, professional development, he suggested, ought to be done in-house.
Professional development is inquiry oriented learning sustained over time. It requires the the use of tools and protocols that help create coherence, sustain learning and make evidence collection manageable. My favourite quotable quote was when Maggioli suggested that professional development should be done WITH teachers not TO teachers. In his own research, he discovered that teachers listed surfing the web to find ideas and free webinars as their top two professional development channels. From the institutional perspective, there was only mandated INSETT in the list of responses but these weren’t focused on teacher needs and were often disconnected from the reality of the classroom. Respondents described INSETT as having the following faults: no follow up, too much talking very little doing, too short, a low level, and a lack of access to resources to apply this learning.
Maggioli portrayed traditional professional development as untimely and not tailored to the career stages of teachers because teachers are seen as having to perform a function which explains the manufacturing-style one size fits all approach. He spoke about he doesn’t perceive career stages as a continuum but a loop and that every time we move into a new role or context, we kick-start the loop again. He suggested that teachers need time, resources and support and that any kind of CPD in the absence of these three things was futile.
He then shared some approaches to CPD.
Mirror coaching – Teacher-initiated; a peer takes notes during your lesson. You are accessing your own behaviour from someone else’s eyes. Maggioli stated that this wasn’t the same as watching a recording.
Co-teaching: You and your co-teacher model behaviours and learn from each other. expert coach – not from deficit perspective (easier said than done).
Expert coaching: The teacher is coached by someone who is acknowledged as an expert in the area that they’re getting feedback on.
Study group: A teacher shares something she did in class with her peers. They then ask her questions which records. The group then goes on a coffee break during which time the teacher prepares to answer questions which is what happens when everyone comes back from the break.
Collaborative action research: groups of teachers who plan and implement interventions.
Exploratory action research
Lesson study: This technique apparently comes from Japan.
Learning circles: Ad hoc professional development meetings that follow a structured process.
These activities sit within a simple but interesting framework for raising teacher awareness and also identifying those within the learning community who can share, coach and/or mentor:
While I don’t think there was anything new or revolutionary in what Maggioli was suggesting, it is I suppose food for thought given how often we talk about CPD, extol CPD frameworks and construct CPD plans. To what extent are these frameworks and activities effective and do they adopt a deficit approach to development?
Dennis Sparks’ book is available as a free download for educators from this link.
I know I’ve been a wee bit quiet for a while. Mark Armstrong was perhaps a little hasty in suggesting I was hyper-prolific. I’ve had so very much on my plate since the beginning of the year. I came back from a holiday in Cambodia to an insane amount of work. Because I work across so many contexts, projects, and organisations, I was spread very thin; to the point of losing my sanity. I suddenly realised how stupid I’d been. I wasted years at university studying economics instead of linguistics because I’d thought I would need a career that would get me job. And here I was doing a rinse and repeat. I was delivering corporate training that I didn’t enjoy. I suddenly found myself teaching things that made no sense to me (some faff called team dynamics and some other crud about leadership). I was being bullied by the organisations I worked with. And yes it all paid really well but money as you well know isn’t everything.
The breadth of activities was useful in giving me insight into the bigger picture and making my approach more multi-disciplinary. However, ELT is where I want to be. It may seem narrow but this is the field I find most fulfilling.
Go shallow and then deep
Regardless of what folks might say about ELT practices not being backed by adequate research, this is, in my opinion, perhaps the only field in the wide spectrum of learning-related disciplines that actually attempts to base its practices on research and tries to (truly) consider the learner experience. If you’re cynical about ELT, you ought to see the crud that passes for learning in corporate contexts. I can barely hide my sneer when I see the words ‘centre of excellence’ on a business card. An overwhelming majority of what’s on offer is thoroughly underwhelming: the latest digital learning (crappy presentations converted into elearning), on-the-go mobile learning (same crappy presentations in a mobile format), and engaging instructor-led training (crappy presentations being talked through by a Charlie who ought to have his lips sown).
Although I have worked across different ELT contexts (General English, Business English, ESP, EAP and teacher training), I feel like I’ve barely skimmed the surface. I haven’t done any formal study since my DELTA three years ago. Teacher training and materials writing are areas I want to explore more deeply. Young learners and refugees are contexts I want to work in. I’d also like to get more involved with professional associations and other ELT collectives and churn out some papers and maybe even an ebook or two. In other words, I gotta dig deeper.
I’ve recently been approved to train CELTA courses as an Assistant Tutor, a move that will help me consolidate my portfolio of activities and facilitate the depth that I’m looking for.
Go fast and then slow
This past year has been frantic. I’ve been spread so thin that I’ve hardly had any time for myself, let alone for people I enjoy spending time with. I didn’t go the theatre, barely got any exercise, didn’t go trekking during the monsoon, didn’t participate in eclectic workshops, didn’t read as much, didn’t learn any new languages, didn’t meet new people … and the list goes on. I even spent my birthday writing training materials, albeit at a beautiful beach side hotel in Goa.
When I quit a full time job, I thought I’d have more time, not less, to do the things I love. It turns out that freelancing, especially when you have niche skills in a market full of people with generic offerings, is just a volley of clients and institutions putting pressure on you to over-commit. As a result, you end up in the perpetual fast lane and half the time you don’t know if you’re coming or going.
I just have a couple more contractual obligations to complete and then I’m taking the first exit. I’m really looking forward to 2017-18 which is going to be filled with meaningful projects, and at least half as much time with family, friends and doing things other than work.
Whereabouts are you now? Are you going wide, shallow and fast? What will it take for you to go narrow, deep and slow?
Last week I co-presented a talk with a colleague at the Learning for a Sustainable Future – Teacher Conference in Delhi. It was my very first plenary presentation and there were hundreds of attendees. We were presenting our initial findings from piloting a new assessment approach (with behaviour libraries within a smartphone app) in a project we are working on in the south of India.
Here are some quick reflections with presenting to a large audience in this format for the first time.
WWW (What went well)
We went after someone who did a fairly high-level talk on the assessment approach we were piloting. This really helped our presentation because many of the attendees came to me later and said that it wasn’t till they heard our experiences from the field that it all made sense to them.
We incorporated a task for the teachers to try their hand at crafting their own criteria, which helped make the session a bit more engaging.
We’d done a shorter version of the same presentation for a group of policymakers the previous day which helped us anticipate questions.
We constructed our presentation in a way that was of value to both teachers who are familiar with approaches to assessing non-academic skills as well as those who were completely new to the topic.
EBI (Even better if)
We made it less impersonal. I reckon we kept it a bit business-like. We could have throw in some humour and perhaps taken advantage of the fact there were two of us on stage and engaged in a more natural dialogue rather than “you take this slide, I’ll take this one.”
We expanded on our experiences and shared more anecdotes because the audience seemed to respond to stories far more than factual information.
We made it even more relevant to the audience who were mostly K12 teachers. This was a bit challenging because we were merely reporting from our project which looks at assessment from the perspective of project outcomes rather than tracking progress of individual learners (which is of course what teachers are interested in). It may have been worth exploring how the teachers could have used the same assessment approach with their own learners.
During a subsequent talk, I noticed that the erudite looking woman sitting in front of me had dropped her notepad. When I was passing it back to her, she suddenly realized that I looked familiar and she said “great presentation” and then added “I shouldn’t say this, I’m a teacher after all, but you have a sexy voice.” Nonetheless, I’m dreading the prospect of watching myself (uggggghhhh) whenever the video gets uploaded to YouTube.
I read this article on how stereotyped ethnic names can sadly be a barrier to workplace entry and was reminded of a course I designed earlier this year. It was for a client who was going to purchase the materials from me. When they reviewed the workbook, they asked me to change all the names to ones that were familiar to people in the Philippines because they were planning on running the program in Manila. So I changed the names to the names of people I worked with on a short stint in the Philippines.
When I resubmitted the materials to my client, they got back to me with a concern that the names would sound too foreign to learners in India because they planned to run the module in both countries. I suggested having two versions. They made noises about standardisation and asked me to incorporate ‘globally acceptable’ names. I tried to put up a fight but I had to finally give in. The final straw was when they told me that they were also planning to launch the program in the US and that the names would need to be globally acceptable to Indians, Americans, Filipinos and anyone else who’d happen to be around.
I changed the names in the text to ones that I kinda thought would be culture and country agnostic (although that’s a fairly erroneous line of thinking in a multicultural, globalised world)
I couldn’t come up with any others. I ended up using Jay in four different texts. I was wondering if anyone else has faced a similar situation. Also what names would you add to this ‘globally acceptable’ list?
Image attribution: O inmost wind of living ecstasy… by haRee | Flickr | CC BY-NC 2.0