[Listening to: ‘My Favorite Year’ by Destroyer]
A community of ‘assessment illiterates’?
Richard Stiggins argued in his seminal 1991 paper, ‘Assessment literacy’, that most of the people involved in making “important instructional and policy decisions” in education “are not sufficiently literate in the basics of assessment to know whether their [data about student achievement] are sound or unsound” (p. 535).
We are a nation of assessment illiterates. We are a society that has come to care very much about high standards of achievement, but we are a society that is incapable of understanding whether those standards are being met. … Assessment illiterates don’t understand what it takes to produce high-quality achievement data and so do not evaluate critically the data they use. Assessment illiterates accept achievement data at face value and can easily be intimidated by apparently technical information and by a complicated presentation of test scores. In short, assessment illiterates lack the tools to be critical consumers of assessment data.
Sadly, a quarter of a century on, we are still a nation of assessment illiterates. Or at the very least we are governed by them.
To be fair, though, it is very easy to be intimidated by technical information, especially when it is presented to us by the ubiquitous employees of powerful testing organisations who may also be our colleagues, managers or mentors. These influential members of our professional community may appear to us as “‘adepts’, steeped in the interpretations and practices of a cult” (Fulcher, 2015, p. 97).
Only they are capable of understanding the true meaning of the words of the scales. The uninitiated are outsiders who can only begin to see the truth if they submit to the required initiation processes. (p. 97-98)
That’s basically how I felt for the first ten years of my teaching and until I successfully completed the required IELTS initiation processes.
What is ‘assessment literacy’?
Fulcher (2012, p. 125) offers a broad definition:
The knowledge, skills and abilities required to design, develop, maintain or evaluate, large-scale standardized and/or classroom based tests, familiarity with test processes, and awareness of principles and concepts that guide and underpin practice, including ethics and codes of practice. The ability to place knowledge, skills, processes, principles and concepts within wider historical, social, political and philosophical frameworks in order in order [to] understand why practices have arisen as they have, and to evaluate the role and impact of testing on society, institutions, and individuals.
That’s a definition I can embrace and I agree with him on the importance of the historical context of assessment. But perhaps Webb’s (2002, in White, 2009, p. 7) would be a more reasonable starting point for many in ELT:
the knowledge about how to assess what students know and can do, interpret the results of these assessments, and apply these results to improve student learning and program effectiveness
How ‘assessment literate’ are we in ELT?
Stiggins (1991, p. 535) writes:
To gauge our assessment literacy … we need only consider the amount of interest, time, and resources we have devoted to creating an assessment-literate society.
How much interest, time and resources have we devoted in ELT to creating an assessment-literate community? Not much, in my experience.
A look at the CELTA syllabus suggests that there is little or nothing in the course on assessment. Graduates of relevant Master’s level courses often complete their degrees without engaging with much scholarship on assessment.
In the last two English Australia Conferences, there have only been six sessions on assessment. In 2015*, there were four. Two of these were by Fiona Barker of Cambridge English Language Assessment and one was by Xiaoming Xi of ETS. In 2016, there were two sessions on assessment; one was about the Pearson Test of English and was sponsored by Pearson; and the other – a ‘cafe session’ – was about IELTS and sponsored by IELTS.
This is a rather poor showing, I think, and reflects the relatively low status that assessment has amongst ELT educators.
pedagogablog 