Jilani-巴基斯坦 Problematizing High School Certificate Exam in Pakistan A Washback Perspective

175 The Reading Matrix © 2009 Volume 9, Number 2, September 2009 Problematizing High School Certificate Exam in Pakistan: A Washback Perspective Raana Jilani Indiana University of Pennsylvania ABSTRACT This paper describes and evaluates the Higher-Secondary School Certificate (HSC) exam in Pakistan that has been in place in its present form for more than thirty years. The author recounts her experience as a teacher of English in a representative high school in Pakistan and, reflecting on the impact of high school public exam, she argues that the exam has a negative washback effect. She then presents a detailed survey of washback literature, provides a description of the context and the test, and analyzes the current testing situation in Pakistan. It is concluded that the exam has had a widespread washback both at the individual and societal level needing serious attention in view of its broader educational and social implications. INTRODUCTION It was the second week of the new academic year and my first week as a high school teacher of English. I was hurrying toward the classroom for my second English class in grade twelve. I was very enthusiastic because I was going to face the challenge of teaching a class as large as 120 students. I had worked hard on my lesson plan and was soon going to see whether it would work. But as I entered the classroom, I was shocked to see only fifteen students sitting there. Having finished calling the roll, I asked why the rest of the class was not there. What they told me was a real eye-opener for me: most students attend classes only on the days when they have labs scheduled. I was dealing with a class of pre-engineering science group students who had labs only on two days of the week and so they would show up on those two days for other classes too. They preferred to study at home in the mornings because in the afternoons they had to attend private coaching classes for the preparation of exams. Those fifteen students were there because they were not taking any exam-preparation classes in the afternoons and thought they needed to come to school. This made me realize for the first time how the examination system in Pakistan affected the education in general and teaching and learning of English in particular. My own observation with regard to the impact of tests on teaching and learning has not been limited to this incident. Having worked in two different educational systems (matriculation and O‟Level1) in Karachi, Pakistan, I have noticed a marked difference in the ways English is taught and learned in schools in these different systems. In addition to some brief descriptions of 176 teaching and learning situation in Pakistani schools (Mansoor, 2005; Rahman, 2002), Shamim (1993, 2006) and Shamim and Allen (2000, cited in Shamim, 2008), the only in-depth classroom studies in schools and at higher education level, argue that the ways in which English is taught in the classrooms depends greatly on the kind of tests used in assessing the student achievement in English. The impact of tests or examination system on the teaching and learning has been widely discussed in the field of education (Morris, 1990; Smith, 1991; Shepard, 1993; Vallette, 1994). Due to their wide use, tests, particularly high-stakes tests, may exert an influence on teachers and students with an associated impact on what happens in classrooms (Morris, 1990). The phenomenon that tests influence what and how teachers teach and students learn is described as the washback effect (Alderson & Wall, 1993). Hugh (2003) defined washback as “a part of the impact a test may have on learners and teachers, on educational systems in general, and on society at large” (p. 53). For decades now, scholars have debated the potential impact of tests. Although some scholars have also considered that tests have the potential to bring about a favorable impact on teaching and learning (e.g., Alderson, 1986; Bailey, 1996; Wall, 2000), Alderson and Banerjee‟s (2001) review article noted that most researchers believe that tests induce deleterious effects. However, in a country where the debate on the importance of learning English as a second language is far from over, the issue of test impact or washback in the teaching/learning gets overlooked. Since the inception of Pakistan, English has been considered an important language. It is the language of technology, business and employment which is necessary for communication across cultures. In Pakistani society, English has always been the language of power and a virtual class marker (Rehman, 1996, 2002). The terms “Urdu-medium” and “English-medium” are used commonly to identify an individual‟s social and educational status. Interviews for high-paying jobs are usually conducted in English. Similarly, its international primacy makes proficiency in English arguably the most fundamental advantage for Pakistani youth seeking attractive employment abroad and in Pakistan. English proficiency also helps an individual rise in the armed forces and civil service. Therefore, the learning of English is mandatory from first to twelfth grade. According to Pinter (2006), the purpose of introducing a language program depends on certain factors such as the language setting (which is affected by a number of „contextual variables‟ like the political influence and the views of education ministry of a given country), educational frameworks, status of English and attitudes toward it. The following points rationalize the status of English as a compulsory subject in the curriculum of grade one to undergraduate level in Pakistan: English being the language of international communication is also a means to higher education and better career opportunities. English, which was used as a tool in power politics in the society, should be used to eliminate discrimination among the classes. Since cognitive, ethical and social development are part of language development, the development of the language which is to become the language of communication, higher learning and better career options must start early. 177 In view of its significance, as outlined above, English has ever since been taught and students‟ competence in it been tested at all levels. Being an English teacher, what bothers me the most about the English language teaching situation in Pakistan is that we do not realize how important assessment has been in shaping the current teaching and learning situation there. I consider the situation one of a serious case of negative washback of the examination system and this paper will briefly discuss why I think so. I will first discuss what washback studies in ESL have revealed and then present my analysis of the situation. The purpose of this study is to articulate my concerns about an important dimension of educational system that to this day has been overlooked in Pakistan and, further, to call the attention of the concerned to what is happening in the classrooms as a result of test impact. I conclude, in this paper, that the negative washback of the examination system is one of the reasons why, in spite of following a twelve- year course of study in English as a second language, the graduates of public high schools in Pakistan fail to exhibit basic English proficiency. THE WASHBACK PERSPECTIVE It has been frequently and vehemently asserted in assessment literature that tests exert a powerful influence on language learners who are preparing to take these exams, and on the teachers who try to help them prepare. Recent studies (e.g., Cheng and Watanabe, 2004) suggest that the most important question is how and why washback is brought about and how we can collect reliable evidence to understand its nature (Cain, 2005). However, of first and foremost importance for our understanding of the issues beforehand is to define washback. The impact of test on teaching and learning is termed as „washback‟ and has been defined in numerous ways in the published research and literature on language testing. One of the most common definitions sees the concept referred to as the influence of testing on teaching and learning (e.g., Alderson & Wall, 1993; Cheng & Curtis 2004). Similarly, Shohamy, Donitsa-Schmidt and Ferman (1996) define washback as “the connections between testing and learning” (p. 298) and Saville (2000) and Hughes (1989) as “the effect of testing on teaching and learning” (p. 4 and p. 1 respectively). Messick (1996), noting that washback can have either harmful or positive effects, describes it as “the extent to which the introduction and use of a test influence language teachers and learners to do things they would not otherwise do that promote or inhibit language learning” (p. 241). According to Alderson and Wall (1993) tests can also be powerful determiners, both positively and negatively, of what happens in classrooms. Andrews and Fullilove (1994), cited in Bailey (1999), note that “educationalists often decry the 'negative' washback effects of examinations and regard washback as an impediment to educational reform or 'progressive' innovation in schools” (pp. 59-60). Since Messick‟s (1989, 1996) emphasis on consequential validity, the effects a test could have on a wide range of teaching and learning factors have now been more acutely recognized as a central factor of test validity. According to Messick (1996), the use of a particular test should be justified by information about and evidence for its relevance and social consequences at large. Pierce (1992) states that “the washback effect, sometimes referred to as the systemic validity of a test, refers to the impact of a test on classroom pedagogy, curriculum development, and educational policy” (p. 687). He describes washback in terms of “how assessment instruments affect educational practices and beliefs” (p. 41). More recently, Bachman and Palmer (1996) have discussed washback as not only the impact of tests on teaching and learning 178 but also as impact on society and educational systems (pp. 29-35). They state that test impact operates at two levels: the micro level (i.e., the effect of the test on individual students and teachers) and the macro level (the impact on society and its educational systems). Bachman and Palmer note, however, that washback is a more complex phenomenon than simply the effect of a test on teaching and learning. Instead, they feel the impact of a test should be evaluated with reference to the contextual variables of society's goals and values, the educational system in which the test is used, and the potential outcomes of its use (p. 35). Messick (1996) makes the more specific point that washback is “not simply good or bad teaching or learning practice that might occur with or without the test, but rather good or bad practice that is evidentially linked to the introduction and use of the test” (p. 254). He points out that tests which promote positive washback are likely to include tasks which are criterion sample –that is, “authentic and direct samples of the communicative behaviors of listening, speaking, reading and writing of the language being learnt” (ibid., p. 241), and, he adds, that the transition from learning exercises to test exercises “should be seamless” (ibid.). High School Certificate Exam in Karachi, Pakistan I have chosen to discuss the issues with the assessment of English at high school level for two reasons: High school certificate exam in Pakistan is a high stakes exam and English scores are counted toward the students‟ final scores on the exam. I have had the opportunity to teach English at this level for four years, in Karachi, Pakistan, where I have observed the situation closely. The high school certificate is officially called Higher-Secondary School Certificate, commonly known as HSC. Every student seeking this certificate has to appear for exam twice, at the end of eleventh and twelfth years of schooling. In both years, they have to prepare for exams in six subjects: three optional and three compulsory subjects, and English is one of the compulsory ones in both the years. All the subjects carry equal weight in the final scores and therefore, students‟ performance in compulsory subjects is as important as in the optional ones. In Karachi, which is the capital of province Sindh and the most highly populated city of Pakistan, this exam is conducted and controlled by an institution called Karachi Board of Intermediate Education, working under the Ministry of Education, Government of Sindh. Every year, the Board appoints a committee of English examiners, for each eleventh and twelfth grade, led by a head examiner. The head examiner sets the English paper for that year, supervises the process of the examination and once the exam has been held, assigns the other examiners the answer scripts to mark. This test is given/taken in two parts: English paper I in the eleventh grade and English paper II in the twelfth grade. For its content, the test draws heavily from the prescribed textbooks 2 which consist of selections from English literature. There are three textbooks for each level: a collection of reading texts and a selection of English poetry in both the years, two one- act plays in the eleventh grade and a novel in the twelfth grade. Although the basic format of the two papers is almost the same, grammar carries more weight in paper I than paper II. A general outline of the test and the distribution of maximum marks (i.e., 100) is presented in Table 1 below. 179 Table 1. General Outline of Test and Distribution of Maximum Marks Question 1 Explain one of the excerpts from the reading texts with context. 15 marks Question 2 Explain one of the excerpts from the play or the novel with reference to context. 10 marks Question 3 Explain one of the excerpts from the poetry with reference to context or write a detailed summary of one of the poems. 10 marks Question 4 Answer a descriptive question about the reading text. 10 marks Question 5 Write the sketch of one of the given characters from the play or the novel. 15 marks Question 6 Do some discrete point grammar exercises. 20 marks Question 7 Write a letter on one of the given topics or translate the given paragraph from English to Urdu or Sindhi. 8 marks Question 8 Write an essay of 250 to 300 words on one of the given topics or translate the given paragraph from Urdu or Sindhi to English. 12 marks Teaching and Learning of English at High School Level in Karachi To present my analysis of the teaching and learning of English in my context, I will use a model of washback that is based on a framework suggested by Hughes (1993, cited in Bailey, 1999). According to Hughes, “[i]n order to clarify our thinking about backwash [a British alternative term to the American term washback], it is helpful, I believe, to distinguish between participants, process and product in teaching and learning, recognizing that all three may be affected by the nature of a test” (p. 2). In the Hughes framework, participants include language learners and teachers, administrators, materials developers, and publishers, “all of whose perceptions and attitudes toward their work may be affected by a test” (ibid.). The term process covers “any actions taken by the participants which may contribute to the process of learning” (ibid.). According to Hughes, such processes include materials development, syllabus design, changes in teaching methods or content, learning and/or test-taking strategies, etc. Finally, in Hughes‟ framework, product refers to “what is learned (facts, skills, etc.) and the quality of learning (fluency, etc.)” (ibid.). In the light of this framework, a critical look at the test described above reveals that it has had significant consequences for its participants, process and product. To begin with, it is worthwhile to sort out the students from the other participants since the washback processes that influence them will directly affect language learning, while the influences on other participants will affect efforts to promote language learning. According to Bachman and Palmer (1996), the test-takers themselves can be affected by (1) “the experience of taking and, in some cases, of preparing for the test; (2) the feedback they receive about their performance on the test; and (3) the decisions that may be made about them on the basis of the test” (p. 31). The test-takers in this case are affected in the ways pointed out in (1) and (3) above. From the point of view of (3), this test is high-stakes and has had a negative washback on the learners. According to Alderson and Wall‟s (1993) washback hypothesis, “[t]ests that have important consequences will have washback” (pp. 120-121) which is the case with Pakistani students. Their scores in the English test are counted toward their final scores from the high school certificate exams which, in turn, are decisive for their admission to professional colleges, especially to medicine, engineering, and business schools. The experience of preparing for the test is also important for the test takers. As it is evident from the format, the test requires the test takers to write a lot in the limited three hour 180 time. The required length of the answer depends on the marks the question carries. Also, there has been a precept shared by teachers with their students: „the lengthier the better.‟ This precept has transformed into a belief and has given rise to the culture of rote learning. Since the test requires students to be able to write long answers to oft repeated questions over the years, students do not feel the need to become proficient in English. All they need to do is to record well-written answers to those questions which they can memorize and reproduce successfully to score high on the test. Since their teachers do not provide notes, this explains the students‟ attraction to test preparation centers as they provide ready-made answers to all the expected questions and give them opportunity to have several mock attempts of the test before they actually take it at the end of the year. Some washback studies have revealed this to be one of the effects of tests in high-stake examination contexts. Students tend to adopt surface approaches to learning as opposed to deep approaches (Crooks, 1988; Entwistle & Entwistle, 1991). As a result, students‟ reasoning power is impeded, rote-memorization is encouraged by concentrating on recall of isolated details and students resist attempts to engage in risky cognitive activities which can prove both effective and potentially beneficial for their future improvement (Black & William, 1998; Dietel, Herman, & Knuth, 1991). In the case presented here, this has made acquiring of high scores in English very easy since the students do not have to work hard on gaining proficiency in the language. Another interesting aspect of this test is its predictability. Since the textbooks and the test have not been revised for several years (roughly thirty five years), students can conveniently refer to past papers and predict, to a great extent accurately, the questions that may come in the next year‟s test. Five of Alderson and Wall‟s (1993) restatements of the washback hypothesis directly address learner washback, and I think, this test influences the learners in all the five ways they have discussed (i.e., „learni