Outline Cheating in the News Cheating in the News
30/03/2015 Outline Teacher Malpractice in Assessment: the International Context Ardeshir Geranpayeh, Ph.D. Head of Psychometrics & Data Services Oxford, 26 March 2015 • Introduction • Cheating (Malpractice): - What, Where, How, Why, Who - Consequences: Threat to Test Validity • Standards and Regulatory Requirements • How to avoid Cheating: – Prevention vs Detection – Policies on punishment • School Collusion: Teacher Malpractice © UCLES 2015 Cheating in the News Cheating in the News Driving test fraud: More learner drivers caught cheating, BBC News, 26 Jan 2015 • 2,440 Chinese students caught cheating in latest high-tech scam Chinese state television reported on Sunday that invigilators detected abnormal radio signals from an illegal frequency during national licensing tests for pharmacists in Shaanxi province. The independent Wednesday, 28 October 2014 1 30/03/2015 Cheating in the News Cheating in the News According to The Harvard Crimson, a survey of this year's incoming freshmen revealed that 42 percent had cheated on high school homework before arriving at the Ivy League institution. The Huffington Post, 18 February 2015 More than 50,000 English language tests taken by overseas students to extend their British visas have been declared invalid or questionable as a result of an official investigation into cheating on a huge scale. Cheating in the News Cheating in the News Cheating in Bihar: India arrests Hundreds over Bihar Blatant Cheating In Schools 'Cheating teachers have lost morality': Dirty tricks on the rise to boost exam grades BBC News 21 March 2015 The Guardian, 24 June 2014 MailOnline 13 June 2014 2 30/03/2015 abcNews abcNews Poll 1 in 3 they themselves have cheated. Rising to 43% of older teens. Most say cheaters don't get caught. What is Cheating? “Any action that violates the rules for administering a test” Cizek, 1999:3 Where does it happen? It can take a variety of forms 3 30/03/2015 Pen Scanners There are YouTube videos on How to Scan a Document on the Go iPod Song names are renamed with notes or test answers for viewing on the screen. Text files can be stored. Text Messaging Examinee can ask questions and get answers from friend during test via text messaging. Toilet Breaks! Calculator Notes are entered into calculators that have memory for storing notes. Audio notes can be stored. Video notes can be stored. 4 30/03/2015 Camera Phone Examinee takes pictures of a test with a camera phone and sends picture to another person who can text message correct answers back. Why does it happen? – Material rewards such as • Access to life chances • Competitiveness • Lack of self confidence • Publication of league tables (Schools) – Unintentional Cheating • Students, material sharing • Teachers and Schools New Technology Google Glass Share your adventures instantly to Google+, Facebook, YouTube … Apple Watch Who is involved? • • • • • Maladministration Imposters Leaked paper / answers Prohibited material Copying 5 30/03/2015 What is at stake? – Threat to test validity – Score obtained by fraudulent means is not valid – Has negative impact on the validity of scores obtained by other candidates – Denying opportunities to others – Where cheating is seen to be widespread, even honestly obtained test results may lose credibility and certificates become devalued Standards for Prevention of Cheating Explicit statements in the Standards for Educational and Psychological Testing (1999) What is at stake? • Test Security is now seen as one of the most important aspects of test Validity • A Test may not be Valid if it is not secure Standards for Prevention of Cheating 1. Protect the security of tests (standard 11.7) 2. Ensure that individuals who administer the tests are proficient in administration procedures and understand the importance of adhering to directions provided by the test developer (standard 13.10) 3. Inform examinees that it is inappropriate for them to have someone else take the test, for them to disclose secure test materials, or engage in any other form of cheating (standard 8.7) 6 30/03/2015 Standards for Prevention of Cheating Cheating detection: Regulatory requirements 4. Ensure that test preparation activities and materials provided to students will not adversely affect the validity of test score inferences (standard 13.11) and 5. Maintain the integrity of test results by eliminating practices designed to raise test scores without improving students, real knowledge, skills, or abilities in the area tested (standard 15.9) Ofqual’s conditions around malpractice • Ofqual is a risk-based regulator of qualifications (other than degrees) that sets the standards that awarding organisations need to meet when they design, deliver and award regulated qualifications, via a number of conditions of recognition. • Ofqual doesn’t tell AO’s how to manage malpractice. It tells AO’s what must result from their behaviour. Condition A8: Malpractice and maladministration Condition A8 of the General Conditions of Recognition covers: Preventing malpractice and maladministration Investigating and managing the effect of malpractice and maladministration Procedures relating to malpractice and maladministration Dealing with malpractice and maladministration 7 30/03/2015 Examples of ‘Positive indicators’ The awarding organisation: has ways of working that reduce risks and incidents occurring makes sure that individuals involved in the delivery of its qualifications understand and routinely follow these ways of working takes all reasonable steps to provide information which is not inaccurate or misleading has policies, practices and/or procedures that reduce the risk of malpractice and maladministration When Ofqual monitored Cambridge English Language Assessment they were happy with how we are managing malpractice. How to avoid Cheating? Examples of ‘Negative indicators’ Using an individual or Centre to assist with an investigation when there is a suspicion or allegation that they were connected to, or responsible for, the case being investigated Not demonstrating steps taken to prevent repeat incidents of previous cases not following its own procedures during investigation not properly evaluating or validating evidence collected during investigation not keeping records/documents relating to the incident for an appropriate period not being able to enforce action taken against those involved, and cannot demonstrate a good reason why it has been unable to do so How to avoid Cheating? • The Best practice is “Prevention” • No Statistical magic can replace good invigilation • Tension between Faster/on demand results and proper data forensic processes • Use variety of ID checks – Candidate ID Verification – Biomedical checks – Hand writing 8 30/03/2015 Policies on punishment Once a cheating is detected, an action has to be put in place to • Stop fraudulent use of test results • Deter future cheaters • Punishment is dependent on the level of cheating, which in turn can depend on 5 levels of cheating detection • Individual candidates, Group of candidates, Teacher Malpractice, Test Centre collusion and Widespread cheating Level of punishment • What about School collusion: Teacher Malpractice • Students implicated may not have been involved in the cheating • Whilst the candidate’s results may be cancelled if school was to be blamed, no further action will normally be taken against candidates • • • • • Withdrawing results/certificate (individual) Re-taking the exam (suspect results) Life Ban (if stakes is high or imposters) Informing stake holders (regulator) Legal action (insider) Teacher Malpractice: the Atlanta case • In 2009, Beverly Hall, Ex-Atlanta school superintendent, was named national superintendent of the year largely based on her district’s improved standardised test scores • She was once credited with turning around some of the city’s lowestperforming schools 9 30/03/2015 Teacher Malpractice: the Atlanta case • Biggest US cheating scandal in US History • Cheating detected on a 2009 standardized state test involving 178 teachers and principles, 56 schools investigated cheated, 43 people were indicted • Prosecutors set out to reveal a culture of fear and pressure existed under Hall • Georgia Governor determination to trace its source • Cheating traced back to 2001 Teacher Malpractice: the Atlanta case • The scandal testifies that cheating is no longer seen as an old-fashioned battle between teachers and students • When the stakes are high, teachers would also be willing to cheat Final Remarks • Use new psychometric techniques to detect school collusion • Erasure analysis detection • Multifaceted approach to monitor teachers’ assessment • Malpractice awareness among teachers • Collaborative dialogue with teachers as the guardians of students’ achievement 10 30/03/2015 Thank you for listening. Geranpayeh.a@CambridgeEnglish.org 11
© Copyright 2024