How To Interpret CAP Proficiency Testing Evaluations Elaine Staley, MT(ASCP) Compliance Officer St. Luke’s Hospital – Mayo Clinic Jacksonville Objectives Identify performance issues by evaluating data provided on the PT evaluation and Participant Summary List investigational techniques for responding to Proficiency Testing Exception Summary (PTES) Assemble a quarterly review of PT scores by using the All Analyte Scorecard e-Lab Solutions The best way to view CAP evaluations is to use e-Lab Solutions  Request access from your institution’s administrator CAP Evaluations 101 Printed Evaluation  Exception Codes Interactive Evaluation Participant Summary CAP Evaluation What Should You Review? Evaluate each analyte and specimen for negative/positive bias, trends or shifts Evaluate ungraded challenges  If an evaluation criteria is not listed in the Participant Summary you can use information provided by manufacturer or use Rhoads http://www.dgrhoads.com/db2004/ae2004.php Education Challenges Code 26 POC.03275 Phase II N/A YES NO Is there evidence of evaluation and, if indicated, corrective action in response to "unacceptable" results on the proficiency testing reports and results of the alternative performance assessment system? NOTE: The evaluation must document the specific reason(s) for the "unacceptable" result(s) and actions taken to reduce the likelihood of recurrence. This must be done within one month after the POCT program receives its evaluation. In addition, each ungraded challenge, each educational challenge, and each episode of nonparticipation must be reviewed and corrective action instituted as appropriate. Review Results to Prevent Failure First Instrument First Instrument 1.5 1 0.5 Difference Upper Limit 0 SO-01 -0.5 -1 -1.5 SO-02 SO-03 SO-04 SO-05 Lower Limit Second Instrument Second Instrument 1.5 1 0.5 Difference 0 Upper Limit SO-01 -0.5 -1 -1.5 SO-02 SO-03 SO-04 SO-05 Lower Limit Evaluation Review Is There Something Wrong? Reviewing Ungraded/Educational Challenges HFC-B Neutrophils 100.0 90.0 80.0 70.0 Result 60.0 Target 50.0 Low er Limt 40.0 Upper Limit 30.0 20.0 10.0 0.0 HFC-03 HFC-04 Reviewing Ungraded/Educational Challenges Continued HFC-B Lymphocytes 60.0 50.0 40.0 Result Target 30.0 Low er Limt Upper Limit 20.0 10.0 0.0 HFC-03 HFC-04 Reviewing Ungraded/Educational Challenges Continued HFC-B Monocytes 9.0 8.0 7.0 6.0 Result 5.0 Target 4.0 Lower Limt 3.0 Upper Limit 2.0 1.0 0.0 HFC-03 HFC-04 Proficiency Testing Exception Summary (PTES) Unsatisfactory PT performance  Failure to attain the minimum satisfactory score for an analyte, test, subspecialty, or specialty for a testing event. Unsuccessful PT performance (2 of 3 testing events)  Failure to attain the minimum satisfactory score for an analyte, test, subspecialty, or specialty for two consecutive or two of three consecutive testing events. Critical PT performance (3 of 4 testing events)  Failure to attain the minimum satisfactory score for an analyte, test, subspecialty, or specialty for three consecutive or three of four consecutive testing events. A laboratory must immediately cease testing for that analyte or the discipline. PTES Response College of American Pathologists How to Respond to Unsatisfactory Proficiency Testing Events Example Reasons for Failure Review Images On-Line Use Images for Training Example All Analyte Score Card What is it and How Can it Help? The all analyte score card can be filtered various ways Results can be downloaded into PDF or Excel format Provides a cumulative summary for quality team Example of All Analyte Score Card CAP Accepted PT Providers AccuTest (Healthmetrx Digital PT) American Association of Bioanalysts (AAB) American Proficiency Institute (API) American Society of Histocompatibility and Immunogenetics (ASHI) College of American Pathologists (CAP) Control Laboratory PT Program South-Eastern Organ Procurement Foundation (SEOPF) Wisconsin State Laboratory of Hygiene Proficiency Testing Program (WSLH) CAP LAP Audioconference POCT and CAP Accreditation Selwyn J. Baptist, MD, FCAP September 20, 2006 1:00 – 2:00 pm Free enrollment to institutions accredited by CAP AACC Southeast POC Chapter Point-of-Care-Testing: Evidenced Based Practices and Regulatory Issues November 2-4, 2006 Francis Marion Hotel Charleston, South Carolina http://www.pointofcare.net/AACC_Southeast_POC_Meeting.htm Central Florida POC Network 6th Annual Health First Point-of-Care Conference and Exhibition Friday, November 17th, 2006 Cape Canaveral Hospital in Cocoa Beach, FL http://www.pointofcare.net/Central_Florida/November2006Meeting.htm References GP27-A Using Proficiency Testing (PT) to Improve the Clinical Laboratory College of American Pathologists How to Respond to Unsatisfactory Proficiency Testing Events What to do With PT Providers – 2004 LAP Audioconference – Gerald Hoeltge, MD CAP Laboratory Accreditation Program Manual Thank You Contact Information Elaine Staley, MT(ASCP) E-mail – staley.elaine@mayo.edu