CALIFORNIA Assessment of Student Performance and Progress Online Testing Appeals User Guide Winter/Spring 2015 Smarter Balanced Summative Assessments for English Language Arts/Literacy and Mathematics Published April 13, 2015 Prepared by the American Institutes for Research© California customization prepared by Educational Testing Service® © Smarter Balanced Assessment Consortium, 2014 Descriptions of the operation of the Test Delivery System, Test Information Distribution Engine, and related systems are property of the American Institutes for Research® (AIR) and are used with permission of AIR. CAASPP System Contact Information California Technical Assistance Center 2731 Systron Drive Concord, CA 94518 Phone: 800-955-2954 Fax: 800-541-8455 E-mail: CalTAC@ets.org Web site: http://caaspp.org/ ii CAASPP System Table of Contents Work with Appeals .............................................................................................................................................................. 1 Understanding Appeals......................................................................................................................................................... 1 User Roles ........................................................................................................................................................................ 1 Create an Appeal ................................................................................................................................................................ 2 Approving, Rejecting, or Retracting Appeals ........................................................................................................................ 5 To retract an appeal: ......................................................................................................................................................... 5 Creating Appeals through File Uploads ................................................................................................................................ 6 Understanding the Appeal Upload File Format ................................................................................................................. 6 Submitting an Appeal Upload File ..................................................................................................................................... 6 Resolving File Upload Errors ............................................................................................................................................. 7 View an Appeal ................................................................................................................................................................... 9 List of Figures Figure 1. TOMS Dashboard ................................................................................................................................................. 2 Figure 2. Appeals Home screen .......................................................................................................................................... 2 Figure 3. Appeals tabs ......................................................................................................................................................... 3 Figure 4. Step 1 to create an appeal (request type) ............................................................................................................. 3 Figure 5. Step 2 to create an appeal (select criterion) ......................................................................................................... 4 Figure 6. Step 3 to create an appeal (enter criterion) .......................................................................................................... 4 Figure 7. View Appeals page ............................................................................................................................................... 4 Figure 8. Selection fields in the View Pending Appeals page .............................................................................................. 5 Figure 9. Retrieved Pending Appeals page ......................................................................................................................... 5 Figure 10. Sample Appeals Upload file ................................................................................................................................ 6 Figure 11. Upload Appeals screen ....................................................................................................................................... 7 Figure 12. Sample error messages ...................................................................................................................................... 7 Figure 13. TOMS Dashboard ............................................................................................................................................... 9 Figure 14. Appeals Home screen ........................................................................................................................................ 9 Figure 15. Appeals tabs ..................................................................................................................................................... 10 Figure 16. Step 1 to view an appeal (request type) ........................................................................................................... 10 Figure 17. Step 2 to view an appeal (select status) ........................................................................................................... 10 Figure 18. Step 3 to view an appeal (select filtering criterion) ............................................................................................. 11 Figure 19. Step 4 to view an appeal (additional values)..................................................................................................... 11 Figure 20. View Appeals screen ........................................................................................................................................ 11 List of Tables Table 1. Table 2. Table 3. Table 4: Table 5. Table 6. Table 7. Table 8. User functions in the Appeals module .................................................................................................................... 1 Appeal types .......................................................................................................................................................... 3 Search criteria ........................................................................................................................................................ 4 Columns in the Appeals Upload File ...................................................................................................................... 6 File upload error icons and resolutions .................................................................................................................. 7 Columns in the “Error Messages” report ................................................................................................................ 8 Appeal status and descriptions ............................................................................................................................ 10 Fields in the View Appeals page .......................................................................................................................... 11 iii CAASPP System Acronyms and Initialisms Used in the Online Testing Appeals User Guide CAASPP CALPADS CalTAC CDE LEA SC SSID TA TDS TOMS California Assessment of Student Performance and Progress California Longitudinal Pupil Achievement Data System California Technical Assistance Center California Department of Education local educational agency CAASPP Test Site Coordinator Statewide Student Identifier Test Administrator test delivery system Test Operations Management System iv Work with Appeals | Understanding Appeals Work with Appeals In the normal flow of an online testing opportunity, a student takes a test in the test delivery system (TDS) via the secure browser and then submits it. The TDS then forwards the test for scoring and reporting. For security incidents that result in a need to reset, reopen, invalidate, or restore individual online student assessments, the request must be approved by the California Department of Education (CDE). In most instances, an appeal will be submitted to address a test security breach or irregularity. In some cases, an appeal may be submitted to address incidents that are not security related such as allowing an extension of the grace period—the allowable period between the time a test is paused and when the test is resumed—to reopen an assessment for a student who becomes ill, has paused the test, and is unable to resume testing until after testing has expired. The local educational agency (LEA) California Assessment of Student Performance and Progress (CAASPP) Coordinator or the CAASPP Test Site Coordinator (SC) may submit appeals in the Test Operations Management System (TOMS). Appeals are a way of interrupting this normal flow when there are testing exceptions or circumstances that interfere with the planned completion of a testing opportunity. For example, a student may need to have a test reset, or a Test Administrator (TA) may need to invalidate a test because of a security incident or impropriety. This section describes how you may view, create, and approve appeals. All individual submitted appeals are available for retrieval and review by appropriately credentialed users within a given organization. Your view of appeals will be restricted according to your user role as established in TOMS. Understanding Appeals This section describes the types of appeals that may be created, and the statuses that may be assigned to these appeals. User Roles Table 1 shows the three user roles in the Appeals module and the functions they are permitted to perform. Submit means that a user has the rights to request an appeal. View means that a user can see a student’s appeal. Edit means that a user has the rights to process, retract, or reject an appeal with a comment. Table 1. User functions in the Appeals module Role LEA CAASPP Coordinator CAASPP Test Site Coordinator TA April 2015 Submit Function View Edit 2015 CAASPP Online Testing Appeals User Guide 1 Customization Copyright © 2015 by the California Department of Education Create an Appeal | Understanding Appeals Create an Appeal Note: Appeals may be created by LEA CAASPP Coordinators and CAASPP Test Site Coordinators only. Open TOMS. 1. 2. 3. Open an Internet browser. Open TOMS by going to http://caaspp.org/ and selecting the [Test Operations Management System] button to access the TOMS Sign In Web form. When you have logged on and the TOMS Dashboard has opened, select the [Appeals] button in the left navigation bar (shown in Figure 1). Figure 1. TOMS Dashboard Select the Function. 1. The Appeals Home screen, shown in Figure 2, opens. Figure 2. Appeals Home screen 2. 3. Select the [Appeals] tab. Select the tab to choose a function—either to create, view, or upload an appeal. Tabs are shown in Figure 3. 2 2015 CAASPP Online Testing Appeals User Guide Customization Copyright © 2015 by the California Department of Education April 2015 Create an Appeal | Understanding Appeals Figure 3. Appeals tabs Create an Appeal. 1. 2. Select the [Create Appeals] tab, which is indicated in Figure 3. Select an appeal type to request from Submit a request to: drop-down list, shown in Figure 4. Table 2 lists the available types of appeals. Figure 4. Step 1 to create an appeal (request type) Table 2. Appeal types Type of Appeal Invalidate Test Reasons for Appeal Test security breach Test administered inconsistently with the Online Test Administration Manual (TAM) Student deliberately did not attempt to respond appropriately to items Results of Approved Appeal Reset Test A student with an individualized Resetting a student’s test removes that test from the system and enables the student to start a new test from the beginning. education program or Section 504 plan started test without his or her designated supports or accommodations Correct test was not available Incorrect test originally opened Reopen a Test Allows for a submitted or expired test to be reopened April 2015 While invalidated tests will be scores and scores will be reported (with a note that an irregularity occurred), these scores will not be included in percent-proficient reporting for federal accountability. Reopening a test allows a student to access a test that has already been expired or been submitted: Expired—If the test has expired, it reopens where student left off. Submitted—If the test has been submitted, it reopens at the last page of the test; student can review items in the current segment, but cannot return to previous segments 2015 CAASPP Online Testing Appeals User Guide 3 Customization Copyright © 2015 by the California Department of Education Create an Appeal | Understanding Appeals Type of Appeal Restore Test (from Reset) Reasons for Appeal A test was inadvertently or inappropriately reset Results of Approved Appeal Grace Period Extension Student became ill and the test Allows the student to review previously answered questions upon logging back on to the test following expiration of the pause rule period. expired Technological difficulty resulted in expiration of the test Unanticipated excused absence or school closure resulted in expiration of test(s) A test that has been reset in error can be restored to its previous status and restarted where the student left off. Warning: Submit test reset and restore requests at least one day prior to the end of a test window to allow students to complete their test opportunity. 3. Select a criterion, listed in Table 3, by which to search for a student from the Search by dropdown list (shown in Figure 5). Figure 5. Step 2 to create an appeal (select criterion) Table 3. Search criteria Field Result ID Session ID SSID 4. Description Result ID is a unique internal identifier assigned by the system and associated with a student’s test opportunity; for system use only. The result ID is not the same as the SSID. Session ID associated with the test opportunity. Student’s Statewide Student Identifier (SSID) associated with the result. Once the criterion is selected, enter the ID number for the field type, shown in Figure 6. Figure 6. Step 3 to create an appeal (enter criterion) 5. Select the [Search Student Results] button, indicated in Figure 6. The system displays the results at the bottom of the View Appeals page (see Figure 7). Figure 7. View Appeals page 4 2015 CAASPP Online Testing Appeals User Guide Customization Copyright © 2015 by the California Department of Education April 2015 Create an Appeal | Approving, Rejecting, or Retracting Appeals 6. For each result for which you want to create an appeal, enter the reason in the text box, and then select [Create]. The system displays a confirmation message. Note: Please add sufficient detail in the Reason box to provide enough information to facilitate a decision about the appropriateness of the type of appeal. For example, if the student’s test expired due to illness, please indicate that in comment area. You may retract any appeal that you have created within the system; see Approving, Rejecting, or Retracting Appeals in the next section for additional details. Approving, Rejecting, or Retracting Appeals Appeals require CDE approval or rejection before the TDS can process them. Caution: You cannot retract or delete an approved or rejected appeal. To retract an appeal: 1. Retrieve the desired student appeals by following these steps: a. Select the Appeals tab, and then select View Pending Appeals. The View Pending Appeals page appears (see Figure 8). Figure 8. Selection fields in the View Pending Appeals page b. From the drop-down lists and in the text field, select or create the search criteria. c. Select Search. The system displays the found appeals at the bottom of the View Pending Appeals page (see Figure 9). Figure 9. Retrieved Pending Appeals page 2. 3. 4. Optional: If needed, you may review your original reason for initiating the appeal by selecting show comment. Mark the checkbox for the appeals you want to retract. A comment box and approval buttons appear. As California users may only retract their created appeals, select [Retract] (other options will be unavailable to select). The system removes the retracted appeal from the list. April 2015 2015 CAASPP Online Testing Appeals User Guide 5 Customization Copyright © 2015 by the California Department of Education Create an Appeal | Creating Appeals through File Uploads Creating Appeals through File Uploads If you have many appeals to create, it may be easier to create them at once using a file upload procedure. This task requires some familiarity with composing comma-separated value (CSV) files or working with Microsoft Excel. The following sections describe how to compose the files and then upload them to the appeals interface. Understanding the Appeal Upload File Format The upload file consists of a Microsoft Excel or CSV file with a dedicated header row and subsequent data rows containing the detail for each of the appeals to be created. Table 4 describes the columns presented in the upload file, and associated valid values. Table 4: Columns in the Appeals Upload File Column Name Type* Description Type of Appeal. Valid Values One of the following: Invalidate Test Reset Test Re-open Test Grace Period Extension Restore Test Search Type* Student field to perform a search on. One of the following: Result ID Session ID SSID Search Value* Search value corresponding to the search type. Up to 1,000 alphanumeric characters. The value must exist in the test delivery system (TDS). For example, specifying a result ID of 123456 requires that this result ID exist in the TDS. Note, too, that an SSID needs to be in the format entered, i.e., CA-XXXXXXXXXX. Up to 1,000 alphanumeric characters. Reason* Reason for submitting an appeal. *Required field. Figure 10 is an example of a simple upload file that reopens a test for a student with the SSID number 99999999 (which uses the format CA-XXXXXXXXXX). Figure 10. Sample Appeals Upload file Submitting an Appeal Upload File 1. Select the [Appeals] tab, and then [Upload Appeals]. The Upload Appeals page, seen in Figure 11, appears. 6 2015 CAASPP Online Testing Appeals User Guide Customization Copyright © 2015 by the California Department of Education April 2015 Create an Appeal | Creating Appeals through File Uploads Figure 11. Upload Appeals screen 2. 3. 4. 5. 6. Download one of the file templates by selecting [Download Excel Template] or [Download CSV Template]. Open the file in a spreadsheet application or text editor, and add a row for each appeal you want to add. Be sure to follow the guidelines in Table 4. Save the file on your computer. To upload the file you created, select [Browse] and navigate to the upload file you created in step 3. Select [Upload File]. The system displays a preview of the uploaded file. Use this preview to verify you uploaded the correct file and whether there are any errors present. For information about resolving error messages, see Resolving File Upload Errors. Select [Commit]. The system commits those records that do not have errors and sends a confirmation e-mail. Resolving File Upload Errors During validation, the appeals system displays messages for the errors it detects, as in the example in Figure 12. Figure 12. Sample error messages The first column in the table of error messages shows an icon that indicates the error’s severity. Table 5 describes those icons and associated resolutions. Table 5. File upload error icons and resolutions Icon April 2015 Description Indicates the record causes the entire upload file to fail. Indicates a record that will not be processed due to an error. The record containing the error will be ignored; the remaining records will process. Resolution Select [Cancel] to abort the file upload. Make the indicated correction, and upload the file again. One of the following: Select [Commit] to submit all records that have no errors. Repair those records that have errors and submit them in a separate file. Select [Cancel] to abort the file upload. Make the indicated corrections, and upload the file again. 2015 CAASPP Online Testing Appeals User Guide 7 Customization Copyright © 2015 by the California Department of Education Create an Appeal | Creating Appeals through File Uploads Icon Description Warning that a record contains an error; The record will be accepted as entered. Resolution One of the following: Select [Commit] to submit all records that have warnings. Review the warning messages, repair the records as necessary, and upload them again in a separate file. Select [Cancel] to abort the file upload. Make the indicated correction, and upload the file again. Table 6 describes the columns appearing in the “Error Messages” report. Table 6. Columns in the “Error Messages” report Column Name Record Number Field Name Field Value Validation Message Description Line in the upload file where the error occurred. Name of the column in which the error occurred. Value that caused the error. Message describing the error. 8 2015 CAASPP Online Testing Appeals User Guide Customization Copyright © 2015 by the California Department of Education April 2015 View an Appeal | Creating Appeals through File Uploads View an Appeal Open TOMS. 1. 2. 3. Open an Internet browser. Open TOMS by going to http://caaspp.org/ and selecting the [Test Operations Management System] button to access the TOMS Sign In Web form. When you have logged on and the TOMS Dashboard has opened, select the [Appeals] button in the left navigation bar (shown in Figure 13). Figure 13. TOMS Dashboard Select the Function. 1. The Appeals Home screen, shown in Figure 14, opens. Figure 14. Appeals Home screen 2. 3. Select the [Appeals] tab. Select the tab to choose a function—either to create, view, or upload an appeal. Tabs are shown in Figure 15. April 2015 2015 CAASPP Online Testing Appeals User Guide 9 Customization Copyright © 2015 by the California Department of Education View an Appeal | Creating Appeals through File Uploads Figure 15. Appeals tabs View an Appeal. 1. Select an appeal type to view from the Submit a request to: drop-down list, shown in Figure 16. Table 2 lists the available types of appeals. Figure 16. Step 1 to view an appeal (request type) 2. Select an appeal status from the Select status: drop-down list, shown in Figure 17. Table 7 lists the possible statuses that may be displayed following initiation of an appeal. Figure 17. Step 2 to view an appeal (select status) Note: An appeal’s status can change throughout its life cycle. Some of the statuses have a corresponding reason or description that you may view in the Comments column of the View Appeals page (see Figure 20). Table 7. Appeal status and descriptions Appeal Status Error Occurred Pending Approval Processed Rejected Rejected by System Requires Resubmission Retracted Submitted for Processing Description of Status A technical or system interruption occurred while the appeal was being processed which caused an error. The appeal may need to be resubmitted. Appeal is pending approval in the approver queue. Appeal was successfully processed and the test opportunity has been updated. Appeal was rejected by the reviewing party. TDS was unable to process the appeal. This may occur when an incongruent appeal request is submitted based upon the current student test status (e.g. attempting to reopen an unexpired test). This appeal must be resubmitted. This may occur when the appeal creation process does not complete correctly due to a system or other technical interruption; the appeal was logged. Originator retracted the appeal. Appeal submitted to TDS for processing. 10 2015 CAASPP Online Testing Appeals User Guide Customization Copyright © 2015 by the California Department of Education April 2015 View an Appeal | Creating Appeals through File Uploads 3. Select a criterion from the Filter by drop-down list shown in Figure 18. Figure 18. Step 3 to view an appeal (select filtering criterion) 4. If you selected something other than “All” in step 3, enter a value for the search field you selected. Figure 19. Step 4 to view an appeal (additional values) 5. 6. Select [Search]. The system displays the found appeals at the bottom of the View Appeals screen (see Figure 20). Optional: To review additional information about an appeal, select show comment, indicated in Figure 20. Figure 20. View Appeals screen Table 8 describes the columns in the Retrieved Appeals screen. Table 8. Fields in the View Appeals page Field Case Number Request Type Action Status Request Date Comments School IRN SSID Result ID First Name Last Name Test Test Opp # April 2015 Description ID number associated with the appeal. One of the appeal types listed in Table 2. A [Resubmit] button will appear here if the action is to resubmit the appeal. One of the status codes listed in Table 7. Date appeal was created. Comment or explanation added by user who created the appeal. 14-digit code that identifies county, district, and school (i.e., CDS code). Student’s SSID associated with the result. Unique internal identifier assigned by the system and associated with a student’s test opportunity; for system use only. The result ID is not the same as the SSID. Student’s first name. Student’s last name. Name of the test associated with the appeal. Test opportunity associated with the result ID. 2015 CAASPP Online Testing Appeals User Guide 11 Customization Copyright © 2015 by the California Department of Education View an Appeal | Creating Appeals through File Uploads Field Test Status Description Approved Completed Denied Expired Invalidated Paused TA approved the student for the session, but the student has not yet started or resumed the test. Student submitted the test for scoring. TA denied the student entry into the session. If the student attempts to enter the session again, this status changes to Pending until the TA approves or denies the student. Student did not complete the opportunity, and cannot resume the test because the test opportunity expired. The test result was invalidated. The test is currently paused as a result of one of the following: Student or TA selected the Pause button on the test. Student or TA idled for too long and the test automatically paused. Test Administrator stopped the student’s session. Test Administrator paused the individual student’s test. Student’s browser or computer shut down or crashed. Pending Requested By Requested By Email Test Start Date Date of Last Activity Student is awaiting TA approval for a new test opportunity. Reported Student’s score for the completed test has been submitted. Review Student has answered all test items and is currently reviewing answers before submission for scoring. Scored Test delivery system processed the student’s answers on the test. Started Student started or resumed the test and is actively testing. Submitted Test was submitted for quality assurance review and scoring before it is sent downstream for reporting. Suspended Student is awaiting TA approval to resume a test opportunity. User who created the appeal. Single sign-on username of the person who made the request. Date student started the test opportunity. The date the appeal was last acted upon. 12 2015 CAASPP Online Testing Appeals User Guide Customization Copyright © 2015 by the California Department of Education April 2015
© Copyright 2024