Teacher Hand Scoring System and Scoring

Scoring the Speaking Domain Locally

The Speaking domain is administered one-on-one with the student and TE and is the only domain that is scored locally, in the moment, by the TE. The TE has two options for capturing the student’s scores in the moment. One option is to log on to the DEI before administering the Speaking domain and enter the student’s scores directly into the DEI while administering the assessment. The second option is to use the Speaking Student Score Sheet (figure 1) in the DFA or, for K–2, the Writing Answer Book, which also includes the Speaking Student Score Sheet to record the student’s scores as the assessment is administered. This second option requires the TE to log on to the DEI as soon as possible after the Speaking domain administration has been completed and enter the scores for the student. If this option is used, the Speaking Student Score Sheet should be kept secure until data entry. In the event the TE misplaces the score sheet prior to entering the scores in the DEI and cannot determine the scores from voice capture, the student might require a Reset Appeal to recapture the student’s responses.

Figure 1. Sample Speaking Student Score Sheet

During the administration of the Speaking domain, the TE should not be concerned about the quality of the voice-capture response but instead should focus on scoring the oral response in the moment. Do not rerecord the student’s response. The Using the DEI chapter of this manual can be used as reference, as it provides the steps that must be followed for entering student Speaking domain scores into the DEI.

Securely destroy the used Speaking Student Score Sheet after a student’s scores have been entered into the DEI.

Audio Capture for the Speaking Domain

During the administration of the Speaking domain, the following guidelines apply to the audio capture tool. The TE should score what the student says, not what is recorded.

Rerecording: If any of the following issues occur, the TE may continue to the next question and does not need to rerecord:

• The TE forgets to record a response.
• The TE records a response on the wrong question.
• There is a red triangle in the review box when ending the test session. If the audio capture times out after two minutes, the TE should allow the student to complete the response and score the complete spoken response and not only what was recorded.

Headsets: Use of headsets with a microphone for voice capture is not recommended by the TE, to enable interaction between the student and TE.

Scoring the Initial ELPAC Writing Domain Locally

K–2

TEs will also need to score the Writing domain responses locally. Students in K–2 will receive a Writing Answer Book. Trained TEs must score the written responses on the Answer Book using the rubric and anchor samples included in the DFA. Scores for each item must be bubbled in the score sheet (figure 2) that is included in the Writing Answer Book. After, these scores must be entered in the DEI for scores to be captured and matched with other domain scores taken on the TDS. The Using the DEI chapter in this manual can also be used as reference, as it provides the steps that must be followed for entering student Writing domain scores into the DEI.

Figure 2. Grade one score sheet in the Answer Book

Grades Three Through Twelve

Students in grades three through twelve enter their Writing domain item responses into the TDS. These responses are then routed to the THSS, where trained TEs can log on to view students’ responses to each item and enter scores.

Initial ELPAC: THSS

Quick Start Instructions to Access the THSS

1. Navigate to the CAASPP & ELPAC Website, select the System Links menu, and then select the THSS—Teacher Hand Scoring System item from the drop-down list.
2. Log on by entering the TOMS username and password on the Logon screen and then selecting the [Secure Logon] button.
3. Select the school.
4. Select the assessment to score; if no assessments are available to be scored, the page will be blank after the headings.
5. Select student item responses to score.
6. Score student item responses.
7. Reassign items (optional).
8. Select the [Submit Scores for Students] button on the Item Scoring screen or the Submit checkbox and then the [Submit Score(s)] button on the Assessment Scoring screen.

More detailed procedures are outlined in the Using the THSS subsection.

Teacher Hand Scoring System User Roles

Scorers

Scorers use the THSS to score the Writing domain item responses for grades three through twelve. Student item responses are assigned to the scorer on the basis of the session ID in which the assessment was completed. Scorers can reassign item responses to other scorers in the same school or score managers from within the school.

TEs are automatically assigned the role of scorer in the THSS. A TE will receive item responses for scoring only for the students to whom the TE administered the assessment or for students whose responses the TE has been reassigned.

Score Managers

There are two types of score managers:

1. LEA ELPAC coordinator score managers, who can view item responses completed by any student associated with their LEA
2. Site ELPAC coordinator score managers, who can view item responses completed by any student associated with their school

Both types of score managers can reassign student item responses to any scorer within their user level. Score managers inherit scorer properties as part of their role, so they can also score item responses that have been reassigned to them. Table 1 describes the hand scoring roles and their abilities.

Using the THSS

Step 1. Access the THSS

To access the THSS, TOMS logon credentials—username and password—are required.

1. Open a web browser and navigate to the CAASPP & ELPAC Website.
2. Select the System Links menu.
3. Select the THSS—Teacher Hand Scoring System item from the drop-down list.

4. Enter the required email address and password in the Email Address and Password fields in the Logon screen (indicated in figure 3), and then select the [Secure Logon] button.

   

   Figure 3. Logon screen

If the password has been forgotten or otherwise needs to be reset, select the [Reset Password] link; instructions for how to proceed to reset it are found in the Resetting a Password subsection in the TOMS User Guide.

Step 3. Select School

After logging on, the system will direct users as follows:

• Test examiners assigned to one school will be sent to the THSS Dashboard screen directly; those with multiple roles will select a school and then select the [Continue] button (figure 4).

  

  Figure 4. Select School screen—test examiners

• Site ELPAC coordinators will be directed to select an option from the District drop-down list (figure 5). After selecting the site listing, the system moves the user to the THSS Dashboard screen.

  

  Figure 5. Select School screen—coordinators

• LEA ELPAC coordinators will be directed to first select an option from the District drop-down list and then to select an option from the School drop-down list (figure 5). After selecting a school, the system moves the user to the THSS Dashboard screen.

Step 3. Select Assessments to Score

After logging on, the scorer will be directed to the Dashboard screen (figure 6). The Dashboard view shows the test programs that have been assigned to the scorer and may include interim assessments. Assessments to score are listed in the Showing all assessments table.

Figure 6. Showing all assessments table on the Dashboard screen

1. Select the [Show Filters] button to open the Filters overlay that can be used to select specific assessment groups (like the Initial ELPAC) or a specific manner of administration (figure 7).

   

   Figure 7. Filters

2. Keyword Search:—This field allows the scorer to search for an assessment or other criteria (such as “Nonstandardized”).
3. [Change School] button—Select the [Change School] button to return to the Select School screen.
4. Assessment Name—Access the selected assessment to score by selecting the link.
5. Manner of Administration—This column is applicable for interim assessments only. It indicates whether an interim assessment was administered as Nonstandardized or Standardized. Initial ELPAC grade-span assessments are marked “Unassigned.”
6. Student Count—This column indicates the number of students who took the assessment.
7. Items to Score—This column indicates the number of items to be scored.
8. Items to Submit—This column indicates the number of scored items to be submitted.
9. Date Last Taken—This column indicates the last date the students took the assessment.
10. Days in System—This column lists how long item responses for an assessment have been in the THSS awaiting scoring. Responses for the Initial ELPAC that have been submitted to the THSS for three or more days are indicated using the [Scoring Reminder] exclamation point icon.
11. Sorting arrows—These arrows are used to sort the content of a column in ascending or descending order. They appear when hovering over the column name. Users can sort the screen by any of the column headers.
    • Select a column header sorting arrow. The table lists the fields by that column in ascending order.
    • To sort the column content in descending order, select the column header sorting arrow again.
12. Navigation arrows—These arrows are used to view item responses on other pages if the item response list includes more assessments to score than what are shown currently on the screen.

Step 4. Select Item Responses to Score

After selecting the assessment in the Assessment Name column, the Assessment Scoring screen will appear (figure 8). The table on this screen is populated with the test items that are assigned to the scorer to be scored.

Figure 8. Assessment Scoring screen

1. [Dashboard] link—Select the [Dashboard] link to return to the Dashboard screen.
2. Submit—Boxes in this column remain suppressed until all the student’s item responses have been scored.
3. Reassign—Boxes in this column are available to reassign scorers to item responses that are to be scored.
4. Assignee—This column shows the name of the scorer to which the item response is assigned.
5. Student Name—This column indicates the student’s name.
6. Student SSID—This column indicates the student’s SSID.
7. Date Taken—This column lists the date and time the student took the assessment.
8. Items to Score—This section of columns is separated on the basis of the Item Sequence Number and Max Points. The [Score] link will change to the points earned number once an item response has been scored.
9. [Score] link—This column provides the scoring status of the item. An assigned scorer can select this link to score the student’s response.
10. Sorting arrows—These arrows are used to sort the content of a column in ascending or descending order. They appear when hovering over the column name. Score managers can sort the screen by assignee, student name, student SSID, and date taken. Score managers can also sort the Assessment Scoring screen by the assigned scorer.
    • Select a column header sorting arrow. The table lists the fields by that column in ascending order.
    • To sort the column content in descending order, select the column header sorting arrow again.
11. [Submit Score(s)] button—The button remains suppressed until the student’s assessment is scored. After a student’s assessment is scored and the Submit checkbox for one or more students has been selected, the user can select this button to submit scores for the student(s).
12. [Reassign] button—This button, which is used to reassign item responses for scoring, remains suppressed until a Reassign checkbox is selected.
13. Navigation arrows—These arrows are used to view item responses on other pages if the list of scores shows more than the designated number of rows of students per page.

Step 5. Score Item Responses

In the Items to Score column of the Assessment Scoring screen (figure 8), select the [Score] link for the response. The Item Scoring screen appears, displaying the student response. Screen features, indicated in figure 9, include the following:

1. Navigation arrows to move between items
2. Drop-down list and navigation arrows to move between students
3. A toggle to open the “Rubric & Resources” section
4. A drop-down list to select a different language for the item and prompt, if available
5. A toggle to view the item using the student’s test settings
6. The “Item Score” section, where the scorer selects the number of points the student earned
7. The “Scoring Notes” section, where the scorer can opt to add a note by selecting the [Add Note] button

8. The item stimulus, prompt, and response, which are blurred in figure 9 (The layout of the item can vary due to a variety of factors, including the item content and whether student settings were toggled.)

   

   Figure 9. Item Scoring screen

Take these actions when navigating the Item Scoring screen:

1. To navigate to the Item Scoring screen for a different item, a scorer may use the Current Item navigation arrows (figure 10), or close out the item being scored by selecting the [Assessment Scoring] link at the top of the page to return to the list of assessments; and then selecting a [Score] link in an Items to Score column (figure 8).

   

   Figure 10. Navigation arrows to move between available items

2. Selecting the Current Student drop-down list or a navigation arrow next to the drop-down list will move to the next assigned student's item (figure 11). The list contains the full set of students; the arrows will scroll through the entire list of assigned students.

   

   Figure 11. Options to move between students

3. To enter a score for the response, select the Points Earned drop-down list to select a numerical score (figure 12). The score does not exceed the value in the Max Points column. When a score is assigned or changed, it will be automatically saved and will appear in the appropriate Items to Score column on the Assessment Scoring screen.

   

   Figure 12. Points Earned drop-down list

4. A collaboration feature in the THSS lets a scorer add a note. Previous notes entered by other scorers cannot be edited, but new notes can be entered into the “Scoring Notes” section. Historical notes are preserved, including reassignment notes. Select the [Add Note] button to add a note (figure 13).

   

   Figure 13. Add a scoring note in the Scoring Note field

5. After selecting the Rubric & Resources toggle (figure 14), the screen layout changes to allow the scorer to review resources appropriate for the grade level and assessment program. Select the toggle again to collapse this view. Information includes the following:
   • Scoring notes
   • Item details
   • Item resource links (exemplars, anchor samples for each score point, rubric training guide)

   The rubric description

   

   Figure 14. The “Rubrics and Resources” section of the Item Scoring screen

6. Use the Item Language drop-down list to view the item and stimuli in a presentation language other than English if another language is available (figure 15). Options can include braille.

   

   Figure 15. Item Language list

7. Select the Student Settings toggle to apply any assigned settings (such as print size) to the item (figure 16). Select the toggle again to return to a standard view.

   

   Figure 16. Student Settings toggle

8. View the item stimulus, prompt, and response section at the bottom of the Item Scoring screen (blurred in figure 17).

   

   Figure 17. Item stimulus, prompt, and response section of the THSS

   The three parts of the item the student accesses in the TDS and that is recreated in the THSS are as follows:

   1. Item prompt—A test question or stimulus presented to a student to elicit a response
   2. Item stimulus—Content passages, short scenarios, or source material provided to the student that give context for assessing student knowledge and skills
   3. Student response area—The designated area for the student’s response to the item prompt

Step 6. Reassign Responses (If Needed)

The THSS automatically assigns student responses for hand scoring to the TE who administered the session in which the student assessment was completed. If a student started an assessment in one session and completed it in another, the response will be assigned to the TE who administered the session in which the student completed the assessment.

Item responses can only be assigned to one scorer at a time, and only one score is saved for a student after it is submitted. An item response cannot be reassigned after the score is submitted unless a STAIRS case that results in a Rescore Appeal has been submitted.

Score managers at the school level (site ELPAC coordinators) can reassign responses to scorers in a school. Those score managers associated with an LEA (LEA ELPAC coordinators) can reassign responses to scorers within the LEA.

What follows is the process for score managers to reassign responses.

1. Select the assessment being scored.

2. On the Assessment Scoring screen (figure 18), mark the checkbox in the Reassign column on the left for any student responses intended to be reassigned.

   

   Figure 18. Reassign checkbox and button on the Assessment Scoring screen

3. Select the [Reassign] button at the bottom of the screen (figure 18). The Reassign to new scorer dialog box appears (figure 19).

   

   Figure 19. Reassign to new scorer dialog box

4. Select a scorer from the Assignee drop-down list shown in figure 19. Only users associated with the user’s school or LEA will appear.
5. Enter text in the Note field in the Reassign to new scorer dialog box (figure 19). The THSS does not track reassignments, but details can be documented in the Note field along with other information for the next scorer. This note will appear in the “Scoring Note” section of the Item Scoring screen.
6. Select the [Reassign] button. The selected responses are reassigned to the selected scorer. The newly assigned scorer’s name will show in the “Scoring Notes” section (figure 20); this display is available to score managers only. Note the following about scorer note options:
   • For a scorer, once responses are reassigned, those responses are removed from the original scorer’s response list.
   • The [Edit] pencil icon and [Delete] trash can icon appear only for the note’s originator and can be used to edit or delete a note.

   

   Figure 20. “Scoring Notes” section on the Item Scoring screen with reassignment notes

Step 7. Submit Student Responses

There are two ways to submit scores for a student: by selecting the [Submit Scores for Student] button on the Item Scoring screen; or by selecting the Submit checkbox and then the [Submit Score(s)] button on the Assessment Scoring screen.

1. After finishing entering scores for all student responses on the Item Scoring screen, select [Submit Scores for Student] (figure 21), to the right of the Current Student navigation arrows. Note that the [Submit Scores for Student] button remains suppressed until all the student’s responses are tentatively scored.

   

   Figure 21. [Submit Scores for Student] button on the Item Scoring screen

   1. Select the [Continue] button on the Score Submit Alert pop-up message box to submit the scores, or select [Cancel] to exit (figure 22).

      

      Figure 22. [Continue] button on the Score Submit Alert message, Item Scoring screen

   2. To navigate to the Item Scoring screen for a different student, select a [Score] link in the Items to Score column (figure 8) on the Assessment Scoring screen. The order of responses is determined by the sorting options chosen on the Assessment Scoring screen.

2. Alternatively, take the following steps to submit scored responses on the Assessment Scoring screen:

   1. Mark the checkbox in the Submit column for one or more students whose responses have all been scored (figure 23). To select all responses, mark the checkbox next to the Submit header or each checkbox in the student row.

      

      Figure 23. Submit scores on the Assessment Scoring screen

   2. Select the [Submit Score(s)] button (figure 23).

   3. Select the [Continue] button on the Score Submit Alert pop-up message box to submit the scores or select [Cancel] to exit (figure 24).

      

      Figure 24. [Continue] button on the Score Submit Alert message, Assessment Scoring screen

Completed THSS scores will be merged with the other domain scores in TOMS. Once all four domain scores are in TOMS, the system will produce the Initial ELPAC SSR, which will be available in TOMS.

In the event a scorer realizes that an error was made scoring a student’s response in the THSS, the scorer must immediately alert the site ELPAC coordinator (if the scorer is the TE or local scoring data entry staff) or the LEA ELPAC coordinator (if the scorer is the LS correspondence administrator). The site ELPAC coordinator or LEA ELPAC coordinator must create a STAIRS case in TOMS.