1. (Optional) Set Agreement Range Threshold
Agreement range thresholds are only applicable to assessments dispatched via the Collective Review tool and assessments of the exact same student submissions by multiple Assessors (Inter-rater Reliability, Assessor Pool distribution to multiple members, or manual transfer of submission to additional assessors).
If the same file or submission is sent to multiple Assessors for assessment and criterion scoring agreement is required or desired, the agreement range threshold setting will generate new assessments to the same Assessors to complete until agreement is reached.
Agreement is determined by the assessment criterion scores selected by each assessor, for a given file or submission, falling within the selected range threshold.
Agreement assessments will be available for reporting in Results Analytics and in the Agreement Reporting Tool, regardless of whether or not all assessors have reached agreement.
Step 1: Access the Assessment Instruments Screen
1. Click on the Main Menu icon.
2. Select Curriculum.
3. Select Assessment Instruments.
Step 2: Edit the Assessment Instrument
Locate the Assessment Instrument to which you would like to enable and apply an Agreement Range Threshold.
4. Click on the Assessment Instrument and select Edit.
Step 3: Enable Agreement Assessments
5. Click on the Settings tab.
6. Select Enable Agreement Assessments, under the 'Collective Review' Agreement heading.
Step 4: Select Agreement Range Threshold
7. Select the Agreement Range Threshold. New assessments will be generated when the agreement range threshold exceeds the scoring range selected.
8. Click Save Settings at the top right corner of the screen.