Measuring Standards of Excellence

How ARTS uses its Standards and Criteria for Excellence

ARTS-COA created the ARTS Standards and Criteria for Excellence which integrate the COA standards with a listing of typical documentation and suggested evaluative questions and criteria. In developing this document ARTS-COA reviewed similar documents published by TRACS, ABHE, PEQAB (Ontario Canada), and ATS. This is a living document, reviewed at the 2023 annual meeting, and will be reviewed annually.

These standards of excellence are rubrics that are used by site visit teams, and the COA, as the basis for evaluating self-studies, during accreditation visits, and the ARTS annual reports. Our criteria contain the following scale values:

  • Standard is met
  • Standard is partially met
  • Standard is not met

However, the COA has had discussions (at the recent annual meeting and during some COA meetings) regarding adding additional gradations to these three scale values. Would it be better to expand our criteria values from 3 to a greater number (such as 5-7)? I thought it would be worth summarizing our discussions and the pros and cons of expanding our scale.

Pros of Expanding the Scale:

  1. Enhanced Precision: A larger scale provides more granularity, allowing for a more precise assessment of performance. This can help in distinguishing between various degrees of achievement, especially in cases where the standard is not clearly met or not met.
  2. Improved Feedback: A larger scale can offer more detailed feedback to institutions, enabling them to identify specific areas where they excel or need improvement. This can lead to more targeted and actionable recommendations.
  3. Fairness and Flexibility: A larger scale allows for a more nuanced evaluation, accommodating situations where an institution might partially meet a standard but excel in some aspects. This added flexibility can be fairer to institutions that don’t neatly fit into a binary “met” or “not met” category.
  4. Encouraging Continuous Improvement: With a larger scale, institutions may be more motivated to strive for higher levels of performance, as there are more gradations to aim for. This can foster a culture of continuous improvement.
  5. Reflecting Complexity: In situations where standards are complex and multifaceted, a larger scale can better capture the complexity of the evaluation process, providing a more accurate picture of an institution’s performance.

Cons of Expanding the Scale:

  1. Increased Subjectivity: A larger scale may introduce more subjectivity in the evaluation process. It can be challenging to reach a consensus when there are numerous gradations, and different evaluators may interpret the criteria differently.
  2. Longer Evaluation Processes: Expanding the scale may require more time and resources for evaluations, as each additional gradation necessitates more detailed analysis and discussion.
  3. Difficulty in Maintaining Consistency: With more gradations, it can be more challenging to maintain consistency in evaluations across different evaluators, leading to potential discrepancies in assessments.
  4. Complexity in Reporting: A larger scale can make reporting and communication of evaluation results more complex, potentially making it harder for stakeholders to understand and act upon the feedback.
  5. Potential for Ambiguity: As the scale becomes more granular, there’s a risk of introducing ambiguity about what each gradation means, making it harder for evaluators to apply the scale consistently.

The current thinking of the COA is that ultimately a school is either meeting or not meeting the standard. During a site visit, or in discussions of an Annual Report submission, participants have the opportunity to explore the nuances of what has been presented, ask questions and seek clarification. It’s an inductive process which goes from the qualitative to a quantitative conclusion. Our current thinking is that we will not change this scale, but continue to think about our rubric and how best to evaluate. We invite further discussion on this topic. Please contact the executive director of ARTS if you would like to make your opinion known.

Education and Training

Our ongoing list of ARTS education and training resources

Since we have many education and training resources at ARTS I decided to create this page to collect these resources in one place. This page will be updated as needed.

Resources for the Annual Reports.

Between March and April of 2023, the COA planned and implemented 5 training sessions on standards 2-6. These Zoom sessions were recorded. These sessions discussed the COA standards and how to use them to improve academic quality at member schools. Starting with Standard 6 we explained quantitative and qualitative methods of evaluation and assessment and then progressed to other standards. The recordings can be found here: Session 1: Standard 6; Session 2: Standard 2, Session 3: Standard 3, Session 4: Standard 5, Session 5: Standard 4.

Links for Training Videos for ARTS-COA Site Visit Teams – 2023 Edition. The following are training videos for the ARTS-COA Commissioners. Each site visit member is required to complete this training.

Annual Meeting Trainings (2023)

Pin It on Pinterest