Notes from “The Design and Development of an Internal Course Review Process”, presented by Lisa Kidder, Idaho State University, at the 2015 Quality Matters (QM) conference.
Idaho State University subscribed to QM in 2008, but have been on a long road through implementation. They have only started doing internal course reviews in 2014, have not done any external reviews yet.
Peer Review Project Overview
Summer 2014 – invited 14 faculty (hand-picked across 5 colleges/divisions), 2 Instructional Technologists, and 6 Course Developers (of the courses that were being reviewed). The 14 faculty were divided across the 6 courses and completed a course review following one of 3 review models. Throughout, they met to discuss the models and had a focus group at the end to discuss the processes. The result was the internal peer review process that they use now.
Reviewers were not all Peer Review trained, but they were QM trained and knowledgeable about online course design.
The 3 Internal Review Models
These are the three models that Idaho State University tested as part of their development process.
1 mentor works with a course developer to use the self-review tool and an alignment matrix.
Benefits – self-review, easy for mentor to find evidence, less stressful
Concerns – not rigorous enough, difficult ot match personalities between mentor and course developer, lacked a subject matter expert (difficult to find a mentor within the same field), and generally seemed better in the early stages
3 reviewers with 1 instructional designer, 1 outside the college, and a team chair from within the college. Followed the QM review process
Benefits – good tech & accessibility knowledge, ID able to fill in some of the gaps in the rubric
Concerns – seemed to be more about the tech center driving the process instead of faculty. Missing subject matter expertise. Have a limited number of ID’s, so the workload would be high
Review team built just like the QM model
Benefits – great prep for an official QM review with sufficient rigor
Concerns – why do an internal review if you are following the full process? It’s a small community at the campus, and worried about bias among colleagues (but good to have a subject matter expert)
They pulled pieces from each of the models to form their review process
- Used the Course Alignment Map from the mentor model (now call it the course map)
- Required evidence of a self-review
- Used 3 reviewers: all outside the college, including 1 instructional designer/technologist
The Dean’s Council has accepted the proposed process and have stated that serving as a peer reviewer counts as service for tenure. No compensation is given to internal peer reviewers.
The application is available online their website (similar to the course worksheet). They have found additional money to have more faculty complete the Peer Reviewer Course.
They are working with the Registrar to identify Quality Courses in the Course Catalog. It has been a long process, so they have spent 3 years so far working on this (working on an image or text code in Banner). During professional development, they are encouraging more faculty to create Course Alignment Maps.
ISU is limiting the number of APPQMR offerings in a hope to increase demand. Going forward they will offer 1 in fall and spring semesters and 3 during the summer (since that is when their faculty prefer to attend).