Uncategorized

Notes from Moving Class Evaluations Online with Enterprise Survey #BbWorld14

Presented by Jennifer Ivie, Richard Shelton, and Stacy Thompson of Northeastern State University

Concerns with online evals
Low response rates
Low completion of comments
**neither proved true in their testing**

Timeline
Started a task force in July 2013, conducted research on online surveys
Educated and advertised in October
Tested in November

Started with 866 courses, 358 instructors
13,258 students received hard copies. Enterprise surveys went to all students (32,043 students)
Hard copy response rate 75%
Online only response rate was 45.833%

Students were more responsive in online evaluations than paper.

Some courses had higher than 75%, turned out that instructors talked about it in class, emailed students, and discussed how feedback would be valued and used.

Strengths and Weaknesses
Hard copy cost $2000 per semester for scantrons, envelopes, and labels, plus 2-3 weeks prep and distribution, collection, and calculation. (Security concerns of paper evaluations carried across campus, and interference from faculty who would threaten students to get positive ratings)

Enterprise surveys included in license, required 30 minutes to set up, and were instant and secure. Faculty received results as soon as grades were submitted.

Ran a second semester of two formats for testing, but students refused to complete both and response rates dropped

Faculty and deans pushed back against online, but administration decided to move forward online only. Student association is supportive of the format and understands the value of students bring able to provide feedback

Strengths:
– no loss of class time
– students have more time to provide lengthy and thoughtful comments
– no significant different in number of students providing written responses
– faster to administer
– cheaper to administer, in the long run
– faster feedback to students
– students have more time
– no need to rely on students
– increased security
– response rates improve over time and with training
– faculty less likely to try and intimidate students for higher ratings
– faculty cannot recognize handwriting in low productivity programs

Weaknesses:
– lower response rates
– anonymity concerns by students
– no way to update evals on the fly
– no way to embed the class into the email notification, so students get confused about which course they are evaluating
– faculty fears about lower scores if taken online
– low performing students less likely to respond
– if combined with hard copy, students refuse to take second online survey
– unable to designate deans/program chairs to automatically have access (actually not true, if you use business units on the institutional hierarchy)
– confusion by faculty and students about switch

Future Directions
All courses starting in Fall 2014 will be evaluated online only
Educate and advertise to instructors and students both
Students will man kiosks and walk around with tablets to get students to complete, will incentivize with water and soda plus a drawing for a tablet.

Other Uses
– student government elections
– surveys for internship supervisors or other outside constituents

Output and Reporting
– pull down data and analyze with SPSS by institutional research

Lessons learned
– don’t duplicate with paper formats simultaneously
– get buy-in from all stakeholders (faculty, deans, provost)
– use proactive marketing by faculty and administration
– use incentives such as extra credit or prize drawing
– use ambassadors to go where the students are with tablets
– send a notice to students warning that evals are coming
– send a notice from provost to faculty asking them to explain the value

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s