April 27, 2009
Incentives for filling out teaching evaluations?
The University of Michigan switched to an online course evaluation system this year (Fall 2008 was the first full run).
One of the primary concerns during the study that went into the decision, and during the implementation, is what would happen to response rates. Selection bias for respondents is a concern for any survey, and there are obvious (and documented) reasons to expect that response is correlated with student satisfaction with a course and a teacher, which definitely would create a bias, the more so the lower is the response rate. With the traditional fill-in-a-form system, if administered during a class late in the semester when attendance is high (perhaps because students are getting concerned about what will be on the exam!), through convenience and peer pressure the response rate can be fairly high (as I recall, the norm at UM was above 70%). With an online system, evals are filled out at the student's convenience. This might catch students who would have missed the evaluation class day, but might lose many others.
The evidence UM collected from other campuses (and from two schools at UM that had already implemented their own online system), response rates do tend to be a bit lower, though not much.
My ICD colleague, Yan Chen, and I have chatted a bit about providing incentives to students to complete the online evaluations. One idea that she'd heard seemed effective in another department is to award a nominal amount of "extra credit" to students who submit an evaluation (in some systems, even though the evaluation content is anonymized, it is possible to track whether a student submitted something).
Here is a blog entry on the use of course evaluation incentives from the Center for Teaching, Learning and Technology at Washington State University. Their impression is that the incentives (including extra credit and randomized cash awards) have had little impact on participation (at least at the level at which the incentives were offered). Their conclusions are mostly based on anecdotes, but they provide a link to a discussion of a comparative study that was done in their College of Engineering which, it appears, was being submitted for publication.
Posted by jmm at April 27, 2009 01:36 PM