Friday, June 11, 2010
Highly credentialed and experienced professors are better at
preparing students for long-term academic success than their
less-experienced counterparts, but that ability isn't necessarily
reflected in their students' teaching evaluations. That's according
to research by a pair of economists published in this month's
Journal of Political Economy.
The study's authors, Scott Carrell of U.C. Davis and James West of
the U.S. Air Force Academy, say their results raise questions about
the value of student evaluations as measures of instructor
quality.
Student evaluations are widely used by colleges in tenure and
promotion decisions, but Carrell and West considered a different
measure of instructor quality. They looked at how well instructors
in introductory courses prepare students for more advanced courses
in related subjects.
Their data come from Calculus I and follow-on classes at the U.S.
Air Force Academy. All Air Force Academy students are required to
take Calculus I, Calculus II, and nine math-based technical courses
regardless of their majors, and professors in all sections of
classes use an identical syllabus and give identical exams. That
gives the researchers a chance to compare instructors on a
relatively even playing field.
The study found that students' achievement in follow-on coursework
was strongly influenced by their Calculus I instructor. Students
who had a seasoned Calculus I professor with a Ph.D. tended to do
better in follow-on coursework than students who had
less-experienced and less-credentialed Calculus I instructors. This
happened despite the fact that students of seasoned professors
tended to have lower grades in Calculus I. The results, the
researchers say, suggest that less experienced instructors have a
tendency to "teach to the test," while more experienced teachers
produce "deep learning" of the subject matter that helps students
down the road.
The findings weren't a result of newer professors being
"easy-graders," because the Calculus I course was designed to
remove as much instructor discretion as possible from their
student's grades. Midterm and final exams are group graded, where
one instructor grades a single question for the entire course to
ensure uniformity of partial credit.
The deep learning produced by more-experienced instructors was not
reflected in their students' teaching evaluations, the study found.
Less-experienced instructors—whose students tended to do better in
the short-term but worse in later classes—received higher ratings
on student evaluations. For example, the instructor who ranked dead
last in "deep learning" in the sample of 91 Calculus I instructors
ranked sixth best in student evaluations.
Taken together, the findings imply that student evaluations give
instructors—especially those who do not have tenure—incentive to
teach in ways that "have great value for raising current scores,
but may have little value for lasting knowledge," the authors
conclude.
###
Thanks to University of Chicago Press Journals for this
article.