Everyone hates student course evaluations. They don’t measure what students are learning or even how much good teaching is occurring (although it is probably not good to have persistent high GPAs and low student evaluations). At best, they tell us what students perceive.

It also turns out that most student evaluation forms have one predictive question, usually some version of “Would you recommend this course to your friends?” or “Your general assessment of this teacher?” All the rest is commentary.

I find it can be useful to see if a professor gets consistently low marks for “supportive of diverse ideas in the classroom,” but “How prepared was your professor for class?” or “Was the professor knowledgeable?” are completely useless.

There are great new survey tools, with tested questions,  and these should be required reading for anyone thinking of redoing their course evaluations, but student course evaluations should mostly be about improving teaching and retention.

1. Your course evaluations should reflect your institutional student goals. Reminding both faculty and students in every course will help bring focus and integration to key campus learning outcomes.

If you want to be the creativity place, then every course should ask “How much did this course improve your ability to think and work creatively?” If you want to improve the critical thinking of your graduates, the easiest and cheapest way is simply to remind students (and faculty!) that they do this in every course. Naming what we do is itself a pedagogy that works.

Every degree and department should have learning outcomes and these too should be reflected in every course evaluation: “How much did this course improve your ability to manage a business?” “How much did this course contribute to your ability to solve complex problems?”

Not every course is aligned with a single student learning outcome, and administrators will need to be sensitive to low scores in courses designed with a different purpose, but having a small and focused list of outcomes and evaluating them in every course is an extremely low cost way of improving focus.

2. Your course evaluations should reflect high impact practices and research-based pedagogies.
“How much did this course expand your appreciation for diverse ideas?” “How much did this course expand your ability to collaborate and work with others?” While these again are perceptions, they are perceptions that matter. I should also be measuring students’ actual improvement in self-regulation or motivation, but I also want to know what they think: “How much did this course expand your ability to think about your own thinking?” “How much did this course bolster your motivation to succeed in college?”

If we want faculty to make more use of active learning then asking “What percentage of class time did you spend in active learning?” (or negatively, “in passive listening?”) This will vary in different types of courses, and again, administrators will need to be sensitive to different goals. Most studio, lab and art courses will score high here, but that is not a measure of their relative quality. Still, if faculty think we lecture only 40% of the time, and students say it is 80%, this is important is good to know.

(I will note that in more than a decade of reading tenure files and sitting through P&T meetings, I find that most faculty and administrators are highly sensitive to the problematic nature of the numeric part of course evaluations. Institutions need to avoid averages, but I have never seen a tenure case fail only because of low numbers. I also think we should use course evaluations primarily for development—in the same way we should use assignments and assessments for how they can help students. I know we credential eventually, but if I wanted to spend my life sorting, I would have joined the postal service.)

If we are trying to improve teaching at our institutions, then maybe we should ask “How useful was the feedback for improving your work?”

None of this is a substitute for real assessment of learning outcomes. We need better tools to measure the actual critical thinking, creativity and cultural sensitivity of our students. At the moment, we often use the poor surrogates of grades or distribution (you took an art class or studied abroad). Requiring something only tells us about quantity or exposure (100% of students were exposed to science or passed a test in a foreign language–—yeah us!) but we need to understand about the quality of student learning. In the meantime, we should keep doing everything we can to improve learning and provide incentives for faculty to try proven pedagogies: course evaluations are low-hanging fruit.

In the end, you are what you measure. When we measure and assess, we set values and priorities. All colleges measure, but what we measure is often irrelevant. Measuring for accreditation will do little to improve our graduation rates. While it is true that much of what we do is hard to measure, or at least hard to quantify, that should not deter us. I’d rather have high standards and improving assessments, but we are the people of judgment. We evaluate all day long. We just need to turn more of our attention to better evaluation of ourselves.

Rankings in higher education don’t measure quality. Mostly they measure the status or brand of a college, so more and better applicants means a higher rank. Beyond the historical elites, most of what the public knows about colleges comes from football and basketball, both of which are essentially advertising programs for colleges. This is part of why you see so much investment here: a more visible sports program does increase applications, and often substantially.

Schools also tout their alumni networks, where the big state schools will always have more quantity and the elite privates more quality. A Harvard or a Texas transcript is about status—neither says anything about what students actually learned.

Students with the right grades and SAT scores in high school can get into a range of good colleges. The best strategy for poor students is to attend the best college they can afford: in other words, most colleges will offer a steep discount to students with better SAT scores than their current students. Students, though, often want to go to the highest ranked college to which they were accepted, and parents will either pay or borrow more money to fill these dreams. This is where colleges make money.

The average debt for graduating seniors is now almost $30,000.  When graduates borrow more money, it is often because of want, not need. At least some of the problem with college debt is because parents and students don’t want to attend a less prestigious school.

So is it worth it? Is your Ivy-league education worth $100,000 or $200,000 more than your state school?

The status of an elite degree surely has some additional value in the job market, but parents increasingly want to know the return on investment. There will always be some parents willing to pay for bragging rights, but more parents will want to know how much learning per dollar you offer. Is there really ten times more learning at a four-year college or is it just ten times the price? Will a Nobel Laureate (or her TA) really be a good teacher for introductory economics? The “draft” plan at Johns Hopkins Krieger School of Arts and Sciences, to reduce the number of graduate students (but pay them higher stipends) has drawn criticism for the suggestion that with fewer grad students to teach and fewer TAs, faculty might need to teach 3 undergraduate + 1 graduate course a year instead of 2+2.

Will a course on a college campus have more interaction than an online course, or will the teacher just stand and deliver? Colleges need to be prepared to answer these questions and how they relate to pricing and value.

University pricing resembles wine pricing: there is enormous price variation in the market, but it is difficult to tell how different the actual products are. Both wine and higher education are subject to complicated rating systems and a belief that the experts can truly tell the difference, but the abundance of exceptions seems to undermine the basic value proposition. With more than 6000 blind tastings as evidence, the relationship between wine ratings and price is small and actually negative: on average even wine experts enjoy more expensive wines less when they do not know the price! It does not matter if expensive wine is not actually better, but it tastes better if we think it costs more.

Similarly, it ultimately does not matter if a student learns more with an Ivy League education. As long as everyone else believes it is better, it will be in high demand and continue to bestow genuine benefits. In the same way that a bottle of wine is about the quality of the experience, a college degree, in the current market, is really more about buying a credential or a degree than it is about buying learning. Both consumers and providers have been willing to continue their shared misconception, since little else could justify the massive price variation in both wines and education.

Still, value will be an increasingly important proposition in the college marketplace. Parents and students already make decisions based upon cost. As competition increases and accountability provides easier ways of comparing outcomes, liberal arts colleges especially will need to justify the added expense. Knowledge is now freely available on the internet, but that only increases the value of being able to find, sort, analyze, evaluate, and synthesize that knowledge. Colleges talk about critical thinking, but we need to do more to ensure we are increasing the complexity of the mental models of our students. Employers are clamoring for graduates with better communication and thinking skills, ethical reasoning and the ability to solve complex problems. That value will only increase.

The liberal arts are under attack largely because they are misunderstood. College is the best opportunity in your life to expand your world view, think about your own thinking, expand your imagination, focus your gaze, and most importantly to prepare for the unknown. That can be delivered in different ways and at different costs, but that is our value: preparing students to explore the unknown, where they will be required to analyze new information, reinterpret what they know and maybe even, change their minds.