Do College Rankings Matter? See What They Measure

Do College Rankings Matter? See What They Measure

Do College Rankings Matter? See What They Measure 1000 1000 Theo Wolf

Do College Rankings Matter


In a recent episode of his popular podcast Revisionist History, Malcolm Gladwell took aim at a constant thorn in the side of anyone who works in the college admissions space: The US News and World Report top college rankings. For those who don’t follow Gladwell or his podcast, in each episode he tackles a different element of society and questions the generally accepted assumptions that surround it. As with much of his work, he can tend toward being reductive, but he always raises good questions and interesting points. The two episodes he devotes to college rankings are no exception. So do college rankings matter? What do they actually tell us? Let’s take a look.

In his episode, Gladwell delves into how the US News formulates its ranking. What goes into the ranking is not secret, US News is transparent about this on their website. However, their formula, how they calculate the actual ranking, is kept under lock and key. But as Gladwell details, some students at Reed College created their own model that appears to be a fairly accurate predictor of US News College Rankings.

How College Reputation Is Calculated

Gladwell then chooses to focus on poking a hole in one specific aspect of the US News College Rankings, the reputation score. This accounts for a whopping 20% of a school’s score (more than any other single metric)! Considering how much weight this metric is given, it must be calculated pretty scientifically, right? That’s the assumption one might go in with, and Gladwell is quick to prove it wrong. Surveys are sent out to high-ranking college officials asking them to review all of the other universities in their category based on their knowledge of those schools. These administrators have never attended most of those schools, and have maybe never even visited them. They’re essentially asked to pull this metric for “how good” other schools are out of vacuum.

So do college rankings matter in terms of a college’s reputation? Often, as the college president that Gladwell interviewed admits, they end up just using the US News College Rankings itself to make their determinations, which results in a self-fulfilling prophecy. Maybe this is why only 36.4% of those who were sent the peer assessment survey ended up filling it out in 2020.

Imagine a ranking of the best restaurants in the country where the scores come from only the head chefs of other restaurants, but those chefs have never eaten at any of the restaurants they’re being asked to rank. Sounds problematic, right?

US News Rankings Prioritize Wealth Over Diversity

Aside from the reputation score, Gladwell mentions the weight of the SAT in the rankings (5%) which can also have issues. The SATs have been shown to be strongly correlated with family income, so incentivizing high SATs (this is one of the key reasons colleges care so much about SAT scores) leads to schools prioritizing higher income students over those lower income or first generation students who are less likely to have high scores.

Beyond those two elements, Gladwell doesn’t talk too much about the other pieces of the rankings. (Though he does call out graduation rates and financial resources for students in passing, which do also advantage wealthy universities and incentivize admitting wealthy students.) But there are many other factors that he easily could have targeted. For instance, 3% of the score comes from the percentage of alumni who donate to the school. US News claims that this measures “student satisfaction and post-graduate engagement.” Yet, does it really do that? Do college rankings matter in measuring how well the alumni and student community love the school?

As a Cornell alumnus, I haven’t given money to the university since graduating. I feel that I paid my dues when I paid for my tuition. Plenty of my friends feel similarly, especially if they’re paying off student loans. It doesn’t mean I didn’t love the school. But someone who comes from an extremely wealthy family might feel differently. This situation, once again, incentivizes the school to accept those who are going to be more likely to donate money. Are you sensing a trend here?

Failing to Capture Class Size

Class size is another interesting factor that Malcolm doesn’t mention but accounts for 8% of the weighting. This one is a little more complicated. I loved being able to participate in small classes. Many of these classes allowed me to develop strong relationships with my professors and have a fantastic learning experience. And in theory, small class sizes are great, but they can be difficult to measure.

US News College Rankings uses a sliding scale which rewards classes that have under 20 students in them, gives fewer points if they have over 20, and none if they go over 50. The challenge is, there’s no tangible difference in a class of 21 vs. 20. But in the US News, 20 is vastly superior to 21. Additionally, it makes no distinction between a class size of 50 and a class size of 1000. My “Introduction To Sociology” class had over 200 students, and it was a completely faceless experience. But my “Introduction To Musical Theater” class had 60 students, and my professor knew every student’s name. This arbitrary cutoff encourages colleges that prioritize rankings to maximize their score here, potentially to the detriment of students. For instance, take two potential schools:

  • University A offers seven classes with 20 students and three classes with 500 students.
  • University B has five classes with 20 students, two classes with 30, two classes with 50, and one class with 300

Which school would you rather attend? Personally, I’d go with University B, which will give a much more personal experience. Yet in US News, University A is superior.

US News College Rankings Punish Socioeconomic Diversity

In Gladwell’s second episode, he explores what would happen to a school if some of its fundamentals changed completely. Specifically, he looks at Dillard University, a historically black university (HBCU) in New Orleans. He asks what happens if Dillard suddenly grows a massive endowment and changes the type of students it accepts to be wealthier and higher scoring and, surprise surprise, it jumps to near the top of the rankings.

Gladwell’s exercise is an interesting thought experiment, but it’s purely hypothetical and not particularly useful. Do college rankings matter in terms of socioeconomic diversity? No, we don’t need a hypothetical to see the problems with the rankings. Schools are incentivized by the rankings to accept high-income students with good SAT scores who are likely to graduate and who create more tuition dollars for the school. If colleges actively work to increase the socioeconomic diversity of their class or to expand their class sizes (as Cornell and Rice have been doing recently), they can be punished by the rankings.

College Rankings Also Poorly Capture Social Mobility

To the credit of the US News College Rankings, they recently added “social mobility” as a measure accounting for 5% of the score, but it’s extremely limited in scope. This factor measures graduation rates of Pell Grant recipients (typically the lowest income students on campus), but it doesn’t take into account how many students are from this group. If University A accepts 2 Pell Grant students and both graduate and University B accepts 1,000 Penn Grant students and 900 graduate, US News favors University A over B, yet again.

There are plenty of other factors that go into the ranking, including school resources and graduate indebtedness, but even those can be flawed. All in all, as you can see, the US News ranking is measuring a wide swath of factors and many of them fall apart when subjected to scrutiny, especially if you’re hoping to find the “best” universities.

Okay, so how should I use this information as a student or parent?

We know the US News College Rankings are always going to be important to our students and their families. To answer the question, do college rankings matter? It depends. Employers can care about them. This is especially true for students who are located internationally, where brand recognition becomes all the more vital. So that’s exactly how we recommend thinking about rankings: as one factor of many (including base starting salary, salary 10 years after graduation, and more) as you think about employment outcomes of the school. Most crucially, you should stop conflating the US News Best National Universities rankings with the actual educational quality of a school.

The Best College Ranking Site

So how can you judge educational quality? Nothing beats doing a deep dive into each individual college’s departments, course offerings, and professors to figure out the best fit for you. But if you do feel you need the kind of “objectivity” that rankings offer, we have two recommendations.

First off, we recommend the Wall Street Journal/Times Higher Education ranking. It’s still flawed, as all rankings are, but it is grounded in more objectivity and is harder to game. Much of it is based on the student experience. To break it down, 20% is student engagement, measured by surveys answered by people who have actually attended the school. Meanwhile, 30% comes from resources (including student to professor ratio, which we like more than the class size metric). Only 10% is based on reputation (vs. US News’ 20%). Again, it’s not perfect, but we like it as an alternative.

Using the WSJ/THE College Rankings can also allow you to see schools that other people who are relying on US News might overlook. For instance on the WSJ/THE rankings, UC Davis, at 38 is only 4 spots under UC Berkeley at 34, but has a much higher acceptance rate. Michigan State at 82 is ranked 4 above Northeastern at 86, and similarly is significantly easier to get into. There is also a lot of overlap between the two rankings, so you can get some sense of which schools have worked to specifically climb the US News Ranking.

Do College Rankings Matter for Individual Academic Departments?

Additionally, we actually do recommend a US News ranking, just not the US News one that you’re used to. Beyond its National Universities and Liberal Arts rankings, US News also tabulates a number of rankings related to specific departments. These are based purely on peer assessment, but instead of asking administrators about schools generally, they are relying on department chairs and professors who regularly interface with other academics in their field at conferences and reviewing research. These faculty members have their fingers on the pulse of where the best research and innovation is taking place, and are being asked very specifically about quality in their field rather than about university quality as a whole. As such, we find that these program-specific rankings can be a reality gauge for students to better understand what schools excel in their areas of interest.

US News does not compile departmental rankings for all fields, but you can find them for Computer Science, Business, Engineering (both at colleges that offer PhDs and those that don’t, as well as within various specific disciplines of engineering), and a wide range. Where they don’t offer undergraduate rankings, the graduate program ones can sometimes be insightful as well, such as in social science and the humanities. Some others I like are Writing Across the Disciplines and Undergraduate Research/Creative Projects.

Our Advice: Focus on Programs Over College Rankings

What we’ve found with our students is that those who prioritize these individual department rankings over the overall school rankings tend to have much better experiences, with their applications, in college, and after. During the application season, they are usually less stressed because they are prioritizing their process differently. Rather than purely gunning for the most highly selective schools, they focus on those that are best for their area of interest, many of which are less selective. If you’re unsure of how to start this process, take a look at our article on how to determine whether a college is a good fit for you.

We’ve had students choose University of Illinois over University of Southern California, University of Michigan over Cornell, and Drexel over Carleton. Do college rankings matter here? Not as much as other factors. When those students get to college, they succeed because they know why they’re there. They find professors in their area of interest, receive great mentorship, and build a path to a great career in their field. Of course, if you’re undecided, this can be harder to do, but in my experience undecided students typically have a few areas they’re interested in exploring—so find a school that does well with those areas!

And perhaps most important of all, don’t go it alone! We recommend working with a trusted advisor who can point you toward schools you may have overlooked and help you to develop the strongest application you can.Not sure how to do college research? Need more college application support? Talk to us to learn more about our programs.