When I was taking classes at UC Irvine to earn my Independent Educational Consultant certificate, my classmates and I held many differing opinions. However, there was one thing we all seemed to agree on: a strong dislike for the U.S. News & World Report Best Colleges rankings. Begun in the 1980s as a way for U.S. News to distinguish itself from Time and Newsweek, the publication of the annual rankings has grown into a highly anticipated event.
U.S. News uses a system of weighted categories to rank colleges and universities. The categories include:
- Outcomes (40% of the ranking): student indebtedness, graduation and retention rates, social mobility
- Faculty Resources (20%): class size, student-to-faculty ratio, average faculty salary, proportion of faculty who are full-time and have earned terminal degrees in their discipline
- Expert Opinion (20%): reputation ranking by presidents, provosts and admission deans
- Financial Resources (10%): academic spending per undergraduate
- Student Excellence (7%): ACT/SAT scores and high school class rank
- Alumni Giving (3%): percentage of bachelor’s degree graduates who donate to their institution each year
How did U.S. News come up with these categories and the weights assigned to each? No one seems to know. In fact, the magazine goes out of its way to keep its algorithm a secret. In an episode of Malcolm Gladwell’s Revisionist History podcast, Gladwell tries to figure it out. He speaks to Peter Bernstein, who was a managing editor at U.S. News during the 80s. In a startling Freudian slip, Bernstein says, “A lot of people have been critical of the rankings and whether this mythology really, this methodology…”
So, are the rankings based on method or myth? Kelly McConville, an assistant professor of statistics at Reed College, asked her students to try to find out. Reed hasn’t supplied data to the U.S. News rankings for years, believing them to be flawed. The magazine, however, continues to rank Reed, and its administrators noticed the college’s rank kept declining. Reed’s administration asked McConville “to recreate a model to see what our [Reed’s] ranking would be if we supplied data. Was Reed being punished with a lower ranking because they chose to opt out?”
According to Gladwell, the algorithm McConville’s students came up with was so good that “when they ran it, their list looked almost exactly like the U.S. News list, with one exception: Reed College.” In 2019, U.S. News ranked Reed 90th among liberal arts colleges. The Reed algorithm placed the college 36th.
The biggest wildcard in the rankings seems to be the Expert Opinion category that counts for 20% of a college’s overall score. Also referred to as the Peer Assessment score, it’s a measure of reputation. The fastest way to move up in the rankings is to increase that score.
According to Robert Morse, Chief Data Strategist for the U.S. News rankings, three surveys are sent to each school. The president, provost and dean of admissions are asked to rate each college on the list on a scale of 1 to 5. If you’re the president of a liberal arts college (LAC) you’re asked to rank over 200 other LACs. If you’re the head of a large university, you’re asked to rank over 300 other big schools.
While it may make sense to ask the presidents of large universities to rank other large universities, how much, for example, do the presidents of different religious institutions really know about each other’s schools? Gladwell wonders, “what does the Rabbi who runs Yeshiva University [a Jewish college] know about Brigham Young University [Mormon]? And, what does the Jesuit priest who oversees Gonzaga [Catholic] know about Yeshiva?” Might Gonzaga’s president rank other Jesuit colleges highly because he believes whole-heartedly in the strength of a Jesuit education? Or, more cynically, might he give fellow Jesuit institutions a low ranking in hopes that Gonzaga will rise to the top of the Jesuit schools?
It’s interesting to hear what college presidents have to say. Ali Houshmand, president of Rowan University (ranked 187th) tells Gladwell, “I can only speak for myself. I can assure you that I really don’t know much about the institutions that I’m asked to rank. I just don’t have enough tools.” Even the president of Princeton University, which U.S. News has ranked #1 for the past 11 years, isn’t a fan of the rankings, as described in a previous blog post.
The U.S. News rankings are based on methodology. But, the importance and value placed on them is nothing but mythology. When looking for the right college, the most important factor should not be its national ranking or reputation, but whether it offers the elements which make it a good fit for you.
