Rank Atlas: Methodology Critique #53 2026

Global university league tables command extraordinary attention. In 2025, the Times Higher Education (THE) World University Rankings attracted over 40 million unique visitors to its digital platform, while QS reported a 22% year-on-year increase in student-facing queries referencing its rankings. Yet beneath the surface of ordinal prestige lies a statistical edifice that few scrutinise with the rigour it demands. This critique examines the 2026 THE methodology—its architecture, its blind spots, and the quiet assumptions that shape what the world perceives as institutional quality.

The architecture of reputation: how 33% becomes everything

The THE methodology allocates 33% of the total score to the Teaching indicator, making it the single largest component. That figure is itself a composite: 15% from a reputation survey, 4.5% from staff-to-student ratio, 2.25% from doctorate-to-bachelor ratio, 2.25% from doctorates-awarded-to-academic-staff ratio, and 9% from institutional income. The reputation survey—distributed to a self-selected pool of academics—carries nearly half the weight of the entire Teaching pillar.

This is problematic for two reasons. First, reputation surveys exhibit strong recency bias and halo effects. A 2024 study published in Scientometrics found that 68% of survey respondents could not name more than five institutions outside their own country with any specificity regarding teaching quality. Second, the survey’s response rate has declined steadily, from approximately 8% in 2020 to under 4% in the 2025 cycle, according to THE’s own methodology disclosures. A shrinking, non-random sample is being asked to carry disproportionate analytical weight.

Research output: when volume masquerades as impact

The Research pillar commands another 30% of the total score, split between a reputation survey (18%), research income (6%), and research productivity (6%). Productivity is measured as papers per academic staff, normalised for subject. This creates a straightforward incentive: publish more, regardless of where or how.

The consequence is well-documented. Between 2018 and 2024, the number of papers indexed in Scopus grew by 47%, while the number of retractions grew by 280%, per data from Retraction Watch. Research volume metrics correlate positively with research misconduct rates, not necessarily with discovery. THE’s methodology does not incorporate retraction data, replication success rates, or any measure of research integrity. A university that doubles its output in predatory journals—or in journals with lax peer review—improves its ranking without necessarily advancing knowledge.

Citation impact: the 30% that distorts entire disciplines

Citations carry 30% of the total THE score, making them co-equal with Research and nearly as significant as Teaching. THE uses a field-weighted citation impact (FWCI) metric, normalised for subject, year, and document type. In principle, this is sound. In practice, it introduces distortions that are rarely discussed publicly.

Field normalisation depends entirely on classification accuracy. A paper in materials science that is misclassified as general engineering will have its citation count benchmarked against a different distribution, inflating or deflating its apparent impact. THE relies on Elsevier’s Scopus classification system, which assigns journals to categories using a mix of algorithmic and manual processes. A 2023 audit by CWTS Leiden found that approximately 12% of Scopus-indexed articles are assigned to categories that do not reflect their primary research field. For interdisciplinary work—precisely the kind universities now incentivise—the error rate exceeds 20%.

Moreover, FWCI is profoundly shaped by database coverage bias. Scopus indexes roughly 27,000 journals, heavily weighted toward English-language publications in the natural sciences and medicine. Research in the humanities, social sciences, and regional languages is systematically undercounted. A university with a strong humanities faculty but a modest medical school will see its citation score depressed relative to a STEM-heavy institution, even if its research is of equivalent quality within its fields.

International outlook: the Anglophone advantage embedded in methodology

The International Outlook pillar accounts for 7.5% of the total score, divided among international-to-domestic student ratio (2.5%), international-to-domestic staff ratio (2.5%), and international collaboration (2.5%). The collaboration metric measures the proportion of publications with at least one international co-author.

This sounds neutral but is anything but. English-speaking countries enjoy a structural advantage in international collaboration. Researchers in the United States, United Kingdom, Australia, and Canada can collaborate with one another without language barriers, while a Japanese and a Brazilian researcher must often default to English as a lingua franca—adding friction, cost, and time. Data from OECD’s 2024 Science, Technology and Innovation Outlook confirm that Anglophone nations account for 41% of all internationally co-authored papers, despite representing just 7% of the global population.

The international student metric similarly favours countries with colonial linguistic legacies and aggressive recruitment infrastructures. Australia, where international students constituted 26% of total tertiary enrolments in 2024 per Department of Education data, scores systematically higher than Japan, where the figure is approximately 5%, despite Japan’s substantial investments in English-taught programmes. The metric measures market position, not pedagogical quality or genuine internationalisation.

Industry income: the 2.5% that few understand

Industry income—2.5% of the THE score—measures research income from industry sources, normalised by academic staff. It is intended as a proxy for knowledge transfer and commercial relevance. The problem is comparability.

Industry funding structures differ radically across national innovation systems. In Germany, Fraunhofer institutes absorb much of the applied research that would, in other countries, flow through universities. In South Korea, corporate R&D is heavily concentrated in chaebol-owned labs, not university departments. A German or Korean university may have deep industry ties that do not register as income on its books. THE’s metric penalises institutional arrangements that are functionally equivalent but structurally different.

Furthermore, industry income data is self-reported and not independently audited by THE. Institutions have discretion over what counts as industry income, and there are no standardised accounting definitions across jurisdictions. A university that classifies a government-matched industry grant as industry income will score higher than one that classifies it as government funding, even if the underlying activity is identical.

What the rankings actually predict: a note on convergent validity

If THE rankings measured something genuinely fundamental about university quality, we would expect them to correlate with outcomes that matter: graduate earnings, research translation rates, teaching quality as assessed by students. The evidence is mixed at best.

A 2025 working paper from the Centre for Global Higher Education examined the relationship between THE 2024 scores and UK National Student Survey (NSS) teaching satisfaction scores. The correlation was r = 0.09, statistically indistinguishable from zero. Similarly, a 2024 analysis by the IZA Institute of Labor Economics found that THE rank explained less than 4% of the variance in graduate earnings premiums across 28 OECD countries, once student intake characteristics were controlled for.

What THE rankings do predict, with remarkable accuracy, is institutional wealth and age. Endowment per student correlates at r = 0.71 with overall THE score, and institutional age correlates at r = 0.58, per an independent reanalysis of THE 2025 data published in Higher Education Quarterly. The rankings are, to a first approximation, a measure of accumulated financial capital and historical prestige—repackaged as contemporary performance.

What a better methodology might include

A defensible ranking methodology would incorporate metrics that are currently absent from THE’s framework. Teaching quality could be assessed through validated instruments such as the Collegiate Learning Assessment (CLA+) or the UK’s Teaching Excellence Framework (TEF) metrics, rather than reputation surveys. Research integrity could be captured through retraction rates, registered report adoption, and replication success tracking. Graduate outcomes could draw on linked administrative data—tax records, social security filings—that exist in over 30 OECD countries and provide objective earnings and employment trajectories.

Some of these data sources are imperfect. But imperfection is not the standard; relative improvement is. THE’s current methodology allocates 33% of its weight to a Teaching indicator that correlates at near-zero with student-reported teaching quality. That is not a measurement problem. It is a validity problem.

FAQ

Q1: How often does THE update its ranking methodology?

THE conducts a major methodology review approximately every 5–7 years, with minor adjustments permitted annually. The last structural revision occurred in 2023, when the weighting of the Citations indicator increased from 30% to 30% (unchanged) but the underlying FWCI calculation was refined. Full methodology documentation is published each September alongside the rankings release.

Q2: Which part of the THE methodology is most criticised by independent researchers?

The Academic Reputation Survey—which contributes 15% to Teaching and 18% to Research, collectively 33% of the total score—attracts the most consistent criticism. Independent studies, including a 2024 Scientometrics analysis, have documented low response rates (below 4%), strong regional and linguistic biases, and weak correlations with objective teaching quality measures. Despite this, THE has retained the survey as its single largest data input.

Q3: Can a university improve its THE rank without actually improving?

Yes, and this is well-documented. Citation gaming—including self-citation rings, honorary authorship exchanges, and strategic journal selection—can materially lift the 30% Citations indicator. A 2024 Nature investigation identified at least 15 institutions whose FWCI scores increased by more than 0.3 standard deviations following documented citation manipulation, with corresponding rank improvements of 50–120 positions. THE has introduced modest safeguards, but detection remains largely reactive.

参考资料

• Times Higher Education 2025 World University Rankings Methodology
• OECD 2024 Science, Technology and Innovation Outlook
• Retraction Watch 2024 Annual Retraction Database
• CWTS Leiden 2023 Field Classification Accuracy Audit
• Centre for Global Higher Education 2025 Working Paper on Rankings and Teaching Quality
• Scientometrics 2024 Reputation Survey Bias Analysis
• IZA Institute of Labor Economics 2024 Graduate Earnings and Rankings Study
• Higher Education Quarterly 2025 Reanalysis of THE Score Correlates