Rank inconsistency undermines university ratings systems

The latest international university rankings have arrived with the winners and the losers there for all to see. But what do these rankings tell us if anything?

The most recent ratings were released last night by Times Higher Education, while last month the QS rankings were published.

The results mismatch completely.

Some 13 performance indicators are used in the latest rankings. They gauge a university’s strengths across parameters such as teaching, research, knowledge transfer and international outlook. This data is collated and interpreted by Thomson Reuters.

The QS system relies heavily on the international reputation of an institution’s academics. It uses indicators such as student/lecturer ratio, publications per faculty and paper citations. These metrics are crunched and a league table is produced.

Upwards and downwards
Yet there is no consistency between the two. Three weeks ago the QS system had Trinity at 61 and moving upwards. Yesterday the Times Higher Education rankings had it at 129 and moving downwards.

Any of the universities with rising placement will crow about the results, while those who saw falling placements will denigrate or dismiss the findings – if they respond at all.

Neither Dublin City University nor University of Limerick issued a statement on the latest rankings, possibly because they didn't make the cut for the top 400 institutions.

The Times rankings describe Trinity’s decline as a cause for alarm, but look at the figures in a different way and you could argue that Ireland has done spectacularly well.

The Irish Universities Association looked at the number of top 200 institutions in each country and ranked them based on head of population. This analysis placed Ireland at seventh in the world given we have two in the top 200 (Trinity and University College Dublin) and a third (University College Cork) challenging to break into that category.

The latest rankings data also highlights Ireland’s performance in this regard, noting there are only 26 countries worldwide with universities above the 200 mark. The top three are the US (77), UK (31) and Netherlands (12). Ireland is joined on two by Singapore, Israel and China. Countries with one top-200 institution include Austria, Finland, New Zealand, Norway, South Africa, Spain and Taiwan.

Full picture
The Irish Universities Association says rankings such as these can never provide a full picture of the quality and diversity of higher education.

The association says there are biases – for example scientific research activity is valued ahead of humanities research – but success in the rankings can come down to money.

“In short, winning the rankings game requires a fat cheque book,” it says.

This of course does not mean the Government escapes the need to invest in a credible way in Ireland’s higher education system.

Investment in people and in world-class facilities will help Ireland perform better against the very metrics used by Times Higher Education and by QS to assess the quality of a university.