Posts
RANKINGS expert Simon Marginson has called on leading universities to band together and expose the weakness of well-known rankings, including the Times Higher Education and QS league tables.
Professor Marginson told a meeting of senior executives from Pacific rim universities that some rankings had little relevance to the real world, were open to manipulation and were "based on bad social science".
"These rankings get a lot of airplay. In social science terms they are rubbish," he told the Association of Pacific Rim Universities meeting being held at Australian National University on Monday.
Professor Simon Marginson, One of the Great Education Thinkers
Professor Marginson, who takes up an appointment as professor of higher education at the University of London at the end of the month, said there was no point calling for a boycott because "they don't work".
"What we should collectively do, in my view, is start to critique and discredit the bad social science at the base of multi-indicator rankings," he said. "We are universities; it is not hard for us to say what is good science and what is bad. We need to push at bad ranking methods or at least weaken their legitimacy."
He told the Canberra meeting that threats by QS to sue him, and the predilection of governments to use rankings as a proxy for quality, made speaking out even more important.
He said the results of questionable rankings "slide in all directions" from year to year because they mix survey and objective data, and adjust arbitrary weightings.
"The link back to the real world is over-determined by indicator selection, weightings, poor survey returns and ignorant respondents, scaling decisions and surface fluctuation that is driven by small changes between almost equally ranked universities," he said.
"The rankers shape the table, not the real state of the sector - or not enough. There is scope for manipulation in conversations between the universities and the rankers."
Professor Marginson, who has been a vocal opponent of survey-based rankings for years, sits on the advisory board of the THE. While that ranking was superior to QS, it was still fatally flawed once outside the top 50 universities, he said.
Representatives from both rankings agencies jumped to the defence. Phil Baty, rankings editor with THE, said: "Thomson Reuters carries out all the data collection and analysis, (meaning) that there is no scope for any manipulation, as Simon suggests, in any conversations between the rankers and the universities."
Ben Sowter, QS head of research, said the ranking was open to scrutiny and went "to great lengths to inform people what it is not (and) what it is".
"Individual indicators can be sorted and accessed separately for those who prefer not to accept the blend of factors we and our advisers have devised.
"Ours is the world's largest survey of academics with the 2013 index built on over 62,000 responses gathered over three years."
A leading authority on institutional league tables, Professor Marginson also serves on the advisory board for the Shanghai-based Academic Ranking of World Universities.
He said the Shanghai ranking, as well as the University of Leiden and the Scimago league tables, used "objective, externally referenced, solid metrics though we can complain about (the counting of a university's) Nobel Laureates".
Source: http://www.theaustralian.com.au/higher-education/ranking-of-unis-is-bad-science-says-simon-marginson/story-e6frgcjx-1226740536595#sthash.MeWA9TE7.dpuf
