League tables: The case against rankings (NST)

By SUZIEANA UDA NAGU
suzie@nst.com.my

UNIVERSITI Sains Malaysia vice chancellor Tan Sri Professor Dzulkifli Abdul Razak

Results of recent league tables have sparked up another round of discussions on the relative merits of global university rankings exercises, writes SUZIEANA UDA NAGU

UNIVERSITI Sains Malaysia vice chancellor Tan Sri Professor Dzulkifli Abdul Razak did not wait with bated breath for the outcomes of the QS World University Rankings 2010 and Times Higher Education (THE) World University Rankings 2010-11 which were announced recently.

The QS and THE lists were released a week apart from one another and Malaysian institutions were nowhere on the Top 200 spots of both.


Dzulkifli does not know what the fuss is all about. As far as he is concerned, the results of both league tables were, at best, predictable.

“It is a new ranking system but that is old hat,” he adds, referring to the revised THE league tables.

THE severed ties with rankings data supplier QS last year and announced its plans to work with research-metrics company Thomson Reuters on revamping its league tables in response to criticisms.


It was slammed in the past for “being based on questionable data and flawed methodology”. The most criticised component of the rankings was “peer review”, which accounted for 40 per cent of the overall score, because only 3,500 researchers had responded.

With a new partner, THE promises that its seventh league table will be different — with clearer and more transparent performance indicators and data.

“Changing partners does not guarantee any significant transformation. Bear in mind that THE-QS had parted ways before (the final separation). I see the change as a business decision based on profitability and commercial interest.


As it turns out the ‘new’ ways are apparently no better than the old ones, given the criticisms levelled at them,” says Dzulkifli.

As educator Koh Soo Ling puts it in her article Why bother?: “Now that the euphoria or the disappointment has somewhat settled, questions remain about the relevance of global rankings” (see H5).

Judging by USM’s non-participation in world rankings exercises as from this year, THE’s recent list is of no consequence.

Dzulkifli says that the university did not receive any invitation from THE to participate in the exercise.

“Even if they were to approach us, we would not be keen — that is our official stand. QS had insisted on ranking us even though we have repeatedly stated our lack of interest in the exercise. That alone makes it suspect,” he says.

Although University College London is placed 22nd on THE’s list, vice chancellor Malcolm Grant describes the activity as “nonsensical” because the application of new metrics and weightings still falls “miles short of capturing the variety, dynamism and diversity of the modern university”, writes Grant in a recent article University world rankings are pointless, UCL president says in The Guardian (http://bit.ly/bxWQQ2).

But THE World University Rankings editor Phil Baty is certain that when universities which have declined to take part see the results and examine the methodology in detail, “they’ll opt in next year”.

Dzulkifli remains sceptical of such schemes.

“The proof of the pudding is in the eating! Until we see the results we cannot be convinced,” he says.

University of Malaya (UM), which also opted out of the exercise this year, will continue to do so until THE responds to its questions.

UM vice chancellor Professor Datuk Ghauth Jasmon says: “We declined because THE did not respond to our questions.”

Although UM concedes that THE’s individual performance indicators combined under five categories — Teaching, Research, Citations and International Mix — “are better”, it disagrees with the “Industry income” component.

Industry income uses data on university earnings from research and knowledge transfer activities.

“This would put universities from poorer countries at an immediate disadvantage,” adds Ghauth.

UM sees the merits of rankings exercises.

“QS World University Rankings have pointed out some (of our) weaknesses and motivated us to review our work culture and aim for better Key Performance Indicators (KPI),” he says.

The formation of the Secretariat for Ranking and Improving Performance at the university two years ago reflects its commitment to “competing globally and to doing well in (rankings exercises)”.

“The unit is responsible for advising me on areas that I need to focus on and address. It also monitors all academic staff performance in terms of ISI journal publications, Citations received and the H-index. This will allow us to benchmark against top universities globally,” he adds.

Ghauth is understandably disappointed that UM was not on the QS top 200 list — it slipped to the 207th spot from 180th last year — despite rigorous efforts to improve its performance.

“We are still studying the cause. However, the university’s management is much happier now in terms of Quality Research and Impact journal publications because there have been marked improvements in this regard in the last two years,” he adds.

Professor A Murad Merican, a strong critic of rankings exercises, cautions local tertiary institutions against missing the big picture and succumbing to being captive minds.

“Mental captivity is characterised by a way of thinking that is imitative and uncritical. The captive mind assumes that the THE list is the ultimate measure of what universities do,” says Murad, who is from Universiti Teknologi PETRONAS’ Management and Humanities Department (see H3).
Dzulkifli agrees.

“We have different aspirations and must find appropriate ways to express these.

It is a mistake to have a one-size-fits-all rankings system as far as universities are concerned. It is, at best, pseudo-science.”

Against this backdrop of discussions on the relevance of league tables, USM is working on an Alternative University Appraisal (AUA) approach that will be embedded into its current evaluation method “as part of a developmental process to meet quality standards predicated on equality, availability, affordability, accessibility and sustainability”.

“The AUA calls for a different approach which is more engaging and meaningful,” says Dzulkifli.