Harry Potter and the Draft Engagement and Impact Guidelines

This week the Australian Research Council released for consultation their draft guidelines for the evaluation of university research engagement and impact. The engagement part of the evaluation is mainly quantitative with a shortlist of indicators around research income from industry and end-users. The impact part of the evaluation is mainly qualitative with research impact case studies providing a narrative around the benefit that university research is having outside of the university sector - including the ways that universities are fostering translation and impact from their research.

Some interesting takeaways from the draft guidelines include:

• A May/June 2018 submission deadline (which follows directly behind the ERA 2018 deadline)
• A maximum of 25 impact case studies per university which includes 23 disciplinary case studies, 1 interdisciplinary case study and 1 Aboriginal research case study
• The introduction of a low volume threshold of 150 weighted outputs (books weighted x5) over which a university must submit information and below which a university may opt-in if they so wish
• A new three point rating scale for impact (high, medium, low) which seems more sensible than the pilot ratings (mature, emerging, limited)
• Impact case studies will now receive 2 ratings each - one for the approach to impact and another for the impact itself

Adding to the sector's resource burden in complying with research evaluation is the introduction of two engagement narratives: one is an engagement indicator explanatory statement of 4,500 characters to accompany engagement indicators and the other is a 7,000 character engagement narrative to accompany each unit of assessment. Now seeing as each unit of assessment is the 2-digit field of research this results in a considerable increase in work for the sector. In ERA 2015 there was a total of 656 2-digit FORs evaluated - so if each one of these is accompanied by a 4,500 character explanatory statement and a 7,000 character engagement narrative this equates to around 7.5 million characters, or around 1.2 million words - for comparison, the entire series of Harry Potter books contain around 1.08 million words.

You can see the guidelines for yourself at the ARC website here.

Gender imbalance in scientific research

In a recent article in Nature researchers have used bibliometrics to highlight the gender disparity in the publication output within the sciences around the world. The study reveals that female scientists are publishing less volume than their male counterparts and that their publications have a lower citation impact as well. The authors touch on a number of well documented imbalances between the genders in the sciences including funding, earnings, hiring and patenting.

One of the issues the researchers had to get around in their study was how to determine gender of the authors of journal articles indexed by Thomson Reuters in the Web of Science. They used a combination of sources to match the name to a gender including social security databases, Wikipedia and even Facebook (the interesting methodology can be read in their supplementary material).

It would be interesting to see the same data normalized for funding. In many cases funding agencies are awarding more money to male scientists than to female scientists (e.g. from the ARC's website for Number of participants on all funded projects Male = 2280 and Female = 622). It would be interesting to know whether perhaps female scientists were achieving "more with less" when it came to publication output compared with funding and opportunity.

While there is no single answer to the problem these researchers are describing they do make a good point about improving the ability for female scientist to travel and collaborate internationally:

"For a country to be scientifically competitive, it needs to maximize its human intellectual capital. Our data suggest that, because collaboration is one of the main drivers of research output and scientific impact, programmes fostering international collaboration for female researchers might help to level the playing field."

Researcher mobility in all fields is a good strategy for any organisation that can afford it - and is certainly critical for any developed nation's research strategic plan.


http://www.nature.com/news/bibliometrics-global-gender-disparities-in-science-1.14321