Forum to discuss evaluation of research at Australian universities www.researchimpact.com.au

I have just launched a new forum called Research Impact for those who would like to discuss the evaluation of university research activity, quality and impact.

Feel free to drop in on the forum if you are interested in discussing topics like the Excellence in Research for Australia (ERA); the evaluation of research impact or the 'impact dividend' or maybe you are interested in discussing the growing number of article retractions in the sciences.

www.researchimpact.com.au

Does ERA 2015 present a gender equity issue for the higher education sector?

Academic staff employed on fractional-time contracts may be disadvantaged in the ERA by the requirement for employees on less than 0.4 FTE to have published with their new university's byline within the already passed ERA reference period.

A recent article in The Australian highlights the 'churning' of academic staff that is occurring in the sector to coincide with the ERA census date (31 March). Researchers who move (poached) to a new institution by the census date have their entire publication track record counted towards their new employer's ERA submission. This is despite the fact that the work was likely supported by their previous employer. The fact that researchers take their performance with them for the ERA results in a lot of 'horse trading' (or churning) between institutions  around the census date as they try to maximize their ERA outcomes. This means that high performing researchers, or often now groups of researchers, become a valuable commodity and can no doubt leverage this newly discovered value to their advantage through negotiations with potential employers.

However, as is often the case with these sorts of initiatives, the ERA guidelines are not quite as simple as this. There is actually a guideline that requires the FTE of the newly employed researcher to be greater than 0.4 FTE. If the contract is greater than 0.4 FTE (e.g. 1.0 FTE) then the researcher and all of their publication activity is automatically eligible for the ERA submission. BUT, if the contract is for less than 0.4 FTE there is an extra requirement that the researcher must show a "publication association" with their new institution, for example with a by-line that states the new institution as their affiliation. Now as the ERA is a retrospective evaluation it is highly unlikely that a new researcher employed close to the census date would have had time publish something listing the new university as their affiliation. With a less than 0.4 FTE contract and no affiliation byline with their new employer suddenly none of their performance counts for the ERA. In ERA terms this then makes full time researchers much more valuable than fractional researchers - and, therefore universities will likely employ full time researchers close to the ERA census date rather than look for fractional appointments.

So why is this inequitable? The process makes full-time staff more attractive to a university around the census date and fractional staff less attractive. So while full time researchers can personally and professionally benefit from the head-hunting, horse-trading and churning that goes on - the fractional researchers may not. The national higher education staff data collection shows that female academics make up a higher proportion of the fractional appointments in the sector than males (for research-only academics the ratio is almost 2:1). It is likely then that not only will fractional researchers suffer because of this but that female academics will be impacted in greater numbers than males.

How much is my publication worth to the university?

The Australian Government funds university research through the Australian Research Council (ARC) and National Health and Medical Research Council (NHMRC) as well as through the research block grant schemes. Research block grant funding, which is now managed by the Department of Education, currently consists of four performance based programs to support research and research training in Australian universities. The four programs are:

• The Sustainable Research Excellence (SRE) initiative;
• the Joint Research Engagement (JRE) program;
• the Research Infrastructure Block Grant (RIBG) scheme; and
• the Research Training Scheme (RTS).

Universities report research performance measures such as publications, research income, HDR load and HDR completions to the Department on an annual basis. These data are used in the calculations for distributing block grants funds to the sector. Publications factor into two of the funding schemes and contribute to the distribution of 10% of the JRE and 10% of the RTS. The fact that these data and the calculation methodology are publicly available means that with a simple calculation a dollar value can be placed upon a research publication.

This table shows that in 2002 each publication point was worth $2,928. In fact a book would have been worth a whopping $14,640 as they are weighted at 5:1 over other publication types. However, this value has rapidly decreased from $2,928 in 2002 to a current value of only $1,745 in 2014 ($8,725 for a book). So, if you are publishing research at a university and you are wondering how much your publication is worth in dollar terms, now you know, $1,745 - BUT, this is for a sole authored paper of course, and if you have co-authored with someone outside of your institution this value halves to $872; co-authored with two others? $582 and so on.

I was shown a paper the other day with the title: "Inclusive Search for Supersymmetry Using Razor Variables in pp Collisions at sqrt[s]=7 Tev"; this paper had 2196 authors listed on it and one of them was from an Australian university. This means that this highly collaborative research paper was worth a grand total of 79c!

Draft ERA 2015 Journal and Conference Lists

The ARC have released their draft ERA 2015 Journal and Conference lists.

Draft ERA 2015 Journal and Conference Lists

Public Consultation on the Draft ERA 2015 Submission Documents

The Australian Research Council (ARC) is currently seeking public feedback on the Draft ERA 2015 submission documents.

One of several changes being suggested in the new guidelines is the addition of output volume information to the published ratings (Outcome Reporting - see section 4.2.3). This means that for evaluated fields of research the ARC will not only publicly report the ERA rating (5-1) but also the volume of outputs aligned to that rating. In the past rounds of ERA the evaluation has highlighted 'excellence wherever it occurs' meaning that a small university could score an ERA rating '5' in the same field of research as a larger university and these were seen as equivalent. This suggested change to the reporting of outcomes will mean that there will be a grading or stratification within ratings - for example an ERA rating of '5' with only 50 outputs will no longer be seen as equivalent to an ERA rating '5' in the same field with 500 outputs; or, in fact, may also be seen as inferior to an ERA rating '4' with 500 outputs.

Another change noted in the consultation documents (but not specifically highlighted so you have to dig to find it) is the slight change to the 'reassignment exception'. The new "warning" in the business rules document states that "The reassignment exception must not be used to more more than 40% of the apportioned journal articles into a single UoE". This may be an attempt to put a stop to the 'gaming' that some people think is occurring in the sector. This new rule will now mean that in a single field of research at least 60% of the journal articles must be assigned a field of research as stated on the ARC's journal list

The public consultation can be accessed by the following link:

Public Consultation on the Draft ERA 2015 Submission Documents

Gender imbalance in scientific research

In a recent article in Nature researchers have used bibliometrics to highlight the gender disparity in the publication output within the sciences around the world. The study reveals that female scientists are publishing less volume than their male counterparts and that their publications have a lower citation impact as well. The authors touch on a number of well documented imbalances between the genders in the sciences including funding, earnings, hiring and patenting.

One of the issues the researchers had to get around in their study was how to determine gender of the authors of journal articles indexed by Thomson Reuters in the Web of Science. They used a combination of sources to match the name to a gender including social security databases, Wikipedia and even Facebook (the interesting methodology can be read in their supplementary material).

It would be interesting to see the same data normalized for funding. In many cases funding agencies are awarding more money to male scientists than to female scientists (e.g. from the ARC's website for Number of participants on all funded projects Male = 2280 and Female = 622). It would be interesting to know whether perhaps female scientists were achieving "more with less" when it came to publication output compared with funding and opportunity.

While there is no single answer to the problem these researchers are describing they do make a good point about improving the ability for female scientist to travel and collaborate internationally:

"For a country to be scientifically competitive, it needs to maximize its human intellectual capital. Our data suggest that, because collaboration is one of the main drivers of research output and scientific impact, programmes fostering international collaboration for female researchers might help to level the playing field."

Researcher mobility in all fields is a good strategy for any organisation that can afford it - and is certainly critical for any developed nation's research strategic plan.


http://www.nature.com/news/bibliometrics-global-gender-disparities-in-science-1.14321

Crowdfunding Research

Researchers at Deakin University have completed a really interesting pilot project using crowd-funding to source money for research projects. The pilot titled, Research My World, was run in partnership with crowd-funding site Pozible.com and is the first of its kind in Australia. Of the 8 projects put forward for funding 6 were funded (75% success rate!). However, not only did the pilot raise funds for the projects it also generated huge media and social-media attention for the projects and the researchers. In addition it provided an excellent introduction for the researchers to the use of social media in engaging the wider community in university research.

A copy of the Research My World pilot project evaluation can be found at this link:
http://www.deakin.edu.au/research/documents/research-my-world.pdf