Developing an ‘Impact and Engagement for Australia’ (IEA) metric

ATSE has floated their idea for a research engagement and impact evaluation based purely on metrics. The metrics suggested are category 3 income and commercialization income. ATSE suggest that this evaluation be run along with the ERA to produce a combined quality and impact rating (quality 5-1 and impact A-D).

It is important for Australia to consider an impact evaluation - and examples such as the Excellence in Innovation for Australia (EIA) have shown that it is possible to evaluate research impact in Australia. The UK have also evaluated impact as part of their national research evaluation exercise. Some argue that the EIA and the REF methodology of case studies and peer review are onerous and expensive. However, this is by no means a reason not to do them - these sorts of evaluations will never be easy because the evaluation of research is not easy. Proper evaluation of research requires time and the people with the right level of expertise to carry out the evaluation.

Here are some of the issues I think the ATSE ERA-linked evaluation might have:

• It assumes that your research input, output and impact all occur in the same FoR code.
• It assumes that impact has occurred concurrently with the research – most impact is not realised that quickly.
• It won’t take into account the difference between pure research and applied research in the same FoR.
• It is focused only on economic impact – which for the commercialisation income might actually be more a measure of the success of the company selling the widget than the quality of the university.
• It also does not take into consideration the amount of gaming that occurs in the ERA.

The ATSE release can be read here: http://www.atse.org.au/atse/content/activity/innovation-content/developing-impact-engagement-australia-metric.aspx

Kardashian Index

Do you know what your Kardashian Index is?

Your Kardashian Index is a measure of the discrepancy between your social media profile and your publication record based on the direct comparison of numbers of citations and Twitter followers.

Anything great than '5'and you are considered a science Kardashian! My index is only 0.1 so I am far from a Kardashian - I think I am going to have to try and get more Twitter followers...

Read Neil Hall's paper: The Kardashian index: a measure of discrepant social media profile for scientists http://genomebiology.com/2014/15/7/424

Research ethics and the ebola epidemic

The recent outbreak of the Ebola virus has presented scientists with an opportunity to test potential vaccines on patients. While the epidemic is terrifying this does not mean that research ethics should be ignored.

http://www.researchimpact.com.au/viewtopic.php?f=19&t=36

Four reasons to stop caring so much about the h-index.

Four reasons to stop caring so much about the h-index.

Interesting blog piece from Stacy Konkiel on the h-index and altmetrics called 'Four reasons to stop caring so much about the h-index'. I like that Stacy doesn't ask us to stop caring all together about the h-index but to just stop caring 'so much' about it.

It seems confusing to me to be comparing h-index directly with Altmetrics. To me they appear to be two different things. H-index attempts to measure the productivity of a person whereas Altmetric attempts to measure the influence of an article. Why not compare the Altmetric to Relative Citation Impact (RCI) for example? The RCI is normalized for the discipline and year the article was published and gives a measure of whether the article has been cited above or below the expected average rate for the discipline. It is not immediately clear how to determine whether an Altmetrics score is relatively high or low for the discipline so it would be useful if it incorporated some sort of benchmark.

More Journal Ranking

Here are some more journal rankings to add to your list:

• Impact per Publication (IPP)
• SCImago Journal Rank (SRJ)
• Source Normalized Impact per Paper (SNIP)

Check them out at http://www.researchimpact.com.au/

Journal Ranking

If you would like to view some of the popular and publicly available journal 'ranking' lists head on over to the researchimpact.com.au forum to check out:

• Eigenfactor.org
• ERA Journal Ranking
• Harzing - journal quality list
• ABDC - journal quality list

Make a comment on these lists or perhaps add some of the ones that you know of or use.

Why was it better to be a researcher in Design Practice and Management than Forestry for ERA 2010?

www.researchimpact.com.au

DECRA not considered a highly regarded form of accolade by the ARC's ERA 2015

The third round of the ARC's Excellence in Research for Australia (ERA 2015) is underway; and, unfortunately, if you have a Discovery Early Career Researcher Award (DECRA) it is not being considered an 'esteem' measure. However, you will be considered 'esteemed' if you hold a NHMRC Early Career Fellowship.

The ERA includes a number of measures of 'esteem' and they are defined as measures that:

constitute recognition of the quality ... and indicate that a researcher is held in particularly high regard by peers in their field of research and/or by other well qualified parties. ...embody a measure of prestige and are recognised by experts within the discipline as a highly desired, highly regarded form of accolade or acknowledgement.

The ERA requests institutions submit data on only five esteem measures (listed below) of which one is 'nationally competitive research fellowships'. Unfortunately it looks like the DECRAs did not make it onto the list this time so are not considered a measure of esteem by the ERA.

The esteem measures eligible for ERA 2015 are the following:

• editor of a prestigious work of reference;
• fellowship of a learned academy and membership of the Australian Institute of Aboriginal and Torres Strait Islander Studies (AIATSIS);
• recipient of a nationally competitive research fellowship; 
• membership of a statutory committee; and 
• recipient of an Australia Council grant or Australia Council fellowship.

And, only nationally competitive research fellowships in the following programs are eligible (where are the DECRAs?):

• ARC Discovery—Australian Laureate Fellowships; 
• ARC Discovery—Federation Fellowships; 
• ARC Discovery—Future Fellowships; 
• ARC Discovery—Indigenous Researchers’ Development; 
• ARC Discovery—Projects (including Australian Professorial Fellowships, Queen Elizabeth II Fellowships, and Australian Postdoctoral Fellowships); 
• ARC Linkage—International; 
• ARC Linkage—Projects (including Australian Postdoctoral (Industry) Fellowships); 
• NHMRC Practitioner Fellowships (Formerly Practitioner Fellowships Scheme); 
• NHMRC Research Fellowships; 
• NHMRC Australia Fellowship (Formerly Australia Fellowship Scheme);
• NHMRC Career Development Fellowships (Formerly Career Development Awards); 
• NHMRC Early Career Fellowships (Formerly Postdoctoral Training Fellowships); 
• NHMRC Sir MacFarlane Burnett Fellowship;
• NHMRC John Cade Fellowship in Mental Health Research; and 
• NHMRC Translating Research Into Practice (TRIP) Fellowships.

To check out the ERA 2015 and related documents yourself you can visit: http://arc.gov.au/era/era_2015/2015_keydocs.htm