Impact Assessment Workshop

The Impact Assessment Workshop, hosted by the Graphene Flagship at Chalmers on 1 March 2018, generated a meaningful discussion on how to measure the impact of innovation projects. Why must we measure a project's impact? What metrics can we use to determine the effectiveness of flagship projects? How can these be generated?

"The workshop clearly demonstrated both the importance and complexity of impact assessment. I was left with the understanding that there is no unique answer to how to assess impacts of a research project such as the Graphene Flagship, but the most appropriate methods vary depending on the purpose of the assessment," said Jari Kinaret, Graphene Flagship director.

While every project is unique and lends itself to discrete metrics, a combination of qualitative and quantitative metrics must be used to illustrate the full impact of any initiative. The challenges arise when attempting to assess the long-term impact of an initiative only a couple of years after the project has launched, as required.

Scientific impact can be quantified by looking at the number of publications produced by researchers, number of times these papers are cited and the number of patents that result from the research initiative. A look at the success of PhD students involved in the flagship and the industries they go on to work in can also help to quantify the scientific impact of a project.

Numbers, however, cannot paint the entire picture. Anecdotal data and case studies can put the rest of the data into context and are often more powerful than statistics.

A look at impact assessment for the Human Brain Project presented by Martin Telefont and the ITER project presented by Laban Coblentz illustrated just how different the assessment process can be. While the Graphene project can demonstrate economic impact through direct interactions with companies across various industries, the Human Brain Project's industry ties are more indirect and often come in the form of spin offs created by PhD students who took part in the project. ITER discovered that anecdotal evidence from industry representatives was more powerful than statistics when demonstrating economic impact to policy makers.  

It is important to note that the goals of a project must evolve as it progresses. While at its inception the Graphene Flagship's primary goal was on research, a focus on graphene's industrial applications will become increasingly important. Thomas Reiss discussed his work on developing an innovation roadmap to determine the steps necessary to bring specific graphene applications to the industry. Through a series of Value Chain Workshops which bring together small groups of industry experts, Reiss has been able to identify specific markets in which graphene could serve as a viable solution. Industry experts shed light on specific industry needs, the current state of the industry and the steps and a realistic timeline for achieving industry integration for a graphene product. From these sessions, a roadmap is born.

"The Graphene Flagship has performed very well so far, as was shown in past reviews with the indicators that are in use and through various success stories. As time progresses, there is a need to revisit the indicators that are used, adapting them along with the evolution of the initiative, and moving beyond initial outcomes to look at longer term impact. The workshop provided many excellent ideas on how to do that," concluded Wide Hogenhout of the EC flagship unit.