The development of KPIs

We developed KPIs to assess the product and process levels of organizational ongoing R&D and RRI dimensions.

Product levels include questions related to the issues of technological risks while process levels include questions focusing on research integrity. To find indicators for both product and process levels, we explored literature in the field of innovation management and responsible innovation. In the first stage, we distilled roughly 250 indicators. Next, after removing redundancies and irrelevant indicators, we ended up with 92 relevant indicators.

Selecting KPIs for your company

This list of 92 KPIs can be customly selected depending on your company’s (or your company’s innovation project) situation. In this case, some KPIs may be relevant and some may not be irrelevant.

Hence, from the stack of 92 KPIs, a company’s representatives (managers, researchers/scientists, innovation project participants) can select which KPIs are relevant by categorizing them into three categories

• • Category 1: Yes if the certain KPIs are relevant
  • Category 2: Maybe if certain KPIs may not be relevant at present but may be relevant in the near future
  • Category 3: No if certain KPIs are irrelevant both at present and in the future

KPIs that fall into Category 3 (No) can be crossed out.
The relevant KPIs are those that you have categorized in the first two categories: Yes and Maybe.

Next step: scoring

To turn these indicators into KPIs, you can score each indicator on a  a 7-point Likert scale (1: strong disagree; 2: disagree; 3: slightly disagree; 4: neither disagree nor agree; 5: slightly agree; 6: agree; and 7: strongly agree).

The score that you enter can give you an idea which KPIs score more positive (score five to seven)  or more negative ( score one to three) or neutral (score four) .

See the list of KPIs with scores here. You can use this list for the peer review assignment 1 (Module 2.9).
See also screenshot below.

Clustering and weighting KPIs

After you have selected the relevant KPIs for your company, focus on the “Yes” category of KPIs on your list (i.e. the most relevant KPIs).

Try to re-group these most relevant KPIs into a number of clusters.  You can create your own cluster name.

As an example, the following KPIs can be grouped into a new cluster called “Competition”:

• There are many competitors in the market for the outcome of the project
• The market for the outcome of the project is characterized by intense price competition
• The market for the outcome of this project is growing very quickly
• This project will contribute to the competitive advantage of our company
• Our project outcome will be clearly superior to competing projects (in other companies) in terms of meeting our customer’s needs
• Compared to competing projects (at other companies), our project will offer a number of unique features or attributes to our customer

Next, you can weigh each cluster by putting weights (in %). The weights should reflect how important you perceive each cluster for your company or your company’s innovation project.

Monitoring

Once you score how well your company has done in certain KPIs of organizational and RRI dimensions (from point 1 to point 7), you can have a better understanding of the drivers and challenges for further improvement in three time frames: short term, medium term and long-term.

Besides drivers and challenges, you can also think of risks and barriers in the three time frames. You can also use the RRI approaches and tools to identify short-term, medium-term, and long-term actions needed to align the innovative products with societal needs. The link can also be made to RRI technologies and products that your company wants to achieve in short-term, medium-term, and long-term periods. These are all the components of RRI roadmap.

In turn, once you have an RRI roadmap, you can also use it to monitor the implementation of RRI periodically by re-assessing the KPIs. In this case, the link between KPIs and roadmap is circular through a feedback loop.

Quick-and-dirty method of selecting KPIs for RRI dimensions

We have shown you a comprehensive way of selecting KPIs of both ongoing R&D and RRI dimensions (out of 92 KPIs) relevant for your company or for your company’s innovation project.

However, particularly for RRI dimensions, there is a quick-and-dirty method of selecting KPIs. Recall that RRI dimensions include: (1) anticipation and reflection; (2) inclusiveness; and (3) responsiveness. The rule of thumb is that from these RRI dimensions, you can select three to four KPIs for each RRI dimension in which you may end up to about ten KPIs of RRI dimensions.

To give you an idea of how to use this quick-and-dirty method, below are the ten KPIs of RRI dimensions which are based on a cross-case comparison of our eight pilots (Note: Naturally, based on your company’s situation, you may have a different set of ten KPIs based on the quick-and-dirty method). These ten KPIs are useful to monitor the implementation of RRI principles at company level.

Examples of quantitative indicators to monitor implementation of RRI principles at the company level: