Monitoring Monitoring

Besides the legal obligations directly deriving from the ESIF regulations, the S3 monitoring system should be understood as a fundamental management tool for the S3 approach. A poorly constructed monitoring system would hinder the capacity to effectively face the development needs of the territory and may even prevent the proper implementation of the strategy. In order to be a proper and effective policy tool, the S3 monitoring system should reflect the logic of intervention of the strategy. In particular, the indicator system should be linked to specific objectives and expected changes explicitly defined and identified for each and all S3 priority areas.

Download the "Monitoring" pdf file

How can the monitoring system be an effective S3 management tool?

Challenge

The S3 monitoring system needs to represent and keep track of all the main elements of the intervention logic at the basis of S3. This means to set up an articulated system of indicators associated to each and all of the priority areas defined in the strategy.

Response

The minimum required elements for a meaningful S3 monitoring system are: (i) output indicators (measuring the actual level of implementation of the policies and related actions), (ii) result indicators (measuring the degree of achievement of the strategy's socio-economic objectives), (iii) their explicit articulation by S3 priority areas, (iv) their logical link with the expected changes and objectives to which they will contribute. The S3 monitoring system may also include (v) structural change and specialisation indicators (measuring the evolution of production systems within and between the S3 areas), and (vi) context indicators (providing a picture of the competitiveness of the regional economy and the evolution of the regional innovation system as a whole).

The current approach of Emilia-Romagna (IT) to smart specialisation focuses on two lines of action: reinforcing and modernising existing clusters as well as discovering emerging ones with a high potential for innovation and employment. The idea is to support the evolution of the industrial system towards a higher capacity for better managing the immaterial/intangible aspects of value chains. ASTER - a consortium for industrial research, technology transfer and innovation - oversees the monitoring activities of the S3 through a system capturing four measurement dimensions:

  1. Implementation (output indicators);
  2. Change of the regional economy in terms of specialisation domains (specialisation and transition indicators);
  3. Effectiveness of the overall strategy (result indicators);
  4. Evolution of the regional economy (context indicators).

 

This differentiated approach allows catering to different target groups. Especially ‘change indicators’ are at the core of Emilia Romagna’s effort to promote specialisation in activity areas with proven strengths and potential. Within this broad category, the ‘specialisation indicators’ cover e.g. patents, research grants, value of business-research contracts, share of new start-ups and number of SMEs per specialisation area. These indicators show how the regional economy is advancing in the selected specialisation areas. They also capture how the regional economy is moving along the selected innovative drivers. An online portal is under construction and will allow the visualisation of monitoring data. This device will be a key communication tool to inform stakeholders and the broader public about the implementation of the S3 in the region, providing freely accessible data.

More information

Emilia Romagna’s presentation at the Peer eXchange and Learning workshop in Bologna (IT), Nov. 2015 (in English)

Keywords

Monitoring, priorities, indicators

Challenge

The result-orientation of the S3 can be achieved by: (i) clearly defining the logic of intervention of the strategy linking policy measures to results and objectives, and (ii) actively managing S3 implementation by means of updating and assessing the indicator panel.

Response

The S3 monitoring system of Galicia (ES) comprises a panel of 74 indicators categorized into output, result and context which will be updated continuously during the programming period covered by the S3. Data will be collected by the Galician Innovation Observatory, the body responsible for analysing the impact of public innovation policies in Galicia, with the support of a team of independent experts and stakeholders.

The indicator structure identified in the Galician S3 is reported in the following chart (own elaboration based on Galicia S3):

 

 

These three interlinked sets of indicators constitute the Galicia S3 scorecard, conceived as the key management tool integrating the strategy’s executive and operational levels with the aim to achieve the S3 mid- and long-term objectives.

Intermediate and target values of indicators were fixed in mutual agreement with the Government departments involved in S3, also taking into account the historical evolution of each indicator and the expected impact of the S3 strategy. Baseline values were defined using different sources, such as the Galician Institute of Statistics (IGE), the Innovation Platform Galician (PINNG) or the Galician Service of Industrial Property (SEGAPI).

There will be an interim and a final assessment in 2018 and 2020, respectively, in which the indicators’ actual values will be contrasted with the target values. The assessments will take into account qualitative information obtained via surveys to beneficiaries and Quadruple Helix discussion groups to further confirm the evidence gathered through quantitative information. The assessments will analyse both the evolution in time of individual indicators and the comparative performance of context indicators in the region as well as in other Spanish and European regions. In case of negative deviations from expected targets, corrective measures will be devised according to a risk analysis. In case of positive deviations, the likely causes will be analysed in order to try to transfer the experience to other areas and inspire future actions.

More information

Galicia's S3 document (in Spanish and in English)

Keywords

Monitoring, priorities, indicators, S3 governance

How to communicate data and monitoring results effectively?

Challenge

Targeted communication and accessibility of monitoring information constitute an important step towards the involvement of stakeholders in the implementation of S3. However, they require specific provisions and ad hoc devices.

Response

A transparent monitoring system that concisely conveys the relevant information about S3 implementation contributes to the credibility and reputation of the ambitious transformational plan contained in the strategy. Stakeholders can either be involved in the follow-up of monitoring activities or be empowered by having access to factual information on progress made. In this way trust, ownership and commitment can be built up and maintained.

The Government of Wales (UK) commissioned the innovation charity NESTA the development of a novel data platform that collects and assesses information about innovation activities in Wales and the interconnectedness between people and organisations. Arloesiadur (meaning “innovation tool” in Welsh) will gather data automatically from very different sources, combining established statistics and web data (company websites, software developing or professional meeting platforms, Twitter accounts, etc.). Learning how to engage constructively with these unconventional data sources for improving innovation policies is part of the entrepreneurial discovery process. This also implies that the public sector has to innovate and rethink current approaches. Valuable lessons can be learnt from this exercise on monitoring developments in S3 priority areas and dealing effectively with the lack of regionalised data from official sources, which are common challenges for national and regional authorities across Europe.

More information

Arloesiadur project webpage (in English)

Keywords

Monitoring, indicators, communication, S3 governance

How to define criteria for assessing indicators' quality and relevance?

Challenge

The monitoring system needs to ensure that the selected indicators (output, result, context, etc.) really measure what they are meant to capture. The problem is twofold: lack of statistics, and/or lack of capacity and skills in the administration. The ERDF funding devoted to S3 offers the possibility to set aside resources to invest in both directions. This opportunity should be fully exploited.

Response

The S3 monitoring requires two distinctive features: (i) a breakdown by priority areas, and (ii) a mechanism and indicator system which allow following the specific developments of priority areas at a greater level of specificity than the operational programme monitoring level.

In the monitoring system of the region Aquitaine (FR), indicators are selected to be realistic within the realm of projects appraisal, and to offer the S3 governance bodies a dashboard enabling an update of the strategy if necessary.

More specifically, S3 indicators need to address the following objectives:

  • Objective 1: Indicators should measure the extent to which the projects to be funded by European Structural Investment Funds (ESIF) or other type of EU funding are aligned with the selected S3 priority areas (i.e. number of projects per S3 priority).
  • Objective 2: As innovation and creation of economic value emerge from the junction of two or more domains, indicators should be able to track cross fertilization (i.e. number of projects covering more than one domain or sector).
  • Objective 3: As one of the most prominent goal of the S3 is to improve firms' innovation output, indicators should reflect the impact of S3 projects on firms' development (i.e. patents, collaboration, training, etc.).

More information

Aquitaine’s S3 webpage (in French)

Keywords

Monitoring, priorities, indicators

No comments yet. Be the first.