S3 interim evaluation – stakeholders’ hands on the steering wheel



The Smart Specialisation approach for the R&I policy in EU already has effects on stakeholder engagement, decision-making processes, monitoring and evaluation mechanisms and many more. The impact is present in R&I systems EU wide (and in some cases, even beyond the European Union) are almost ready for the next generation of S3. However, there is little evidence about the impact of S3 for the transformation of the economy for a particular region. Even more important, projected impact assessment and evaluation mechanisms - the tool at stake to understand the gathered evidence - tend to be used in ex-post scenario, rather than at an interim stage, when there is still a chance for adjustment of the strategy.

A well prepared S3 had to follow 6 consequent steps, advised by the S3 Platform's “Guide to Research and Innovation Strategies for Smart Specialisation (RIS3)”. The final step was “Integration of monitoring and evaluation mechanisms”, which was implemented quite successfully, as demonstrated by many good examples of interactive monitoring systems from numerous regions. Currently, just a small fraction of regions are performing an interim evaluation. Consequently, the “lessons learned” will be visible in the next programming period, when new S3 will be designed or revised. During the design of Lithuanian S3 in 2012-2014, an interim evaluation and possible correction of the programme were hard-coded into the Government decree, creating not only a mandate to perform it, but a real anticipation among all stakeholders as well.

Lithuania has allocated around €800 million (with ESIF) to its Smart Specialisation Strategy during the life-span of 2014-2020 programming period. Main implementing ministries, the Ministry of Economy and Innovation and the Ministry of Education, Science and Sport – have initiated 22 measures, targeting businesses, researchers, research institutions, as well as business – research collaboration projects. The design of S3 was based on 6 broad priority areas, which translated into 20 specific priorities, going down to the level of technologies. More than 600 business sector projects and 200 research projects have been funded so far, and about half of the allocated budget was contracted until the end of the 2018. Only one priority did not attract more than 100 projects, and only 9 priorities out of 20 had accumulated funds above €10 million. Priorities that received most of the funding were: molecular technologies (€32 mil), functional materials (€28 mil), and laser technologies (€17 mil). The monitoring function can deliver similar information and serves the accountability function. This is useful for keeping track of execution progress, but will not reveal possible bottlenecks or explain the behaviour of the actors during implementation.

The interim evaluation took place at the end of 2018, as foreseen in the regulation. Although it is too early to discuss the impact of S3 for the economy, it is possible to understand the “traction and direction” of implementation, to search for various signals and take action. Along evaluation of available data from monitoring, the official EDP process was re-established, involving relevant stakeholders (some already involved in the earliest phases of the process). Overall, 130 participants from research and business fields participated and 42 workshops were organized, in view of assessing the policy mix, the relevance of the priorities and other criteria.

We found out, that the concentration of investments is insignificant, mainly due to low intensity of financing. 45% of the applications were rejected, mainly due to their proposals' lack of R&I activities. Too detailed a specification of priorities in the official documents can become an obstacle for innovative ideas, because the evaluators would look for certain keywords rather than the logical connection. “Super-priorities”, meaning those that would perform in every aspect of intervention, have not emerged yet.

The main decision was to abandon the logic of two-level hierarchy S3 priorities and stay with the broader and more inclusive single level. Out of 6 priority areas and 20 priorities, 7 priorities were reformulated with the involvement of EDP stakeholders. It was decided to renew the list of corresponding technologies (as the previous one was from 2012) in the new priorities, although their titles/designations remain almost the same. The project selection should be based on the potential to address important problems/societal challenges rather than integrate priorities' specification. This will allow to receive more applications and encourage further cross-sectorial approaches.

The capacity to gather evidence and the on-going exchange with stakeholders in the context of continuous EDP are crucial elements, in order to extract meaningful interpretations/conclusions from data. A valid evaluation, performed in timely fashion can improve the traction or even change the direction of S3 implementation.

background quote

The review of S3 generated a valuable set of data and recommendations", Dr. Gintatas Valinčius, director of Life Science Center, Vilnius University

Additional Information

Photo by Evaldas Lasys