S3 Targeted Support – Open data & Open Science for S3 monitoring

  • 02 Jul 2019 to 02 Jul 2019
  • Generalitat de Catalunya Carrer Gran Via de les Corts Catalanes, 639 Barcelona

As well as having academic and policy-oriented presentation, the workshop addressed, with hands on exercises: the use of EU and national open data for S3 monitoring. the drafting and assessing ToRs for S3 evaluation.

Agenda and Presentations

S3 Targeted Support – Open data & Open Science for S3 monitoring


2nd July 2019

Generalitat de Catalunya

Carrer Gran Via de les Corts Catalanes, 639



1st July 2019


20:30-22:00         Networking dinner (XUP XUP Restaurant -Paseo Marítimo de la Barceloneta, s/n)


2nd July 2019

08:45 - 09:00 Welcome Coffee and registration

09:00 – 09:15 – Introduction to the day


Mark Boden (JRC): Opening remarks

Elisabetta Marinelli (JRC): Introduction to the day  

09:15-10:15 Open Data and S3 Monitoring – New cases, new issues


Nicola de Chiara (Agid, Agenzia per l’Italia digitale): The ONCE observatory (20 mins)

Ismael Rafols (Ingenio, Spain): On the potential effects of monitoring systems in STI policy (20 mins)


Q&A (20 mins)

10:15- 11:30 Participatory exercise – part 1

  • Country-specific issues in open data for monitoring


Country-Specific table discussions:

With the help of the moderator the working tables will discuss the following aspects, based on the exercises just conducted.

  • For each data-source
    • Potential for S3 analysis and monitoring
    • Main obstacles encountered to access / analyse the data source

 Accessing the data

 Managing / analysing the data

  • Data granularity requirements
  • Data openness


 Accessible / Machine processable / Interoperable



  • Potential improvements and paths forward (e.g. Best practice exchanges between administrations and data platforms // Useful dimensions/data currently not accessible that  could be made public)
  • choose the best example to report exercise 2






Ana Fernandez


Elisabetta Marinelli


Michalis Metaxas


Hugo Pinto


Petra Szávics/Participant

Bulgaria, Croatia and Hungary

Lajos Nyiri



11:30 - 12:00 Coffee Break

12:00 -13:00 Participatory exercise – part 2


3 working groups will be organised, ensuring a good representation of different countries.


  • Peer sharing - short oral presentation of 2-3 best practices (by the moderator/rapporteur)
  • Comments on the two EU-level initiatives (relevance for S3 monitoring and data openness)
  • (If there is time) Discussion on the potential effects of monitoring systems in STI policy






Group 1

Yannis Tolias

Hugo Pinto, 

Group 2

Elisabetta Marinelli

Michalis Metaxas

Group 3

 Mathieu Doussineau 

Petra Szavics



13:00 - 13:30 Reporting back and discussion

  • All the rapporteurs of the different groups report back to open-up a discussion


13:30 - 13:40 Proposal from our Critical Friend Attila Varga: Seeking for regions to pilot S3 evaluation.


13:40 – 14:30 Lunch break

14:30 – 15:15 Real-life experiences with S3 evaluation



15:15 – 16:00


  • Commenting examples of S3 mid-term evaluation ToRs


ToR elements: Are the following aspects clear? Are they appropriate?

  • General information:
    • the background, rationale and objectives of the policy to be evaluated, its target recipients, delivery method and intended outcomes;
    • the audience and intended use of the evaluation;
  •  Technical information:
    • the extent of the existing evidence base related to the policy;
    •   the evaluation objectives and research questions;
    • the possible evaluation approach, research design and methods;
    • the required capabilities, skills and experience of the proposed evaluation and team;
  • Outputs and deliverables
    • the required evaluation outputs (including datasets) and the milestones to be met;
    • data archiving requirements;
  • Organisational and Economic information
    • the indicative budget;
    • the evaluation timetable

Conceptual reflections:

  • OP vs S3: To what extent this OP evaluation can inform the RIS3 mid-term review?
  • OP vs S3: Are the OP evaluation questions sufficient to evaluate a RIS3? What is missing?
  • Discuss the trade-off between overspecification and bidding on the price vs more degrees of freedom in proposing a methodology and bidding on price/performance.
  • To conclude, what would you expect from those ToR?
  • 16:00 – 16:30  - Reporting back + comments by Ken Guy



16:30  - 16:45 Closing remarks

Mark Boden (JRC): Conclusions from the day

Elisabetta Marinelli (JRC): Next steps          


Practical Information

02 Jul 2019 to 02 Jul 2019
Generalitat de Catalunya Carrer Gran Via de les Corts Catalanes, 639 Barcelona


Elisabetta Marnelli