Written by Tereza Szybisty

Moderator Natalia Manola (OpenAIRE) opened the session by reframing the premise. Before asking how we measure Open Science, she invited speakers and the audience alike to ask why we measure it at all. The answer, she argued, is not trivial. We invest in Open Science because we believe it changes science and culture, and that change is precisely what indicators need to capture, not just the outputs that are easiest to count. This framing set the tone for everything that followed.

OpenAIRE Monitor: transparency as a foundation

Ioanna Grypari (OpenAIRE) presented the OpenAIRE Monitor, now serving 77 research institutions, 25 research initiatives, 13 funders, and a growing number of national monitors. The service draws on the OpenAIRE Graph, the largest open scholarly knowledge graph in Europe. Two points from Ioanna's presentation generated lively discussion. First, data quality is not a background concern, it is the product. The Irish Monitor, for example, has used the very act of surfacing unexpected indicator values to prompt institutions to clean and improve their own data, turning measurement into a quality improvement loop. Second, the same number means different things to different readers: a 62% open access rate is a benchmark for one institution, a mandate compliance check for a funder, and a national policy progress signal for a ministry. Indicators are not neutral; their meaning is shaped by who reads them and why.

CoARA Working Group on responsible use of metrics and indicators: start with "why"

Katarzyna Nawrot (Co-chair of the CoARA Working Group on Responsible Use of Metrics and Indicators) shared findings from the CoARA working group, including a large-scale survey (learn more about the results of the survey here). Her core argument was methodological: before choosing any indicator, an institution must first ask why it is evaluating, whom or what it is evaluating, and, critically, what latent construct the indicator is supposed to capture.

The working group identifies a pattern they call the "naive non-responsible workflow": picking available indicators without first establishing what you want to measure. This leads, Katarzyna argued, to inappropriate use, not because the indicators are bad, but because the reasoning behind choosing them was never made explicit. The recommended workflow starts with values, moves to assessment goals, defines the construct to be measured, and only then selects indicators, complementing quantitative measures with qualitative assessment throughout.

The discussion touched on a broader provocation: do we measure what we treasure, or do we treasure what we measure? Katarzyna's call for indicators that guide us toward higher-quality research, rather than simply describing current practices, resonated strongly with the audience.

Open Science Monitoring Initiative (OSMI): aligning content providers

Iratxe Puebla (Co-chair of the OSMI Working Group 3) presented the Open Science Monitoring Initiative's work with scholarly content providers, publishers, repositories, and data centres. A key distinction in Iratxe's presentation: this work is explicitly not about building rankings or passing judgment on levels of openness. It is about generating evidence that allows informed decisions, for example, whether a journal should update its open access policy, or how a funder like NIH can monitor data-sharing practices across generalist repositories.

Iratxe raised a challenge that will resonate with anyone working in this space: as more platforms adopt open science monitoring, they are applying different methodologies to calculate indicators for similar practices within open science. Some level of standardisation, or at least consensual definitions, will be needed to prevent a future in which many indicators exist but none are comparable. She also called explicitly for content providers to share their indicator practices openly, so the community can audit and refine them. Learn more about the results of the OSMI Working Group 3 survey here. 

EOSC Open Science Observatory: policy to practice

Tereza Szybisty (OpenAIRE, EOSC Track / EOSC Open Science Observatory) presented the Observatory as a policy intelligence platform built in close collaboration with the EOSC Steering Board and European Commission. Its purpose is to provide national policymakers, country representatives, and the research community with a data-driven picture of where Europe stands on Open Science, across policies, practices, and impacts. The monitoring framework covers eight categories: policies, practices and impacts.

One of the most striking illustrations from Tereza's presentation was the gap between policy existence and researcher coverage. The policy exists on paper; whether it reaches researchers is a different question entirely. This, she argued, is precisely why the Observatory combines quantitative data with qualitative country narratives written by National Open Science Desks - the numbers tell you that a gap exists; the narrative tells you why.

Tereza also surfaced three structural challenges facing the Observatory: comparability (countries answering the same survey questions with different understandings of basic definitions), the gap between policy coverage and live practice, and sustainability, how to maintain continuity of a monitoring mechanism as EOSC moves into its post-2027 governance phase, while simultaneously evolving the framework to match shifting policy ambitions.

The moderated roundtable and audience Q&A surfaced several threads worth carrying forward.

Data quality is a first-order concern. Deduplication, affiliation disambiguation, licence metadata, these are not technical footnotes but conditions for any meaningful indicator. Multiple speakers noted that the human-in-the-loop remains essential, even as algorithmic approaches improve.

Indicators are often most useful as a diagnostic tool. They show what the data behind a policy actually looks like. As Ioanna noted, this matters in its own right. A low number may point to where investment is needed, in repositories, identifiers, or curation, rather than who is falling short. It may also signal incomplete data, not necessarily poor practice. In this way, indicators do more than guide decisions; they help clarify what kind of decision is needed. 

Impact measurement is the missing piece. Several speakers acknowledged that while output indicators (publications, datasets, repositories) are increasingly mature, indicators for scientific and societal impact remain underdeveloped. The PathOS Handbook of Open Science Impact Indicators was highlighted as a resource that the community should collectively work to keep alive and extend.

Openness applies to the monitoring infrastructure itself. A recurring theme was that the credibility of any monitoring initiative depends on whether its methodology, data sources, and indicator definitions are openly available for scrutiny. The EOSC Open Science Observatory makes its data freely available for reuse; the OpenAIRE Graph is publicly accessible; OSMI publishes its principles that can help us to align on Open Science monitoring.

Asked what they hoped to see in two years, the speakers offered a set of complementary ambitions:

• Katarzyna called for Open Science indicators that genuinely guide quality , not just describe coverage, but point toward what matters.
• Iratxe named adoption, maturity, and openness: more content providers monitoring, with indicators that have reached sufficient quality to be useful beyond their originators.
• Ioanna asked for more granular indicators that capture the whole picture of open science: meaningful uptake rather than open-washing, consequences across different groups and contexts, and, with the help of AI, actual use and impact of scientific outputs beyond access alone. 
• Tereza urged a shift from measuring what is easy to count toward measuring what actually changes , how Open Science is making research more useful, accessible, equitable, and impactful for researchers, policymakers, and society.

Across 130 participants and four initiatives, the session surfaced something that is easy to overlook in the day-to-day work of building monitors and designing frameworks: the indicators we choose are never just descriptions. They are arguments about what matters. They shape what institutions report, what researchers prioritise, and what policymakers reward. Which raises a question worth sitting with long after the webinar ended, not are we measuring Open Science the right way, but whose version of Open Science are our indicators quietly encoding?

Written by Tereza Szybisty

The webinar opened against the backdrop of a defining policy moment. The Letta report on the Single Market, the Draghi report on European competitiveness, and the Heitor expert group on research and innovation have each placed Open Science and research data infrastructure at the heart of Europe's strategic agenda. That message is now shaping the negotiations for Horizon Europe 2028–2034, which names Open Science as one of its core principles. In April 2026, the EOSC Tripartite Governance issued a joint narrative paper framing EOSC as a foundation for Europe's data sovereignty, security and competitiveness; only weeks later, the EOSC Steering Board followed with paper positioning EOSC as an indispensable component of the FP10 Partnership portfolio.

But ambition needs evidence. The presentations that followed brought precisely that.

Gareth O'Neill (Technopolis Group, EOSC Track) introduced the Monitoring Framework for National Contributions to EOSC and Open Science, now in its second version, which underpins the annual survey run on behalf of the EOSC Steering Board. The framework organises evidence around three core dimensions (policies, practices and impacts) and across eight categories relevant for EOSC and Open Science including open access publicationsa, FAIR data, software, services, infrastructure, skills/training, assessment, and engagement. The 2024 survey cycle is the third formal iteration, with 31 countries responding including 22 Member States of the European Union, and the data shows a clear trajectory of gradual maturation: more national policies in place, a growing share of them mandatory, and rising national investment, with the most embedded areas (such as open access) leading and newer policy areas (such as FAIR data and open source software) catching up over successive cycles. Detailed findings, including country-level breakdowns and historical comparisons, are available in the full Survey 2024 analysis on the Zenodo community, and the methodology behind the Steering Board monitoring is documented on the EOSC Open Science Observatory.

Ilaria Nardello (EOSC Association) presented the parallel evidence from the EOSC Partnership side. The EOSC Association and its members have committed to €500 million in additional activities by 2030; as of today, contributions stand at €1.6 billion well ahead of the expected halfway mark. In 2025 alone, members invested €340 million in Additional Activities, with a notable share supporting the establishment of national and thematic EOSC nodes, including significant in-kind staff contributions. The 2026 plans show investments in coordination roles for the establishment of the EOSC Federation and its nodes rising sharply, reflecting the operational reality of building the EOSC Federation. Nardello underlined that the Partnership's monitoring captures institutional contributions while the Steering Board Survey captures national-level commitments, and that the additionality between the two is where the leverage effect of the EOSC initiative becomes visible. Further information on EOSC Association monitoring activities can be found on the EOSC Association website.

The panel discussion, moderated by Tereza Szybisty (OpenAIRE) brought together four speakers, Gareth O'Neill, Ilaria Nardello, Pilar Rico Castro (EOSC Steering Board, Spain / FECYT) and Ana Teresa Maia Mota (European Commission).

Monitoring as a learning exercise

Both Gareth O'Neill and Pilar Rico Castro emphasised that the value of the Survey lies not only in the data it captures, but in the convergence of understanding it has built among Member State representatives. Common definitions of "policy," "financial strategy," or "investment" have taken years to stabilise. Pilar Rico Castro added a critical practitioner's perspective: heterogeneity in national data collection strategies (federated in some countries, centralised in others) remains a real constraint on comparability, and harmonising methodologies is a task still very much in progress

The future of Open Science monitoring

Ilaria Nardello argued that the experience of the past few years since the beginning of the partnership has shown the broad differences between the monitoring of the EOSC deployment, its performance and impact and the monitoring of the National Contributions, providing arguments why the two efforts may hardly be coordinated through a single monitoring effort. EOSC is an infrastructure for research, encompassing an ecosystem of resource providers and uptakers, while Open Science is an outcome of this policy initiative; while different types of performance and impact  indicators are required, a common vision for what the monitoring efforts are intended to provide is necessary. Pilar Rico Castro suggested a forward-looking proposal: as national open science monitoring frameworks mature, e.g. in France, Finland, Ireland, the Netherlands, Denmark and now Spain, a federated  model anchored in Member State initiatives (aligned with the Open Science Monitoring Initiative - OSMI) would offer greater long-term sustainability than reliance on the current Tripartite Governance arrangements alone.

EOSC as a connecting backbone

Ana Teresa Maia Mota set out how EOSC, as the Common European Data Space for Research and Innovation, is embedded  into the Commission's policy landscape:  the Competitiveness Compass, the European Strategy on Research and Technology Infrastructures, the AI Continent Action Plan, the European Strategy for AI in Science, and the Data Union Strategy. The challenge ahead is translating this strategic positioning into concrete instruments under FP10: long-term operational funding beyond short project cycles, stronger links between EOSC, EuroHPC, research infrastructures and sectoral data spaces, and targeted action on skills, data stewardship and AI-ready data management.

The piece of evidence that should travel furthest

Panelists were asked which single piece of evidence from the webinar should most directly shape the FP10 negotiations, the panellists offered complementary answers. Ana Teresa Maia Mota pointed to the persistent fragmentation between national investments, policies and implementation as a signal for where alignment is most needed. Ilaria Nardello underlined the synergies and additionality generated by the Partnership as evidence that EOSC is a necessary element of the next Framework Programme. Pilar Rico Castro made the sharpest point: monitoring should never become the purpose itself, and the problems faced in monitoring (scalability and sustainability above all) are the same problems the policy itself must address. Gareth O'Neill concluded that the scale of national investment and engagement now visible in the data calls for matching policy and financial support from FP10 itself.

If there was a single thread running through the webinar, it was this: monitoring is not a technical activity that runs alongside the policy, it is a mirror of the policy itself. The challenges that emerged in the discussion, like the fragmentation between national and European investments, uneven data collection practices across Member States, or the difficulty of aligning EOSC monitoring with the broader Open Science monitoring landscape, are not problems of measurement alone. They are the same scalability and sustainability questions that FP10 will need to answer for EOSC itself.

The next two years will be decisive. The Horizon Europe 2028–2034 negotiations, the revision of the EOSC Strategic Research and Innovation Agenda, the expansion of the EOSC Federation, and the maturing of national Open Science monitoring frameworks across Europe are all happening in parallel. Whether these workstreams reinforce one another, or pull in different directions, will depend on the kind of dialogue this webinar made visible: between Member States and European governance, between the Partnership and the Steering Board, and between those who design policy and those who measure its effects.

Join the next EOSC Track webinar, on the promises and pitfalls of open science indicators, where we will continue this conversation

Register here

Written by Tereza Szybisty

The Registry brings together a curated collection of key resources supporting Open Science across Europe , such as national policies, strategies, best practices, templates, and impact stories. Each entry includes detailed metadata and, where available, access to the full text, so you can move quickly from discovery to deeper reading.

Filters by country, date, topic, and document type make it straightforward to locate what matters for your context, whether you are drafting an institutional policy, benchmarking national approaches, or simply mapping the landscape.

Policies and strategies are foundational to a collaborative, transparent, and accessible research environment, but a list of links is rarely enough. The Registry goes further, surfacing structured information on each document: responsible organisations, tags, summaries, and connections to related resources. This added context helps users understand a policy's scope, implementation, and impact, not just its title.

Behind the scenes, every resource moves through a transparent workflow. The current main data source is the Survey on National Contributions to EOSC and Open Science, complemented by additional policy resources. Documents are ingested, pre-processed (including text segmentation and automatic translation where needed), and enriched with AI-powered tagging, summarisation, and metadata extraction. Crucially, human review and approval sit at the heart of the process - AI accelerates the work, but curators ensure quality and relevance before anything is published.

Looking ahead, the Registry plans to expand its sources to include initiatives such as the OECD STIP Compass and selected national platforms, and to allow the community to flag missing documents directly through the system.

The best way to understand what the Registry offers is to use it. Explore the European Open Science Resources Registry, filter by your country, browse by topic, and see how Open Science is taking shape across Europe.

Poster from the Open Science Conference in Hamburg 2025. Check out the full poster in Zenodo.