Written by Tereza Szybisty

The report's central finding will sound familiar to anyone following EOSC Track's work: Member States differ substantially in policy instruments, legal bindingness, embargo periods, copyright mechanisms, repository requirements, and critically monitoring arrangements. This fragmentation isn't just an administrative hassle - it actively complicates cross-border research collaboration and makes it nearly impossible to see, at the EU level, whether Open Access policy is actually working.

The report identifies what does work: multi-instrument frameworks (national legislation, funder mandates, and institutional support reinforcing each other) consistently outperform voluntary approaches, and linking Open Access to research assessment drives measurable behavioural change. But it's equally clear about what doesn't: policies without institutional follow-through tend not to move the needle.

Underlying all of this is a line worth sitting with: what cannot be measured cannot be managed. Spichtinger treats an EU-level monitoring and reporting framework not as a nice-to-have, but as a prerequisite for evidence-based governance of the "fifth freedom" - the free circulation of knowledge that the ERA is meant to guarantee.

This is where the report gets specific in a way that matters directly to us. It describes the EOSC Open Science Observatory as "the most technically advanced EU-level monitoring infrastructure currently operational". The report recognises its three-layer approach: quantitative indicators drawn from the OpenAIRE Graph and Eurostat, structured country narratives co-developed with National Open Access Desks (OpenAIRE national experts), and a developing AI-enabled policy resource registry.

Two of the report's formal recommendations build directly on this. Recommendation EU-6 calls on the Commission to continue investing in the EOSC federation, explicitly including the Open Science Observatory, with clearer bridges to national and institutional repositories and targeted support for Member States with limited infrastructure capacity. Recommendation EU-11, the report's preferred path for implementation, goes further: it proposes using the ERA Act as the binding legislative vehicle to transform the current National Points of Reference (NPR) mechanism (until now a voluntary, self-reporting exercise) into a genuine framework for mutual learning, aligned with the EOSC Open Science Observatory.

The report is also candid about the Observatory's current limitations, and we think it's worth being equally candid back. The report notes that the Observatory, like most national monitors, relies partly on self-reported data and could be clearer about which indicators draw on which sources. Further, the Observatory's quantitative layer draws primarily on the OpenAIRE Graph. The Graph is not a static database but part of Europe's open science infrastructure. It methodically harvests from a wide range of sources, including DBLP for computer science, Europe PMC for life sciences, arXiv and Diamond OA hubs, and it feeds into the EOSC Open Science Observatory. Still, monitoring is only as good as the data coming in. Publications that exist only in proprietary CRIS systems, or on publisher platforms without open DOI-based metadata, are harder to capture. Evidence suggests a mixed but encouraging picture. A recent SciMago comparison of open and commercial bibliographic infrastructures found that, among the open platforms, OpenAIRE has the highest coverage of persistent identifiers (73.2% for DOIs, 59.7% for ISSNs) and the lowest share of records missing affiliations (40.55%). It also flagged issues we need to address, such as duplicated DOIs and unclassified document types

How widely the OpenAIRE Graph is relied on across the monitoring landscape adds another angle. When the Open Science Monitoring Initiative (OSMI) Working Group 2 mapped 90 Open Science monitoring initiatives, the OpenAIRE Graph was one of the most frequently cited data sources, mentioned by 22 of them, while OpenAIRE Monitor instances accounted for another 13 entries. That says nothing about data quality on its own, but it shows how many monitoring efforts are already built on this infrastructure.

The report also asks us to be clearer about which indicator draws on which source, and we agree. Improving the transparency of our methodology and looking at how to bring more institutional and national data sources into the picture, so coverage extends further, is an active work in progress. We're also putting real effort into hearing directly from the people who use the Observatory: over the summer, we launched a feedback survey to understand what's working, what's missing, and what would make the platform more useful for researchers, institutions and policymakers alike. That feedback is already shaping what we prioritise next.

None of this is new territory for us. When the European Commission opened its Call for Evidence on the ERA Act, EOSC Track submitted feedback arguing precisely this. We argue that Open Science should not be a side pillar of the ERA but embedded at its core, and that a dedicated set of co-created Open Science indicators belongs in the ERA's governance framework. We backed that position with early lessons from our own monitoring work in a policy brief published alongside our submission.

This new, independent study reaches much the same conclusion and its central recommendation, EU-1, spells out what that framework should look like: a systematic, periodic ERA-level monitoring and reporting mechanism, built on a common indicator framework with standardised definitions and collection methods, with all monitoring data public, machine-readable, and openly licensed.

That's the direction we've been pushing toward, and it's encouraging to see it laid out so explicitly in an EC-commissioned study. If you want to see what EU-level Open Science monitoring can look like in practice today, the Observatory is open to explore.

References: 

European Commission: Directorate-General for Research and Innovation and Spichtinger, D., Open access policies in EU Member States and beyond – Final report, Publications Office of the European Union, 2026, https://data.europa.eu/doi/10.2777/8921907

Szybisty, T. (2025). Monitoring Open Science across ERA: Early Lessons from the EOSC Open Science Observatory [EOSC Track Policy Brief]. Zenodo. https://doi.org/10.5281/zenodo.16991651

Written by Tereza Szybisty

Moderator Natalia Manola (OpenAIRE) opened the session by reframing the premise. Before asking how we measure Open Science, she invited speakers and the audience alike to ask why we measure it at all. The answer, she argued, is not trivial. We invest in Open Science because we believe it changes science and culture, and that change is precisely what indicators need to capture, not just the outputs that are easiest to count. This framing set the tone for everything that followed.

OpenAIRE Monitor: transparency as a foundation

Ioanna Grypari (OpenAIRE) presented the OpenAIRE Monitor, now serving 77 research institutions, 25 research initiatives, 13 funders, and a growing number of national monitors. The service draws on the OpenAIRE Graph, the largest open scholarly knowledge graph in Europe. Two points from Ioanna's presentation generated lively discussion. First, data quality is not a background concern, it is the product. The Irish Monitor, for example, has used the very act of surfacing unexpected indicator values to prompt institutions to clean and improve their own data, turning measurement into a quality improvement loop. Second, the same number means different things to different readers: a 62% open access rate is a benchmark for one institution, a mandate compliance check for a funder, and a national policy progress signal for a ministry. Indicators are not neutral; their meaning is shaped by who reads them and why.

CoARA Working Group on responsible use of metrics and indicators: start with "why"

Katarzyna Nawrot (Co-chair of the CoARA Working Group on Responsible Use of Metrics and Indicators) shared findings from the CoARA working group, including a large-scale survey (learn more about the results of the survey here). Her core argument was methodological: before choosing any indicator, an institution must first ask why it is evaluating, whom or what it is evaluating, and, critically, what latent construct the indicator is supposed to capture.

The working group identifies a pattern they call the "naive non-responsible workflow": picking available indicators without first establishing what you want to measure. This leads, Katarzyna argued, to inappropriate use, not because the indicators are bad, but because the reasoning behind choosing them was never made explicit. The recommended workflow starts with values, moves to assessment goals, defines the construct to be measured, and only then selects indicators, complementing quantitative measures with qualitative assessment throughout.

The discussion touched on a broader provocation: do we measure what we treasure, or do we treasure what we measure? Katarzyna's call for indicators that guide us toward higher-quality research, rather than simply describing current practices, resonated strongly with the audience.

Open Science Monitoring Initiative (OSMI): aligning content providers

Iratxe Puebla (Co-chair of the OSMI Working Group 3) presented the Open Science Monitoring Initiative's work with scholarly content providers, publishers, repositories, and data centres. A key distinction in Iratxe's presentation: this work is explicitly not about building rankings or passing judgment on levels of openness. It is about generating evidence that allows informed decisions, for example, whether a journal should update its open access policy, or how a funder like NIH can monitor data-sharing practices across generalist repositories.

Iratxe raised a challenge that will resonate with anyone working in this space: as more platforms adopt open science monitoring, they are applying different methodologies to calculate indicators for similar practices within open science. Some level of standardisation, or at least consensual definitions, will be needed to prevent a future in which many indicators exist but none are comparable. She also called explicitly for content providers to share their indicator practices openly, so the community can audit and refine them. Learn more about the results of the OSMI Working Group 3 survey here. 

EOSC Open Science Observatory: policy to practice

Tereza Szybisty (OpenAIRE, EOSC Track / EOSC Open Science Observatory) presented the Observatory as a policy intelligence platform built in close collaboration with the EOSC Steering Board and European Commission. Its purpose is to provide national policymakers, country representatives, and the research community with a data-driven picture of where Europe stands on Open Science, across policies, practices, and impacts. The monitoring framework covers eight categories: policies, practices and impacts.

One of the most striking illustrations from Tereza's presentation was the gap between policy existence and researcher coverage. The policy exists on paper; whether it reaches researchers is a different question entirely. This, she argued, is precisely why the Observatory combines quantitative data with qualitative country narratives written by National Open Science Desks - the numbers tell you that a gap exists; the narrative tells you why.

Tereza also surfaced three structural challenges facing the Observatory: comparability (countries answering the same survey questions with different understandings of basic definitions), the gap between policy coverage and live practice, and sustainability, how to maintain continuity of a monitoring mechanism as EOSC moves into its post-2027 governance phase, while simultaneously evolving the framework to match shifting policy ambitions.

The moderated roundtable and audience Q&A surfaced several threads worth carrying forward.

Data quality is a first-order concern. Deduplication, affiliation disambiguation, licence metadata, these are not technical footnotes but conditions for any meaningful indicator. Multiple speakers noted that the human-in-the-loop remains essential, even as algorithmic approaches improve.

Indicators are often most useful as a diagnostic tool. They show what the data behind a policy actually looks like. As Ioanna noted, this matters in its own right. A low number may point to where investment is needed, in repositories, identifiers, or curation, rather than who is falling short. It may also signal incomplete data, not necessarily poor practice. In this way, indicators do more than guide decisions; they help clarify what kind of decision is needed. 

Impact measurement is the missing piece. Several speakers acknowledged that while output indicators (publications, datasets, repositories) are increasingly mature, indicators for scientific and societal impact remain underdeveloped. The PathOS Handbook of Open Science Impact Indicators was highlighted as a resource that the community should collectively work to keep alive and extend.

Openness applies to the monitoring infrastructure itself. A recurring theme was that the credibility of any monitoring initiative depends on whether its methodology, data sources, and indicator definitions are openly available for scrutiny. The EOSC Open Science Observatory makes its data freely available for reuse; the OpenAIRE Graph is publicly accessible; OSMI publishes its principles that can help us to align on Open Science monitoring.

Asked what they hoped to see in two years, the speakers offered a set of complementary ambitions:

• Katarzyna called for Open Science indicators that genuinely guide quality , not just describe coverage, but point toward what matters.
• Iratxe named adoption, maturity, and openness: more content providers monitoring, with indicators that have reached sufficient quality to be useful beyond their originators.
• Ioanna asked for more granular indicators that capture the whole picture of open science: meaningful uptake rather than open-washing, consequences across different groups and contexts, and, with the help of AI, actual use and impact of scientific outputs beyond access alone. 
• Tereza urged a shift from measuring what is easy to count toward measuring what actually changes , how Open Science is making research more useful, accessible, equitable, and impactful for researchers, policymakers, and society.

Across 130 participants and four initiatives, the session surfaced something that is easy to overlook in the day-to-day work of building monitors and designing frameworks: the indicators we choose are never just descriptions. They are arguments about what matters. They shape what institutions report, what researchers prioritise, and what policymakers reward. Which raises a question worth sitting with long after the webinar ended, not are we measuring Open Science the right way, but whose version of Open Science are our indicators quietly encoding?

Written by Tereza Szybisty

The webinar opened against the backdrop of a defining policy moment. The Letta report on the Single Market, the Draghi report on European competitiveness, and the Heitor expert group on research and innovation have each placed Open Science and research data infrastructure at the heart of Europe's strategic agenda. That message is now shaping the negotiations for Horizon Europe 2028–2034, which names Open Science as one of its core principles. In April 2026, the EOSC Tripartite Governance issued a joint narrative paper framing EOSC as a foundation for Europe's data sovereignty, security and competitiveness; only weeks later, the EOSC Steering Board followed with paper positioning EOSC as an indispensable component of the FP10 Partnership portfolio.

But ambition needs evidence. The presentations that followed brought precisely that.

Gareth O'Neill (Technopolis Group, EOSC Track) introduced the Monitoring Framework for National Contributions to EOSC and Open Science, now in its second version, which underpins the annual survey run on behalf of the EOSC Steering Board. The framework organises evidence around three core dimensions (policies, practices and impacts) and across eight categories relevant for EOSC and Open Science including open access publicationsa, FAIR data, software, services, infrastructure, skills/training, assessment, and engagement. The 2024 survey cycle is the third formal iteration, with 31 countries responding including 22 Member States of the European Union, and the data shows a clear trajectory of gradual maturation: more national policies in place, a growing share of them mandatory, and rising national investment, with the most embedded areas (such as open access) leading and newer policy areas (such as FAIR data and open source software) catching up over successive cycles. Detailed findings, including country-level breakdowns and historical comparisons, are available in the full Survey 2024 analysis on the Zenodo community, and the methodology behind the Steering Board monitoring is documented on the EOSC Open Science Observatory.

Ilaria Nardello (EOSC Association) presented the parallel evidence from the EOSC Partnership side. The EOSC Association and its members have committed to €500 million in additional activities by 2030; as of today, contributions stand at €1.6 billion well ahead of the expected halfway mark. In 2025 alone, members invested €340 million in Additional Activities, with a notable share supporting the establishment of national and thematic EOSC nodes, including significant in-kind staff contributions. The 2026 plans show investments in coordination roles for the establishment of the EOSC Federation and its nodes rising sharply, reflecting the operational reality of building the EOSC Federation. Nardello underlined that the Partnership's monitoring captures institutional contributions while the Steering Board Survey captures national-level commitments, and that the additionality between the two is where the leverage effect of the EOSC initiative becomes visible. Further information on EOSC Association monitoring activities can be found on the EOSC Association website.

The panel discussion, moderated by Tereza Szybisty (OpenAIRE) brought together four speakers, Gareth O'Neill, Ilaria Nardello, Pilar Rico Castro (EOSC Steering Board, Spain / FECYT) and Ana Teresa Maia Mota (European Commission).

Monitoring as a learning exercise

Both Gareth O'Neill and Pilar Rico Castro emphasised that the value of the Survey lies not only in the data it captures, but in the convergence of understanding it has built among Member State representatives. Common definitions of "policy," "financial strategy," or "investment" have taken years to stabilise. Pilar Rico Castro added a critical practitioner's perspective: heterogeneity in national data collection strategies (federated in some countries, centralised in others) remains a real constraint on comparability, and harmonising methodologies is a task still very much in progress

The future of Open Science monitoring

Ilaria Nardello argued that the experience of the past few years since the beginning of the partnership has shown the broad differences between the monitoring of the EOSC deployment, its performance and impact and the monitoring of the National Contributions, providing arguments why the two efforts may hardly be coordinated through a single monitoring effort. EOSC is an infrastructure for research, encompassing an ecosystem of resource providers and uptakers, while Open Science is an outcome of this policy initiative; while different types of performance and impact  indicators are required, a common vision for what the monitoring efforts are intended to provide is necessary. Pilar Rico Castro suggested a forward-looking proposal: as national open science monitoring frameworks mature, e.g. in France, Finland, Ireland, the Netherlands, Denmark and now Spain, a federated  model anchored in Member State initiatives (aligned with the Open Science Monitoring Initiative - OSMI) would offer greater long-term sustainability than reliance on the current Tripartite Governance arrangements alone.

EOSC as a connecting backbone

Ana Teresa Maia Mota set out how EOSC, as the Common European Data Space for Research and Innovation, is embedded  into the Commission's policy landscape:  the Competitiveness Compass, the European Strategy on Research and Technology Infrastructures, the AI Continent Action Plan, the European Strategy for AI in Science, and the Data Union Strategy. The challenge ahead is translating this strategic positioning into concrete instruments under FP10: long-term operational funding beyond short project cycles, stronger links between EOSC, EuroHPC, research infrastructures and sectoral data spaces, and targeted action on skills, data stewardship and AI-ready data management.

The piece of evidence that should travel furthest

Panelists were asked which single piece of evidence from the webinar should most directly shape the FP10 negotiations, the panellists offered complementary answers. Ana Teresa Maia Mota pointed to the persistent fragmentation between national investments, policies and implementation as a signal for where alignment is most needed. Ilaria Nardello underlined the synergies and additionality generated by the Partnership as evidence that EOSC is a necessary element of the next Framework Programme. Pilar Rico Castro made the sharpest point: monitoring should never become the purpose itself, and the problems faced in monitoring (scalability and sustainability above all) are the same problems the policy itself must address. Gareth O'Neill concluded that the scale of national investment and engagement now visible in the data calls for matching policy and financial support from FP10 itself.

If there was a single thread running through the webinar, it was this: monitoring is not a technical activity that runs alongside the policy, it is a mirror of the policy itself. The challenges that emerged in the discussion, like the fragmentation between national and European investments, uneven data collection practices across Member States, or the difficulty of aligning EOSC monitoring with the broader Open Science monitoring landscape, are not problems of measurement alone. They are the same scalability and sustainability questions that FP10 will need to answer for EOSC itself.

The next two years will be decisive. The Horizon Europe 2028–2034 negotiations, the revision of the EOSC Strategic Research and Innovation Agenda, the expansion of the EOSC Federation, and the maturing of national Open Science monitoring frameworks across Europe are all happening in parallel. Whether these workstreams reinforce one another, or pull in different directions, will depend on the kind of dialogue this webinar made visible: between Member States and European governance, between the Partnership and the Steering Board, and between those who design policy and those who measure its effects.

Join the next EOSC Track webinar, on the promises and pitfalls of open science indicators, where we will continue this conversation

Register here