The Neotec Shadow Experiment: AI-Assisted Review in Public Funding

Problem

The Spanish Innovation Agency (CDTI) struggled to effectively assess and prioritise the social impact of early-stage technology projects within its flagship Neotec funding programme. While CDTI's technical staff possessed strong scientific and technological expertise, social impact is multidimensional, difficult to measure, and historically carried low weight in evaluations, leaving applicants with little incentive to provide high-quality data.

Solution

In collaboration with the Innovation Growth Lab (IGL) and an academic from Humboldt University, CDTI designed a "shadow experiment"—a parallel, risk-free testing environment running alongside the standard Neotec selection process. To support the assessment, IGL developed a dedicated two-step AI system: a predictive language model to generate a numerical social impact score, and a generative AI model (Google Gemini) to produce written justifications based on proposal texts.

Implementation

To avoid legal and fairness concerns regarding public funding allocation, the shadow evaluations began only after the formal funding decisions for the 2025 applications had concluded. The experiment compared evaluations from three distinct groups: CDTI technical staff, internal CDTI volunteers, and external social impact experts. Proposals were anonymised before being processed by the generative AI system to protect data confidentiality.

Outcomes

Predicting the future social impact of early-stage innovations proved highly challenging across all evaluator groups due to inherent uncertainty. However, the integration of AI-generated information produced small but consistent improvements in the evaluators' predictive capacity. The project demonstrated that AI can serve as a valuable decision-support tool, while highlighting that successful public-sector experimentation relies heavily on institutional trust, flexibility, and operational adaptation.

The initiative was conducted within the Neotec programme under the existing membership agreement between CDTI and IGL. Legal restrictions prevented the use of randomisation in official funding decisions, requiring the project to be implemented as a shadow experiment. A confidentiality agreement was signed, and all data were processed in compliance with applicable data protection requirements.