Model validation with synthetic data in the public sector

Why consider synthetic data in the public sector

Testing and auditing AI models is essential, but often we run into sensitive personal data (grant applications, land registry reports, administrative records). Using real data limits testing, complicates sharing between teams and creates legal risks. Synthetic data — datasets generated to reproduce the statistical and relational properties of real data without containing real identifiers — make it possible to validate models and processes while preserving privacy and traceability.

This article provides a practical methodology, legal boundaries and concrete controls to integrate synthetic data into the AI lifecycle of a public entity.

Regulatory framework and minimum controls

• GDPR: synthetic data can still be considered personal data if it allows re-identification. Before generating or sharing, assess the re-identification risk and document a Data Protection Impact Assessment (DPIA) when appropriate.
• ENS (RD 311/2022): applies to the systems and environments where synthetic data are stored and processed; access controls, encryption in transit/at rest and audit logging are mandatory according to the system’s level.
• EU AI Act: if the system is "high risk" (for example, automated administrative decisions that affect rights), the generation and use of synthetic data must be recorded in the technical documentation and risk management obligations.

Always involve the legal/data protection team and the security officer from the design phase.

Practical methodology: from requirements to the synthetic dataset

1. Map requirements and risks

   • Identify the attributes needed for tests (e.g. dates, amounts, procedure codes) and classify whether they contain personal data or special categories.
   • Determine whether your goal is functional testing, statistical validation, bias auditing or referential integrity testing.

2. Design the synthetic schema

   • Define the field catalog, relationships between tables and business rules (uniqueness, foreign keys, temporal sequences).
   • Include deliberate edge cases (errors, null values) to test robustness.

3. Generation

   • Methods: statistical sampling, rule-based simulation, or generative models for tabular/text data. Choose based on relational complexity and the need to preserve correlations.
   • Keep traceability: document the generator version, random seed and parameters.

4. Technical validation

   • Utility: compare marginal and joint distributions, temporal patterns and relevant correlations. Check that models trained on synthetic data reproduce the expected behavior.
   • Privacy: run re-identification and membership inference tests. If there’s any doubt, apply additional techniques (e.g. adding noise, limiting fidelity).
   • Audit: retain logs of tests, metrics and differences between models trained on real vs synthetic data.

Concrete use cases in municipalities

• Automated grant application evaluation: test business rules and scoring systems without exposing real applicants.
• NLP testing for regulatory checks: train and refine extraction and classification pipelines using synthesized documents that preserve terminology and structure.
• Load and case-flow simulation: generate large volumes with realistic temporal patterns to test performance and continuity.

Integration into the lifecycle and operations

• Separate environments: generate and store synthetic data in development/test environments; never mix with production data without controls.
• CI/CD: include test suites that use validated synthetic datasets to detect regressions before deployment.
• Monitoring: once in production, compare model performance on real data with synthetic test results to detect drift.
• Artifact retention: version synthetic datasets, generation scripts and validation results to meet traceability and audit requirements.

Common risks and mistakes

• Confusing 'no personal data' with 'no risk': statistical similarity can enable re-identification.
• Creating datasets that don’t reproduce critical dependencies (e.g. foreign keys), leading to unrepresentative tests.
• Failing to document generation parameters; this hinders reproducibility and audits.
• Using synthetic data only for development and neglecting tests with minimally anonymized real samples when required by regulation.

Immediate action checklist (recommended actions)

• 1. Map your data and carry out a scoped DPIA for the use of synthetic data in your use case.
• 2. Define the goal of the tests and the minimum schema required (fields, relationships, business rules).
• 3. Select an appropriate synthesis technique and document parameters/versions.
• 4. Run utility and privacy tests; involve data protection to validate residual risk.
• 5. Incorporate synthetic datasets into CI/CD pipelines and keep records for audit (EU AI Act / ENS).
• 6. Monitor by comparing synthetic vs production behavior and update the generator as needed.

Conclusion: synthetic data are a practical tool to expand testing, improve error detection and protect privacy when applied with methodology, controls and documentation. For municipal teams, well-governed synthetic data reduce friction between technical, legal and operational teams; solutions like OptimGov can provide templates and traceability artifacts for this process. Take action today: plan a pilot for synthetic generation and validation with a concrete use case (e.g. validation of grant rules) and document the result as part of the DPIA.