Implement single source of truth for each index

[k-allagbe]

Note

This is a temporary measure, until our data lake is ready.

Description

Set up a central repository which will be the single source of truth for each index to facilitate automated index updates and improve data reliability.

Tasks

• Identify existing data sources.
• Set up a central data repository, most likely unstructured like Blob, for each index.
• Enforce read and write policies: only system admins should write.
• Ensure private data stay on private infrastructure.
• Manually update the indexes with the new source.

Acceptance Criteria

• All indexes are up to date with the sources of truth.
• Repository access is correctly restricted: writable only by admins.
• Security measures are implemented and effective.