Repository navigation
Developer Brief Digital File Library
Digital File Library (DFL) Developer Planning Brief DEMI Initiative — Environmental Assessment Office (EAO) Version: Working Draft — April 2026 Status: Pre-Walkthrough Distribution
This document is your pre-read for the DFL walkthrough session. It is not a technical specification and does not define your architecture. Its purpose is to give you enough business context, scope clarity, and working-rule grounding that you can arrive at the walkthrough with informed questions and a basis for roadmap planning.
| What this document gives you | What this document does not give you |
|---|---|
| The business problem the DFL must solve and the strategic context it sits in The confirmed v1 scope boundary: what is in, what is deferred, and why The core business rules that must shape the data model (status, visibility, source precedence, intake) A locked minimal metadata contract (§16) that ETL and migration can proceed on, independent of the IA workstream The provisional role and permission model A set of suggested work packages to seed your roadmap A project artifact register (Appendix C) identifying every planning and delivery document this project needs to produce | A finalized metadata schema — §16 is the ETL floor; the full schema comes from the IA workstream A confirmed technology stack — confirmed at the walkthrough A finalized permissions matrix — provisional roles are described here; the full matrix is a named delivery artifact (Appendix C) Approved KPIs or acceptance criteria — draft success measures are in §23; pending EAO sponsor review A migration runbook or cutover checklist — named delivery artifacts to be produced during Work Package 2 |
Artifacts that will follow this document: This planning brief is the first DFL-specific document after the DEMI strategic plan. The project will produce a defined set of downstream artifacts. See Appendix C for the full artifact register with owners and sequencing. Key artifacts include: Permissions Matrix, Minimal Metadata Contract (locked in §16), Migration Runbook and Cutover Checklist, and Test and Acceptance Plan.
| Your primary contacts during delivery Product Owner: Geoff McDonald — business-rule interpretation and exception decisions DST Liaison and Scrum Lead: Alyd Llewellyn (DST) — day-to-day technical liaison and scrum lead System Architect: EPIC.system: Saravan Azhikoden — EPIC.system architecture Technical Lead: Mark Lisé (CIM Team) Product Owner: EPIC.system: Carly Greene Hill — EPIC.system product alignment and prioritization GIS Lead: Kate McAnally — geospatial project linkage and spatial metadata authority Privacy / Security Representative: TBD — must be assigned before design begins Records Management Advisors: To be identified — ORCS, retention, and disposition decisions |
|---|
This document is the business-aligned planning brief that should guide your architecture decisions, backlog creation, milestone planning, and phased delivery. It is not a final technical specification. The DFL sits at the centre of DEMI — the EAO's digital ecosystem modernization initiative. It is the EAO's future single, authoritative repository for operational project files. EPIC.centre is the primary internal and partner-facing interface. EPIC.eagle and EPIC.public surface public files. SharePoint remains the place for collaboration and drafting, not the long-term file of record. Post-launch operational ownership of all DEMI components transfers to the Digital Services Team (DST).
The EAO's project records are currently spread across multiple systems: EDRMS, EPIC holdings, the I:/ Drive, SharePoint, Teams, and legacy paper records. This fragmentation makes it hard to find the right file, slows down operational work, creates duplication, and increases risk in records management, public access, Freedom of Information response, and long-term continuity. DEMI was created to address this by building a unified digital ecosystem. Within DEMI, the DFL is the official file-of-record repository for operational project files. It holds final authoritative copies, supports consistent metadata and retention rules, enables internal and public discovery, and reduces dependence on fragmented legacy repositories.
| The plain-language goal The DFL is where the EAO should be able to say, with confidence: "this is the file of record." |
|---|
Note on paper records: The EAO holds an estimated 27,000 physical paper records in offsite storage that will undergo scanning, OCR, and AICT classification as a separate future project. DEMI — including the DFL and AICT — is a dependency of that scanning project, not the reverse. The paper records workstream is out of scope for DFL v1 but must not be foreclosed by v1 architecture decisions.
The Digital File Library will be a secure, scalable, cloud-based repository for the EAO's operational project files. It will store authoritative digital files across project lifecycles, support metadata and retention management, preserve version lineage, and allow files to be surfaced appropriately to staff, government users, and the public. See Appendix B for the full DEMI component relationship diagram.
| Component | Role |
|---|---|
| DFL (this system) | Central file-of-record repository and storage layer |
| EPIC.centre | Primary interface for EAO staff and government partner users |
| EPIC.eagle | Legacy public-facing service; being replaced by EPIC.public over time |
| EPIC.public | Modern public-facing service surfacing DFL files marked for public release |
| EPIC.submit | Intake channel for externally submitted project files |
| ETL pipelines | Migration and ingestion from legacy source systems |
| SharePoint / Teams | Collaboration and drafting only — not the file of record |
| EDRMS / I:/ Drive | Legacy sources; to be migrated into DFL over time |
| EPIC.engage | Out of scope for DEMI v1 and v2; may contribute files in v3 or later |
| Architecture note for the walkthrough A conceptual architecture diagram showing the DFL at the centre, with inflows from EPIC.submit and legacy sources, and outflows to EPIC.centre and EPIC.public, will be presented at the walkthrough session. See Appendix B. The specific technology stack — including storage platform — will be confirmed at that session. |
|---|
The day-to-day business owner for the DFL is Geoff McDonald. Until the Information Architecture and EPIC.centre permissions work are fully complete, Geoff acts as the escalation point for business-rule interpretation and exception decisions. Post-launch operational ownership of all DEMI components transfers to the Digital Services Team (DST).
| Role | Person / Group | Notes |
|---|---|---|
| Product Owner | Geoff McDonald | Business-rule interpretation and exception decisions |
| Product Owner: EPIC.system | Carly Greene Hill | EPIC.system product alignment and prioritization |
| DST Liaison and Scrum Lead | Alyd Llewellyn (DST) | Day-to-day technical liaison and scrum lead |
| System Architect: EPIC.system | Saravan Azhikoden | EPIC.system architecture |
| Technical Lead | Mark Lisé (CIM Team) | Senior technical oversight |
| GIS Lead | Kate McAnally | Geospatial project linkage and spatial metadata authority |
| Privacy / Security Representative | TBD — must be assigned before design begins | Privacy impact and security sign-off |
| Records Management Advisors | To be identified | ORCS, retention, and disposition decisions |
| Post-launch operational owner | Digital Services Team (DST) | All DEMI components; effective at go-live |
Developer liaison: During delivery, developers liaise day-to-day with Geoff McDonald and Alyd Llewellyn.
The DFL is the authoritative repository for the EAO's operational project record. It is specifically for files that form part of the operational record of a project, not a general repository for every file that mentions one.
| Bucket | Description |
|---|---|
| Decision-path materials | Anything necessary to reach a project decision: files submitted to, created by, or considered by the EAO. |
| Project communications | Communications about the project (within government and externally) that form part of the operational record. |
| Other non-administrative project materials | Other materials relevant to the project that are not merely administrative records. |
The DFL is not intended to hold:
- General administrative records governed under ARCS
- Indigenous Knowledge files
- Compliance and Enforcement investigation materials
- Purely administrative support records (travel receipts, unrelated internal administration documents)
| Business rule: Upload blocking in v1 The system must not hard-block uploads based on content type in v1. The DFL uses a light-touch model, relying on policy, training, guidance, and AI flagging. Mixed-content or questionable records are handled case-by-case. |
|---|
| Admission and file-of-record rule During Metadata Verification Hold, the file may be physically stored in a staging area within the DFL platform, but it is not admitted as a DFL record (file of record) until verification is completed. Admission is the moment the record becomes searchable (per permissions), receives its final visibility, and is governed by the DFL status/visibility model. |
|---|
A file becomes a file of record when it enters the DFL through an accepted intake path, provided it is a legitimate government-held version and not an erroneous or inappropriate submission. A file does not need to be fully approved in the everyday business sense. What matters is whether it has entered the operational record properly.
| Capability | Notes |
|---|---|
| Secure file storage | Cloud-based; platform confirmed at walkthrough |
| Metadata tagging | Minimal contract locked (§16); AI-assisted enrichment supported |
| ORCS classification and retention attributes | Baseline tagging; advanced disposition automation deferred |
| Version history and lineage | Full version chain preserved |
| Search via EPIC.centre | Top-5 search priorities defined in §15 |
| Visibility and sharing controls | Four-level model; see §11 |
| Single-file upload | Batch upload deferred |
| EPIC.submit integration | Three intake types; see §13 |
| Public surfacing via EPIC.public / EPIC.eagle | Near-real-time; target 30 seconds or less |
| Priority 1A ETL and cutover | ~65,000 EPIC.eagle public files; see §8 |
| Duplicate detection for historical records | Candidate matching with human review; see §9 |
| Minimum logging and traceability | Defined set; see §18 |
- Front-end batch upload capability
- SharePoint promote-to-DFL workflow
- Broader legacy ingestion front-ends beyond Priority 1A
- Finalized permissions matrix (delivery artifact — see Appendix C)
- Finalized enterprise metadata model beyond the §16 floor
- Full advanced audit framework beyond minimum operational logging
- External user groups beyond the defined v1 access model
- Advanced disposition automation beyond ORCS baseline tagging
| Priority | Source System | Notes |
|---|---|---|
| 1A (v1) | EPIC.eagle existing public files (S3) | ~65,000 files; first ETL and cutover stream. These files are the primary AICT training corpus. |
| 1B (v1) | Additional EPIC.eagle public files (gap fill) | Est. 65,000–75,000 digital records total across 1A and 1B |
| 2 (v1) | EPIC.submit | Ongoing intake integration |
| 3 (v1) | EPIC.centre manual upload | In v1 scope |
| 4 (post-v1) | EDRMS | Legacy migration; authoritative source — EDRMS wins on conflict |
| 5 (post-v1) | I:/ Drive | Legacy migration |
| 6 (post-v1) | SharePoint | Legacy migration |
| 7 (post-v1) | MS Teams | Legacy migration |
| Default precedence rule EDRMS wins unless manually overridden. Where the same historical file exists in both EPIC.eagle and EDRMS and the two do not match, EDRMS is the authoritative source. Some EPIC.eagle files were fragmented or altered to meet older technical limits (e.g., broken into 10MB chunks), reinforcing the need for caution during reconciliation. AICT must never override source-precedence decisions made by ETL. |
|---|
Override authority: The OpsDiv Executive responsible for the project decides which version of a conflicting file enters the DFL. The DEMI team will work with EAO Executive on these decisions. Override decisions are executed by DFL SuperUsers acting on that direction. Required v1 system behaviour:
- Automated duplicate detection (matching algorithm and confidence thresholds defined in the Migration Runbook — Appendix C)
- Candidate matching logic sufficient to support human review
- No automated overwrite of authoritative historical records
- Human business review must remain part of the override path at all times
- All override decisions must be logged (who, when, which version chosen, reason)
| Role | Description |
|---|---|
| Project Team Member | Team-level access to files for their specific project |
| EAO Staff | Broader internal access across projects |
| DFL SuperUser | Advanced operational permissions across all non-admin roles. ~6–10 in v1. Can change file status, manage visibility, resolve conflicts, edit protected metadata, and manage exception cases. |
| System Administrator / Developer | Full system-level access for administration and support |
| Permissions matrix: open item The full permissions matrix by role and action is a named delivery artifact (see Appendix C). It is a dependency that must be resolved before public-facing cutover. In the interim, default visibility rules per intake type (see §13) govern what SuperUsers and staff can and cannot expose. |
|---|
Visibility defines which audience group can see a file. It is one of two independent dimensions governing file access — the other is Status (§12). Changes must be controlled and logged.
| Standard paragraph: Intended vs effective visibility rule During hold, intended visibility may be captured but is not effective. On admission, default visibility is Project Team Only unless a permitted user explicitly sets a different visibility. |
|---|
| Visibility Level | Description |
|---|---|
| Project Team Only | Used for unapproved or in-progress operational records. Accessible only to project team members. Default on admission. |
| All EAO | Used for approved internal records. Accessible to all EAO staff. |
| All IDIR | Used for records intended for broader government sharing. Accessible to all BC Government IDIR users. |
| Public | Used for records released through EPIC.public / EPIC.eagle. Accessible to the general public. Requires explicit permitted human action and logging. |
| Status \ Visibility | Team Only | All EAO | All IDIR | Public |
|---|---|---|---|---|
| Active | Yes | |||
| Superseded | Yes | Rare | No | |
| Transitory | Yes | Rare | No | |
| Retracted | Yes | Rare | Redacted only |
| Legend | Meaning |
|---|---|
| Yes | Normal and expected combination |
| Rare | Technically possible but uncommon; requires explicit justification |
| No | Not a supported combination in v1 |
| Redacted only | File may appear publicly only if it carries a formal Redacted flag |
Status defines the current standing of a file within the operational record. It is one of two independent dimensions governing file access — the other is Visibility (§11).
| Status | Meaning | Notes |
|---|---|---|
| Active | The current valid file of record. | Default status on entry through any accepted intake path. |
| Superseded | An older version replaced by a newer valid version. | Remains in the record unless disposition rules determine otherwise. |
| Transitory | A file that should not remain in the enduring record. | Must include a reason. May be flagged for culling through records management tools. |
| Retracted | A file no longer primary for use or sharing. | Must include a reason. Retraction withdraws the file from EPIC.public unless officially redacted. |
Status changes may be applied by members of the team that posted the file, or by DFL SuperUsers.
Superseded and retracted records must remain understandable in context. Users must be able to see that a file was once primary and is no longer primary, and why.
The three types below correspond to existing EPIC.submit process classifications.
| Type | Enters DFL | Default Visibility | Version Behaviour |
|---|---|---|---|
| Type 1 | At submission | Project Team Only | May be managed through EPIC.centre as part of the operational record if it joins the decision path. |
| Type 2 | At submission | Project Team Only (broadens after staff confirmation) | Visibility expands once a staff member confirms the file is the expected item. |
| Type 3 | Each version at submission | Project Team Only | Earlier versions marked Superseded. A SuperUser may later apply ORCS rules to determine if superseded versions are Transitory. |
The DFL holds the file. EPIC.public / EPIC.eagle displays files marked "share to public."
- At upload through EPIC.centre
- At approval through EPIC.submit
- At any time later through the share/access interface in EPIC.centre
Once marked for public access, the file must appear in EPIC.public near real-time. Target: 30 seconds or less.
- EPIC.public shows nothing, unless the file has been officially redacted.
- If officially redacted, EPIC.public displays the file name with a Redacted flag.
| Security dependency The must-not-fail condition "no exposure of internal-only files" depends directly on the permissions matrix being locked before any public-facing cutover. This is a hard gate. See Appendix C. |
|---|
The confirmed top-five search priorities for DFL v1:
| Priority | Search Requirement |
|---|---|
| 1 | Find all files for a specific project |
| 2 | Find the current file of record for a document type within a project |
| 3 | Find all public files for a project |
| 4 | Find all files of a given file type across all projects |
| 5 | Find all files with a given topic metatag across all projects |
| Performance target Search results must appear within 10 seconds or less. A visible progress indicator is required where results take more than 2 seconds. Load profiles and SLOs will be defined in the Test and Acceptance Plan (Appendix C). |
|---|
Metadata is foundational to the DFL. All files must carry common tags and ORCS-linked attributes. The AI Classifier (AICT) applies curated tags and retention-related metadata using controlled vocabularies that are maintainable by EAO staff and re-runnable against backfiles.
| Status: LOCKED for migration purposes This contract is locked and independent of the IA workstream. ETL and migration may proceed on these fields. If the full IA metadata schema is delayed, this contract is frozen as-is and migration proceeds on it. The full enterprise metadata model extends these fields but does not replace them. |
|---|
Required fields — all 12 must be populated on every ingestion:
| Field | Type | Cardinality | Controlled Vocabulary / Example |
|---|---|---|---|
| record_id | string (UUID) | 1 | System-generated. e.g., dfl-uuid-1234 |
| title | string | 1 | Human-readable title of the file |
| project_id | string | 1 | EAO canonical project code |
| file_type | string | 1 | pdf │ docx │ geotiff │ zip │ msg |
| creator | string | 1..* | Person or organization; name (ORCID/email optional) |
| created_date | datetime (ISO 8601) | 1 | Original file creation date |
| source_system | string | 1 | EPIC.public │ EPIC.submit │ EDRMS │ IDrive │ manual-upload |
| ingestion_date | datetime (ISO 8601) | 1 | Date/time file entered the DFL |
| version_id | string | 1 | Version token; links to version lineage |
| status | string | 1 | Active │ Superseded │ Transitory │ Retracted |
| visibility | string | 1 | ProjectTeam │ AllEAO │ AllIDIR │ Public |
| retention_class | string | 1 | ORCS tag or schedule reference. e.g., ORCS:Operational-7yrs |
Optional but strongly recommended fields:
| Field | Type | Notes |
|---|---|---|
| checksum | string (SHA-256) | File integrity verification; critical for migration validation. Treat as effectively required for Priority 1A. |
| file_size | integer (bytes) | Required for storage planning and upload validation |
| mime_type | string | IANA media type; e.g., application/pdf |
| keywords/tags | string[] | Topic metatags; enables top-5 search priority #5 |
| public_flag | boolean | Explicit flag indicating public release status |
| redaction_flag | boolean + reason | Indicates official redaction; drives EPIC.public display behaviour |
The full enterprise metadata model — controlled vocabularies, extended attributes, and AI classifier configuration — is coming from the IA workstream and must be delivered before Work Package 6 begins. The minimal contract above is the build floor until then.
- v1 must support a metadata structure extensible without redesign
- AI-assisted enrichment must be supported where possible
- Protected metadata editing must be available to SuperUsers
For v1, the DFL must support all file types, including: PDF, Word/Excel/PowerPoint, images, video and audio, GIS and spatial files, zipped packages, email message files, and very large application packages.
| Requirement | Detail |
|---|---|
| Maximum file size | Potentially gigabytes. Validate average and distribution from EPIC.eagle and EDRMS during design. |
| Malware / virus scanning | Standard security controls required on every upload. |
| Upload performance target | Normal single-file upload within 20 seconds. |
Full enterprise audit capability is not required on day one. The following minimum operational log must be preserved from the start:
| Event | What to Log |
|---|---|
| File upload / staging | Who uploaded, when, source system |
| File admission (verification completion) | Who completed verification, when, intended vs effective visibility set |
| Visibility change | Who changed visibility, from what level to what level, when |
| Retraction | Who retracted the file, when, reason |
| Version supersession | Which version superseded which, and when |
| Source precedence override | Who made the decision, which version was chosen, reason given |
| Public publication | Date and time of publication to EPIC.public |
| Public withdrawal | Date and time of withdrawal from EPIC.public, reason if applicable |
The DFL follows a standard DEV → TEST → PROD release path. v1 rollout sequence:
- Complete build and integration in DEV
- Validate in TEST (intake rules, classifier behaviour, visibility changes, load handling)
- Execute Priority 1A ETL migration into the DFL
- Perform cutover
- Make the DFL-backed EPIC.public path live Priority 1A is the critical first migration stream. Public continuity is a core cutover condition. A Migration Runbook and Cutover Checklist will be produced during Work Package 2 and must be signed off before go/no-go. See Appendix C.
Post-launch operational ownership of all DEMI components transfers to the Digital Services Team (DST).
| Function | Owner |
|---|---|
| Day-to-day administration | OpsDiv |
| User support | Digital Services Team (DST) |
| Metadata and rules maintenance | DST |
| Retention and disposition decisions | Records Management |
| Requirement | Target |
|---|---|
| Storage infrastructure | Secure, government-approved cloud infrastructure |
| File volume and size | Large file volumes; very large individual files (potentially GBs) |
| Metadata model | Extensible without redesign |
| Integration | Compatible with EPIC.centre, EPIC.submit, and EPIC.public |
| Traceability | Minimum log defined in §18, operational from day one |
| Migration resilience | Stable operation during migration and cutover windows |
| Security scanning | Malware and virus scanning on every upload |
| Search response | 10 seconds or less |
| Single file upload | 20 seconds or less |
| Public publishing | 30 seconds or less |
| Availability | 5% or less acceptable downtime |
| Visibility controls | Controlled, logged state transitions |
- Misalignment with EPIC release schedules
- Metadata drift: migrating with provisional tags creates inconsistent search and retention behaviour
- Security exposure: undefined permission mapping combined with automated ETL risks accidental public exposure of internal files
- SuperUser capacity: 6–10 SuperUsers may be insufficient for manual conflict resolution at scale during Priority 1A
- IA dependency: delayed metadata contract blocks Work Package 6 and may require freezing the §16 floor earlier than planned
- Over-engineering before IA and permissions work is settled
| These are hard gate criteria. They must be built into testing, migration validation, and the Cutover Checklist. No go/no-go decision may proceed without sign-off on all four. 1. No public file loss 2. No interruption to EPIC.public document availability 3. No exposure of internal-only files to public channels 4. No broken project-to-file relationships |
|---|
Must-not-fail condition #3 has a direct dependency: the permissions matrix must be locked and tested before any public-facing cutover can proceed.
Success will be judged by: repository consolidation, discoverability, metadata quality, reduced duplication, and more efficient operational use.
- EPIC.eagle public files are successfully moved into the DFL without loss
- Public document access continues without disruption
- Staff can reliably find current versions of project files
- New submitted files flow into the DFL correctly
- Visibility controls work for EAO staff
- Duplicate repositories begin to reduce
| Draft KPI candidates — PENDING EAO sponsor review The following are working candidates only. They have not been reviewed or approved by EAO project sponsors and must not be treated as acceptance criteria. Formal acceptance criteria — with measurement methods, owners, data sources, and pass/fail thresholds — will be defined in the Test and Acceptance Plan (see Appendix C). • 100% of Priority 1A files migrated with validated file counts and project links • 0 confirmed incidents of internal-only files exposed publicly at cutover • Public-facing document availability maintained during and after cutover • 95%+ of test search tasks completed successfully by pilot users • 95%+ of new EPIC.submit files routed correctly to the DFL during pilot • Measurable reduction in ongoing use of duplicate storage for pilot projects |
|---|
These items are in progress. They are not blockers to roadmapping, but each will constrain specific design decisions as it resolves. See Appendix C for the artifact register that tracks resolution ownership and sequencing.
| Open Item | Current Assumption | Blocks |
|---|---|---|
| Full metadata schema and controlled vocabularies | §16 minimal contract is the ETL floor until IA delivers | Work Package 6 |
| Permissions matrix by role and action | Provisional role model in §10; default intake visibilities in §13 | Public-facing cutover (hard gate) |
| Technology stack / storage platform | Azure strongly expected (BC GOV Microsoft contract) | Confirmed at walkthrough |
| EPIC.centre search and share interface design | Top-5 search priorities defined in §15 | Work Package 4 detailed design |
| Privacy and security advisor assignment | TBD — must be assigned before design begins | Work Package 7 and cutover security sign-off |
| Disposition automation rules | ORCS baseline tagging only for v1 | Post-v1 phase |
| Exception handling for mixed-content / sensitive records | Case-by-case; SuperUser + policy guidance | Operational policy document; before launch |
| KPI and acceptance criteria sign-off | Draft candidates in §23 | Test and Acceptance Plan; before testing phase |
These work packages seed your roadmap. Each maps directly to the v1 scope defined in §7. Effort sizing, sprint assignment, and detailed ownership will be established at and after the walkthrough session.
| Work Package | Description | v1 Scope Note | Key Dependency |
|---|---|---|---|
| WP1: Core repository and data model | Build the DFL storage model, file entity structure, status model, visibility model, version lineage, and minimum logging. | Fully in v1 scope | §16 metadata contract (locked) |
| WP2: Priority 1A migration and reconciliation | Design and implement ETL for EPIC.eagle public S3 files, including duplicate detection, project linking, validation, and cutover readiness. Produce the Migration Runbook and Cutover Checklist. | Fully in v1 scope. Priority 1B follows. | §16 metadata contract; permissions matrix |
| WP3: EPIC.submit integration | Design submission intake pathways, type-based file handling, version behaviour, and permission state transitions for all three submission types. | Fully in v1 scope | WP1 complete |
| WP4: EPIC.centre internal interface | Enable search (top-5 priorities), manual upload, metadata display, visibility control, and status handling for internal and government users. | Fully in v1 scope | WP1 complete; permissions matrix |
| WP5: EPIC.public publication path | Enable near-real-time public surfacing of DFL files based on public-share flags, including redaction behaviour. | Fully in v1 scope | Permissions matrix (hard gate before go-live) |
| WP6: Metadata and AI integration | Implement the baseline metadata contract and prepare for AI classifier enrichment and ORCS tagging. | Partially in v1 scope. Full enterprise metadata model and advanced disposition deferred. | IA workstream metadata contract; falls back to §16 floor if delayed |
| WP7: Security, operations, and support readiness | Implement upload security controls (malware/virus scanning), monitoring, support handoff, and operational admin functions. | Fully in v1 scope | Privacy/security advisor assignment (currently TBD) |
The Digital File Library is the foundational repository layer for DEMI. It is the mechanism by which the EAO will establish a single, authoritative operational project record across internal, government, and public contexts. The first release should focus on a practical, high-value outcome: move the current EPIC.eagle public file base into the DFL safely, preserve continuity of public access, enable proper intake from EPIC.submit, provide internal search and visibility control, and establish the core rules for status, versioning, metadata, and source precedence.
| Next step Bring your questions from this document to the walkthrough session. The walkthrough will confirm the technology stack, walk through the architecture, and give you the opportunity to identify gaps, raise risks, and align on the planning approach before roadmapping begins. After the walkthrough, priority artifact production is: Permissions Matrix, Migration Runbook and Cutover Checklist, and Test and Acceptance Plan. See Appendix C. |
|---|
The terms below are canonical across all four DEMI developer planning briefs. Use these definitions wherever the terms appear.
| Standard paragraph: Admission and file-of-record rule During Metadata Verification Hold, the file may be physically stored in a staging area within the DFL platform, but it is not admitted as a DFL record (file of record) until verification is completed. Admission is the moment the record becomes searchable (per permissions), receives its final visibility, and is governed by the DFL status/visibility model. |
|---|
| Standard paragraph: Intended vs effective visibility rule During hold, intended visibility may be captured but is not effective. On admission, default visibility is Project Team Only unless a permitted user explicitly sets a different visibility. |
|---|
| Term | Definition | Notes |
|---|---|---|
| Admission | The moment a staged file becomes a DFL record. At admission the record becomes searchable (per permissions), receives effective visibility, and is governed by the DFL status/visibility model. | See standard paragraphs above |
| AICT | AI Classifier Tool. An assistive metadata enrichment and classification service. AICT never makes business-critical decisions autonomously. | |
| DFL | Digital File Library. The authoritative repository and storage layer for EAO operational project files. | |
| DFL record / File of record | A file admitted into the DFL through an accepted intake path, governed by DFL metadata, status, visibility, and logging rules. | |
| EDRMS | Electronic Document and Records Management System. The BC Government-sanctioned system of record. EDRMS files take precedence over transformed or fragmented counterparts in other systems. | |
| Effective visibility | The visibility level that actually controls access to a record. Begins at admission. | |
| EPIC.centre | The internal and government-partner operational interface for search, upload, metadata verification, and visibility management. | Canonical spelling: 'centre' |
| EPIC.eagle | The legacy public-facing service and historic upload pathway, being replaced by EPIC.public over time. | |
| EPIC.engage | An EPIC.system application not in scope for DEMI v1 or v2. May contribute files to the DFL in v3 or later. | Out of scope — v3+ |
| EPIC.public | The modern public-facing service that surfaces publicly accessible DFL files. | |
| EPIC.submit | The invitation-only external intake portal for Proponents and Certificate Holders. Files enter the DFL after AICT verification and EPIC.submit approval. | |
| EPIC.system Integration | The DEMI workstream delivering EPIC.centre changes for DFL search, upload, verification, and visibility management. Informally: EPIC Integration. | |
| ETL | Extract, Transform, Load. The pipeline process for migrating files from legacy source systems into the DFL. | |
| File ETL | The DEMI workstream migrating legacy files from EDRMS, EPIC.eagle, I:/ Drive, SharePoint, Teams, and other sources into the DFL. | |
| Intended visibility | A visibility selection captured during Metadata Verification Hold. Not effective until admission. | |
| Metadata Verification Hold | A workflow hold state (informal: pause). File is staged but not yet admitted and not broadly searchable. | Not a DFL Status value |
| Public surfacing | A DFL record with Public effective visibility displayed in EPIC.public. Requires explicit permitted human action; logged. | |
| Status (DFL) | The lifecycle state of a DFL record: Active, Superseded, Transitory, or Retracted. | |
| Visibility (DFL) | The audience access level of a DFL record: Project Team Only, All EAO, All IDIR, or Public. |
The diagram below shows the full DEMI ecosystem: intake channels that bring files into EPIC.centre and the ETL pipelines, the AI Categorizer (AICT) that enriches them, the Digital File Library that stores authoritative records, and the public and operational interfaces through which files are surfaced. Note on EPIC.engage: EPIC.engage is an EPIC.system application not in scope for DEMI v1 or v2. It appears in this diagram for completeness. In a future version (v3 or later), files collected in EPIC.engage may flow into the DFL. No developer action is required in the current scope.
This register identifies every planning and delivery document the DFL project needs to produce. Owners, dates, and sequencing will be confirmed at and after the walkthrough session.
| This document in context This Developer Planning Brief is the first DFL-specific document after the DEMI Strategic Plan. All artifacts listed below are downstream of this brief. |
|---|
Artifact sequencing legend:
- Phase 1: Pre-build / design (must exist before build begins)
- Phase 2: During build (produced as part of delivery)
- Phase 3: Pre-cutover (must exist and be signed off before go/no-go)
- Phase 4: Post-launch (operational and ongoing)
| Artifact | Description | Phase | Owner | Status |
|---|---|---|---|---|
| DFL Developer Planning Brief (this document) | Business context, scope, business rules, minimal metadata contract, work packages. | 1 | Geoff McDonald | DRAFT — pre-walkthrough |
| Permissions Matrix | Full mapping of roles to actions, including visibility transition rights, status change rights, and protected metadata access. Hard gate before public cutover. | 1 | Geoff McDonald + Alyd Llewellyn | NOT STARTED |
| Full Enterprise Metadata Model | Extended metadata schema, controlled vocabularies, AI classifier configuration. Delivered by IA workstream. | 1 | IA Workstream | IN PROGRESS |
| Information Architecture (IA) Document | Full IA including taxonomy, metadata schema, controlled vocabularies, and EPIC.centre integration points. | 1 | IA Workstream | IN PROGRESS |
| Privacy and Security Assessment | Privacy impact assessment and security review for DFL design and public surfacing path. | 1 | Privacy/Security Advisor (TBD) | BLOCKED — advisor not yet assigned |
| Architecture Decision Record(s) | Key architecture decisions: storage platform, integration patterns, ETL approach. | 2 | Saravan Azhikoden + Alyd Llewellyn | NOT STARTED |
| Migration Runbook | ETL approach for Priority 1A (~65,000 files): staging validation, reconciliation algorithm, confidence thresholds, SuperUser review queue, throughput targets. | 2 | WP2 Lead | NOT STARTED |
| Duplicate Detection and Conflict Resolution Spec | Matching algorithm detail, confidence thresholds, human-review queue design, SLA for manual review, and audit log requirements. | 2 | WP2 Lead + Geoff McDonald | NOT STARTED |
| EPIC.submit Integration Spec | Detailed intake rules for Type 1/2/3 files, including state transitions, visibility defaults, and version handling. | 2 | WP3 Lead + Geoff McDonald | NOT STARTED |
| SuperUser Operational Playbook | Quick reference for SuperUsers: how to retract, supersede, redact, resolve conflicts, and log decisions. | 2 | Geoff McDonald + OpsDiv | NOT STARTED |
| Test and Acceptance Plan | Formal acceptance criteria for each KPI. Covers migration validation, security validation, search validation, and operational readiness. | 3 | Geoff McDonald + Alyd Llewellyn | NOT STARTED — pending sponsor KPI sign-off |
| Cutover Checklist | Step-by-step cutover sequence with named approvers and go/no-go gate criteria. Must include must-not-fail conditions from §22.2. | 3 | WP2 Lead + Geoff McDonald | NOT STARTED |
| Security Sign-off | Formal security clearance for public-facing cutover. Covers permissions matrix testing and internal/public file separation. | 3 | Privacy/Security Advisor | BLOCKED — advisor not yet assigned |
| Operational Runbook | Day-to-day operational procedures for OpsDiv and DST: administration, user support, metadata maintenance, incident escalation. | 4 | OpsDiv + DST | NOT STARTED |
| Retention and Disposition Rules | Formalized ORCS-aligned disposition rules for DFL records. Extends the v1 ORCS baseline. | 4 | Records Management | NOT STARTED |
| Immediate priorities after the walkthrough 1. Assign the Privacy and Security Advisor — unblocks WP7 and the Security Sign-off 2. Begin the Permissions Matrix — hard gate for public-facing cutover 3. Confirm KPIs with EAO sponsors — unblocks the Test and Acceptance Plan 4. Begin the Migration Runbook — Priority 1A ETL design cannot be finalized without it |
|---|
The following are external inputs or approvals that the DFL development team must design to accommodate. These are not open architecture questions — they are dependencies with named owners and timing requirements.
| Dependency | Required for | Owner / Source | Timing |
|---|---|---|---|
| Information Architecture workstream — controlled vocabularies and extended metadata model | Final metadata field contract; vocabulary grounding for AICT | IA workstream (EAO) | Must precede Work Package 6 |
| BC Government ORCS manual and retention schedules | ORCS and retention tagging | BC Government / GIMO | Must be in hand before ORCS classification development |
| Permissions Matrix (full role-to-action mapping) | Public-facing cutover gate | Geoff McDonald + Alyd Llewellyn | Hard gate before EPIC.public go-live |
| Privacy and Security Advisor assignment | Privacy Impact Assessment and security sign-off | EAO / Ministry | Must be assigned before design begins |
| MISO approval — EPIC.public corpus as AI training data | AICT model training | Ministry Information Security Officer | Must be confirmed before AICT training commences |
| EPIC.track — First Nations names list | First Nations name classification input for AICT | EPIC.track / EPIC.system team | Must be available before AICT goes to production |
| Test and Acceptance Plan | Formal acceptance criteria and performance thresholds | Development team + DST | To be produced during delivery |
- Environment Reality & Gotchas
- Architecture
- Seeding — filling Cosmos from Track and Eagle
- Sources and Status — every source, one state
- Search & Retrieval — overview
- Search Index Reference
- Search Query Construction
- Search Measurements
- Extraction Pipeline
- Extraction Quality — coverage, word-joining, tiling
- Identity Model — which id means what
- Attribute-Level Access — field-visibility levels, catalog
- Sync Architecture — how records get in
- Eagle Mirror Reference — mirrored kinds, refusals, drift classes
- ENGAGE Rows Reference — ENGAGE-owned comment periods, reconcile
- Public-Read Backfill — filling and repairing the mirror containers
-
Short Links —
/s/{code}redirect and/api/links -
Recent Uploads —
/api/documents/recent-uploads, eagle-public homepage panel - Reports — comment export and BCGW CSV reads
-
User Data — saved map areas, preferences,
/meroutes -
Bulk Download —
/api/bulk-downloads, zip jobs - Bulk Download: Operations — enable, poison-queue runbook
- Document Intake — upload link, malware scan, release
- Connecting an Application to DEMI
- Azure Environments
- Document Backup — archive copy of originals, run and check
- Document Backup: Revert and Drill — put documents back from the archive
- PDF Titles — public name written into stored PDFs, run, restore
- Secret Sync — Key Vault, secret-sync Function app, required names
-
Operating DEMI in Azure — deploy, verify
- Operating the DEMI Data Plane — devbox access, probes, audit and cost reads
- Edge ban detector — hourly timer, modes, kill switch, soak and promote
-
CI Workflows — every GitHub Actions workflow, trigger, identity, roles
- CI Deploy Workflows — eagle-demi staging and production deploys
- CI Admin Frontend Workflows — eagle-demi-admin checks and deploys
- Release Process — cutting, tagging, shipping and rolling back a version
- Runbook: Search Outage — index/app schema drift, diagnose, fix, roll back
- BC Gov Landing Zone
- Business Rules
- EPIC.search Business Rules — where DEMI differs from EPIC.search
- 001: Typesense for DFL Search — superseded
- 002: Async Processing (MongoDB Queue) — superseded
- 003: Rule-Based Classification
- 004: Read ACL Authorization Model
- 005: Cosmos Full-Text Search Ruled Out
- 006: AI Summarization over BM25
- 007: Service-to-Service Credentials
- 008: Audit and Usage Analytics Store
- 009: Track Feed — proposed
- 010: Attribute-Level Access Control
- 011: Public Read Surface
- 012: Public Config Surface
- 013: Stored Project Summaries — proposed
- 014: Sync-out to Eagle
- 015: Malware scanning on intake