Add grounded safety intelligence generation

[Ashish331-naik]

Summary

Adds evidence-grounded safety explanations and recommendations using the existing risk engine and RAG pipeline.

Included

• Grounded safety intelligence service
• Mock and Groq LLM providers
• Deterministic recommendation validation
• Confidence scoring
• Safe fallback behavior
• RiskEngineInput-to-SafetyIntelligenceResponse pipeline
• Example runner
• Offline tests
• Documentation

Safety

• Recommendations are advisory
• Human approval remains mandatory
• Unsupported equipment IDs and thresholds are rejected
• Retrieved document text is treated as untrusted reference material
• Provider failures return a cautious fallback response

Validation

• Full backend tests pass
• Mock intelligence example works
• Every recommendation cites valid evidence

[Copilot]

Pull request overview

This PR introduces an evidence-grounded “safety intelligence” generation layer on top of the existing risk + retrieval pipeline, with offline-deterministic behavior (mock provider, deterministic embedder), validated public schemas, and documentation/examples to support local development and testing.

Changes:

• Adds new public Pydantic contracts (RiskEngineInput, SafetyIntelligenceResponse, evidence/action/incident models) plus validation tests and JSON examples.
• Implements a deterministic local RAG ingestion + retrieval pipeline (loader/cleaner/chunker/embedder/vector store/reranker + CLI).
• Adds a grounded intelligence orchestration service with mock + Groq providers, deterministic recommendation validation, and confidence scoring.

Reviewed changes

Copilot reviewed 53 out of 71 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
requirements.txt	Adds Groq + PDF parsing deps and documents optional PDF support.
requirements-rag.txt	Adds optional local semantic embedding dependency set.
requirements-dev.txt	Adds dev/test dependency set (pytest).
README.md	Replaces placeholder text with full project overview + run instructions.
pytest.ini	Configures pytest import path + test discovery.
docs/SCHEMA_ARCHITECTURE.md	Adds detailed schema/contract architecture guide and conventions.
docs/SAFETY_INTELLIGENCE.md	Documents grounded intelligence module design, validation, and ops constraints.
docs/RAG_KNOWLEDGE_BASE.md	Documents RAG ingestion/retrieval design, provenance, and limitations.
docs/examples/shift_log_entry.json	Adds validated schema example payload.
docs/examples/safety_intelligence_response.json	Adds validated schema example payload.
docs/examples/risk_engine_input.json	Adds validated schema example payload.
docs/examples/historical_incident.json	Adds validated schema example payload.
docs/examples/extracted_safety_event.json	Adds validated schema example payload.
backend/tests/test_schemas.py	Adds schema validation tests and docs/examples validation.
backend/tests/test_rag.py	Adds deterministic offline tests for ingestion, embeddings, and retrieval/reranking.
backend/tests/test_intelligence_service.py	Adds offline tests for grounding, fallback, confidence, and Groq retry behavior.
backend/examples/risk_to_retrieval.py	Adds runnable example for RiskEngineInput → retrieval handoff.
backend/examples/generate_safety_intelligence.py	Adds runnable example for end-to-end grounded intelligence generation.
backend/data/knowledge/vector_store/.gitkeep	Keeps vector store directory in repo without committing generated indexes.
backend/data/knowledge/raw/synthetic_sops/ventilation_failure_sop.md	Adds synthetic SOP source for retrieval demos/tests.
backend/data/knowledge/raw/synthetic_sops/line_breaking_sop.md	Adds synthetic SOP source for retrieval demos/tests.
backend/data/knowledge/raw/synthetic_sops/hot_work_sop.md	Adds synthetic SOP source for retrieval demos/tests.
backend/data/knowledge/raw/synthetic_sops/gas_testing_sop.md	Adds synthetic SOP source for retrieval demos/tests.
backend/data/knowledge/raw/synthetic_sops/equipment_isolation_sop.md	Adds synthetic SOP source for retrieval demos/tests.
backend/data/knowledge/raw/synthetic_sops/emergency_evacuation_sop.md	Adds synthetic SOP source for retrieval demos/tests.
backend/data/knowledge/raw/regulations/.gitkeep	Keeps regulations folder in repo.
backend/data/knowledge/raw/case_studies/.gitkeep	Keeps case studies folder in repo.
backend/data/knowledge/processed/.gitkeep	Keeps processed folder in repo.
backend/data/knowledge/manifests/documents.json	Adds manifest entries (synthetic + local/pending-review docs).
backend/app/schemas/shift_log.py	Adds shift log + extracted event contracts.
backend/app/schemas/maintenance.py	Adds canonical maintenance contract (and legacy shim support).
backend/app/schemas/maintainance.py	Replaces legacy schema with compatibility re-export shim.
backend/app/schemas/intelligence.py	Adds risk input + evidence/action/intelligence response contracts.
backend/app/schemas/incident.py	Adds incident/near-miss/search/similarity contracts.
backend/app/schemas/common.py	Adds shared enums + constrained string types + base schema config.
backend/app/schemas/init.py	Exposes public contracts from app.schemas.
backend/app/rag/vector_store.py	Adds persistent JSON vector store implementation.
backend/app/rag/text_cleaner.py	Adds conservative safety text cleaning utilities.
backend/app/rag/retriever.py	Adds risk→query conversion + deterministic metadata-aware reranking.
backend/app/rag/models.py	Adds internal ingestion/retrieval models (pages/chunks/results/manifest).
backend/app/rag/metadata.py	Adds manifest parsing + deterministic keyword metadata enrichment.
backend/app/rag/embedder.py	Adds deterministic embedder + optional sentence-transformers adapter.
backend/app/rag/document_loader.py	Adds local .md/.txt/.pdf loader with provenance retention.
backend/app/rag/cli.py	Adds CLI for inspect/ingest/query over the knowledge base.
backend/app/rag/chunker.py	Adds deterministic, section-aware chunking with overlap.
backend/app/rag/init.py	Exposes RAG primitives as a package surface.
backend/app/intelligence/service.py	Adds orchestration service (retrieve → prompt → provider → ground → response).
backend/app/intelligence/providers/mock_provider.py	Adds deterministic mock provider for offline tests/demos.
backend/app/intelligence/providers/groq_provider.py	Adds Groq provider adapter with bounded retries.
backend/app/intelligence/providers/base.py	Adds provider interface + provider error type.
backend/app/intelligence/providers/init.py	Exposes provider implementations.
backend/app/intelligence/prompt_builder.py	Adds grounded prompt assembly with explicit untrusted boundary.
backend/app/intelligence/models.py	Adds intermediate provider output schema + internal evidence model.
backend/app/intelligence/grounding.py	Adds deterministic action grounding/validation rules.
backend/app/intelligence/confidence.py	Adds inspectable confidence heuristic.
backend/app/intelligence/init.py	Exposes intelligence service + provider factory.
AGENTS.md	Adds contributor/repo notes (schemas path, test command, compatibility constraints).
.gitignore	Ignores generated vector stores, caches, and digital twin outputs.
.env.example	Adds environment template for provider configuration.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.