Skip to content

Allow injection of custom Elasticsearch/OpenSearch mappings via configuration #544

@bountx

Description

@bountx

Currently, the application uses a hardcoded default mapping configuration (ES_ITEMS_MAPPINGS). This creates two main issues:

  • Rigidity: Supporting specific STAC extensions (SAR, Cube) or custom fields requires modifying source code or relying on dynamic templates (which may guess types incorrectly).
  • Performance: Dynamic mapping is enabled by default, meaning Elasticsearch indexes every field in the ingested JSON. For large datasets with extensive metadata that isn't queried, this creates unnecessary overhead.

Proposed Solution:
We need a configuration-based approach to extend mappings and control indexing behavior without code changes.

  1. Custom Mappings Injection
  • Introduce STAC_FASTAPI_ES_CUSTOM_MAPPINGS (env var).
  • Accepts a JSON string representing a properties dictionary.
  • Behavior: This dictionary will be merged into the default properties.
  • Override Policy: Custom mappings will overwrite defaults if keys collide. This allows users to refine types but ensures core fields (geometry, datetime) remain unless explicitly redefined (which should be done with caution).
  1. Dynamic Mapping Control
  • Introduce STAC_FASTAPI_ES_DYNAMIC_MAPPING (env var).
  • Default: true (Maintains backward compatibility).
  • Option: false (or strict).
  • Benefit: Setting this to false allows users to define only the fields they want to query via the custom mappings variable, preventing the database from creating mappings for unused data and improving ingestion performance.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions