Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
- GET `/collections` collection search fields extension ex. `/collections?fields=id,title`. [#465](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/465)
- Improved error messages for sorting on unsortable fields in collection search, including guidance on how to make fields sortable. [#465](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/465)
- Added field alias for `temporal` to enable easier sorting by temporal extent, alongside `extent.temporal.interval`. [#465](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/465)
- Added `ENABLE_COLLECTIONS_SEARCH` environment variable to make collection search extensions optional (defaults to enabled). [#465](https://github.com/stac-utils/stac-fastapi-elasticsearch-opensearch/pull/465)

### Changed

Expand Down
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,8 @@ SFEOS implements extended capabilities for the `/collections` endpoint, allowing

These extensions make it easier to build user interfaces that display and navigate through collections efficiently.

> **Configuration**: Collection search extensions can be disabled by setting the `ENABLE_COLLECTIONS_SEARCH` environment variable to `false`. By default, these extensions are enabled.

> **Note**: Sorting is only available on fields that are indexed for sorting in Elasticsearch/OpenSearch. With the default mappings, you can sort on:
> - `id` (keyword field)
> - `extent.temporal.interval` (date field)
Expand Down Expand Up @@ -267,6 +269,7 @@ You can customize additional settings in your `.env` file:
| `ENABLE_DIRECT_RESPONSE` | Enable direct response for maximum performance (disables all FastAPI dependencies, including authentication, custom status codes, and validation) | `false` | Optional |
| `RAISE_ON_BULK_ERROR` | Controls whether bulk insert operations raise exceptions on errors. If set to `true`, the operation will stop and raise an exception when an error occurs. If set to `false`, errors will be logged, and the operation will continue. **Note:** STAC Item and ItemCollection validation errors will always raise, regardless of this flag. | `false` | Optional |
| `DATABASE_REFRESH` | Controls whether database operations refresh the index immediately after changes. If set to `true`, changes will be immediately searchable. If set to `false`, changes may not be immediately visible but can improve performance for bulk operations. If set to `wait_for`, changes will wait for the next refresh cycle to become visible. | `false` | Optional |
| `ENABLE_COLLECTIONS_SEARCH` | Enable collection search extensions (sort, fields). | `true` | Optional |
| `ENABLE_TRANSACTIONS_EXTENSIONS` | Enables or disables the Transactions and Bulk Transactions API extensions. If set to `false`, the POST `/collections` route and related transaction endpoints (including bulk transaction operations) will be unavailable in the API. This is useful for deployments where mutating the catalog via the API should be prevented. | `true` | Optional |
| `STAC_ITEM_LIMIT` | Sets the environment variable for result limiting to SFEOS for the number of returned items and STAC collections. | `10` | Optional |
| `STAC_INDEX_ASSETS` | Controls if Assets are indexed when added to Elasticsearch/Opensearch. This allows asset fields to be included in search queries. | `false` | Optional |
Expand Down Expand Up @@ -413,6 +416,10 @@ The system uses a precise naming convention:
- **Root Path Configuration**: The application root path is the base URL by default.
- For AWS Lambda with Gateway API: Set `STAC_FASTAPI_ROOT_PATH` to match the Gateway API stage name (e.g., `/v1`)

- **Feature Configuration**: Control which features are enabled:
- `ENABLE_COLLECTIONS_SEARCH`: Set to `true` (default) to enable collection search extensions (sort, fields). Set to `false` to disable.
- `ENABLE_TRANSACTIONS_EXTENSIONS`: Set to `true` (default) to enable transaction extensions. Set to `false` to disable.


## Collection Pagination

Expand Down
44 changes: 25 additions & 19 deletions stac_fastapi/elasticsearch/stac_fastapi/elasticsearch/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,9 @@
logger = logging.getLogger(__name__)

TRANSACTIONS_EXTENSIONS = get_bool_env("ENABLE_TRANSACTIONS_EXTENSIONS", default=True)
ENABLE_COLLECTIONS_SEARCH = get_bool_env("ENABLE_COLLECTIONS_SEARCH", default=True)
logger.info("TRANSACTIONS_EXTENSIONS is set to %s", TRANSACTIONS_EXTENSIONS)
logger.info("ENABLE_COLLECTIONS_SEARCH is set to %s", ENABLE_COLLECTIONS_SEARCH)

settings = ElasticsearchSettings()
session = Session.create_from_settings(settings)
Expand Down Expand Up @@ -115,25 +117,26 @@

extensions = [aggregation_extension] + search_extensions

# Create collection search extensions
# Only sort extension is enabled for now
collection_search_extensions = [
# QueryExtension(conformance_classes=[QueryConformanceClasses.COLLECTIONS]),
SortExtension(conformance_classes=[SortConformanceClasses.COLLECTIONS]),
FieldsExtension(conformance_classes=[FieldsConformanceClasses.COLLECTIONS]),
# CollectionSearchFilterExtension(
# conformance_classes=[FilterConformanceClasses.COLLECTIONS]
# ),
# FreeTextExtension(conformance_classes=[FreeTextConformanceClasses.COLLECTIONS]),
]

# Initialize collection search with its extensions
collection_search_ext = CollectionSearchExtension.from_extensions(
collection_search_extensions
)
collections_get_request_model = collection_search_ext.GET
# Create collection search extensions if enabled
if ENABLE_COLLECTIONS_SEARCH:
# Create collection search extensions
collection_search_extensions = [
# QueryExtension(conformance_classes=[QueryConformanceClasses.COLLECTIONS]),
SortExtension(conformance_classes=[SortConformanceClasses.COLLECTIONS]),
FieldsExtension(conformance_classes=[FieldsConformanceClasses.COLLECTIONS]),
# CollectionSearchFilterExtension(
# conformance_classes=[FilterConformanceClasses.COLLECTIONS]
# ),
# FreeTextExtension(conformance_classes=[FreeTextConformanceClasses.COLLECTIONS]),
]

# Initialize collection search with its extensions
collection_search_ext = CollectionSearchExtension.from_extensions(
collection_search_extensions
)
collections_get_request_model = collection_search_ext.GET

extensions.append(collection_search_ext)
extensions.append(collection_search_ext)

database_logic.extensions = [type(ext).__name__ for ext in extensions]

Expand Down Expand Up @@ -170,10 +173,13 @@
"search_get_request_model": create_get_request_model(search_extensions),
"search_post_request_model": post_request_model,
"items_get_request_model": items_get_request_model,
"collections_get_request_model": collections_get_request_model,
"route_dependencies": get_route_dependencies(),
}

# Add collections_get_request_model if collection search is enabled
if ENABLE_COLLECTIONS_SEARCH:
app_config["collections_get_request_model"] = collections_get_request_model

api = StacApi(**app_config)


Expand Down
44 changes: 25 additions & 19 deletions stac_fastapi/opensearch/stac_fastapi/opensearch/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,9 @@
logger = logging.getLogger(__name__)

TRANSACTIONS_EXTENSIONS = get_bool_env("ENABLE_TRANSACTIONS_EXTENSIONS", default=True)
ENABLE_COLLECTIONS_SEARCH = get_bool_env("ENABLE_COLLECTIONS_SEARCH", default=True)
logger.info("TRANSACTIONS_EXTENSIONS is set to %s", TRANSACTIONS_EXTENSIONS)
logger.info("ENABLE_COLLECTIONS_SEARCH is set to %s", ENABLE_COLLECTIONS_SEARCH)

settings = OpensearchSettings()
session = Session.create_from_settings(settings)
Expand Down Expand Up @@ -115,25 +117,26 @@

extensions = [aggregation_extension] + search_extensions

# Create collection search extensions
# Only sort extension is enabled for now
collection_search_extensions = [
# QueryExtension(conformance_classes=[QueryConformanceClasses.COLLECTIONS]),
SortExtension(conformance_classes=[SortConformanceClasses.COLLECTIONS]),
FieldsExtension(conformance_classes=[FieldsConformanceClasses.COLLECTIONS]),
# CollectionSearchFilterExtension(
# conformance_classes=[FilterConformanceClasses.COLLECTIONS]
# ),
# FreeTextExtension(conformance_classes=[FreeTextConformanceClasses.COLLECTIONS]),
]

# Initialize collection search with its extensions
collection_search_ext = CollectionSearchExtension.from_extensions(
collection_search_extensions
)
collections_get_request_model = collection_search_ext.GET
# Create collection search extensions if enabled
if ENABLE_COLLECTIONS_SEARCH:
# Create collection search extensions
collection_search_extensions = [
# QueryExtension(conformance_classes=[QueryConformanceClasses.COLLECTIONS]),
SortExtension(conformance_classes=[SortConformanceClasses.COLLECTIONS]),
FieldsExtension(conformance_classes=[FieldsConformanceClasses.COLLECTIONS]),
# CollectionSearchFilterExtension(
# conformance_classes=[FilterConformanceClasses.COLLECTIONS]
# ),
# FreeTextExtension(conformance_classes=[FreeTextConformanceClasses.COLLECTIONS]),
]

# Initialize collection search with its extensions
collection_search_ext = CollectionSearchExtension.from_extensions(
collection_search_extensions
)
collections_get_request_model = collection_search_ext.GET

extensions.append(collection_search_ext)
extensions.append(collection_search_ext)

database_logic.extensions = [type(ext).__name__ for ext in extensions]

Expand Down Expand Up @@ -167,13 +170,16 @@
post_request_model=post_request_model,
landing_page_id=os.getenv("STAC_FASTAPI_LANDING_PAGE_ID", "stac-fastapi"),
),
"collections_get_request_model": collections_get_request_model,
"search_get_request_model": create_get_request_model(search_extensions),
"search_post_request_model": post_request_model,
"items_get_request_model": items_get_request_model,
"route_dependencies": get_route_dependencies(),
}

# Add collections_get_request_model if collection search is enabled
if ENABLE_COLLECTIONS_SEARCH:
app_config["collections_get_request_model"] = collections_get_request_model

api = StacApi(**app_config)


Expand Down
167 changes: 167 additions & 0 deletions stac_fastapi/tests/api/test_collections_search_env.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
"""Test the ENABLE_COLLECTIONS_SEARCH environment variable."""

import os
import uuid
from unittest import mock

import pytest

from ..conftest import create_collection, refresh_indices


@pytest.mark.asyncio
@mock.patch.dict(os.environ, {"ENABLE_COLLECTIONS_SEARCH": "false"})
async def test_collections_search_disabled(app_client, txn_client, load_test_data):
"""Test that collection search extensions are disabled when ENABLE_COLLECTIONS_SEARCH=false."""
# Create multiple collections with different ids to test sorting
base_collection = load_test_data("test_collection.json")

# Use unique prefixes to avoid conflicts between tests
test_prefix = f"disabled-{uuid.uuid4().hex[:8]}"
collection_ids = [f"{test_prefix}-c", f"{test_prefix}-a", f"{test_prefix}-b"]

for i, coll_id in enumerate(collection_ids):
test_collection = base_collection.copy()
test_collection["id"] = coll_id
test_collection["title"] = f"Test Collection {i}"
await create_collection(txn_client, test_collection)

# Refresh indices to ensure collections are searchable
await refresh_indices(txn_client)

# When collection search is disabled, sortby parameter should be ignored
resp = await app_client.get(
"/collections",
params=[("sortby", "+id")],
)
assert resp.status_code == 200

# Verify that results are NOT sorted by id (should be in insertion order or default order)
resp_json = resp.json()
collections = [
c for c in resp_json["collections"] if c["id"].startswith(test_prefix)
]

# Extract the ids in the order they were returned
returned_ids = [c["id"] for c in collections]

# If sorting was working, they would be in alphabetical order: a, b, c
# But since sorting is disabled, they should be in a different order
# We can't guarantee the exact order, but we can check they're not in alphabetical order
sorted_ids = sorted(returned_ids)
assert (
returned_ids != sorted_ids or len(collections) < 2
), "Collections appear to be sorted despite ENABLE_COLLECTIONS_SEARCH=false"

# Fields parameter should also be ignored
resp = await app_client.get(
"/collections",
params=[("fields", "id")], # Request only id field
)
assert resp.status_code == 200

# Verify that all fields are still returned, not just id
resp_json = resp.json()
for collection in resp_json["collections"]:
if collection["id"].startswith(test_prefix):
# If fields filtering was working, only id would be present
# Since it's disabled, other fields like title should still be present
assert (
"title" in collection
), "Fields filtering appears to be working despite ENABLE_COLLECTIONS_SEARCH=false"


@pytest.mark.asyncio
@mock.patch.dict(os.environ, {"ENABLE_COLLECTIONS_SEARCH": "true"})
async def test_collections_search_enabled(app_client, txn_client, load_test_data):
"""Test that collection search extensions work when ENABLE_COLLECTIONS_SEARCH=true."""
# Create multiple collections with different ids to test sorting
base_collection = load_test_data("test_collection.json")

# Use unique prefixes to avoid conflicts between tests
test_prefix = f"enabled-{uuid.uuid4().hex[:8]}"
collection_ids = [f"{test_prefix}-c", f"{test_prefix}-a", f"{test_prefix}-b"]

for i, coll_id in enumerate(collection_ids):
test_collection = base_collection.copy()
test_collection["id"] = coll_id
test_collection["title"] = f"Test Collection {i}"
await create_collection(txn_client, test_collection)

# Refresh indices to ensure collections are searchable
await refresh_indices(txn_client)

# Test that sortby parameter works - sort by id ascending
resp = await app_client.get(
"/collections",
params=[("sortby", "+id")],
)
assert resp.status_code == 200

# Verify that results are sorted by id in ascending order
resp_json = resp.json()
collections = [
c for c in resp_json["collections"] if c["id"].startswith(test_prefix)
]

# Extract the ids in the order they were returned
returned_ids = [c["id"] for c in collections]

# Verify they're in ascending order
assert returned_ids == sorted(
returned_ids
), "Collections are not sorted by id ascending"

# Test that sortby parameter works - sort by id descending
resp = await app_client.get(
"/collections",
params=[("sortby", "-id")],
)
assert resp.status_code == 200

# Verify that results are sorted by id in descending order
resp_json = resp.json()
collections = [
c for c in resp_json["collections"] if c["id"].startswith(test_prefix)
]

# Extract the ids in the order they were returned
returned_ids = [c["id"] for c in collections]

# Verify they're in descending order
assert returned_ids == sorted(
returned_ids, reverse=True
), "Collections are not sorted by id descending"

# Test that fields parameter works - request only id field
resp = await app_client.get(
"/collections",
params=[("fields", "id")],
)
assert resp.status_code == 200
resp_json = resp.json()

# When fields=id is specified, collections should only have id field
for collection in resp_json["collections"]:
if collection["id"].startswith(test_prefix):
assert "id" in collection, "id field is missing"
assert (
"title" not in collection
), "title field should be excluded when fields=id"

# Test that fields parameter works - request multiple fields
resp = await app_client.get(
"/collections",
params=[("fields", "id,title")],
)
assert resp.status_code == 200
resp_json = resp.json()

# When fields=id,title is specified, collections should have both fields but not others
for collection in resp_json["collections"]:
if collection["id"].startswith(test_prefix):
assert "id" in collection, "id field is missing"
assert "title" in collection, "title field is missing"
assert (
"description" not in collection
), "description field should be excluded when fields=id,title"