Implement GET /collections/{id} endpoint with validation and QueryBuilder integration - #186
Merged
Conversation
…match 05_indexes.sql correctly
- Updated `buildCollectionSearchQuery` to support pagination and improved text search with English language settings. - Modified tests in `buildCollectionsSearchQuery.basic.test.js`, `collections-pagination.test.js`, and `collections-sort.test.js` to reflect new query behavior and validation logic. - Enhanced sort validation in `validators.test.js` and `collectionSearchParams.js` to map API fields to database column names. - Implemented total count retrieval for matched results in `collections.js`.
…ub Repository Secrets to not publish any private Logins and stuff.
…eeds a internal .env file too.
Updated the documentation for: - the buildCollectionSearchQuery function - the fulltextsearch
Changed the SELECT part to match our bid and the database shema. Updated comments and for clarity. Changed full-text search to use 'simple' configuration instead of 'english'.
small typo Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
Removed TODO comment about sorting based on sortby parameter.
Update validateSortby function to explicitly return undefined for normalized when sortby is not provided.
Change plainto_tsquery language from 'english' to 'simple'
Remove duplicate 'SELECT' keyword in SQL query.
Refactor validateSortby function to handle optional sortby parameter and improve validation logic.
…eout Run Jest in CI with --runInBand and a higher default --testTimeout to stabilize database-backed integration tests. Multiple Jest workers were competing for the same PostgreSQL connection pool and some long-running /collections queries exceeded the default 5s timeout, causing failures in existing test suites (e.g. collectionSearch and DBconnection).
- Added a global Teardown for jest and force-exited the tests to prevent leaking. - Made a change to db_APIconnection to only log the pool-(dis)connection if it isn't run in a test enviroment.
- Now we only check once for a empty sortby - And added a test which distinguish between `sortby=""` and `sortby="+"`
Removed the TODO about switching from mock-data to the real db
RobinGummels
requested changes
Dec 10, 2025
RobinGummels
left a comment
RobinGummels
left a comment
Contributor
There was a problem hiding this comment.
Looks pretty good. Please have a look into my remarks and request a Re-Review when you are finished.
…test" in the validator.
…to dev-api-vincent
- Renamed `collection.id` to `c.collection.id`
…to dev-api-vincent
i encoded the "-1" value in the negative ID test instead of directly putting it into the path.
Contributor
Author
|
The tests are now running and I have addressed all your review comments. |
RobinGummels
approved these changes
Dec 18, 2025
RobinGummels
left a comment
RobinGummels
left a comment
Contributor
There was a problem hiding this comment.
Looks pretty good. I like it. Maybe remove the one german comment and then write me, so i will merge it.
RobinGummels
added a commit
that referenced
this pull request
Dec 31, 2025
…and more Queryables-Parameters (#204) * Updated Query-Builder to get all necessary fields from all db_tables for collections. Added some tests and fixed some already existing tests, becuase now the tablenames start with the alias `c.`. * Updated Query-Builder to get all necessary fields from all db_tables for collections. Added some tests and fixed some already existing tests, becuase now the tablenames start with the alias `c.`. * Added `openapi.yaml` (now http://localhost:3000/api-docs/ is working). - needed to do some modifying to the app.js * Added discription on how to use `stac-api-validator`. Currently we are onyl valid to `core`. * Changed API-Version name to 1.1.0 instead of 1.0.0 * Revert "API is now responding with all necessary fields for each collection" (#195) Reverts #185 @SonkeHoffmann accidentally didn't squash correctly. * Revert "Revert "API is now responding with all necessary fields for each collection"" (#185) (#195) (#196) dev-api: prepare v1.1.0 + API docs + query builder fixes - Change API version to 1.1.0 - Add OpenAPI spec so /api-docs works locally - Document stac-api-validator usage - Update api/.env.example - Query builder: select required fields for collections across db_tables; adjust tests (alias `c.`) Commits included: - 34bf962 Changed API-Version name to 1.1.0 instead of 1.0.0 - b047389 Added description on how to use `stac-api-validator` (currently only valid for `core`) - b811288 Added `openapi.yaml` (so http://localhost:3000/api-docs/ works); modified app.js accordingly - 6e5ab3e Merge branch 'dev-api-robin' of github.com:SpatioCore/STAC-Atlas into dev-api-robin - 70dc043 Updated Query-Builder to get all necessary fields from all db_tables for collections. Added tests and fixed existing tests (table names now start with alias `c.`) - d83eeb4 Update api/.env.example - 5a7af5b Updated Query-Builder to get all necessary fields from all db_tables for collections. Added tests and fixed existing tests (table names now start with alias `c.`) Co-authored-by: Robin Tammo Gummels <github@gummels.eu> * Implement more Queryable-Fields and add the keywords-field to q fulltext search (#200) * Add provider and license filters to collection search API - Updated buildCollectionSearchQuery to include provider and license parameters for filtering collections. - Enhanced validateCollectionSearchParams middleware to validate provider and license query parameters. - Modified collections route to handle new provider and license filters in search queries. - Implemented validation functions for provider and license parameters in collectionSearchParams. * Add validation tests for provider and license * Enhance full-text search by including keywords in the tsvector expression and update related tests * Add provider and license to query parameter extraction in collection search validation * Revert "Enhance full-text search by including keywords in the tsvector expression and update related tests" This reverts commit 872443d. * Implement GET /collections/{id} endpoint with validation and QueryBuilder integration (#186) * added SQLQuery-builder with these parameters: q,bbox,datetime,sortby,limit and token * finalised bbox and datetime * adapted to DB, QueryBuilder and added helperfunction runQuery * added question-TODOs * added bbox+datetime to the Query-Builder from Jonas * added tests for Query-Builder from Jonas * added tests from George * added falsely deleted TODOs again * fixed collumn names to match our DB and adjusted full text search to match 05_indexes.sql correctly * Used a formatter and linter on `buildCollectionSearchQuery.js * Did some major and minor fixes to the collection search. - Updated `buildCollectionSearchQuery` to support pagination and improved text search with English language settings. - Modified tests in `buildCollectionsSearchQuery.basic.test.js`, `collections-pagination.test.js`, and `collections-sort.test.js` to reflect new query behavior and validation logic. - Enhanced sort validation in `validators.test.js` and `collectionSearchParams.js` to map API fields to database column names. - Implemented total count retrieval for matched results in `collections.js`. * Added a internal .env creation in the CI/CD Pipeline. It utilzes GitHub Repository Secrets to not publish any private Logins and stuff. * Forgot that the second Job of the CI/CD pipeline runs seperatly and needs a internal .env file too. * Enhance documentation for buildCollectionSearchQuery Updated the documentation for: - the buildCollectionSearchQuery function - the fulltextsearch * Refactor buildCollectionSearchQuery and updated SELECT part Changed the SELECT part to match our bid and the database shema. Updated comments and for clarity. Changed full-text search to use 'simple' configuration instead of 'english'. * Update api/routes/collections.js small typo Co-authored-by: Robin Tammo Gummels <github@gummels.eu> * Remove sorting TODO from collections route Removed TODO comment about sorting based on sortby parameter. * Explicitly return undefined for normalized in validateSortby Update validateSortby function to explicitly return undefined for normalized when sortby is not provided. * small fix in buildCollectionSearch.fulltext.test.js Change plainto_tsquery language from 'english' to 'simple' * Fix duplicate SELECT keyword in query Remove duplicate 'SELECT' keyword in SQL query. * Fix missing newline at end of collectionSearchParams.js * Fixed missing bracket in collectionSearchParams.js * Refactor validateSortby for optional parameter handling Refactor validateSortby function to handle optional sortby parameter and improve validation logic. * Stabilize API test pipeline by running Jest in-band with extended timeout Run Jest in CI with --runInBand and a higher default --testTimeout to stabilize database-backed integration tests. Multiple Jest workers were competing for the same PostgreSQL connection pool and some long-running /collections queries exceeded the default 5s timeout, causing failures in existing test suites (e.g. collectionSearch and DBconnection). * Fixed leaking tests that blocked CI/CD-Pipeline. - Added a global Teardown for jest and force-exited the tests to prevent leaking. - Made a change to db_APIconnection to only log the pool-(dis)connection if it isn't run in a test enviroment. * Did a minimum amount of Formatting to the discription * Used `npm audit fix --force` to fix all vulnerabilties in our used packages. * Fixed curious doublechecking for empty Strings for the sortby-Parameter. - Now we only check once for a empty sortby - And added a test which distinguish between `sortby=""` and `sortby="+"` * Update api/routes/collections.js Removed the TODO about switching from mock-data to the real db * Removed globalTeardown as i brought up some problems corresponding to long db-queries (for example BBOX). Instead i increased the maximal testTimeout. * added validator for collections{id} and correctly implemented collections{id} * added test for collections{id} * removed unnecessary parameter * added id parameter to the Query (temporary fix) * test-fixes to match our current tests and a fix to the baseURL for collection{id} * test fix * fixed problem with tests in api.test.js and adjusted the "invalid-id-test" in the validator. * Update api/routes/collections.js - Renamed `collection.id` to `c.collection.id` * added test for negative ids * deleted the whole "existing links" part and build base Links * fixed bug in validateCollectionId.js * Refactor negative ID test i encoded the "-1" value in the negative ID test instead of directly putting it into the path. * Removed a german comment in `api/routes/collections.js` --------- Co-authored-by: Robin Tammo Gummels <github@gummels.eu> --------- Co-authored-by: Sönke Hoffmann <hoffmann.sonke@gmx.de> Co-authored-by: JonasK <156602337+BrokeJ@users.noreply.github.com> Co-authored-by: Vincent Kühn <vkuehn@uni-muenster.de>
jkrumboe
added a commit
that referenced
this pull request
Jan 1, 2026
* feat(api): initialize STAC Atlas API with collections, conformance, and queryables routes - Added package.json for project dependencies and scripts. - Implemented GET endpoint for collections. - Created conformance endpoint to list supported conformance classes. - Developed landing page for the API with links to collections and documentation. - Added queryables endpoint to return queryable properties for collections. (If i'm correct this can be removed) * Added all Remarks to the bid and finished it (#74) * changed Texts 1,2 and 8 according to the remarks of the customer * Did my fixes to 4. and 10.3 * added remark why we want to save every catalog * deleted keywords for catalog * changed everything related to the database component * Updated 3.1, 3.3, 3,4 (References to Lastenheft/small other changes) * Added small Graph to 3. Produktumgebung * Update bid.md 7.3STAC-Validator added the handling of collections that cannot be validated automatically. * Update bid.md 7.3STAC-API-Validator changed the way we validate the collection search extension. * Update bid.md 9.3.2Endpunkte small fix collection search extension. * added Skizze for 3, and updated 6.1, 10.1 * Updated 3. Produktumgebung * Update bid.md 7.Qualitätsanforderungen minor fixes * Update 6.4.2 added remark about loading feedback * Minor change to Table in "11 Zeitplan". A collum was missing in the head, therefore the table wasn't rendering correctly.. --------- Co-authored-by: Jakob <vertrox78@gmail.com> Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de> Co-authored-by: jklaer <jklaer@uni-muenster.de> Co-authored-by: VincentKuehn <vkuehn@uni-muenster.de> Co-authored-by: mammutor <mammutor@gmail.com> Co-authored-by: Humam <44206081+Mammutor@users.noreply.github.com> Co-authored-by: Justin K <justinkrumbhmer@yahoo.com> * fix(api): enhance STAC API landing page and conformance links - Overhaul of first idea landing page - Added some more tests for the required elements in the landingpage-Catalog * feat(api): implement shared conformance URIs and add tests for conformance endpoint - implemented condormance endpoint * Implemented 2.3 and 2.4 (#111) * Temporary mock data for testing and frontend development * Added API middleware layer for error handling and validation * TODOs ready? pls review * Added API utilities for query parsing, validation, and response formatting * Added swagger and openapi.yaml * Update queryables.js Refactor queryables endpoint into /collections/queryables * Renamed the collections.js file to mocks-collections.js to better reflect its purpose and improve project clarity * changed README "Projektstruktur" * restart from dev-api 22.11..2025 * API: 2.3 Implement Collections List Endpoint done (added explanations as comments in the code) * API: 2.4 Implement Single Collection Endpoint (added explanations as comments in the code) * Update api/routes/collections.js Co-authored-by: Robin Tammo Gummels <github@gummels.eu> * Changed some of the code with the comments on Github (i will finish it tomorrow morning) * Implement most of the feedback and comments (need to talk about some other changes) * Update api/routes/index.js Changed wording from `/collections/queryables` to `/collections-queryables` * Update api/README.md Changed wording from `/collections/queryables` to `/collections-queryables` * Update api/README.md Removed missing folder * Update api/routes/collections.js Removed TODOs from wrong lines * Update api/routes/collections.js Added TODOs * Update api/routes/queryables.js Changed wording from `/collections/queryables` to `/collections-queryables` * Update api/routes/queryables.js Changed wording from `/collections/queryables` to `/collections-queryables` --------- Co-authored-by: VincentKuehn <vkuehn@uni-muenster.de> Co-authored-by: Robin Tammo Gummels <github@gummels.eu> * feat(api): add collection search parameters and validation middleware (#159) * feat(api): add collection search parameters and validation middleware * Added unit-test for validator-functions and integration-tests for `GET /collections`-Querys. - Also minor bugfix, because the validator accepted deecimals as tokens. * API: 3 Database Integration first version (#161) * database connection in implementated. The parameters for the connection have to added in the .env-file. Also there is test-file for testing and console messages (installed `pg`) * support for spatial queries via postgis + error handling for datatbase operations changed language to english * error handling * added DATABASE_URL There is an issue with the distance query. Changed the error handling and testing, the console messages are now way better structured * found the Problem with the distance query. The layer are so big, that they reach over the 180° long (PostgGIS can't handel that). Now the calc is done by degree and not meters. * The two files `test-data-retrieval.js` and `verify-schema.js` have been added. `test-data-retrieval` (theoretical, checks against the spezification): ``` Discovers all tables and columns and validates against expected schema. ``` The second files `verify-schema.js` (practical, checks against the real data): ``` Discovers all tables and columns, validates against expected schema ``` * pooling error hanling and log imporoved. renamed tests files to actual test-files * standalone node tests were convertad into JEST * write file `validateRequest.js`. Validates every incoming API request, whether the request is valid and logical. * commented `stac_id` from the tests, it is not in both databases, so the tests for `stac_id` will always fail Added explanation to the `.env.example`, which port is which database * added example pattern for API - database connection. * deleted `validateRequest` cause it's already implemented by @RobinGummels --------- Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de> * Added a CI/CD Pipeline to prevent pull-requests without functioning tests and proper linting. * fixed errors suggested by the linter. - Some lines used tab and spaces... * Implemented Collection Search extension including a DB-Connection (#165) * added SQLQuery-builder with these parameters: q,bbox,datetime,sortby,limit and token * finalised bbox and datetime * adapted to DB, QueryBuilder and added helperfunction runQuery * added question-TODOs * added bbox+datetime to the Query-Builder from Jonas * added tests for Query-Builder from Jonas * added tests from George * added falsely deleted TODOs again * fixed collumn names to match our DB and adjusted full text search to match 05_indexes.sql correctly * Used a formatter and linter on `buildCollectionSearchQuery.js * Did some major and minor fixes to the collection search. - Updated `buildCollectionSearchQuery` to support pagination and improved text search with English language settings. - Modified tests in `buildCollectionsSearchQuery.basic.test.js`, `collections-pagination.test.js`, and `collections-sort.test.js` to reflect new query behavior and validation logic. - Enhanced sort validation in `validators.test.js` and `collectionSearchParams.js` to map API fields to database column names. - Implemented total count retrieval for matched results in `collections.js`. * Added a internal .env creation in the CI/CD Pipeline. It utilzes GitHub Repository Secrets to not publish any private Logins and stuff. * Forgot that the second Job of the CI/CD pipeline runs seperatly and needs a internal .env file too. * Enhance documentation for buildCollectionSearchQuery Updated the documentation for: - the buildCollectionSearchQuery function - the fulltextsearch * Refactor buildCollectionSearchQuery and updated SELECT part Changed the SELECT part to match our bid and the database shema. Updated comments and for clarity. Changed full-text search to use 'simple' configuration instead of 'english'. * Update api/routes/collections.js small typo Co-authored-by: Robin Tammo Gummels <github@gummels.eu> * Remove sorting TODO from collections route Removed TODO comment about sorting based on sortby parameter. * Explicitly return undefined for normalized in validateSortby Update validateSortby function to explicitly return undefined for normalized when sortby is not provided. * small fix in buildCollectionSearch.fulltext.test.js Change plainto_tsquery language from 'english' to 'simple' * Fix duplicate SELECT keyword in query Remove duplicate 'SELECT' keyword in SQL query. * Fix missing newline at end of collectionSearchParams.js * Fixed missing bracket in collectionSearchParams.js * Refactor validateSortby for optional parameter handling Refactor validateSortby function to handle optional sortby parameter and improve validation logic. * Stabilize API test pipeline by running Jest in-band with extended timeout Run Jest in CI with --runInBand and a higher default --testTimeout to stabilize database-backed integration tests. Multiple Jest workers were competing for the same PostgreSQL connection pool and some long-running /collections queries exceeded the default 5s timeout, causing failures in existing test suites (e.g. collectionSearch and DBconnection). * Fixed leaking tests that blocked CI/CD-Pipeline. - Added a global Teardown for jest and force-exited the tests to prevent leaking. - Made a change to db_APIconnection to only log the pool-(dis)connection if it isn't run in a test enviroment. * Did a minimum amount of Formatting to the discription * Used `npm audit fix --force` to fix all vulnerabilties in our used packages. * Fixed curious doublechecking for empty Strings for the sortby-Parameter. - Now we only check once for a empty sortby - And added a test which distinguish between `sortby=""` and `sortby="+"` * Update api/routes/collections.js Removed the TODO about switching from mock-data to the real db * Removed globalTeardown as i brought up some problems corresponding to long db-queries (for example BBOX). Instead i increased the maximal testTimeout. --------- Co-authored-by: Robin Tammo Gummels <github@gummels.eu> * Update api/.env.example * latest database Version (#187) with `stac_id` and changed definition of `primary Keys` * added `.env` * added environment for docker-compose.yml now every connection-details are inside an `.env`. There is an `example.env` for better understanding which need to be set as connection details * added description of how to use the `.env` and `example.env` in the `README.md` * changed a few things e.g. DB_PORT --> ${DB_PORT} * now, everthing should be done. my god, help. sorry * layout issues fixed * Fixed Typo/incomplete Sentence in README.md * added `stac_id` for collections * all IDs are now written in the newer PostgrSQL standart: ```SQL id SERIAL PRIMARY KEY, ``` changed to ```SQL id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY, ``` * changed `extend` to `extent`. * Changed language used in `./api/README.md` from german to english. I wanted to thsi anyway at some point, but this is now more like a Test-commit to see if the CI/CD Pipeline triggers... --------- Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de> Co-authored-by: Robin Tammo Gummels <github@gummels.eu> * chore: removed Helloworld * feat: collection page, mockup data for visual feedback chore: styling corrected, adjusted based on the mockup * style: removed old css names * chore: removed the mockup data * feat: update button styles and add contact popover in CollectionDetail view * style: fixed witdh of the apply filter button * style: map button, apply button behave like the others now * chore: removed mockup data * feat: route gitignore * feat: showing the collections from /collections in the result list * feat: enhance filter section layout and improve search result card tag display * style: refine search result card layout and improve text overflow handling * feat: implement pagination for collections in Home.vue and adjust max-height in search results * style: remove fixed height from card header and add styles for first button in card footer * style: enhance active pagination button styles for better visibility * feat: implement loading and error states in CollectionDetail.vue, enhance API integration for fetching collection data * style: width problem collection page, scolll items * Added `GET /collections/{id}`-Endpoint, more Fields in the responses and more Queryables-Parameters (#204) * Updated Query-Builder to get all necessary fields from all db_tables for collections. Added some tests and fixed some already existing tests, becuase now the tablenames start with the alias `c.`. * Updated Query-Builder to get all necessary fields from all db_tables for collections. Added some tests and fixed some already existing tests, becuase now the tablenames start with the alias `c.`. * Added `openapi.yaml` (now http://localhost:3000/api-docs/ is working). - needed to do some modifying to the app.js * Added discription on how to use `stac-api-validator`. Currently we are onyl valid to `core`. * Changed API-Version name to 1.1.0 instead of 1.0.0 * Revert "API is now responding with all necessary fields for each collection" (#195) Reverts #185 @SonkeHoffmann accidentally didn't squash correctly. * Revert "Revert "API is now responding with all necessary fields for each collection"" (#185) (#195) (#196) dev-api: prepare v1.1.0 + API docs + query builder fixes - Change API version to 1.1.0 - Add OpenAPI spec so /api-docs works locally - Document stac-api-validator usage - Update api/.env.example - Query builder: select required fields for collections across db_tables; adjust tests (alias `c.`) Commits included: - 34bf962 Changed API-Version name to 1.1.0 instead of 1.0.0 - b047389 Added description on how to use `stac-api-validator` (currently only valid for `core`) - b811288 Added `openapi.yaml` (so http://localhost:3000/api-docs/ works); modified app.js accordingly - 6e5ab3e Merge branch 'dev-api-robin' of github.com:SpatioCore/STAC-Atlas into dev-api-robin - 70dc043 Updated Query-Builder to get all necessary fields from all db_tables for collections. Added tests and fixed existing tests (table names now start with alias `c.`) - d83eeb4 Update api/.env.example - 5a7af5b Updated Query-Builder to get all necessary fields from all db_tables for collections. Added tests and fixed existing tests (table names now start with alias `c.`) Co-authored-by: Robin Tammo Gummels <github@gummels.eu> * Implement more Queryable-Fields and add the keywords-field to q fulltext search (#200) * Add provider and license filters to collection search API - Updated buildCollectionSearchQuery to include provider and license parameters for filtering collections. - Enhanced validateCollectionSearchParams middleware to validate provider and license query parameters. - Modified collections route to handle new provider and license filters in search queries. - Implemented validation functions for provider and license parameters in collectionSearchParams. * Add validation tests for provider and license * Enhance full-text search by including keywords in the tsvector expression and update related tests * Add provider and license to query parameter extraction in collection search validation * Revert "Enhance full-text search by including keywords in the tsvector expression and update related tests" This reverts commit 872443d. * Implement GET /collections/{id} endpoint with validation and QueryBuilder integration (#186) * added SQLQuery-builder with these parameters: q,bbox,datetime,sortby,limit and token * finalised bbox and datetime * adapted to DB, QueryBuilder and added helperfunction runQuery * added question-TODOs * added bbox+datetime to the Query-Builder from Jonas * added tests for Query-Builder from Jonas * added tests from George * added falsely deleted TODOs again * fixed collumn names to match our DB and adjusted full text search to match 05_indexes.sql correctly * Used a formatter and linter on `buildCollectionSearchQuery.js * Did some major and minor fixes to the collection search. - Updated `buildCollectionSearchQuery` to support pagination and improved text search with English language settings. - Modified tests in `buildCollectionsSearchQuery.basic.test.js`, `collections-pagination.test.js`, and `collections-sort.test.js` to reflect new query behavior and validation logic. - Enhanced sort validation in `validators.test.js` and `collectionSearchParams.js` to map API fields to database column names. - Implemented total count retrieval for matched results in `collections.js`. * Added a internal .env creation in the CI/CD Pipeline. It utilzes GitHub Repository Secrets to not publish any private Logins and stuff. * Forgot that the second Job of the CI/CD pipeline runs seperatly and needs a internal .env file too. * Enhance documentation for buildCollectionSearchQuery Updated the documentation for: - the buildCollectionSearchQuery function - the fulltextsearch * Refactor buildCollectionSearchQuery and updated SELECT part Changed the SELECT part to match our bid and the database shema. Updated comments and for clarity. Changed full-text search to use 'simple' configuration instead of 'english'. * Update api/routes/collections.js small typo Co-authored-by: Robin Tammo Gummels <github@gummels.eu> * Remove sorting TODO from collections route Removed TODO comment about sorting based on sortby parameter. * Explicitly return undefined for normalized in validateSortby Update validateSortby function to explicitly return undefined for normalized when sortby is not provided. * small fix in buildCollectionSearch.fulltext.test.js Change plainto_tsquery language from 'english' to 'simple' * Fix duplicate SELECT keyword in query Remove duplicate 'SELECT' keyword in SQL query. * Fix missing newline at end of collectionSearchParams.js * Fixed missing bracket in collectionSearchParams.js * Refactor validateSortby for optional parameter handling Refactor validateSortby function to handle optional sortby parameter and improve validation logic. * Stabilize API test pipeline by running Jest in-band with extended timeout Run Jest in CI with --runInBand and a higher default --testTimeout to stabilize database-backed integration tests. Multiple Jest workers were competing for the same PostgreSQL connection pool and some long-running /collections queries exceeded the default 5s timeout, causing failures in existing test suites (e.g. collectionSearch and DBconnection). * Fixed leaking tests that blocked CI/CD-Pipeline. - Added a global Teardown for jest and force-exited the tests to prevent leaking. - Made a change to db_APIconnection to only log the pool-(dis)connection if it isn't run in a test enviroment. * Did a minimum amount of Formatting to the discription * Used `npm audit fix --force` to fix all vulnerabilties in our used packages. * Fixed curious doublechecking for empty Strings for the sortby-Parameter. - Now we only check once for a empty sortby - And added a test which distinguish between `sortby=""` and `sortby="+"` * Update api/routes/collections.js Removed the TODO about switching from mock-data to the real db * Removed globalTeardown as i brought up some problems corresponding to long db-queries (for example BBOX). Instead i increased the maximal testTimeout. * added validator for collections{id} and correctly implemented collections{id} * added test for collections{id} * removed unnecessary parameter * added id parameter to the Query (temporary fix) * test-fixes to match our current tests and a fix to the baseURL for collection{id} * test fix * fixed problem with tests in api.test.js and adjusted the "invalid-id-test" in the validator. * Update api/routes/collections.js - Renamed `collection.id` to `c.collection.id` * added test for negative ids * deleted the whole "existing links" part and build base Links * fixed bug in validateCollectionId.js * Refactor negative ID test i encoded the "-1" value in the negative ID test instead of directly putting it into the path. * Removed a german comment in `api/routes/collections.js` --------- Co-authored-by: Robin Tammo Gummels <github@gummels.eu> --------- Co-authored-by: Sönke Hoffmann <hoffmann.sonke@gmx.de> Co-authored-by: JonasK <156602337+BrokeJ@users.noreply.github.com> Co-authored-by: Vincent Kühn <vkuehn@uni-muenster.de> * chore: switched to the apis collections/:id in the ui * style: calender icon response to dark & light mode * chroe: removed svg for language button style: slightly thicker border for nav-buttons * style: added a slight highlight to input/ interaction buttons * style: update dark mode colors for text and adjust tag background * style: smaller vertical padding for select options * feat: source and contact are shown when clicked * feat: search functionality * style: centered the paging icon chore: view css folder to seperate style and content * feat: removed the unnecessary Filter Header * feat: add bounding box drawing functionality and integrate with filter section --------- Co-authored-by: RobinGummels <github@gummels.eu> Co-authored-by: Jakob <vertrox78@gmail.com> Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de> Co-authored-by: jklaer <jklaer@uni-muenster.de> Co-authored-by: VincentKuehn <vkuehn@uni-muenster.de> Co-authored-by: mammutor <mammutor@gmail.com> Co-authored-by: Humam <44206081+Mammutor@users.noreply.github.com> Co-authored-by: Georgios Voulgaris <gvoulgar@uni-muenster.de> Co-authored-by: Sönke Hoffmann <hoffmann.sonke@gmx.de> Co-authored-by: JonasK <156602337+BrokeJ@users.noreply.github.com>
LennKru
pushed a commit
that referenced
this pull request
Jan 15, 2026
* feat(api): initialize STAC Atlas API with collections, conformance, and queryables routes - Added package.json for project dependencies and scripts. - Implemented GET endpoint for collections. - Created conformance endpoint to list supported conformance classes. - Developed landing page for the API with links to collections and documentation. - Added queryables endpoint to return queryable properties for collections. (If i'm correct this can be removed) * Added all Remarks to the bid and finished it (#74) * changed Texts 1,2 and 8 according to the remarks of the customer * Did my fixes to 4. and 10.3 * added remark why we want to save every catalog * deleted keywords for catalog * changed everything related to the database component * Updated 3.1, 3.3, 3,4 (References to Lastenheft/small other changes) * Added small Graph to 3. Produktumgebung * Update bid.md 7.3STAC-Validator added the handling of collections that cannot be validated automatically. * Update bid.md 7.3STAC-API-Validator changed the way we validate the collection search extension. * Update bid.md 9.3.2Endpunkte small fix collection search extension. * added Skizze for 3, and updated 6.1, 10.1 * Updated 3. Produktumgebung * Update bid.md 7.Qualitätsanforderungen minor fixes * Update 6.4.2 added remark about loading feedback * Minor change to Table in "11 Zeitplan". A collum was missing in the head, therefore the table wasn't rendering correctly.. --------- Co-authored-by: Jakob <vertrox78@gmail.com> Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de> Co-authored-by: jklaer <jklaer@uni-muenster.de> Co-authored-by: VincentKuehn <vkuehn@uni-muenster.de> Co-authored-by: mammutor <mammutor@gmail.com> Co-authored-by: Humam <44206081+Mammutor@users.noreply.github.com> Co-authored-by: Justin K <justinkrumbhmer@yahoo.com> * fix(api): enhance STAC API landing page and conformance links - Overhaul of first idea landing page - Added some more tests for the required elements in the landingpage-Catalog * feat(api): implement shared conformance URIs and add tests for conformance endpoint - implemented condormance endpoint * Implemented 2.3 and 2.4 (#111) * Temporary mock data for testing and frontend development * Added API middleware layer for error handling and validation * TODOs ready? pls review * Added API utilities for query parsing, validation, and response formatting * Added swagger and openapi.yaml * Update queryables.js Refactor queryables endpoint into /collections/queryables * Renamed the collections.js file to mocks-collections.js to better reflect its purpose and improve project clarity * changed README "Projektstruktur" * restart from dev-api 22.11..2025 * API: 2.3 Implement Collections List Endpoint done (added explanations as comments in the code) * API: 2.4 Implement Single Collection Endpoint (added explanations as comments in the code) * Update api/routes/collections.js Co-authored-by: Robin Tammo Gummels <github@gummels.eu> * Changed some of the code with the comments on Github (i will finish it tomorrow morning) * Implement most of the feedback and comments (need to talk about some other changes) * Update api/routes/index.js Changed wording from `/collections/queryables` to `/collections-queryables` * Update api/README.md Changed wording from `/collections/queryables` to `/collections-queryables` * Update api/README.md Removed missing folder * Update api/routes/collections.js Removed TODOs from wrong lines * Update api/routes/collections.js Added TODOs * Update api/routes/queryables.js Changed wording from `/collections/queryables` to `/collections-queryables` * Update api/routes/queryables.js Changed wording from `/collections/queryables` to `/collections-queryables` --------- Co-authored-by: VincentKuehn <vkuehn@uni-muenster.de> Co-authored-by: Robin Tammo Gummels <github@gummels.eu> * feat(api): add collection search parameters and validation middleware (#159) * feat(api): add collection search parameters and validation middleware * Added unit-test for validator-functions and integration-tests for `GET /collections`-Querys. - Also minor bugfix, because the validator accepted deecimals as tokens. * API: 3 Database Integration first version (#161) * database connection in implementated. The parameters for the connection have to added in the .env-file. Also there is test-file for testing and console messages (installed `pg`) * support for spatial queries via postgis + error handling for datatbase operations changed language to english * error handling * added DATABASE_URL There is an issue with the distance query. Changed the error handling and testing, the console messages are now way better structured * found the Problem with the distance query. The layer are so big, that they reach over the 180° long (PostgGIS can't handel that). Now the calc is done by degree and not meters. * The two files `test-data-retrieval.js` and `verify-schema.js` have been added. `test-data-retrieval` (theoretical, checks against the spezification): ``` Discovers all tables and columns and validates against expected schema. ``` The second files `verify-schema.js` (practical, checks against the real data): ``` Discovers all tables and columns, validates against expected schema ``` * pooling error hanling and log imporoved. renamed tests files to actual test-files * standalone node tests were convertad into JEST * write file `validateRequest.js`. Validates every incoming API request, whether the request is valid and logical. * commented `stac_id` from the tests, it is not in both databases, so the tests for `stac_id` will always fail Added explanation to the `.env.example`, which port is which database * added example pattern for API - database connection. * deleted `validateRequest` cause it's already implemented by @RobinGummels --------- Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de> * Added environment variables and a `.env` for `docker-compose.yml` (#164) * added `.env` * added environment for docker-compose.yml now every connection-details are inside an `.env`. There is an `example.env` for better understanding which need to be set as connection details * added description of how to use the `.env` and `example.env` in the `README.md` * changed a few things e.g. DB_PORT --> ${DB_PORT} * now, everthing should be done. my god, help. sorry * layout issues fixed * Fixed Typo/incomplete Sentence in README.md --------- Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de> Co-authored-by: Robin Tammo Gummels <github@gummels.eu> * Added a CI/CD Pipeline to prevent pull-requests without functioning tests and proper linting. * fixed errors suggested by the linter. - Some lines used tab and spaces... * Implemented Collection Search extension including a DB-Connection (#165) * added SQLQuery-builder with these parameters: q,bbox,datetime,sortby,limit and token * finalised bbox and datetime * adapted to DB, QueryBuilder and added helperfunction runQuery * added question-TODOs * added bbox+datetime to the Query-Builder from Jonas * added tests for Query-Builder from Jonas * added tests from George * added falsely deleted TODOs again * fixed collumn names to match our DB and adjusted full text search to match 05_indexes.sql correctly * Used a formatter and linter on `buildCollectionSearchQuery.js * Did some major and minor fixes to the collection search. - Updated `buildCollectionSearchQuery` to support pagination and improved text search with English language settings. - Modified tests in `buildCollectionsSearchQuery.basic.test.js`, `collections-pagination.test.js`, and `collections-sort.test.js` to reflect new query behavior and validation logic. - Enhanced sort validation in `validators.test.js` and `collectionSearchParams.js` to map API fields to database column names. - Implemented total count retrieval for matched results in `collections.js`. * Added a internal .env creation in the CI/CD Pipeline. It utilzes GitHub Repository Secrets to not publish any private Logins and stuff. * Forgot that the second Job of the CI/CD pipeline runs seperatly and needs a internal .env file too. * Enhance documentation for buildCollectionSearchQuery Updated the documentation for: - the buildCollectionSearchQuery function - the fulltextsearch * Refactor buildCollectionSearchQuery and updated SELECT part Changed the SELECT part to match our bid and the database shema. Updated comments and for clarity. Changed full-text search to use 'simple' configuration instead of 'english'. * Update api/routes/collections.js small typo Co-authored-by: Robin Tammo Gummels <github@gummels.eu> * Remove sorting TODO from collections route Removed TODO comment about sorting based on sortby parameter. * Explicitly return undefined for normalized in validateSortby Update validateSortby function to explicitly return undefined for normalized when sortby is not provided. * small fix in buildCollectionSearch.fulltext.test.js Change plainto_tsquery language from 'english' to 'simple' * Fix duplicate SELECT keyword in query Remove duplicate 'SELECT' keyword in SQL query. * Fix missing newline at end of collectionSearchParams.js * Fixed missing bracket in collectionSearchParams.js * Refactor validateSortby for optional parameter handling Refactor validateSortby function to handle optional sortby parameter and improve validation logic. * Stabilize API test pipeline by running Jest in-band with extended timeout Run Jest in CI with --runInBand and a higher default --testTimeout to stabilize database-backed integration tests. Multiple Jest workers were competing for the same PostgreSQL connection pool and some long-running /collections queries exceeded the default 5s timeout, causing failures in existing test suites (e.g. collectionSearch and DBconnection). * Fixed leaking tests that blocked CI/CD-Pipeline. - Added a global Teardown for jest and force-exited the tests to prevent leaking. - Made a change to db_APIconnection to only log the pool-(dis)connection if it isn't run in a test enviroment. * Did a minimum amount of Formatting to the discription * Used `npm audit fix --force` to fix all vulnerabilties in our used packages. * Fixed curious doublechecking for empty Strings for the sortby-Parameter. - Now we only check once for a empty sortby - And added a test which distinguish between `sortby=""` and `sortby="+"` * Update api/routes/collections.js Removed the TODO about switching from mock-data to the real db * Removed globalTeardown as i brought up some problems corresponding to long db-queries (for example BBOX). Instead i increased the maximal testTimeout. --------- Co-authored-by: Robin Tammo Gummels <github@gummels.eu> * Update api/.env.example * latest database Version (#187) with `stac_id` and changed definition of `primary Keys` * added `.env` * added environment for docker-compose.yml now every connection-details are inside an `.env`. There is an `example.env` for better understanding which need to be set as connection details * added description of how to use the `.env` and `example.env` in the `README.md` * changed a few things e.g. DB_PORT --> ${DB_PORT} * now, everthing should be done. my god, help. sorry * layout issues fixed * Fixed Typo/incomplete Sentence in README.md * added `stac_id` for collections * all IDs are now written in the newer PostgrSQL standart: ```SQL id SERIAL PRIMARY KEY, ``` changed to ```SQL id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY, ``` * changed `extend` to `extent`. * Changed language used in `./api/README.md` from german to english. I wanted to thsi anyway at some point, but this is now more like a Test-commit to see if the CI/CD Pipeline triggers... --------- Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de> Co-authored-by: Robin Tammo Gummels <github@gummels.eu> * Added `GET /collections/{id}`-Endpoint, more Fields in the responses and more Queryables-Parameters (#204) * Updated Query-Builder to get all necessary fields from all db_tables for collections. Added some tests and fixed some already existing tests, becuase now the tablenames start with the alias `c.`. * Updated Query-Builder to get all necessary fields from all db_tables for collections. Added some tests and fixed some already existing tests, becuase now the tablenames start with the alias `c.`. * Added `openapi.yaml` (now http://localhost:3000/api-docs/ is working). - needed to do some modifying to the app.js * Added discription on how to use `stac-api-validator`. Currently we are onyl valid to `core`. * Changed API-Version name to 1.1.0 instead of 1.0.0 * Revert "API is now responding with all necessary fields for each collection" (#195) Reverts #185 @SonkeHoffmann accidentally didn't squash correctly. * Revert "Revert "API is now responding with all necessary fields for each collection"" (#185) (#195) (#196) dev-api: prepare v1.1.0 + API docs + query builder fixes - Change API version to 1.1.0 - Add OpenAPI spec so /api-docs works locally - Document stac-api-validator usage - Update api/.env.example - Query builder: select required fields for collections across db_tables; adjust tests (alias `c.`) Commits included: - 34bf962 Changed API-Version name to 1.1.0 instead of 1.0.0 - b047389 Added description on how to use `stac-api-validator` (currently only valid for `core`) - b811288 Added `openapi.yaml` (so http://localhost:3000/api-docs/ works); modified app.js accordingly - 6e5ab3e Merge branch 'dev-api-robin' of github.com:SpatioCore/STAC-Atlas into dev-api-robin - 70dc043 Updated Query-Builder to get all necessary fields from all db_tables for collections. Added tests and fixed existing tests (table names now start with alias `c.`) - d83eeb4 Update api/.env.example - 5a7af5b Updated Query-Builder to get all necessary fields from all db_tables for collections. Added tests and fixed existing tests (table names now start with alias `c.`) Co-authored-by: Robin Tammo Gummels <github@gummels.eu> * Implement more Queryable-Fields and add the keywords-field to q fulltext search (#200) * Add provider and license filters to collection search API - Updated buildCollectionSearchQuery to include provider and license parameters for filtering collections. - Enhanced validateCollectionSearchParams middleware to validate provider and license query parameters. - Modified collections route to handle new provider and license filters in search queries. - Implemented validation functions for provider and license parameters in collectionSearchParams. * Add validation tests for provider and license * Enhance full-text search by including keywords in the tsvector expression and update related tests * Add provider and license to query parameter extraction in collection search validation * Revert "Enhance full-text search by including keywords in the tsvector expression and update related tests" This reverts commit 872443d. * Implement GET /collections/{id} endpoint with validation and QueryBuilder integration (#186) * added SQLQuery-builder with these parameters: q,bbox,datetime,sortby,limit and token * finalised bbox and datetime * adapted to DB, QueryBuilder and added helperfunction runQuery * added question-TODOs * added bbox+datetime to the Query-Builder from Jonas * added tests for Query-Builder from Jonas * added tests from George * added falsely deleted TODOs again * fixed collumn names to match our DB and adjusted full text search to match 05_indexes.sql correctly * Used a formatter and linter on `buildCollectionSearchQuery.js * Did some major and minor fixes to the collection search. - Updated `buildCollectionSearchQuery` to support pagination and improved text search with English language settings. - Modified tests in `buildCollectionsSearchQuery.basic.test.js`, `collections-pagination.test.js`, and `collections-sort.test.js` to reflect new query behavior and validation logic. - Enhanced sort validation in `validators.test.js` and `collectionSearchParams.js` to map API fields to database column names. - Implemented total count retrieval for matched results in `collections.js`. * Added a internal .env creation in the CI/CD Pipeline. It utilzes GitHub Repository Secrets to not publish any private Logins and stuff. * Forgot that the second Job of the CI/CD pipeline runs seperatly and needs a internal .env file too. * Enhance documentation for buildCollectionSearchQuery Updated the documentation for: - the buildCollectionSearchQuery function - the fulltextsearch * Refactor buildCollectionSearchQuery and updated SELECT part Changed the SELECT part to match our bid and the database shema. Updated comments and for clarity. Changed full-text search to use 'simple' configuration instead of 'english'. * Update api/routes/collections.js small typo Co-authored-by: Robin Tammo Gummels <github@gummels.eu> * Remove sorting TODO from collections route Removed TODO comment about sorting based on sortby parameter. * Explicitly return undefined for normalized in validateSortby Update validateSortby function to explicitly return undefined for normalized when sortby is not provided. * small fix in buildCollectionSearch.fulltext.test.js Change plainto_tsquery language from 'english' to 'simple' * Fix duplicate SELECT keyword in query Remove duplicate 'SELECT' keyword in SQL query. * Fix missing newline at end of collectionSearchParams.js * Fixed missing bracket in collectionSearchParams.js * Refactor validateSortby for optional parameter handling Refactor validateSortby function to handle optional sortby parameter and improve validation logic. * Stabilize API test pipeline by running Jest in-band with extended timeout Run Jest in CI with --runInBand and a higher default --testTimeout to stabilize database-backed integration tests. Multiple Jest workers were competing for the same PostgreSQL connection pool and some long-running /collections queries exceeded the default 5s timeout, causing failures in existing test suites (e.g. collectionSearch and DBconnection). * Fixed leaking tests that blocked CI/CD-Pipeline. - Added a global Teardown for jest and force-exited the tests to prevent leaking. - Made a change to db_APIconnection to only log the pool-(dis)connection if it isn't run in a test enviroment. * Did a minimum amount of Formatting to the discription * Used `npm audit fix --force` to fix all vulnerabilties in our used packages. * Fixed curious doublechecking for empty Strings for the sortby-Parameter. - Now we only check once for a empty sortby - And added a test which distinguish between `sortby=""` and `sortby="+"` * Update api/routes/collections.js Removed the TODO about switching from mock-data to the real db * Removed globalTeardown as i brought up some problems corresponding to long db-queries (for example BBOX). Instead i increased the maximal testTimeout. * added validator for collections{id} and correctly implemented collections{id} * added test for collections{id} * removed unnecessary parameter * added id parameter to the Query (temporary fix) * test-fixes to match our current tests and a fix to the baseURL for collection{id} * test fix * fixed problem with tests in api.test.js and adjusted the "invalid-id-test" in the validator. * Update api/routes/collections.js - Renamed `collection.id` to `c.collection.id` * added test for negative ids * deleted the whole "existing links" part and build base Links * fixed bug in validateCollectionId.js * Refactor negative ID test i encoded the "-1" value in the negative ID test instead of directly putting it into the path. * Removed a german comment in `api/routes/collections.js` --------- Co-authored-by: Robin Tammo Gummels <github@gummels.eu> --------- Co-authored-by: Sönke Hoffmann <hoffmann.sonke@gmx.de> Co-authored-by: JonasK <156602337+BrokeJ@users.noreply.github.com> Co-authored-by: Vincent Kühn <vkuehn@uni-muenster.de> * Dev database: trigger function for better keyword handling (#209) * added `.env` * added environment for docker-compose.yml now every connection-details are inside an `.env`. There is an `example.env` for better understanding which need to be set as connection details * added description of how to use the `.env` and `example.env` in the `README.md` * changed a few things e.g. DB_PORT --> ${DB_PORT} * now, everthing should be done. my god, help. sorry * layout issues fixed * Fixed Typo/incomplete Sentence in README.md * added `stac_id` for collections * all IDs are now written in the newer PostgrSQL standart: ```SQL id SERIAL PRIMARY KEY, ``` changed to ```SQL id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY, ``` * changed `extend` to `extent`. * Changed language used in `./api/README.md` from german to english. I wanted to thsi anyway at some point, but this is now more like a Test-commit to see if the CI/CD Pipeline triggers... * added triggering function for an auto-update search_vector, both for collections and catalogs. The search_vector includes title, description and keywords * changed the CI-Pipeline. Now also Changes in the /db will be acceped by the Pipeline --------- Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de> Co-authored-by: Robin Tammo Gummels <github@gummels.eu> * Dev database: added different users (for api and crawler) (#214) * added `.env` * added environment for docker-compose.yml now every connection-details are inside an `.env`. There is an `example.env` for better understanding which need to be set as connection details * added description of how to use the `.env` and `example.env` in the `README.md` * changed a few things e.g. DB_PORT --> ${DB_PORT} * now, everthing should be done. my god, help. sorry * layout issues fixed * Fixed Typo/incomplete Sentence in README.md * added `stac_id` for collections * all IDs are now written in the newer PostgrSQL standart: ```SQL id SERIAL PRIMARY KEY, ``` changed to ```SQL id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY, ``` * changed `extend` to `extent`. * Changed language used in `./api/README.md` from german to english. I wanted to thsi anyway at some point, but this is now more like a Test-commit to see if the CI/CD Pipeline triggers... * added triggering function for an auto-update search_vector, both for collections and catalogs. The search_vector includes title, description and keywords * changed the CI-Pipeline. Now also Changes in the /db will be acceped by the Pipeline * added different users for the api and crawler groups. The api has read-only acces and the crawler user full acces to the database. The acces to the database is now possible by using the users `stac_api`or `stac_crawler`. The admin user (`postgres_user`) is still available but shouldn't be used --------- Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de> Co-authored-by: Robin Tammo Gummels <github@gummels.eu> * Dev database: source_url and filter trigger (#220) * added `.env` * added environment for docker-compose.yml now every connection-details are inside an `.env`. There is an `example.env` for better understanding which need to be set as connection details * added description of how to use the `.env` and `example.env` in the `README.md` * changed a few things e.g. DB_PORT --> ${DB_PORT} * now, everthing should be done. my god, help. sorry * layout issues fixed * Fixed Typo/incomplete Sentence in README.md * added `stac_id` for collections * all IDs are now written in the newer PostgrSQL standart: ```SQL id SERIAL PRIMARY KEY, ``` changed to ```SQL id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY, ``` * changed `extend` to `extent`. * Changed language used in `./api/README.md` from german to english. I wanted to thsi anyway at some point, but this is now more like a Test-commit to see if the CI/CD Pipeline triggers... * added triggering function for an auto-update search_vector, both for collections and catalogs. The search_vector includes title, description and keywords * changed the CI-Pipeline. Now also Changes in the /db will be acceped by the Pipeline * added different users for the api and crawler groups. The api has read-only acces and the crawler user full acces to the database. The acces to the database is now possible by using the users `stac_api`or `stac_crawler`. The admin user (`postgres_user`) is still available but shouldn't be used * Refactor(database): SQL trigger definitions for catalog and collection keywords. Moved triggers to 06_triggers.sql for better organization, as they depend on the respective tables created in earlier scripts. * added source_url for collections and catalogs. Now the full_json doesn`t has to be used for getting the url * resolved a Problem I had with git by hand cause I didn't found the function --------- Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de> Co-authored-by: Robin Tammo Gummels <github@gummels.eu> Co-authored-by: mammutor <mammutor@gmail.com> --------- Co-authored-by: RobinGummels <github@gummels.eu> Co-authored-by: Jakob <vertrox78@gmail.com> Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de> Co-authored-by: jklaer <jklaer@uni-muenster.de> Co-authored-by: VincentKuehn <vkuehn@uni-muenster.de> Co-authored-by: Justin K <justinkrumbhmer@yahoo.com> Co-authored-by: Georgios Voulgaris <gvoulgar@uni-muenster.de> Co-authored-by: Sönke Hoffmann <hoffmann.sonke@gmx.de> Co-authored-by: JonasK <156602337+BrokeJ@users.noreply.github.com>
Mammutor
added a commit
that referenced
this pull request
Jan 31, 2026
* added `.env`
* added environment for docker-compose.yml
now every connection-details are inside an `.env`. There is an `example.env` for better understanding which need to be set as connection details
* added description of how to use the `.env` and `example.env` in the `README.md`
* changed a few things
e.g. DB_PORT --> ${DB_PORT}
* now, everthing should be done.
my god, help. sorry
* layout issues fixed
* Fixed Typo/incomplete Sentence in README.md
* added `stac_id` for collections
* all IDs are now written in the newer PostgrSQL standart:
```SQL
id SERIAL PRIMARY KEY,
```
changed to
```SQL
id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY,
```
* changed `extend` to `extent`.
* Updated Query-Builder to get all necessary fields from all db_tables for collections. Added some tests and fixed some already existing tests, becuase now the tablenames start with the alias `c.`.
* Updated Query-Builder to get all necessary fields from all db_tables for collections. Added some tests and fixed some already existing tests, becuase now the tablenames start with the alias `c.`.
* Added `openapi.yaml` (now http://localhost:3000/api-docs/ is working).
- needed to do some modifying to the app.js
* Added discription on how to use `stac-api-validator`. Currently we are onyl valid to `core`.
* Changed API-Version name to 1.1.0 instead of 1.0.0
* Changed language used in `./api/README.md` from german to english.
I wanted to thsi anyway at some point, but this is now more like a Test-commit to see if the CI/CD Pipeline triggers...
* Revert "API is now responding with all necessary fields for each collection" (#195)
Reverts #185
@SonkeHoffmann accidentally didn't squash correctly.
* Revert "Revert "API is now responding with all necessary fields for each collection"" (#185) (#195) (#196)
dev-api: prepare v1.1.0 + API docs + query builder fixes
- Change API version to 1.1.0
- Add OpenAPI spec so /api-docs works locally
- Document stac-api-validator usage
- Update api/.env.example
- Query builder: select required fields for collections across db_tables; adjust tests (alias `c.`)
Commits included:
- 34bf962 Changed API-Version name to 1.1.0 instead of 1.0.0
- b047389 Added description on how to use `stac-api-validator` (currently only valid for `core`)
- b811288 Added `openapi.yaml` (so http://localhost:3000/api-docs/ works); modified app.js accordingly
- 6e5ab3e Merge branch 'dev-api-robin' of github.com:SpatioCore/STAC-Atlas into dev-api-robin
- 70dc043 Updated Query-Builder to get all necessary fields from all db_tables for collections. Added tests and fixed existing tests (table names now start with alias `c.`)
- d83eeb4 Update api/.env.example
- 5a7af5b Updated Query-Builder to get all necessary fields from all db_tables for collections. Added tests and fixed existing tests (table names now start with alias `c.`)
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* Implement more Queryable-Fields and add the keywords-field to q fulltext search (#200)
* Add provider and license filters to collection search API
- Updated buildCollectionSearchQuery to include provider and license parameters for filtering collections.
- Enhanced validateCollectionSearchParams middleware to validate provider and license query parameters.
- Modified collections route to handle new provider and license filters in search queries.
- Implemented validation functions for provider and license parameters in collectionSearchParams.
* Add validation tests for provider and license
* Enhance full-text search by including keywords in the tsvector expression and update related tests
* Add provider and license to query parameter extraction in collection search validation
* Revert "Enhance full-text search by including keywords in the tsvector expression and update related tests"
This reverts commit 872443d.
* Implement GET /collections/{id} endpoint with validation and QueryBuilder integration (#186)
* added SQLQuery-builder with these parameters: q,bbox,datetime,sortby,limit and token
* finalised bbox and datetime
* adapted to DB, QueryBuilder and added helperfunction runQuery
* added question-TODOs
* added bbox+datetime to the Query-Builder from Jonas
* added tests for Query-Builder from Jonas
* added tests from George
* added falsely deleted TODOs again
* fixed collumn names to match our DB and adjusted full text search to match 05_indexes.sql correctly
* Used a formatter and linter on `buildCollectionSearchQuery.js
* Did some major and minor fixes to the collection search.
- Updated `buildCollectionSearchQuery` to support pagination and improved text search with English language settings.
- Modified tests in `buildCollectionsSearchQuery.basic.test.js`, `collections-pagination.test.js`, and `collections-sort.test.js` to reflect new query behavior and validation logic.
- Enhanced sort validation in `validators.test.js` and `collectionSearchParams.js` to map API fields to database column names.
- Implemented total count retrieval for matched results in `collections.js`.
* Added a internal .env creation in the CI/CD Pipeline. It utilzes GitHub Repository Secrets to not publish any private Logins and stuff.
* Forgot that the second Job of the CI/CD pipeline runs seperatly and needs a internal .env file too.
* Enhance documentation for buildCollectionSearchQuery
Updated the documentation for:
- the buildCollectionSearchQuery function
- the fulltextsearch
* Refactor buildCollectionSearchQuery and updated SELECT part
Changed the SELECT part to match our bid and the database shema. Updated comments and for clarity. Changed full-text search to use 'simple' configuration instead of 'english'.
* Update api/routes/collections.js
small typo
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* Remove sorting TODO from collections route
Removed TODO comment about sorting based on sortby parameter.
* Explicitly return undefined for normalized in validateSortby
Update validateSortby function to explicitly return undefined for normalized when sortby is not provided.
* small fix in buildCollectionSearch.fulltext.test.js
Change plainto_tsquery language from 'english' to 'simple'
* Fix duplicate SELECT keyword in query
Remove duplicate 'SELECT' keyword in SQL query.
* Fix missing newline at end of collectionSearchParams.js
* Fixed missing bracket in collectionSearchParams.js
* Refactor validateSortby for optional parameter handling
Refactor validateSortby function to handle optional sortby parameter and improve validation logic.
* Stabilize API test pipeline by running Jest in-band with extended timeout
Run Jest in CI with --runInBand and a higher default --testTimeout to stabilize database-backed integration tests.
Multiple Jest workers were competing for the same PostgreSQL connection pool and some long-running /collections queries exceeded the default 5s timeout, causing failures in existing test suites (e.g. collectionSearch and DBconnection).
* Fixed leaking tests that blocked CI/CD-Pipeline.
- Added a global Teardown for jest and force-exited the tests to prevent leaking.
- Made a change to db_APIconnection to only log the pool-(dis)connection if it isn't run in a test enviroment.
* Did a minimum amount of Formatting to the discription
* Used `npm audit fix --force` to fix all vulnerabilties in our used packages.
* Fixed curious doublechecking for empty Strings for the sortby-Parameter.
- Now we only check once for a empty sortby
- And added a test which distinguish between `sortby=""` and `sortby="+"`
* Update api/routes/collections.js
Removed the TODO about switching from mock-data to the real db
* Removed globalTeardown as i brought up some problems corresponding to long db-queries (for example BBOX). Instead i increased the maximal testTimeout.
* added validator for collections{id} and correctly implemented collections{id}
* added test for collections{id}
* removed unnecessary parameter
* added id parameter to the Query (temporary fix)
* test-fixes to match our current tests and a fix to the baseURL for collection{id}
* test fix
* fixed problem with tests in api.test.js and adjusted the "invalid-id-test" in the validator.
* Update api/routes/collections.js
- Renamed `collection.id` to `c.collection.id`
* added test for negative ids
* deleted the whole "existing links" part and build base Links
* fixed bug in validateCollectionId.js
* Refactor negative ID test
i encoded the "-1" value in the negative ID test instead of directly putting it into the path.
* Removed a german comment in `api/routes/collections.js`
---------
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* added triggering function for an auto-update search_vector, both for collections and catalogs.
The search_vector includes title, description and keywords
* changed the CI-Pipeline.
Now also Changes in the /db will be acceped by the Pipeline
* feat(api): Implement complete CQL2 filtering for Collection Search (#208)
This commit implements comprehensive CQL2 (Common Query Language 2) filtering support for the STAC Atlas Collection Search API, enabling advanced queries on collection metadata.
## New Features
### CQL2 Parser Integration
- Integrated cql2-wasm (Rust compiled to WebAssembly) for parsing CQL2
- Support for both CQL2-Text and CQL2-JSON encodings
- Dynamic ESM import to maintain Jest compatibility with CommonJS
### Basic CQL2 Operators
- Comparison operators: =, <, >, <=, >=, <>
- Logical operators: AND, OR, NOT
- Advanced comparison: BETWEEN, IN, IS NULL
### Spatial Operators (PostGIS)
- S_INTERSECTS: Find collections whose geometry intersects with GeoJSON
- S_WITHIN: Find collections completely within a geometry
- S_CONTAINS: Find collections containing a geometry
- Uses ST_GeomFromGeoJSON for geometry parsing
### Temporal Operators
- T_INTERSECTS: Find collections with overlapping temporal extents
- T_BEFORE: Find collections before a timestamp
- T_AFTER: Find collections after a timestamp
- Support for open-ended intervals (..)
### Column Mappings
- Maps CQL2 properties to database columns with table aliases
- Core fields: id, title, description, license, type, etc.
- Aggregated fields: keywords, stac_extensions, providers, assets, summaries
- Fallback to JSONB full_json column for custom properties
## Files Added or Modified
- utils/cql2.js: WASM initialization and CQL2 parsing wrapper
- utils/cql2ToSql.js: CQL2 JSON AST to PostgreSQL WHERE clause converter
- middleware/validateCollectionSearch.js: Request validation with filter support
- docs/cql2-filtering.md: Comprehensive CQL2 documentation
- routes/collections.js: Integrated CQL2 filter processing
- utils/buildCollectionSearchQuery.js: Added cqlWhere parameter support
- config/conformanceURIS.js: Added all CQL2 conformance class URIs
- README.md: Added CQL2 section and updated implementation status
## Tests Added
- __tests__/cql2ToSql.test.js: Unit tests for SQL conversion (17 tests)
- __tests__/cql2.integration.test.js: Integration tests with database (18 tests)
- __tests__/buildCollectionSearchQuery_cql.test.js: Query builder CQL2 tests
## Technical Notes
### ESM Compatibility
The cql2-wasm package is an ES Module. To maintain compatibility with Jest
(CommonJS), the module is loaded via dynamic import() instead of require().
This allows the WASM to be initialized lazily when first needed.
### SQL Injection Prevention
All CQL2 filters are converted to parameterized queries with $1, $2, etc.
placeholders. Values are passed separately to pg-pool, preventing injection.
## Conformance Classes Implemented
- http://www.opengis.net/spec/cql2/1.0/conf/basic-cql2
- http://www.opengis.net/spec/cql2/1.0/conf/advanced-comparison-operators
- http://www.opengis.net/spec/cql2/1.0/conf/cql2-json
- http://www.opengis.net/spec/cql2/1.0/conf/cql2-text
- http://www.opengis.net/spec/cql2/1.0/conf/basic-spatial-functions
- http://www.opengis.net/spec/cql2/1.0/conf/spatial-functions
- http://www.opengis.net/spec/cql2/1.0/conf/temporal-functions
## Dependencies Added
- cql2-wasm@0.4.2: WASM-based CQL2 parser from cql2-rs
* added different users for the api and crawler groups.
The api has read-only acces and the crawler user full acces to the database. The acces to the database is now possible by using the users `stac_api`or `stac_crawler`.
The admin user (`postgres_user`) is still available but shouldn't be used
* Refactor(database): SQL trigger definitions for catalog and collection keywords. Moved triggers to 06_triggers.sql for better organization, as they depend on the respective tables created in earlier scripts.
* added source_url for collections and catalogs. Now the full_json doesn`t has to be used for getting the url
* resolved a Problem I had with git by hand cause I didn't found the function
* to be stac conform the stac_id is not allowed to throw an error when asking for a string. So the stac_id in the database is saved as a TEXT and no longer as a INTEGER
* Implemented structured Error-Handling, Error-Messaging and Error-Logging incl. request tracking - all according to RFC 7807 (#213)
* feat(api): implement RFC 7807 error handling with request tracking
Implement standardized error responses and global error handler
to improve API error reporting and debugging capabilities.
Resolves API 7.1 (Implement Error Response Format) #119
Resolves API 7.3 (Implement Global Error Handler) #121
Changes:
- Add RFC 7807 Problem Details error response format
* Standard fields: type, title, status, detail, instance, requestId
* Backwards compatibility: maintained code/description fields
* Error type URIs: https://stacspec.org/errors/{code}
- Implement request ID tracking system
* UUID v4 generation for request tracing
* Support for client-provided X-Request-ID header
* Request ID included in all error responses
- Add global error handler with intelligent logging
* Severity-based logging (500+: full details, 400+: basic info)
* Error message sanitization (removes passwords, tokens, secrets)
* Production-safe error messages
- Update error responses across codebase
* validateCollectionSearch: InvalidParameterValue errors
* validateCollectionId: InvalidParameter errors
* collections route: NotFound errors
* 404 handler: throw errors instead of direct response
- Add comprehensive error handler test suite
* RFC 7807 compliance validation
* Request ID generation and propagation
* Error code consistency checks
* Message sanitization verification
* Removed old mock-data `/api/data/collections.js` as it is no longer used
* Added Docker-Setup for API-Component (#246)
* database connection in implementated. The parameters for the connection have to added in the .env-file.
Also there is test-file for testing and console messages
(installed `pg`)
* support for spatial queries via postgis + error handling for datatbase operations
changed language to english
* error handling
* added DATABASE_URL
There is an issue with the distance query. Changed the error handling and testing, the console messages are now way better structured
* found the Problem with the distance query. The layer are so big, that they reach over the 180° long (PostgGIS can't handel that). Now the calc is done by degree and not meters.
* The two files `test-data-retrieval.js` and `verify-schema.js` have been added.
`test-data-retrieval` (theoretical, checks against the spezification):
```
Discovers all tables and columns and validates against expected schema.
```
The second files `verify-schema.js` (practical, checks against the real data):
```
Discovers all tables and columns, validates against expected schema
```
* pooling error hanling and log imporoved.
renamed tests files to actual test-files
* standalone node tests were convertad into JEST
* write file `validateRequest.js`. Validates every incoming API request, whether the request is valid and logical.
* commented `stac_id` from the tests, it is not in both databases, so the tests for `stac_id` will always fail
Added explanation to the `.env.example`, which port is which database
* added example pattern for API - database connection.
* deleted `validateRequest` cause it's already implemented by @RobinGummels
* added everything related to containerisation for the API-component. The docker compose starts the whole API folder and starts the whole API funcunality
* Deleted `./api/example/README.md` because the same file already exists under a different name in `./api/docs/`
---------
Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de>
Co-authored-by: RobinGummels <github@gummels.eu>
* Implement rate limiting middleware and update README with rate limit details (#253)
* Enhance full-text search by including keywords in the tsvector expression and update related tests
* Revert "Enhance full-text search by including keywords in the tsvector expression and update related tests"
This reverts commit 872443d.
* Enhance full-text search by including keywords in the tsvector expression and update related tests
* Revert "Enhance full-text search by including keywords in the tsvector expression and update related tests"
This reverts commit 872443d.
* Enhance full-text search by including keywords in the tsvector expression and update related tests
* Revert "Enhance full-text search by including keywords in the tsvector expression and update related tests"
This reverts commit 872443d.
* Enhance full-text search by including keywords in the tsvector expression and update related tests
* Revert "Enhance full-text search by including keywords in the tsvector expression and update related tests"
This reverts commit 872443d.
* Implement rate limiting middleware and update README with rate limit details
* Fixed wrong Errorhandling. Now Ratelimit-Errors will be handled the same, as all other Errors according to RFC7807
* Removed sample Errorresponse in `README.md`.
---------
Co-authored-by: RobinGummels <github@gummels.eu>
* STAC API Validator Compliance, Tests Alignment, and CI Integration (#216)
* Add STAC API Validator workflow and enhance collection retrieval logic
- Introduced a new CI job for STAC API validation in the GitHub Actions workflow.
- Updated collection retrieval endpoints to support both numeric and string IDs.
- Improved validation middleware for collection IDs to ensure proper formatting and length.
- Enhanced test cases for collection endpoints to reflect new validation rules and response structures.
- Added documentation for STAC API Validator results.
* refactor(api): Remove unused cqlFilter parameter from buildCollectionSearchQuery function
* feat(api): Add falsely removed cqlFilter parameter back to buildCollectionSearchQuery function
* feat(api): Add queryables schema for STAC Atlas collections
* Add 'parent' link check in collections test
Add test to check for 'parent' link in collections response
* Update api/__tests__/collections-id.test.js
removed (non-numeric) as its outdated
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* Remove test case for negative id 404 response
Removed test for non-existing negative id.
* Update api/middleware/validateCollectionId.js
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* Update api/middleware/validateCollectionId.js
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* Update api/middleware/validateCollectionId.js
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* Add validation tests for collection ID format
Added tests for validation of collection IDs including length, invalid characters, and empty/whitespace cases.
* Add TODOs for full_json and extent handling
Added TODO comments for future database schema updates.
* Change error code from 'InvalidParameter' to 'NotFound'
* Fix error messages for id parameter validation
* Correct error response in validateCollectionId
Fix error response structure in validateCollectionId middleware.
* move helper function from collections.js out of block
* Disable parent link test in collections response
Comment out the test for 'parent' link in collections response.
* bugfix Change error response from 400 to 404 for invalid parameter
* Update validation to return error response
Return a 400 status with an error response instead of calling next() when validation fails.
* Fix duplicate error response in validateCollectionId
* Fixed: Incorrect middelware handling (missed `return next()`) and added `parent` link into the response of `GET /collections` and brought back helperfunction inside of the old base-code-block.
---------
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* Dev docker: container to start every component container at once (#259)
* added docker-compose for the whole project. Via the docker-comand include every docker-sompose from the under-foldrs can be started.
* changed structure of the docker-compose. Now the dontainer is more resistant against issues
* Database: Docker config for using the same network, STAC_ID now unique and stac_url is now in collections (#262)
* Add .gitignore for environment and dependencies; update docker-compose to include network configuration
* Update docker-compose.yml to rename network from 'stac_network' to 'stac-network' for consistency.
* db(change) source_url is now in collection and stac_id is now unique
* type is no longer needes, since we only need to save collections
* added is_valid. So we can save every collection be can also be stac_conform since we can easily sort by valid collections
* Database: Added source_url in crawllog (#274)
* Add .gitignore for environment and dependencies; update docker-compose to include network configuration
* Update docker-compose.yml to rename network from 'stac_network' to 'stac-network' for consistency.
* db(change) source_url is now in collection and stac_id is now unique
* database(feature)
Enhance crawllog_collection table: make collection_id optional and add source_url field for identifying collections not in the collection table. Update documentation accordingly and create an index on source_url for improved query performance.
* changed the whole structure of the catalogs. Now the catalogs can be seen as a crawlog for teh crawler. The crawllog can be now used in case the crawling gets cancelled and the crawler can use the crawllog for its own
* We asked Mohr on the handling with valid and unvalid collection. We don't need to save the unvalid collections, so also don't need the collumn that points that out
* the reference for the crawllog_collections weren't right
---------
Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de>
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
Co-authored-by: JonasK <156602337+BrokeJ@users.noreply.github.com>
Co-authored-by: Vincent Kühn <vkuehn@uni-muenster.de>
Co-authored-by: mammutor <mammutor@gmail.com>
Co-authored-by: Humam <44206081+Mammutor@users.noreply.github.com>
Mammutor
added a commit
that referenced
this pull request
Feb 3, 2026
* Node init
* Basic Crawling of STAC Index API
* feat: every object in the stac index, will be converted to an dynamic array
* added crawling to collection level, including nested catalogs
* Closes #55; added crawling to collection level, including nested catalogs
* feat: implement database connection with node pg for catalog management
* closes #53, set up docker for the crawler component
* fixes #55, restructured the crawler component into catalog and api crawling
* feat: add dotenv for environment variable management and improved database connection
* deleted .env
* updated crawling with use uf stac-js
* feat: basic api crawling
* implements #76, stac api validation for catalog crawling
* fix: updated crawler project to be ESM compliant
I had an issue on the server, where it doesnt want to start, because stac-js is ESM only and the project isnt
* refactor: improve catalog handling in crawler by restructuring catalog data extraction and ensuring consistent property ordering
* feat: enhance database operations:
insertOrUpdateCatalog and insertOrUpdateCollection
* fix: adjust API crawling limit to include all found APIs instead of the first five
* updated pghost
function to insert keywords
* feat: changed database connection configuration and add insertStacExtensions helper function
* removed the api valication #76
* feat: implement helper functions for inserting STAC extensions, summaries, providers, and assets
* feat: enhance crawler configuration with CLI and environment variable support
* feat: functions to save crawled catalogs, collections and apis in the database
* #154, refactored the crawler to use crawlee Framework, improved performance and structure
* #53,example environment configuration file for database and crawler settings
* added `.env`
* added environment for docker-compose.yml
now every connection-details are inside an `.env`. There is an `example.env` for better understanding which need to be set as connection details
* added description of how to use the `.env` and `example.env` in the `README.md`
* changed a few things
e.g. DB_PORT --> ${DB_PORT}
* now, everthing should be done.
my god, help. sorry
* layout issues fixed
* Fixed Typo/incomplete Sentence in README.md
* converted to es modules, added --no-db mode for debugging and moved cli parsing to own file
* Update .env.example
Co-authored-by: Humam <44206081+Mammutor@users.noreply.github.com>
* Refactor database interactions by moving db helper to utils and implementing insert/update functions for catalogs and collections
* #176, fixed logic for max apis, fixed double db init
* #167, fixed logic for max apis, fixed double db init
* #153, refactor: update API crawling logic to use Crawlee and enhance result structure
* Update .gitignore and enhance catalog/collection database handling
* psst
* Update .gitignore and enhance catalog/collection database handling
* added `stac_id` for collections
* all IDs are now written in the newer PostgrSQL standart:
```SQL
id SERIAL PRIMARY KEY,
```
changed to
```SQL
id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY,
```
* changed `extend` to `extent`.
* Scheduling and time logic for recrawling (#190)
* added crawling time statistics
* #189, added 7 day scheduling, created time util for easier managment/readability
* Changed language used in `./api/README.md` from german to english.
I wanted to thsi anyway at some point, but this is now more like a Test-commit to see if the CI/CD Pipeline triggers...
* fix: time utility was missing and improved main module check logic (#193)
* fix: time utility was missing and improved main module check logic
* chore: add stac-network to crawler and db services in docker-compose files
* chore: update .gitignore and .env.example for crawler configuration
* added triggering function for an auto-update search_vector, both for collections and catalogs.
The search_vector includes title, description and keywords
* changed the CI-Pipeline.
Now also Changes in the /db will be acceped by the Pipeline
* bug fix: extent was written wrong
* A quick cleanup and fix for the crawler to work for docker (#210)
* fix: time utility was missing and improved main module check logic
* chore: add stac-network to crawler and db services in docker-compose files
* chore: update .gitignore and .env.example for crawler configuration
* fix: enhance database connection error handling and logging in initDb function
* bug fix: extent was written wrong (#212)
* feat(api): initialize STAC Atlas API with collections, conformance, and queryables routes
- Added package.json for project dependencies and scripts.
- Implemented GET endpoint for collections.
- Created conformance endpoint to list supported conformance classes.
- Developed landing page for the API with links to collections and documentation.
- Added queryables endpoint to return queryable properties for collections. (If i'm correct this can be removed)
* Added all Remarks to the bid and finished it (#74)
* changed Texts 1,2 and 8 according to the remarks of the customer
* Did my fixes to 4. and 10.3
* added remark why we want to save every catalog
* deleted keywords for catalog
* changed everything related to the database component
* Updated 3.1, 3.3, 3,4 (References to Lastenheft/small other changes)
* Added small Graph to 3. Produktumgebung
* Update bid.md 7.3STAC-Validator
added the handling of collections that cannot be validated automatically.
* Update bid.md 7.3STAC-API-Validator
changed the way we validate the collection search extension.
* Update bid.md 9.3.2Endpunkte
small fix collection search extension.
* added Skizze for 3, and updated 6.1, 10.1
* Updated 3. Produktumgebung
* Update bid.md 7.Qualitätsanforderungen
minor fixes
* Update 6.4.2 added remark about loading feedback
* Minor change to Table in "11 Zeitplan". A collum was missing in the head, therefore the table wasn't rendering correctly..
---------
Co-authored-by: Jakob <vertrox78@gmail.com>
Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de>
Co-authored-by: jklaer <jklaer@uni-muenster.de>
Co-authored-by: VincentKuehn <vkuehn@uni-muenster.de>
Co-authored-by: mammutor <mammutor@gmail.com>
Co-authored-by: Humam <44206081+Mammutor@users.noreply.github.com>
Co-authored-by: Justin K <justinkrumbhmer@yahoo.com>
* fix(api): enhance STAC API landing page and conformance links
- Overhaul of first idea landing page
- Added some more tests for the required elements in the landingpage-Catalog
* feat(api): implement shared conformance URIs and add tests for conformance endpoint
- implemented condormance endpoint
* Implemented 2.3 and 2.4 (#111)
* Temporary mock data for testing and frontend development
* Added API middleware layer for error handling and validation
* TODOs ready? pls review
* Added API utilities for query parsing, validation, and response formatting
* Added swagger and openapi.yaml
* Update queryables.js
Refactor queryables endpoint into /collections/queryables
* Renamed the collections.js file to mocks-collections.js to better reflect its purpose and improve project clarity
* changed README "Projektstruktur"
* restart from dev-api 22.11..2025
* API: 2.3 Implement Collections List Endpoint done (added explanations as comments in the code)
* API: 2.4 Implement Single Collection Endpoint (added explanations as comments in the code)
* Update api/routes/collections.js
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* Changed some of the code with the comments on Github (i will finish it tomorrow morning)
* Implement most of the feedback and comments (need to talk about some other changes)
* Update api/routes/index.js
Changed wording from `/collections/queryables` to `/collections-queryables`
* Update api/README.md
Changed wording from `/collections/queryables` to `/collections-queryables`
* Update api/README.md
Removed missing folder
* Update api/routes/collections.js
Removed TODOs from wrong lines
* Update api/routes/collections.js
Added TODOs
* Update api/routes/queryables.js
Changed wording from `/collections/queryables` to `/collections-queryables`
* Update api/routes/queryables.js
Changed wording from `/collections/queryables` to `/collections-queryables`
---------
Co-authored-by: VincentKuehn <vkuehn@uni-muenster.de>
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* feat(api): add collection search parameters and validation middleware (#159)
* feat(api): add collection search parameters and validation middleware
* Added unit-test for validator-functions and integration-tests for `GET /collections`-Querys.
- Also minor bugfix, because the validator accepted deecimals as tokens.
* API: 3 Database Integration first version (#161)
* database connection in implementated. The parameters for the connection have to added in the .env-file.
Also there is test-file for testing and console messages
(installed `pg`)
* support for spatial queries via postgis + error handling for datatbase operations
changed language to english
* error handling
* added DATABASE_URL
There is an issue with the distance query. Changed the error handling and testing, the console messages are now way better structured
* found the Problem with the distance query. The layer are so big, that they reach over the 180° long (PostgGIS can't handel that). Now the calc is done by degree and not meters.
* The two files `test-data-retrieval.js` and `verify-schema.js` have been added.
`test-data-retrieval` (theoretical, checks against the spezification):
```
Discovers all tables and columns and validates against expected schema.
```
The second files `verify-schema.js` (practical, checks against the real data):
```
Discovers all tables and columns, validates against expected schema
```
* pooling error hanling and log imporoved.
renamed tests files to actual test-files
* standalone node tests were convertad into JEST
* write file `validateRequest.js`. Validates every incoming API request, whether the request is valid and logical.
* commented `stac_id` from the tests, it is not in both databases, so the tests for `stac_id` will always fail
Added explanation to the `.env.example`, which port is which database
* added example pattern for API - database connection.
* deleted `validateRequest` cause it's already implemented by @robinGummels
---------
Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de>
* Added environment variables and a `.env` for `docker-compose.yml` (#164)
* added `.env`
* added environment for docker-compose.yml
now every connection-details are inside an `.env`. There is an `example.env` for better understanding which need to be set as connection details
* added description of how to use the `.env` and `example.env` in the `README.md`
* changed a few things
e.g. DB_PORT --> ${DB_PORT}
* now, everthing should be done.
my god, help. sorry
* layout issues fixed
* Fixed Typo/incomplete Sentence in README.md
---------
Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de>
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* Added a CI/CD Pipeline to prevent pull-requests without functioning tests and proper linting.
* fixed errors suggested by the linter.
- Some lines used tab and spaces...
* Implemented Collection Search extension including a DB-Connection (#165)
* added SQLQuery-builder with these parameters: q,bbox,datetime,sortby,limit and token
* finalised bbox and datetime
* adapted to DB, QueryBuilder and added helperfunction runQuery
* added question-TODOs
* added bbox+datetime to the Query-Builder from Jonas
* added tests for Query-Builder from Jonas
* added tests from George
* added falsely deleted TODOs again
* fixed collumn names to match our DB and adjusted full text search to match 05_indexes.sql correctly
* Used a formatter and linter on `buildCollectionSearchQuery.js
* Did some major and minor fixes to the collection search.
- Updated `buildCollectionSearchQuery` to support pagination and improved text search with English language settings.
- Modified tests in `buildCollectionsSearchQuery.basic.test.js`, `collections-pagination.test.js`, and `collections-sort.test.js` to reflect new query behavior and validation logic.
- Enhanced sort validation in `validators.test.js` and `collectionSearchParams.js` to map API fields to database column names.
- Implemented total count retrieval for matched results in `collections.js`.
* Added a internal .env creation in the CI/CD Pipeline. It utilzes GitHub Repository Secrets to not publish any private Logins and stuff.
* Forgot that the second Job of the CI/CD pipeline runs seperatly and needs a internal .env file too.
* Enhance documentation for buildCollectionSearchQuery
Updated the documentation for:
- the buildCollectionSearchQuery function
- the fulltextsearch
* Refactor buildCollectionSearchQuery and updated SELECT part
Changed the SELECT part to match our bid and the database shema. Updated comments and for clarity. Changed full-text search to use 'simple' configuration instead of 'english'.
* Update api/routes/collections.js
small typo
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* Remove sorting TODO from collections route
Removed TODO comment about sorting based on sortby parameter.
* Explicitly return undefined for normalized in validateSortby
Update validateSortby function to explicitly return undefined for normalized when sortby is not provided.
* small fix in buildCollectionSearch.fulltext.test.js
Change plainto_tsquery language from 'english' to 'simple'
* Fix duplicate SELECT keyword in query
Remove duplicate 'SELECT' keyword in SQL query.
* Fix missing newline at end of collectionSearchParams.js
* Fixed missing bracket in collectionSearchParams.js
* Refactor validateSortby for optional parameter handling
Refactor validateSortby function to handle optional sortby parameter and improve validation logic.
* Stabilize API test pipeline by running Jest in-band with extended timeout
Run Jest in CI with --runInBand and a higher default --testTimeout to stabilize database-backed integration tests.
Multiple Jest workers were competing for the same PostgreSQL connection pool and some long-running /collections queries exceeded the default 5s timeout, causing failures in existing test suites (e.g. collectionSearch and DBconnection).
* Fixed leaking tests that blocked CI/CD-Pipeline.
- Added a global Teardown for jest and force-exited the tests to prevent leaking.
- Made a change to db_APIconnection to only log the pool-(dis)connection if it isn't run in a test enviroment.
* Did a minimum amount of Formatting to the discription
* Used `npm audit fix --force` to fix all vulnerabilties in our used packages.
* Fixed curious doublechecking for empty Strings for the sortby-Parameter.
- Now we only check once for a empty sortby
- And added a test which distinguish between `sortby=""` and `sortby="+"`
* Update api/routes/collections.js
Removed the TODO about switching from mock-data to the real db
* Removed globalTeardown as i brought up some problems corresponding to long db-queries (for example BBOX). Instead i increased the maximal testTimeout.
---------
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* Update api/.env.example
* latest database Version (#187) with `stac_id` and changed definition of `primary Keys`
* added `.env`
* added environment for docker-compose.yml
now every connection-details are inside an `.env`. There is an `example.env` for better understanding which need to be set as connection details
* added description of how to use the `.env` and `example.env` in the `README.md`
* changed a few things
e.g. DB_PORT --> ${DB_PORT}
* now, everthing should be done.
my god, help. sorry
* layout issues fixed
* Fixed Typo/incomplete Sentence in README.md
* added `stac_id` for collections
* all IDs are now written in the newer PostgrSQL standart:
```SQL
id SERIAL PRIMARY KEY,
```
changed to
```SQL
id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY,
```
* changed `extend` to `extent`.
* Changed language used in `./api/README.md` from german to english.
I wanted to thsi anyway at some point, but this is now more like a Test-commit to see if the CI/CD Pipeline triggers...
---------
Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de>
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* bug fix: extent was written wrong
---------
Co-authored-by: RobinGummels <github@gummels.eu>
Co-authored-by: Jakob <vertrox78@gmail.com>
Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de>
Co-authored-by: jklaer <jklaer@uni-muenster.de>
Co-authored-by: VincentKuehn <vkuehn@uni-muenster.de>
Co-authored-by: mammutor <mammutor@gmail.com>
Co-authored-by: Humam <44206081+Mammutor@users.noreply.github.com>
Co-authored-by: Justin K <justinkrumbhmer@yahoo.com>
Co-authored-by: Georgios Voulgaris <gvoulgar@uni-muenster.de>
Co-authored-by: Sönke Hoffmann <hoffmann.sonke@gmx.de>
* refactor(api): simplify collections link handling in STAC API root function
- Streamlined the logic for retrieving the collections link by directly using the href property if available, improving code clarity and reducing unnecessary checks.
* chore(crawler): comment out restart policy in docker-compose.yml
* fix(crawler): update .env.example and normalize base URL handling
- Added new environment variables for Postgres/PostGis configuration and crawler settings in .env.example.
- Improved URL handling in tryCollectionEndpoints by normalizing the base URL to prevent double slashes.
* added different users for the api and crawler groups.
The api has read-only acces and the crawler user full acces to the database. The acces to the database is now possible by using the users `stac_api`or `stac_crawler`.
The admin user (`postgres_user`) is still available but shouldn't be used
* chore(db): update docker-compose.yml for network configuration
- Changed the networks section to use array syntax for consistency.
- Added a driver specification for the stac-network to use bridge mode.
* Refactor(database): SQL trigger definitions for catalog and collection keywords. Moved triggers to 06_triggers.sql for better organization, as they depend on the respective tables created in earlier scripts.
* feat(crawler): implement collection flushing to database
- Added a new utility function `flushCollectionsToDb` to handle batch saving of collections to the database.
- Integrated flushing logic into the crawling process to save collections periodically and at the end of the crawl.
- Updated statistics to track saved and failed collections during the flush operation.
- Enhanced logging for better visibility of the flushing process.
* added source_url for collections and catalogs. Now the full_json doesn`t has to be used for getting the url
* resolved a Problem I had with git by hand cause I didn't found the function
* feat(crawler): enhance concurrency and storage configuration for crawling
- Updated `crawlApis` and `crawlCatalogs` functions to use in-memory storage, preventing file lock race conditions under high concurrency.
- Set `maxConcurrency` to 20 and `maxRequestsPerMinute` to 200 to limit request rates and improve stability during high-load scenarios.
- Imported `Configuration` from `crawlee` to manage global settings effectively.
* feat(crawler): implement max depth configuration and batch flushing for collections
- Updated the crawler to support a maximum depth for nested catalogs, preventing excessive recursion.
- Enhanced the collection handling by implementing batch flushing to the database during the crawl process.
- Improved logging to provide detailed statistics on collections found, saved, and failed during the crawl.
- Added CLI options for configuring maximum depth and updated the configuration defaults accordingly.
* Crawler: implemented rate limiting, and implements some bug fixes (#211)
* added crawling time statistics
* #189, added 7 day scheduling, created time util for easier managment/readability
* #202, feat(crawler): add rate limiting and STAC link discovery, fixed bug with wndpoint logic and additional MIME types
* #227, fixed bug catalogs that where listed as catlogs were treated as apis, no will be handled by the catalog crawler
* to be stac conform the stac_id is not allowed to throw an error when asking for a string. So the stac_id in the database is saved as a TEXT and no longer as a INTEGER
* feat: add source URL extraction for catalogs and collections
* feat: update insertOrUpdateCatalog to only process catalogs for traversal, no longer saving to database
* merge from dev to dev-crawler-humam and now to dev-crawler (#243)
* feat(api): initialize STAC Atlas API with collections, conformance, and queryables routes
- Added package.json for project dependencies and scripts.
- Implemented GET endpoint for collections.
- Created conformance endpoint to list supported conformance classes.
- Developed landing page for the API with links to collections and documentation.
- Added queryables endpoint to return queryable properties for collections. (If i'm correct this can be removed)
* Added all Remarks to the bid and finished it (#74)
* changed Texts 1,2 and 8 according to the remarks of the customer
* Did my fixes to 4. and 10.3
* added remark why we want to save every catalog
* deleted keywords for catalog
* changed everything related to the database component
* Updated 3.1, 3.3, 3,4 (References to Lastenheft/small other changes)
* Added small Graph to 3. Produktumgebung
* Update bid.md 7.3STAC-Validator
added the handling of collections that cannot be validated automatically.
* Update bid.md 7.3STAC-API-Validator
changed the way we validate the collection search extension.
* Update bid.md 9.3.2Endpunkte
small fix collection search extension.
* added Skizze for 3, and updated 6.1, 10.1
* Updated 3. Produktumgebung
* Update bid.md 7.Qualitätsanforderungen
minor fixes
* Update 6.4.2 added remark about loading feedback
* Minor change to Table in "11 Zeitplan". A collum was missing in the head, therefore the table wasn't rendering correctly..
---------
Co-authored-by: Jakob <vertrox78@gmail.com>
Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de>
Co-authored-by: jklaer <jklaer@uni-muenster.de>
Co-authored-by: VincentKuehn <vkuehn@uni-muenster.de>
Co-authored-by: mammutor <mammutor@gmail.com>
Co-authored-by: Humam <44206081+Mammutor@users.noreply.github.com>
Co-authored-by: Justin K <justinkrumbhmer@yahoo.com>
* fix(api): enhance STAC API landing page and conformance links
- Overhaul of first idea landing page
- Added some more tests for the required elements in the landingpage-Catalog
* feat(api): implement shared conformance URIs and add tests for conformance endpoint
- implemented condormance endpoint
* Implemented 2.3 and 2.4 (#111)
* Temporary mock data for testing and frontend development
* Added API middleware layer for error handling and validation
* TODOs ready? pls review
* Added API utilities for query parsing, validation, and response formatting
* Added swagger and openapi.yaml
* Update queryables.js
Refactor queryables endpoint into /collections/queryables
* Renamed the collections.js file to mocks-collections.js to better reflect its purpose and improve project clarity
* changed README "Projektstruktur"
* restart from dev-api 22.11..2025
* API: 2.3 Implement Collections List Endpoint done (added explanations as comments in the code)
* API: 2.4 Implement Single Collection Endpoint (added explanations as comments in the code)
* Update api/routes/collections.js
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* Changed some of the code with the comments on Github (i will finish it tomorrow morning)
* Implement most of the feedback and comments (need to talk about some other changes)
* Update api/routes/index.js
Changed wording from `/collections/queryables` to `/collections-queryables`
* Update api/README.md
Changed wording from `/collections/queryables` to `/collections-queryables`
* Update api/README.md
Removed missing folder
* Update api/routes/collections.js
Removed TODOs from wrong lines
* Update api/routes/collections.js
Added TODOs
* Update api/routes/queryables.js
Changed wording from `/collections/queryables` to `/collections-queryables`
* Update api/routes/queryables.js
Changed wording from `/collections/queryables` to `/collections-queryables`
---------
Co-authored-by: VincentKuehn <vkuehn@uni-muenster.de>
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* feat(api): add collection search parameters and validation middleware (#159)
* feat(api): add collection search parameters and validation middleware
* Added unit-test for validator-functions and integration-tests for `GET /collections`-Querys.
- Also minor bugfix, because the validator accepted deecimals as tokens.
* API: 3 Database Integration first version (#161)
* database connection in implementated. The parameters for the connection have to added in the .env-file.
Also there is test-file for testing and console messages
(installed `pg`)
* support for spatial queries via postgis + error handling for datatbase operations
changed language to english
* error handling
* added DATABASE_URL
There is an issue with the distance query. Changed the error handling and testing, the console messages are now way better structured
* found the Problem with the distance query. The layer are so big, that they reach over the 180° long (PostgGIS can't handel that). Now the calc is done by degree and not meters.
* The two files `test-data-retrieval.js` and `verify-schema.js` have been added.
`test-data-retrieval` (theoretical, checks against the spezification):
```
Discovers all tables and columns and validates against expected schema.
```
The second files `verify-schema.js` (practical, checks against the real data):
```
Discovers all tables and columns, validates against expected schema
```
* pooling error hanling and log imporoved.
renamed tests files to actual test-files
* standalone node tests were convertad into JEST
* write file `validateRequest.js`. Validates every incoming API request, whether the request is valid and logical.
* commented `stac_id` from the tests, it is not in both databases, so the tests for `stac_id` will always fail
Added explanation to the `.env.example`, which port is which database
* added example pattern for API - database connection.
* deleted `validateRequest` cause it's already implemented by @robinGummels
---------
Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de>
* Added environment variables and a `.env` for `docker-compose.yml` (#164)
* added `.env`
* added environment for docker-compose.yml
now every connection-details are inside an `.env`. There is an `example.env` for better understanding which need to be set as connection details
* added description of how to use the `.env` and `example.env` in the `README.md`
* changed a few things
e.g. DB_PORT --> ${DB_PORT}
* now, everthing should be done.
my god, help. sorry
* layout issues fixed
* Fixed Typo/incomplete Sentence in README.md
---------
Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de>
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* Added a CI/CD Pipeline to prevent pull-requests without functioning tests and proper linting.
* fixed errors suggested by the linter.
- Some lines used tab and spaces...
* Implemented Collection Search extension including a DB-Connection (#165)
* added SQLQuery-builder with these parameters: q,bbox,datetime,sortby,limit and token
* finalised bbox and datetime
* adapted to DB, QueryBuilder and added helperfunction runQuery
* added question-TODOs
* added bbox+datetime to the Query-Builder from Jonas
* added tests for Query-Builder from Jonas
* added tests from George
* added falsely deleted TODOs again
* fixed collumn names to match our DB and adjusted full text search to match 05_indexes.sql correctly
* Used a formatter and linter on `buildCollectionSearchQuery.js
* Did some major and minor fixes to the collection search.
- Updated `buildCollectionSearchQuery` to support pagination and improved text search with English language settings.
- Modified tests in `buildCollectionsSearchQuery.basic.test.js`, `collections-pagination.test.js`, and `collections-sort.test.js` to reflect new query behavior and validation logic.
- Enhanced sort validation in `validators.test.js` and `collectionSearchParams.js` to map API fields to database column names.
- Implemented total count retrieval for matched results in `collections.js`.
* Added a internal .env creation in the CI/CD Pipeline. It utilzes GitHub Repository Secrets to not publish any private Logins and stuff.
* Forgot that the second Job of the CI/CD pipeline runs seperatly and needs a internal .env file too.
* Enhance documentation for buildCollectionSearchQuery
Updated the documentation for:
- the buildCollectionSearchQuery function
- the fulltextsearch
* Refactor buildCollectionSearchQuery and updated SELECT part
Changed the SELECT part to match our bid and the database shema. Updated comments and for clarity. Changed full-text search to use 'simple' configuration instead of 'english'.
* Update api/routes/collections.js
small typo
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* Remove sorting TODO from collections route
Removed TODO comment about sorting based on sortby parameter.
* Explicitly return undefined for normalized in validateSortby
Update validateSortby function to explicitly return undefined for normalized when sortby is not provided.
* small fix in buildCollectionSearch.fulltext.test.js
Change plainto_tsquery language from 'english' to 'simple'
* Fix duplicate SELECT keyword in query
Remove duplicate 'SELECT' keyword in SQL query.
* Fix missing newline at end of collectionSearchParams.js
* Fixed missing bracket in collectionSearchParams.js
* Refactor validateSortby for optional parameter handling
Refactor validateSortby function to handle optional sortby parameter and improve validation logic.
* Stabilize API test pipeline by running Jest in-band with extended timeout
Run Jest in CI with --runInBand and a higher default --testTimeout to stabilize database-backed integration tests.
Multiple Jest workers were competing for the same PostgreSQL connection pool and some long-running /collections queries exceeded the default 5s timeout, causing failures in existing test suites (e.g. collectionSearch and DBconnection).
* Fixed leaking tests that blocked CI/CD-Pipeline.
- Added a global Teardown for jest and force-exited the tests to prevent leaking.
- Made a change to db_APIconnection to only log the pool-(dis)connection if it isn't run in a test enviroment.
* Did a minimum amount of Formatting to the discription
* Used `npm audit fix --force` to fix all vulnerabilties in our used packages.
* Fixed curious doublechecking for empty Strings for the sortby-Parameter.
- Now we only check once for a empty sortby
- And added a test which distinguish between `sortby=""` and `sortby="+"`
* Update api/routes/collections.js
Removed the TODO about switching from mock-data to the real db
* Removed globalTeardown as i brought up some problems corresponding to long db-queries (for example BBOX). Instead i increased the maximal testTimeout.
---------
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* Update api/.env.example
* latest database Version (#187) with `stac_id` and changed definition of `primary Keys`
* added `.env`
* added environment for docker-compose.yml
now every connection-details are inside an `.env`. There is an `example.env` for better understanding which need to be set as connection details
* added description of how to use the `.env` and `example.env` in the `README.md`
* changed a few things
e.g. DB_PORT --> ${DB_PORT}
* now, everthing should be done.
my god, help. sorry
* layout issues fixed
* Fixed Typo/incomplete Sentence in README.md
* added `stac_id` for collections
* all IDs are now written in the newer PostgrSQL standart:
```SQL
id SERIAL PRIMARY KEY,
```
changed to
```SQL
id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY,
```
* changed `extend` to `extent`.
* Changed language used in `./api/README.md` from german to english.
I wanted to thsi anyway at some point, but this is now more like a Test-commit to see if the CI/CD Pipeline triggers...
---------
Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de>
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* Added `GET /collections/{id}`-Endpoint, more Fields in the responses and more Queryables-Parameters (#204)
* Updated Query-Builder to get all necessary fields from all db_tables for collections. Added some tests and fixed some already existing tests, becuase now the tablenames start with the alias `c.`.
* Updated Query-Builder to get all necessary fields from all db_tables for collections. Added some tests and fixed some already existing tests, becuase now the tablenames start with the alias `c.`.
* Added `openapi.yaml` (now http://localhost:3000/api-docs/ is working).
- needed to do some modifying to the app.js
* Added discription on how to use `stac-api-validator`. Currently we are onyl valid to `core`.
* Changed API-Version name to 1.1.0 instead of 1.0.0
* Revert "API is now responding with all necessary fields for each collection" (#195)
Reverts #185
@SonkeHoffmann accidentally didn't squash correctly.
* Revert "Revert "API is now responding with all necessary fields for each collection"" (#185) (#195) (#196)
dev-api: prepare v1.1.0 + API docs + query builder fixes
- Change API version to 1.1.0
- Add OpenAPI spec so /api-docs works locally
- Document stac-api-validator usage
- Update api/.env.example
- Query builder: select required fields for collections across db_tables; adjust tests (alias `c.`)
Commits included:
- 34bf962 Changed API-Version name to 1.1.0 instead of 1.0.0
- b047389 Added description on how to use `stac-api-validator` (currently only valid for `core`)
- b811288 Added `openapi.yaml` (so http://localhost:3000/api-docs/ works); modified app.js accordingly
- 6e5ab3e Merge branch 'dev-api-robin' of github.com:SpatioCore/STAC-Atlas into dev-api-robin
- 70dc043 Updated Query-Builder to get all necessary fields from all db_tables for collections. Added tests and fixed existing tests (table names now start with alias `c.`)
- d83eeb4 Update api/.env.example
- 5a7af5b Updated Query-Builder to get all necessary fields from all db_tables for collections. Added tests and fixed existing tests (table names now start with alias `c.`)
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* Implement more Queryable-Fields and add the keywords-field to q fulltext search (#200)
* Add provider and license filters to collection search API
- Updated buildCollectionSearchQuery to include provider and license parameters for filtering collections.
- Enhanced validateCollectionSearchParams middleware to validate provider and license query parameters.
- Modified collections route to handle new provider and license filters in search queries.
- Implemented validation functions for provider and license parameters in collectionSearchParams.
* Add validation tests for provider and license
* Enhance full-text search by including keywords in the tsvector expression and update related tests
* Add provider and license to query parameter extraction in collection search validation
* Revert "Enhance full-text search by including keywords in the tsvector expression and update related tests"
This reverts commit 872443d8e83c55834ef5f0d275c54fefb4b74e2d.
* Implement GET /collections/{id} endpoint with validation and QueryBuilder integration (#186)
* added SQLQuery-builder with these parameters: q,bbox,datetime,sortby,limit and token
* finalised bbox and datetime
* adapted to DB, QueryBuilder and added helperfunction runQuery
* added question-TODOs
* added bbox+datetime to the Query-Builder from Jonas
* added tests for Query-Builder from Jonas
* added tests from George
* added falsely deleted TODOs again
* fixed collumn names to match our DB and adjusted full text search to match 05_indexes.sql correctly
* Used a formatter and linter on `buildCollectionSearchQuery.js
* Did some major and minor fixes to the collection search.
- Updated `buildCollectionSearchQuery` to support pagination and improved text search with English language settings.
- Modified tests in `buildCollectionsSearchQuery.basic.test.js`, `collections-pagination.test.js`, and `collections-sort.test.js` to reflect new query behavior and validation logic.
- Enhanced sort validation in `validators.test.js` and `collectionSearchParams.js` to map API fields to database column names.
- Implemented total count retrieval for matched results in `collections.js`.
* Added a internal .env creation in the CI/CD Pipeline. It utilzes GitHub Repository Secrets to not publish any private Logins and stuff.
* Forgot that the second Job of the CI/CD pipeline runs seperatly and needs a internal .env file too.
* Enhance documentation for buildCollectionSearchQuery
Updated the documentation for:
- the buildCollectionSearchQuery function
- the fulltextsearch
* Refactor buildCollectionSearchQuery and updated SELECT part
Changed the SELECT part to match our bid and the database shema. Updated comments and for clarity. Changed full-text search to use 'simple' configuration instead of 'english'.
* Update api/routes/collections.js
small typo
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* Remove sorting TODO from collections route
Removed TODO comment about sorting based on sortby parameter.
* Explicitly return undefined for normalized in validateSortby
Update validateSortby function to explicitly return undefined for normalized when sortby is not provided.
* small fix in buildCollectionSearch.fulltext.test.js
Change plainto_tsquery language from 'english' to 'simple'
* Fix duplicate SELECT keyword in query
Remove duplicate 'SELECT' keyword in SQL query.
* Fix missing newline at end of collectionSearchParams.js
* Fixed missing bracket in collectionSearchParams.js
* Refactor validateSortby for optional parameter handling
Refactor validateSortby function to handle optional sortby parameter and improve validation logic.
* Stabilize API test pipeline by running Jest in-band with extended timeout
Run Jest in CI with --runInBand and a higher default --testTimeout to stabilize database-backed integration tests.
Multiple Jest workers were competing for the same PostgreSQL connection pool and some long-running /collections queries exceeded the default 5s timeout, causing failures in existing test suites (e.g. collectionSearch and DBconnection).
* Fixed leaking tests that blocked CI/CD-Pipeline.
- Added a global Teardown for jest and force-exited the tests to prevent leaking.
- Made a change to db_APIconnection to only log the pool-(dis)connection if it isn't run in a test enviroment.
* Did a minimum amount of Formatting to the discription
* Used `npm audit fix --force` to fix all vulnerabilties in our used packages.
* Fixed curious doublechecking for empty Strings for the sortby-Parameter.
- Now we only check once for a empty sortby
- And added a test which distinguish between `sortby=""` and `sortby="+"`
* Update api/routes/collections.js
Removed the TODO about switching from mock-data to the real db
* Removed globalTeardown as i brought up some problems corresponding to long db-queries (for example BBOX). Instead i increased the maximal testTimeout.
* added validator for collections{id} and correctly implemented collections{id}
* added test for collections{id}
* removed unnecessary parameter
* added id parameter to the Query (temporary fix)
* test-fixes to match our current tests and a fix to the baseURL for collection{id}
* test fix
* fixed problem with tests in api.test.js and adjusted the "invalid-id-test" in the validator.
* Update api/routes/collections.js
- Renamed `collection.id` to `c.collection.id`
* added test for negative ids
* deleted the whole "existing links" part and build base Links
* fixed bug in validateCollectionId.js
* Refactor negative ID test
i encoded the "-1" value in the negative ID test instead of directly putting it into the path.
* Removed a german comment in `api/routes/collections.js`
---------
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
---------
Co-authored-by: Sönke Hoffmann <hoffmann.sonke@gmx.de>
Co-authored-by: JonasK <156602337+BrokeJ@users.noreply.github.com>
Co-authored-by: Vincent Kühn <vkuehn@uni-muenster.de>
* Dev database: trigger function for better keyword handling (#209)
* added `.env`
* added environment for docker-compose.yml
now every connection-details are inside an `.env`. There is an `example.env` for better understanding which need to be set as connection details
* added description of how to use the `.env` and `example.env` in the `README.md`
* changed a few things
e.g. DB_PORT --> ${DB_PORT}
* now, everthing should be done.
my god, help. sorry
* layout issues fixed
* Fixed Typo/incomplete Sentence in README.md
* added `stac_id` for collections
* all IDs are now written in the newer PostgrSQL standart:
```SQL
id SERIAL PRIMARY KEY,
```
changed to
```SQL
id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY,
```
* changed `extend` to `extent`.
* Changed language used in `./api/README.md` from german to english.
I wanted to thsi anyway at some point, but this is now more like a Test-commit to see if the CI/CD Pipeline triggers...
* added triggering function for an auto-update search_vector, both for collections and catalogs.
The search_vector includes title, description and keywords
* changed the CI-Pipeline.
Now also Changes in the /db will be acceped by the Pipeline
---------
Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de>
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* Dev database: added different users (for api and crawler) (#214)
* added `.env`
* added environment for docker-compose.yml
now every connection-details are inside an `.env`. There is an `example.env` for better understanding which need to be set as connection details
* added description of how to use the `.env` and `example.env` in the `README.md`
* changed a few things
e.g. DB_PORT --> ${DB_PORT}
* now, everthing should be done.
my god, help. sorry
* layout issues fixed
* Fixed Typo/incomplete Sentence in README.md
* added `stac_id` for collections
* all IDs are now written in the newer PostgrSQL standart:
```SQL
id SERIAL PRIMARY KEY,
```
changed to
```SQL
id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY,
```
* changed `extend` to `extent`.
* Changed language used in `./api/README.md` from german to english.
I wanted to thsi anyway at some point, but this is now more like a Test-commit to see if the CI/CD Pipeline triggers...
* added triggering function for an auto-update search_vector, both for collections and catalogs.
The search_vector includes title, description and keywords
* changed the CI-Pipeline.
Now also Changes in the /db will be acceped by the Pipeline
* added different users for the api and crawler groups.
The api has read-only acces and the crawler user full acces to the database. The acces to the database is now possible by using the users `stac_api`or `stac_crawler`.
The admin user (`postgres_user`) is still available but shouldn't be used
---------
Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de>
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* Dev database: source_url and filter trigger (#220)
* added `.env`
* added environment for docker-compose.yml
now every connection-details are inside an `.env`. There is an `example.env` for better understanding which need to be set as connection details
* added description of how to use the `.env` and `example.env` in the `README.md`
* changed a few things
e.g. DB_PORT --> ${DB_PORT}
* now, everthing should be done.
my god, help. sorry
* layout issues fixed
* Fixed Typo/incomplete Sentence in README.md
* added `stac_id` for collections
* all IDs are now written in the newer PostgrSQL standart:
```SQL
id SERIAL PRIMARY KEY,
```
changed to
```SQL
id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY,
```
* changed `extend` to `extent`.
* Changed language used in `./api/README.md` from german to english.
I wanted to thsi anyway at some point, but this is now more like a Test-commit to see if the CI/CD Pipeline triggers...
* added triggering function for an auto-update search_vector, both for collections and catalogs.
The search_vector includes title, description and keywords
* changed the CI-Pipeline.
Now also Changes in the /db will be acceped by the Pipeline
* added different users for the api and crawler groups.
The api has read-only acces and the crawler user full acces to the database. The acces to the database is now possible by using the users `stac_api`or `stac_crawler`.
The admin user (`postgres_user`) is still available but shouldn't be used
* Refactor(database): SQL trigger definitions for catalog and collection keywords. Moved triggers to 06_triggers.sql for better organization, as they depend on the respective tables created in earlier scripts.
* added source_url for collections and catalogs. Now the full_json doesn`t has to be used for getting the url
* resolved a Problem I had with git by hand cause I didn't found the function
---------
Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de>
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
Co-authored-by: mammutor <mammutor@gmail.com>
---------
Co-authored-by: RobinGummels <github@gummels.eu>
Co-authored-by: Jakob <vertrox78@gmail.com>
Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de>
Co-authored-by: jklaer <jklaer@uni-muenster.de>
Co-authored-by: VincentKuehn <vkuehn@uni-muenster.de>
Co-authored-by: Justin K <justinkrumbhmer@yahoo.com>
Co-authored-by: Georgios Voulgaris <gvoulgar@uni-muenster.de>
Co-authored-by: Sönke Hoffmann <hoffmann.sonke@gmx.de>
Co-authored-by: JonasK <156602337+BrokeJ@users.noreply.github.com>
* Refactor: crawler configuration and memory management
- Updated rate limiting options in `HttpCrawler` to prevent memory buildup from queue overflow.
- Introduced a new constant `CATALOG_CLEAR_BATCH_SIZE` to periodically clear the catalogs array, reducing memory usage.
- Enhanced `handleCatalog` function to accept a configuration object, allowing for dynamic max depth checks during catalog processing.
- Improved logging to provide clearer insights into catalog processing and memory management.
* Crawler added source_url to collection_summaries and changed stac_id, just write collections (#245)
* feat(api): initialize STAC Atlas API with collections, conformance, and queryables routes
- Added package.json for project dependencies and scripts.
- Implemented GET endpoint for collections.
- Created conformance endpoint to list supported conformance classes.
- Developed landing page for the API with links to collections and documentation.
- Added queryables endpoint to return queryable properties for collections. (If i'm correct this can be removed)
* Added all Remarks to the bid and finished it (#74)
* changed Texts 1,2 and 8 according to the remarks of the customer
* Did my fixes to 4. and 10.3
* added remark why we want to save every catalog
* deleted keywords for catalog
* changed everything related to the database component
* Updated 3.1, 3.3, 3,4 (References to Lastenheft/small other changes)
* Added small Graph to 3. Produktumgebung
* Update bid.md 7.3STAC-Validator
added the handling of collections that cannot be validated automatically.
* Update bid.md 7.3STAC-API-Validator
changed the way we validate the collection search extension.
* Update bid.md 9.3.2Endpunkte
small fix collection search extension.
* added Skizze for 3, and updated 6.1, 10.1
* Updated 3. Produktumgebung
* Update bid.md 7.Qualitätsanforderungen
minor fixes
* Update 6.4.2 added remark about loading feedback
* Minor change to Table in "11 Zeitplan". A collum was missing in the head, therefore the table wasn't rendering correctly..
---------
Co-authored-by: Jakob <vertrox78@gmail.com>
Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de>
Co-authored-by: jklaer <jklaer@uni-muenster.de>
Co-authored-by: VincentKuehn <vkuehn@uni-muenster.de>
Co-authored-by: mammutor <mammutor@gmail.com>
Co-authored-by: Humam <44206081+Mammutor@users.noreply.github.com>
Co-authored-by: Justin K <justinkrumbhmer@yahoo.com>
* fix(api): enhance STAC API landing page and conformance links
- Overhaul of first idea landing page
- Added some more tests for the required elements in the landingpage-Catalog
* feat(api): implement shared conformance URIs and add tests for conformance endpoint
- implemented condormance endpoint
* Implemented 2.3 and 2.4 (#111)
* Temporary mock data for testing and frontend development
* Added API middleware layer for error handling and validation
* TODOs ready? pls review
* Added API utilities for query parsing, validation, and response formatting
* Added swagger and openapi.yaml
* Update queryables.js
Refactor queryables endpoint into /collections/queryables
* Renamed the collections.js file to mocks-collections.js to better reflect its purpose and improve project clarity
* changed README "Projektstruktur"
* restart from dev-api 22.11..2025
* API: 2.3 Implement Collections List Endpoint done (added explanations as comments in the code)
* API: 2.4 Implement Single Collection Endpoint (added explanations as comments in the code)
* Update api/routes/collections.js
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* Changed some of the code with the comments on Github (i will finish it tomorrow morning)
* Implement most of the feedback and comments (need to talk about some other changes)
* Update api/routes/index.js
Changed wording from `/collections/queryables` to `/collections-queryables`
* Update api/README.md
Changed wording from `/collections/queryables` to `/collections-queryables`
* Update api/README.md
Removed missing folder
* Update api/routes/collections.js
Removed TODOs from wrong lines
* Update api/routes/collections.js
Added TODOs
* Update api/routes/queryables.js
Changed wording from `/collections/queryables` to `/collections-queryables`
* Update api/routes/queryables.js
Changed wording from `/collections/queryables` to `/collections-queryables`
---------
Co-authored-by: VincentKuehn <vkuehn@uni-muenster.de>
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* feat(api): add collection search parameters and validation middleware (#159)
* feat(api): add collection search parameters and validation middleware
* Added unit-test for validator-functions and integration-tests for `GET /collections`-Querys.
- Also minor bugfix, because the validator accepted deecimals as tokens.
* API: 3 Database Integration first version (#161)
* database connection in implementated. The parameters for the connection have to added in the .env-file.
Also there is test-file for testing and console messages
(installed `pg`)
* support for spatial queries via postgis + error handling for datatbase operations
changed language to english
* error handling
* added DATABASE_URL
There is an issue with the distance query. Changed the error handling and testing, the console messages are now way better structured
* found the Problem with the distance query. The layer are so big, that they reach over the 180° long (PostgGIS can't handel that). Now the calc is done by degree and not meters.
* The two files `test-data-retrieval.js` and `verify-schema.js` have been added.
`test-data-retrieval` (theoretical, checks against the spezification):
```
Discovers all tables and columns and validates against expected schema.
```
The second files `verify-schema.js` (practical, checks against the real data):
```
Discovers all tables and columns, validates against expected schema
```
* pooling error hanling and log imporoved.
renamed tests files to actual test-files
* standalone node tests were convertad into JEST
* write file `validateRequest.js`. Validates every incoming API request, whether the request is valid and logical.
* commented `stac_id` from the tests, it is not in both databases, so the tests for `stac_id` will always fail
Added explanation to the `.env.example`, which port is which database
* added example pattern for API - database connection.
* deleted `validateRequest` cause it's already implemented by @robinGummels
---------
Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de>
* Added environment variables and a `.env` for `docker-compose.yml` (#164)
* added `.env`
* added environment for docker-compose.yml
now every connection-details are inside an `.env`. There is an `example.env` for better understanding which need to be set as connection details
* added description of how to use the `.env` and `example.env` in the `README.md`
* changed a few things
e.g. DB_PORT --> ${DB_PORT}
* now, everthing should be done.
my god, help. sorry
* layout issues fixed
* Fixed Typo/incomplete Sentence in README.md
---------
Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de>
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* Added a CI/CD Pipeline to prevent pull-requests without functioning tests and proper linting.
* fixed errors suggested by the linter.
- Some lines used tab and spaces...
* Implemented Collection Search extension including a DB-Connection (#165)
* added SQLQuery-builder with these parameters: q,bbox,datetime,sortby,limit and token
* finalised bbox and datetime
* adapted to DB, QueryBuilder and added helperfunction runQuery
* added question-TODOs
* added bbox+datetime to the Query-Builder from Jonas
* added tests for Query-Builder from Jonas
* added tests from George
* added falsely deleted TODOs again
* fixed collumn names to match our DB and adjusted full text search to match 05_indexes.sql correctly
* Used a formatter and linter on `buildCollectionSearchQuery.js
* Did some major and minor fixes to the collection search.
- Updated `buildCollectionSearchQuery` to support pagination and improved text search with English language settings.
- Modified tests in `buildCollectionsSearchQuery.basic.test.js`, `collections-pagination.test.js`, and `collections-sort.test.js` to reflect new query behavior and validation logic.
- Enhanced sort validation in `validators.test.js` and `collectionSearchParams.js` to map API fields to database column names.
- Implemented total count retrieval for matched results in `collections.js`.
* Added a internal .env creation in the CI/CD Pipeline. It utilzes GitHub Repository Secrets to not publish any private Logins and stuff.
* Forgot that the second Job of the CI/CD pipeline runs seperatly and needs a internal .env file too.
* Enhance documentation for buildCollectionSearchQuery
Updated the documentation for:
- the buildCollectionSearchQuery function
- the fulltextsearch
* Refactor buildCollectionSearchQuery and updated SELECT part
Changed the SELECT part to match our bid and the database shema. Updated comments and for clarity. Changed full-text search to use 'simple' configuration instead of 'english'.
* Update api/routes/collections.js
small typo
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* Remove sorting TODO from collections route
Removed TODO comment about sorting based on sortby parameter.
* Explicitly return undefined for normalized in validateSortby
Update validateSortby function to explicitly return undefined for normalized when sortby is not provided.
* small fix in buildCollectionSearch.fulltext.test.js
Change plainto_tsquery language from 'english' to 'simple'
* Fix duplicate SELECT keyword in query
Remove duplicate 'SELECT' keyword in SQL query.
* Fix missing newline at end of collectionSearchParams.js
* Fixed missing bracket in collectionSearchParams.js
* Refactor validateSortby for optional parameter handling
Refactor validateSortby function to handle optional sortby parameter and improve validation logic.
* Stabilize API test pipeline by running Jest in-band with extended timeout
Run Jest in CI with --runInBand and a higher default --testTimeout to stabilize database-backed integration tests.
Multiple Jest workers were competing for the same PostgreSQL connection pool and some long-running /collections queries exceeded the default 5s timeout, causing failures in existing test suites (e.g. collectionSearch and DBconnection).
* Fixed leaking tests that blocked CI/CD-Pipeline.
- Added a global Teardown for jest and force-exited the tests to prevent leaking.
- Made a change to db_APIconnection to only log the pool-(dis)connection if it isn't run in a test enviroment.
* Did a minimum amount of Formatting to the discription
* Used `npm audit fix --force` to fix all vulnerabilties in our used packages.
* Fixed curious doublechecking for empty Strings for the sortby-Parameter.
- Now we only check once for a empty sortby
- And added a test which distinguish between `sortby=""` and `sortby="+"`
* Update api/routes/collections.js
Removed the TODO about switching from mock-data to the real db
* Removed globalTeardown as i brought up some problems corresponding to long db-queries (for example BBOX). Instead i increased the maximal testTimeout.
---------
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* Update api/.env.example
* latest database Version (#187) with `stac_id` and changed definition of `primary Keys`
* added `.env`
* added environment for docker-compose.yml
now every connection-details are inside an `.env`. There is an `example.env` for better understanding which need to be set as connection details
* added description of how to use the `.env` and `example.env` in the `README.md`
* changed a few things
e.g. DB_PORT --> ${DB_PORT}
* now, everthing should be done.
my god, help. sorry
* layout issues fixed
* Fixed Typo/incomplete Sentence in README.md
* added `stac_id` for collections
* all IDs are now written in the newer PostgrSQL standart:
```SQL
id SERIAL PRIMARY KEY,
```
changed to
```SQL
id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY,
```
* changed `extend` to `extent`.
* Changed language used in `./api/README.md` from german to english.
I wanted to thsi anyway at some point, but this is now more like a Test-commit to see if the CI/CD Pipeline triggers...
---------
Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de>
Co-authored-by: Robin Tammo Gummels <github@gummels.eu>
* bug fix: extent was written wrong
* feat: add source URL extraction for catalogs and collections
* feat: update insertOrUpdateCatalog to only process catalogs for traversal, no longer saving to database
---------
Co-authored-by: RobinGummels <github@gummels.eu>
Co-authored-by: Jakob <vertrox78@gmail.com>
Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de>
Co-authored-by: jklaer <jklaer@uni-muenster.de>
Co-authored-by: VincentKuehn <vkuehn@uni-muenster.de>
Co-authored-by: mammutor <mammutor@gmail.com>
Co-authored-by: Humam <44206081+Mammutor@users.noreply.github.com>
Co-authored-by: Justin K <justinkrumbhmer@yahoo.com>
Co-authored-by: Georgios Voulgaris <gvoulgar@uni-muenster.de>
Co-authored-by: Sönke Hoffmann <hoffmann.sonke@gmx.de>
* Enhance memory management in API and handlers
- Introduced periodic clearing of the `apis` array in `checkAndFlushApi` to free memory, with a defined batch size.
- Updated `checkAndFlush` in `handlers.js` to ensure the `catalogs` array is cleared only if it exists, preventing potential errors.
- Improved logging for memory management actions to provide better insights during API operations.
* Refactor database operations in insertOrUpdateCollection and insertSummary functions
- Removed the source_url parameter from the insertOrUpdateCollection function, simplifying the SQL query.
- Updated the insertSummary function to include source_url in the database insert statement.
- Adjusted related calls to insertSummary to pass the new source_url parameter.
- Enhanced the handling of collection summaries to ensure proper data insertion.
* Delete .env.example in root
we have to decide later what exactly to do
* fix(crawler-db): Enhance collection extent handling in insertOrUpdateCollection function
- Updated the function to support both normalized (bbox) and original STAC format (extent.spatial.bbox) for spatial extent.
- Improved temporal extent parsing to accommodate both normalized (temporal) and original STAC format (extent.temporal.interval).
- Refactored code for better clarity and maintainability.
* fix(crawler-api): update STAC object creation to disable URL migration
- Modified the `create` function calls in `handleApiRoot`, `handleApiCollection`, and `handleCatalog` to set the second parameter to `false`, preventing URL migration.
- Added comments to clarify the change and its implications for STAC compliance validation.
* fix(crawler-api): add defensive JSON validation in API and catalog handlers
- Implemented checks in `handleApiRoot` and `handleCatalog` to ensure the JSON response is valid before processing.
- Added logging for invalid JSON responses to improve error tracking and handling.
- Updated comments to clarify the purpose of disabling URL migration in STAC object creation.
* chore(crawler): update Node.js version in Dockerfile to 20-alpine
* fix(crawler): enhance JSON parsing in API and catalog request handlers
- Updated request handlers in `crawlApis` and `crawlCatalogs` to include a fallback mechanism for manually parsing JSON responses when automatic parsing fails, because some servers like the DLR usese 14 seconds.
- Added logging for successful and failed manual parsing attempts to improve debugging and error tracking.
- Adjusted comments to clarify the purpose of the new parsing logic.
* fix(crawler): add S3 URL handling and relative URL conversion in API and handlers
- Implemented conversion of S3 protocol URLs to HTTPS format in `handleApiRoot`, `tryCollectionEndpoints`, and `handleCatalog` functions.
- Added logging for successful conversions and warnings for malformed S3 URLs.
- Enhanced handling of relative URLs to ensure they are converted to absolute URLs based on the request context.
- Improved validation checks for URLs to skip invalid entries with appropriate logging.
* fix(crawler): update Dockerfile to omit development dependencies during npm install
- Changed the npm install command to use the --omit=dev flag, ensuring that only production dependencies are installed in the Docker image.
* feat(crawler): implement parallel crawling for APIs and catalogs
- Added support for parallel crawling of multiple domains in both API and catalog crawlers.
- Introduced new configuration options for parallel domains, max requests per minute per domain, and max concurrency per domain.
- Enhanced logging to provide detailed statistics on parallel crawling performance and domain distribution.
- Refactored existing crawling functions to accommodate the new parallel execution model, improving overall efficiency and throughput.
- Updated CLI arguments and configuration to support the new parallel crawling features.
* fix(crawler): enhance normalization of collection metadata extraction
- Improved the `normalizeCollection` function to utilize raw data from stac-js objects for more robust metadata extraction.
- Added fallback mechanisms for bounding box, temporal extent, self URL, and other properties to ensure reliable data retrieval from both stac-js methods and raw data.
- Updated the handling of collection properties to prioritize raw data when available, enhancing overall data integrity.
* fix(crawler): enhance normalization of collection to include additional fields for database insertion
- Updated the `normalizeCollection` function to extract and preserve additional fields such as links, summaries, and extensions from collection objects.
- Ensured compatibility with both stac-js and raw data formats for comprehensive metadata extraction.
- Improved overall data integrity by including all necessary fields for database insertion.
* fix(crawler): increase max concurrency for improved throughput in API and catalog crawlers
- Updated the maxConcurrencyPerDomain setting from 10 to 20 to enhance performance and avoid bottlenecks during crawling.
- Adjusted logging to provide detailed information on the number of APIs and catalogs being crawled, including concurrency and rate limits.
- Modified rate limiting calculations to ensure minimal delay between requests, allowing for better handling of slow responses.
* fix(crawler): optimize concurrency settings for API and catalog crawlers
- Introduced configurable concurrency settings to enhance crawling performance, allowing for immediate scaling and minimal delays between requests.
- Updated logging to reflect the new concurrency model and removed unnecessary delay parameters for improved throughput.
- Adjusted rate limiting calculations to rely solely on maxReques…
RobinGummels
pushed a commit
that referenced
this pull request
Feb 3, 2026
* feat: initialize Vue 3 project with Vite and packages * feat: Added project folder structure * feat, styling: css vars, base, reset * feat: Add README, project structure, and styling guide documentation * UI component structure (#182) * feat: initialize Vue 3 project with Vite and packages * feat: Added project folder structure * feat, styling: css vars, base, reset * feat: Add README, project structure, and styling guide documentation * feat: lucide icons * feat: removed the default HelloWorld component * feat: filter, navbar, search result component and search section * feat: box-shadow for the section divider style: cleaned up some artifacts from the logo * feat: changed title and package name to STAC Atlas * chore: removed vite.svg * chore: removed vue.svg * feat: Implement InfoCard, ItemCard and SearchResultCard components (#194) Cards for the display of the Search Result and also for the collection/ catalog page * chore: removed Helloworld * feat: collection page, mockup data for visual feedback chore: styling corrected, adjusted based on the mockup * style: removed old css names * chore: removed the mockup data * UI | Mockup recreated: Collections page, little adjuments to style base etc (#198) * chore: removed Helloworld * feat: collection page, mockup data for visual feedback chore: styling corrected, adjusted based on the mockup * style: removed old css names * chore: removed the mockup data * feat: update button styles and add contact popover in CollectionDetail view * style: fixed witdh of the apply filter button * style: map button, apply button behave like the others now * UI | Style fixes, Copy element for contact (#199) * chore: removed Helloworld * feat: collection page, mockup data for visual feedback chore: styling corrected, adjusted based on the mockup * style: removed old css names * chore: removed the mockup data * feat: update button styles and add contact popover in CollectionDetail view * style: fixed witdh of the apply filter button * style: map button, apply button behave like the others now * chore: removed mockup data * feat: route gitignore * feat: showing the collections from /collections in the result list * feat: enhance filter section layout and improve search result card tag display * style: refine search result card layout and improve text overflow handling * feat: implement pagination for collections in Home.vue and adjust max-height in search results * style: remove fixed height from card header and add styles for first button in card footer * style: enhance active pagination button styles for better visibility * feat: implement loading and error states in CollectionDetail.vue, enhance API integration for fetching collection data * style: width problem collection page, scolll items * chore: switched to the apis collections/:id in the ui * style: calender icon response to dark & light mode * chroe: removed svg for language button style: slightly thicker border for nav-buttons * style: added a slight highlight to input/ interaction buttons * style: update dark mode colors for text and adjust tag background * style: smaller vertical padding for select options * feat: source and contact are shown when clicked * feat: search functionality * style: centered the paging icon chore: view css folder to seperate style and content * feat: removed the unnecessary Filter Header * feat: add bounding box drawing functionality and integrate with filter section * API integrated in the UI (#206) * feat(api): initialize STAC Atlas API with collections, conformance, and queryables routes - Added package.json for project dependencies and scripts. - Implemented GET endpoint for collections. - Created conformance endpoint to list supported conformance classes. - Developed landing page for the API with links to collections and documentation. - Added queryables endpoint to return queryable properties for collections. (If i'm correct this can be removed) * Added all Remarks to the bid and finished it (#74) * changed Texts 1,2 and 8 according to the remarks of the customer * Did my fixes to 4. and 10.3 * added remark why we want to save every catalog * deleted keywords for catalog * changed everything related to the database component * Updated 3.1, 3.3, 3,4 (References to Lastenheft/small other changes) * Added small Graph to 3. Produktumgebung * Update bid.md 7.3STAC-Validator added the handling of collections that cannot be validated automatically. * Update bid.md 7.3STAC-API-Validator changed the way we validate the collection search extension. * Update bid.md 9.3.2Endpunkte small fix collection search extension. * added Skizze for 3, and updated 6.1, 10.1 * Updated 3. Produktumgebung * Update bid.md 7.Qualitätsanforderungen minor fixes * Update 6.4.2 added remark about loading feedback * Minor change to Table in "11 Zeitplan". A collum was missing in the head, therefore the table wasn't rendering correctly.. * fix(api): enhance STAC API landing page and conformance links - Overhaul of first idea landing page - Added some more tests for the required elements in the landingpage-Catalog * feat(api): implement shared conformance URIs and add tests for conformance endpoint - implemented condormance endpoint * Implemented 2.3 and 2.4 (#111) * Temporary mock data for testing and frontend development * Added API middleware layer for error handling and validation * TODOs ready? pls review * Added API utilities for query parsing, validation, and response formatting * Added swagger and openapi.yaml * Update queryables.js Refactor queryables endpoint into /collections/queryables * Renamed the collections.js file to mocks-collections.js to better reflect its purpose and improve project clarity * changed README "Projektstruktur" * restart from dev-api 22.11..2025 * API: 2.3 Implement Collections List Endpoint done (added explanations as comments in the code) * API: 2.4 Implement Single Collection Endpoint (added explanations as comments in the code) * Update api/routes/collections.js * Changed some of the code with the comments on Github (i will finish it tomorrow morning) * Implement most of the feedback and comments (need to talk about some other changes) * Update api/routes/index.js Changed wording from `/collections/queryables` to `/collections-queryables` * Update api/README.md Changed wording from `/collections/queryables` to `/collections-queryables` * Update api/README.md Removed missing folder * Update api/routes/collections.js Removed TODOs from wrong lines * Update api/routes/collections.js Added TODOs * Update api/routes/queryables.js Changed wording from `/collections/queryables` to `/collections-queryables` * Update api/routes/queryables.js Changed wording from `/collections/queryables` to `/collections-queryables` * feat(api): add collection search parameters and validation middleware (#159) * feat(api): add collection search parameters and validation middleware * Added unit-test for validator-functions and integration-tests for `GET /collections`-Querys. - Also minor bugfix, because the validator accepted deecimals as tokens. * API: 3 Database Integration first version (#161) * database connection in implementated. The parameters for the connection have to added in the .env-file. Also there is test-file for testing and console messages (installed `pg`) * support for spatial queries via postgis + error handling for datatbase operations changed language to english * error handling * added DATABASE_URL There is an issue with the distance query. Changed the error handling and testing, the console messages are now way better structured * found the Problem with the distance query. The layer are so big, that they reach over the 180° long (PostgGIS can't handel that). Now the calc is done by degree and not meters. * The two files `test-data-retrieval.js` and `verify-schema.js` have been added. `test-data-retrieval` (theoretical, checks against the spezification): ``` Discovers all tables and columns and validates against expected schema. ``` The second files `verify-schema.js` (practical, checks against the real data): ``` Discovers all tables and columns, validates against expected schema ``` * pooling error hanling and log imporoved. renamed tests files to actual test-files * standalone node tests were convertad into JEST * write file `validateRequest.js`. Validates every incoming API request, whether the request is valid and logical. * commented `stac_id` from the tests, it is not in both databases, so the tests for `stac_id` will always fail Added explanation to the `.env.example`, which port is which database * added example pattern for API - database connection. * deleted `validateRequest` cause it's already implemented by @RobinGummels * Added a CI/CD Pipeline to prevent pull-requests without functioning tests and proper linting. * fixed errors suggested by the linter. - Some lines used tab and spaces... * Implemented Collection Search extension including a DB-Connection (#165) * added SQLQuery-builder with these parameters: q,bbox,datetime,sortby,limit and token * finalised bbox and datetime * adapted to DB, QueryBuilder and added helperfunction runQuery * added question-TODOs * added bbox+datetime to the Query-Builder from Jonas * added tests for Query-Builder from Jonas * added tests from George * added falsely deleted TODOs again * fixed collumn names to match our DB and adjusted full text search to match 05_indexes.sql correctly * Used a formatter and linter on `buildCollectionSearchQuery.js * Did some major and minor fixes to the collection search. - Updated `buildCollectionSearchQuery` to support pagination and improved text search with English language settings. - Modified tests in `buildCollectionsSearchQuery.basic.test.js`, `collections-pagination.test.js`, and `collections-sort.test.js` to reflect new query behavior and validation logic. - Enhanced sort validation in `validators.test.js` and `collectionSearchParams.js` to map API fields to database column names. - Implemented total count retrieval for matched results in `collections.js`. * Added a internal .env creation in the CI/CD Pipeline. It utilzes GitHub Repository Secrets to not publish any private Logins and stuff. * Forgot that the second Job of the CI/CD pipeline runs seperatly and needs a internal .env file too. * Enhance documentation for buildCollectionSearchQuery Updated the documentation for: - the buildCollectionSearchQuery function - the fulltextsearch * Refactor buildCollectionSearchQuery and updated SELECT part Changed the SELECT part to match our bid and the database shema. Updated comments and for clarity. Changed full-text search to use 'simple' configuration instead of 'english'. * Update api/routes/collections.js small typo * Remove sorting TODO from collections route Removed TODO comment about sorting based on sortby parameter. * Explicitly return undefined for normalized in validateSortby Update validateSortby function to explicitly return undefined for normalized when sortby is not provided. * small fix in buildCollectionSearch.fulltext.test.js Change plainto_tsquery language from 'english' to 'simple' * Fix duplicate SELECT keyword in query Remove duplicate 'SELECT' keyword in SQL query. * Fix missing newline at end of collectionSearchParams.js * Fixed missing bracket in collectionSearchParams.js * Refactor validateSortby for optional parameter handling Refactor validateSortby function to handle optional sortby parameter and improve validation logic. * Stabilize API test pipeline by running Jest in-band with extended timeout Run Jest in CI with --runInBand and a higher default --testTimeout to stabilize database-backed integration tests. Multiple Jest workers were competing for the same PostgreSQL connection pool and some long-running /collections queries exceeded the default 5s timeout, causing failures in existing test suites (e.g. collectionSearch and DBconnection). * Fixed leaking tests that blocked CI/CD-Pipeline. - Added a global Teardown for jest and force-exited the tests to prevent leaking. - Made a change to db_APIconnection to only log the pool-(dis)connection if it isn't run in a test enviroment. * Did a minimum amount of Formatting to the discription * Used `npm audit fix --force` to fix all vulnerabilties in our used packages. * Fixed curious doublechecking for empty Strings for the sortby-Parameter. - Now we only check once for a empty sortby - And added a test which distinguish between `sortby=""` and `sortby="+"` * Update api/routes/collections.js Removed the TODO about switching from mock-data to the real db * Removed globalTeardown as i brought up some problems corresponding to long db-queries (for example BBOX). Instead i increased the maximal testTimeout. * Update api/.env.example * latest database Version (#187) with `stac_id` and changed definition of `primary Keys` * added `.env` * added environment for docker-compose.yml now every connection-details are inside an `.env`. There is an `example.env` for better understanding which need to be set as connection details * added description of how to use the `.env` and `example.env` in the `README.md` * changed a few things e.g. DB_PORT --> ${DB_PORT} * now, everthing should be done. my god, help. sorry * layout issues fixed * Fixed Typo/incomplete Sentence in README.md * added `stac_id` for collections * all IDs are now written in the newer PostgrSQL standart: ```SQL id SERIAL PRIMARY KEY, ``` changed to ```SQL id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY, ``` * changed `extend` to `extent`. * Changed language used in `./api/README.md` from german to english. I wanted to thsi anyway at some point, but this is now more like a Test-commit to see if the CI/CD Pipeline triggers... * chore: removed Helloworld * feat: collection page, mockup data for visual feedback chore: styling corrected, adjusted based on the mockup * style: removed old css names * chore: removed the mockup data * feat: update button styles and add contact popover in CollectionDetail view * style: fixed witdh of the apply filter button * style: map button, apply button behave like the others now * chore: removed mockup data * feat: route gitignore * feat: showing the collections from /collections in the result list * feat: enhance filter section layout and improve search result card tag display * style: refine search result card layout and improve text overflow handling * feat: implement pagination for collections in Home.vue and adjust max-height in search results * style: remove fixed height from card header and add styles for first button in card footer * style: enhance active pagination button styles for better visibility * feat: implement loading and error states in CollectionDetail.vue, enhance API integration for fetching collection data * style: width problem collection page, scolll items * Added `GET /collections/{id}`-Endpoint, more Fields in the responses and more Queryables-Parameters (#204) * Updated Query-Builder to get all necessary fields from all db_tables for collections. Added some tests and fixed some already existing tests, becuase now the tablenames start with the alias `c.`. * Updated Query-Builder to get all necessary fields from all db_tables for collections. Added some tests and fixed some already existing tests, becuase now the tablenames start with the alias `c.`. * Added `openapi.yaml` (now http://localhost:3000/api-docs/ is working). - needed to do some modifying to the app.js * Added discription on how to use `stac-api-validator`. Currently we are onyl valid to `core`. * Changed API-Version name to 1.1.0 instead of 1.0.0 * Revert "API is now responding with all necessary fields for each collection" (#195) Reverts #185 @SonkeHoffmann accidentally didn't squash correctly. * Revert "Revert "API is now responding with all necessary fields for each collection"" (#185) (#195) (#196) dev-api: prepare v1.1.0 + API docs + query builder fixes - Change API version to 1.1.0 - Add OpenAPI spec so /api-docs works locally - Document stac-api-validator usage - Update api/.env.example - Query builder: select required fields for collections across db_tables; adjust tests (alias `c.`) Commits included: - 34bf962 Changed API-Version name to 1.1.0 instead of 1.0.0 - b047389 Added description on how to use `stac-api-validator` (currently only valid for `core`) - b811288 Added `openapi.yaml` (so http://localhost:3000/api-docs/ works); modified app.js accordingly - 6e5ab3e Merge branch 'dev-api-robin' of github.com:SpatioCore/STAC-Atlas into dev-api-robin - 70dc043 Updated Query-Builder to get all necessary fields from all db_tables for collections. Added tests and fixed existing tests (table names now start with alias `c.`) - d83eeb4 Update api/.env.example - 5a7af5b Updated Query-Builder to get all necessary fields from all db_tables for collections. Added tests and fixed existing tests (table names now start with alias `c.`) * Implement more Queryable-Fields and add the keywords-field to q fulltext search (#200) * Add provider and license filters to collection search API - Updated buildCollectionSearchQuery to include provider and license parameters for filtering collections. - Enhanced validateCollectionSearchParams middleware to validate provider and license query parameters. - Modified collections route to handle new provider and license filters in search queries. - Implemented validation functions for provider and license parameters in collectionSearchParams. * Add validation tests for provider and license * Enhance full-text search by including keywords in the tsvector expression and update related tests * Add provider and license to query parameter extraction in collection search validation * Revert "Enhance full-text search by including keywords in the tsvector expression and update related tests" This reverts commit 872443d. * Implement GET /collections/{id} endpoint with validation and QueryBuilder integration (#186) * added SQLQuery-builder with these parameters: q,bbox,datetime,sortby,limit and token * finalised bbox and datetime * adapted to DB, QueryBuilder and added helperfunction runQuery * added question-TODOs * added bbox+datetime to the Query-Builder from Jonas * added tests for Query-Builder from Jonas * added tests from George * added falsely deleted TODOs again * fixed collumn names to match our DB and adjusted full text search to match 05_indexes.sql correctly * Used a formatter and linter on `buildCollectionSearchQuery.js * Did some major and minor fixes to the collection search. - Updated `buildCollectionSearchQuery` to support pagination and improved text search with English language settings. - Modified tests in `buildCollectionsSearchQuery.basic.test.js`, `collections-pagination.test.js`, and `collections-sort.test.js` to reflect new query behavior and validation logic. - Enhanced sort validation in `validators.test.js` and `collectionSearchParams.js` to map API fields to database column names. - Implemented total count retrieval for matched results in `collections.js`. * Added a internal .env creation in the CI/CD Pipeline. It utilzes GitHub Repository Secrets to not publish any private Logins and stuff. * Forgot that the second Job of the CI/CD pipeline runs seperatly and needs a internal .env file too. * Enhance documentation for buildCollectionSearchQuery Updated the documentation for: - the buildCollectionSearchQuery function - the fulltextsearch * Refactor buildCollectionSearchQuery and updated SELECT part Changed the SELECT part to match our bid and the database shema. Updated comments and for clarity. Changed full-text search to use 'simple' configuration instead of 'english'. * Update api/routes/collections.js small typo * Remove sorting TODO from collections route Removed TODO comment about sorting based on sortby parameter. * Explicitly return undefined for normalized in validateSortby Update validateSortby function to explicitly return undefined for normalized when sortby is not provided. * small fix in buildCollectionSearch.fulltext.test.js Change plainto_tsquery language from 'english' to 'simple' * Fix duplicate SELECT keyword in query Remove duplicate 'SELECT' keyword in SQL query. * Fix missing newline at end of collectionSearchParams.js * Fixed missing bracket in collectionSearchParams.js * Refactor validateSortby for optional parameter handling Refactor validateSortby function to handle optional sortby parameter and improve validation logic. * Stabilize API test pipeline by running Jest in-band with extended timeout Run Jest in CI with --runInBand and a higher default --testTimeout to stabilize database-backed integration tests. Multiple Jest workers were competing for the same PostgreSQL connection pool and some long-running /collections queries exceeded the default 5s timeout, causing failures in existing test suites (e.g. collectionSearch and DBconnection). * Fixed leaking tests that blocked CI/CD-Pipeline. - Added a global Teardown for jest and force-exited the tests to prevent leaking. - Made a change to db_APIconnection to only log the pool-(dis)connection if it isn't run in a test enviroment. * Did a minimum amount of Formatting to the discription * Used `npm audit fix --force` to fix all vulnerabilties in our used packages. * Fixed curious doublechecking for empty Strings for the sortby-Parameter. - Now we only check once for a empty sortby - And added a test which distinguish between `sortby=""` and `sortby="+"` * Update api/routes/collections.js Removed the TODO about switching from mock-data to the real db * Removed globalTeardown as i brought up some problems corresponding to long db-queries (for example BBOX). Instead i increased the maximal testTimeout. * added validator for collections{id} and correctly implemented collections{id} * added test for collections{id} * removed unnecessary parameter * added id parameter to the Query (temporary fix) * test-fixes to match our current tests and a fix to the baseURL for collection{id} * test fix * fixed problem with tests in api.test.js and adjusted the "invalid-id-test" in the validator. * Update api/routes/collections.js - Renamed `collection.id` to `c.collection.id` * added test for negative ids * deleted the whole "existing links" part and build base Links * fixed bug in validateCollectionId.js * Refactor negative ID test i encoded the "-1" value in the negative ID test instead of directly putting it into the path. * Removed a german comment in `api/routes/collections.js` * chore: switched to the apis collections/:id in the ui * style: calender icon response to dark & light mode * chroe: removed svg for language button style: slightly thicker border for nav-buttons * style: added a slight highlight to input/ interaction buttons * style: update dark mode colors for text and adjust tag background * style: smaller vertical padding for select options * feat: source and contact are shown when clicked * feat: search functionality * style: centered the paging icon chore: view css folder to seperate style and content * feat: removed the unnecessary Filter Header * feat: add bounding box drawing functionality and integrate with filter section * style: bigger metadata section, hover state for items feat: item thumbnail * feat: routing the home when clicking on the logo and app title * feat: enhance button states and update provider button in CollectionDetail * style: scrollbar styling for dark mode * feat: docker * feat: enhance collection API integration and UI components with queryables support * feat: docker dev container hot reload, removed "Page" text in front of pagination-info * feat: add .vite to .gitignore * fix: update scrollbar track background to transparent * feat: add feedback for copy to clipboard action with visual indication * fix: change justify-content to start for better alignment in collection detail layout * fix: update favicon * feat: items in collection page * feat: add CQL2 filter support in filter section and API integration * chore: markdown lint, removed vue.svg * refactor: remove development Dockerfile and associated docker-compose configuration * feat: add active and API status filters to FilterSection and update filter store * style: update spacing in filter section and set height for right section in collection detail * fix: docker start / build * fix: add missing closing div in FilterSection and clean up imports in CollectionDetail * Refactor code structure for improved readability and maintainability * style: bigger search cards * feat: licences & providers queryables * fix: update API status labels for clarity * style: gap in metadata-item for seperation * fix: update active filter default value and placeholder for clarity * feat: enhance pagination with input jump and update items per page * feat: enhance loading and no-results UI with spinner and messages * style: add ellipsis for overflowing text in source and provider URLs * style: comment out information button in navbar for future implementation * feat: i18n support across components and views, documentation * feat: enhance pagination with input jump, update items per page, and improve item count display * docs: update README with comprehensive structure, deployment instructions, and environment variable details * feat: add loading indicator for items count and additional properties section in CollectionDetail * feat: implement collapsible additional properties section in CollectionDetail * feat: update German translations for consistency in Collection terminology * feat: add selfUrl property to ItemCard and CollectionDetail for external link support * feat: enhance item fetching to set selfUrl based on resolved URLs * fix: update German translations for consistency in terminology * fix: remove max-width constraint from search result card styles * feat: enhance source link handling to prioritize source_root and open in STAC Browser --------- Co-authored-by: Simon Benjamin Imfeld <simfeld@uni-muenster.de>
Mammutor
added a commit
that referenced
this pull request
Jun 30, 2026
* feat: initialize Vue 3 project with Vite and packages * feat: Added project folder structure * feat, styling: css vars, base, reset * feat: Add README, project structure, and styling guide documentation * UI component structure (#182) * feat: initialize Vue 3 project with Vite and packages * feat: Added project folder structure * feat, styling: css vars, base, reset * feat: Add README, project structure, and styling guide documentation * feat: lucide icons * feat: removed the default HelloWorld component * feat: filter, navbar, search result component and search section * feat: box-shadow for the section divider style: cleaned up some artifacts from the logo * feat: changed title and package name to STAC Atlas * chore: removed vite.svg * chore: removed vue.svg * feat: Implement InfoCard, ItemCard and SearchResultCard components (#194) Cards for the display of the Search Result and also for the collection/ catalog page * chore: removed Helloworld * feat: collection page, mockup data for visual feedback chore: styling corrected, adjusted based on the mockup * style: removed old css names * chore: removed the mockup data * UI | Mockup recreated: Collections page, little adjuments to style base etc (#198) * chore: removed Helloworld * feat: collection page, mockup data for visual feedback chore: styling corrected, adjusted based on the mockup * style: removed old css names * chore: removed the mockup data * feat: update button styles and add contact popover in CollectionDetail view * style: fixed witdh of the apply filter button * style: map button, apply button behave like the others now * UI | Style fixes, Copy element for contact (#199) * chore: removed Helloworld * feat: collection page, mockup data for visual feedback chore: styling corrected, adjusted based on the mockup * style: removed old css names * chore: removed the mockup data * feat: update button styles and add contact popover in CollectionDetail view * style: fixed witdh of the apply filter button * style: map button, apply button behave like the others now * chore: removed mockup data * feat: route gitignore * feat: showing the collections from /collections in the result list * feat: enhance filter section layout and improve search result card tag display * style: refine search result card layout and improve text overflow handling * feat: implement pagination for collections in Home.vue and adjust max-height in search results * style: remove fixed height from card header and add styles for first button in card footer * style: enhance active pagination button styles for better visibility * feat: implement loading and error states in CollectionDetail.vue, enhance API integration for fetching collection data * style: width problem collection page, scolll items * chore: switched to the apis collections/:id in the ui * style: calender icon response to dark & light mode * chroe: removed svg for language button style: slightly thicker border for nav-buttons * style: added a slight highlight to input/ interaction buttons * style: update dark mode colors for text and adjust tag background * style: smaller vertical padding for select options * feat: source and contact are shown when clicked * feat: search functionality * style: centered the paging icon chore: view css folder to seperate style and content * feat: removed the unnecessary Filter Header * feat: add bounding box drawing functionality and integrate with filter section * API integrated in the UI (#206) * feat(api): initialize STAC Atlas API with collections, conformance, and queryables routes - Added package.json for project dependencies and scripts. - Implemented GET endpoint for collections. - Created conformance endpoint to list supported conformance classes. - Developed landing page for the API with links to collections and documentation. - Added queryables endpoint to return queryable properties for collections. (If i'm correct this can be removed) * Added all Remarks to the bid and finished it (#74) * changed Texts 1,2 and 8 according to the remarks of the customer * Did my fixes to 4. and 10.3 * added remark why we want to save every catalog * deleted keywords for catalog * changed everything related to the database component * Updated 3.1, 3.3, 3,4 (References to Lastenheft/small other changes) * Added small Graph to 3. Produktumgebung * Update bid.md 7.3STAC-Validator added the handling of collections that cannot be validated automatically. * Update bid.md 7.3STAC-API-Validator changed the way we validate the collection search extension. * Update bid.md 9.3.2Endpunkte small fix collection search extension. * added Skizze for 3, and updated 6.1, 10.1 * Updated 3. Produktumgebung * Update bid.md 7.Qualitätsanforderungen minor fixes * Update 6.4.2 added remark about loading feedback * Minor change to Table in "11 Zeitplan". A collum was missing in the head, therefore the table wasn't rendering correctly.. --------- Co-authored-by: Jakob <vertrox78@gmail.com> Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de> Co-authored-by: jklaer <jklaer@uni-muenster.de> Co-authored-by: VincentKuehn <vkuehn@uni-muenster.de> Co-authored-by: mammutor <mammutor@gmail.com> Co-authored-by: Humam <44206081+Mammutor@users.noreply.github.com> Co-authored-by: Justin K <justinkrumbhmer@yahoo.com> * fix(api): enhance STAC API landing page and conformance links - Overhaul of first idea landing page - Added some more tests for the required elements in the landingpage-Catalog * feat(api): implement shared conformance URIs and add tests for conformance endpoint - implemented condormance endpoint * Implemented 2.3 and 2.4 (#111) * Temporary mock data for testing and frontend development * Added API middleware layer for error handling and validation * TODOs ready? pls review * Added API utilities for query parsing, validation, and response formatting * Added swagger and openapi.yaml * Update queryables.js Refactor queryables endpoint into /collections/queryables * Renamed the collections.js file to mocks-collections.js to better reflect its purpose and improve project clarity * changed README "Projektstruktur" * restart from dev-api 22.11..2025 * API: 2.3 Implement Collections List Endpoint done (added explanations as comments in the code) * API: 2.4 Implement Single Collection Endpoint (added explanations as comments in the code) * Update api/routes/collections.js Co-authored-by: Robin Tammo Gummels <github@gummels.eu> * Changed some of the code with the comments on Github (i will finish it tomorrow morning) * Implement most of the feedback and comments (need to talk about some other changes) * Update api/routes/index.js Changed wording from `/collections/queryables` to `/collections-queryables` * Update api/README.md Changed wording from `/collections/queryables` to `/collections-queryables` * Update api/README.md Removed missing folder * Update api/routes/collections.js Removed TODOs from wrong lines * Update api/routes/collections.js Added TODOs * Update api/routes/queryables.js Changed wording from `/collections/queryables` to `/collections-queryables` * Update api/routes/queryables.js Changed wording from `/collections/queryables` to `/collections-queryables` --------- Co-authored-by: VincentKuehn <vkuehn@uni-muenster.de> Co-authored-by: Robin Tammo Gummels <github@gummels.eu> * feat(api): add collection search parameters and validation middleware (#159) * feat(api): add collection search parameters and validation middleware * Added unit-test for validator-functions and integration-tests for `GET /collections`-Querys. - Also minor bugfix, because the validator accepted deecimals as tokens. * API: 3 Database Integration first version (#161) * database connection in implementated. The parameters for the connection have to added in the .env-file. Also there is test-file for testing and console messages (installed `pg`) * support for spatial queries via postgis + error handling for datatbase operations changed language to english * error handling * added DATABASE_URL There is an issue with the distance query. Changed the error handling and testing, the console messages are now way better structured * found the Problem with the distance query. The layer are so big, that they reach over the 180° long (PostgGIS can't handel that). Now the calc is done by degree and not meters. * The two files `test-data-retrieval.js` and `verify-schema.js` have been added. `test-data-retrieval` (theoretical, checks against the spezification): ``` Discovers all tables and columns and validates against expected schema. ``` The second files `verify-schema.js` (practical, checks against the real data): ``` Discovers all tables and columns, validates against expected schema ``` * pooling error hanling and log imporoved. renamed tests files to actual test-files * standalone node tests were convertad into JEST * write file `validateRequest.js`. Validates every incoming API request, whether the request is valid and logical. * commented `stac_id` from the tests, it is not in both databases, so the tests for `stac_id` will always fail Added explanation to the `.env.example`, which port is which database * added example pattern for API - database connection. * deleted `validateRequest` cause it's already implemented by @RobinGummels --------- Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de> * Added a CI/CD Pipeline to prevent pull-requests without functioning tests and proper linting. * fixed errors suggested by the linter. - Some lines used tab and spaces... * Implemented Collection Search extension including a DB-Connection (#165) * added SQLQuery-builder with these parameters: q,bbox,datetime,sortby,limit and token * finalised bbox and datetime * adapted to DB, QueryBuilder and added helperfunction runQuery * added question-TODOs * added bbox+datetime to the Query-Builder from Jonas * added tests for Query-Builder from Jonas * added tests from George * added falsely deleted TODOs again * fixed collumn names to match our DB and adjusted full text search to match 05_indexes.sql correctly * Used a formatter and linter on `buildCollectionSearchQuery.js * Did some major and minor fixes to the collection search. - Updated `buildCollectionSearchQuery` to support pagination and improved text search with English language settings. - Modified tests in `buildCollectionsSearchQuery.basic.test.js`, `collections-pagination.test.js`, and `collections-sort.test.js` to reflect new query behavior and validation logic. - Enhanced sort validation in `validators.test.js` and `collectionSearchParams.js` to map API fields to database column names. - Implemented total count retrieval for matched results in `collections.js`. * Added a internal .env creation in the CI/CD Pipeline. It utilzes GitHub Repository Secrets to not publish any private Logins and stuff. * Forgot that the second Job of the CI/CD pipeline runs seperatly and needs a internal .env file too. * Enhance documentation for buildCollectionSearchQuery Updated the documentation for: - the buildCollectionSearchQuery function - the fulltextsearch * Refactor buildCollectionSearchQuery and updated SELECT part Changed the SELECT part to match our bid and the database shema. Updated comments and for clarity. Changed full-text search to use 'simple' configuration instead of 'english'. * Update api/routes/collections.js small typo Co-authored-by: Robin Tammo Gummels <github@gummels.eu> * Remove sorting TODO from collections route Removed TODO comment about sorting based on sortby parameter. * Explicitly return undefined for normalized in validateSortby Update validateSortby function to explicitly return undefined for normalized when sortby is not provided. * small fix in buildCollectionSearch.fulltext.test.js Change plainto_tsquery language from 'english' to 'simple' * Fix duplicate SELECT keyword in query Remove duplicate 'SELECT' keyword in SQL query. * Fix missing newline at end of collectionSearchParams.js * Fixed missing bracket in collectionSearchParams.js * Refactor validateSortby for optional parameter handling Refactor validateSortby function to handle optional sortby parameter and improve validation logic. * Stabilize API test pipeline by running Jest in-band with extended timeout Run Jest in CI with --runInBand and a higher default --testTimeout to stabilize database-backed integration tests. Multiple Jest workers were competing for the same PostgreSQL connection pool and some long-running /collections queries exceeded the default 5s timeout, causing failures in existing test suites (e.g. collectionSearch and DBconnection). * Fixed leaking tests that blocked CI/CD-Pipeline. - Added a global Teardown for jest and force-exited the tests to prevent leaking. - Made a change to db_APIconnection to only log the pool-(dis)connection if it isn't run in a test enviroment. * Did a minimum amount of Formatting to the discription * Used `npm audit fix --force` to fix all vulnerabilties in our used packages. * Fixed curious doublechecking for empty Strings for the sortby-Parameter. - Now we only check once for a empty sortby - And added a test which distinguish between `sortby=""` and `sortby="+"` * Update api/routes/collections.js Removed the TODO about switching from mock-data to the real db * Removed globalTeardown as i brought up some problems corresponding to long db-queries (for example BBOX). Instead i increased the maximal testTimeout. --------- Co-authored-by: Robin Tammo Gummels <github@gummels.eu> * Update api/.env.example * latest database Version (#187) with `stac_id` and changed definition of `primary Keys` * added `.env` * added environment for docker-compose.yml now every connection-details are inside an `.env`. There is an `example.env` for better understanding which need to be set as connection details * added description of how to use the `.env` and `example.env` in the `README.md` * changed a few things e.g. DB_PORT --> ${DB_PORT} * now, everthing should be done. my god, help. sorry * layout issues fixed * Fixed Typo/incomplete Sentence in README.md * added `stac_id` for collections * all IDs are now written in the newer PostgrSQL standart: ```SQL id SERIAL PRIMARY KEY, ``` changed to ```SQL id INTEGER PRIMARY KEY GENERATED ALWAYS AS IDENTITY, ``` * changed `extend` to `extent`. * Changed language used in `./api/README.md` from german to english. I wanted to thsi anyway at some point, but this is now more like a Test-commit to see if the CI/CD Pipeline triggers... --------- Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de> Co-authored-by: Robin Tammo Gummels <github@gummels.eu> * chore: removed Helloworld * feat: collection page, mockup data for visual feedback chore: styling corrected, adjusted based on the mockup * style: removed old css names * chore: removed the mockup data * feat: update button styles and add contact popover in CollectionDetail view * style: fixed witdh of the apply filter button * style: map button, apply button behave like the others now * chore: removed mockup data * feat: route gitignore * feat: showing the collections from /collections in the result list * feat: enhance filter section layout and improve search result card tag display * style: refine search result card layout and improve text overflow handling * feat: implement pagination for collections in Home.vue and adjust max-height in search results * style: remove fixed height from card header and add styles for first button in card footer * style: enhance active pagination button styles for better visibility * feat: implement loading and error states in CollectionDetail.vue, enhance API integration for fetching collection data * style: width problem collection page, scolll items * Added `GET /collections/{id}`-Endpoint, more Fields in the responses and more Queryables-Parameters (#204) * Updated Query-Builder to get all necessary fields from all db_tables for collections. Added some tests and fixed some already existing tests, becuase now the tablenames start with the alias `c.`. * Updated Query-Builder to get all necessary fields from all db_tables for collections. Added some tests and fixed some already existing tests, becuase now the tablenames start with the alias `c.`. * Added `openapi.yaml` (now http://localhost:3000/api-docs/ is working). - needed to do some modifying to the app.js * Added discription on how to use `stac-api-validator`. Currently we are onyl valid to `core`. * Changed API-Version name to 1.1.0 instead of 1.0.0 * Revert "API is now responding with all necessary fields for each collection" (#195) Reverts #185 @SonkeHoffmann accidentally didn't squash correctly. * Revert "Revert "API is now responding with all necessary fields for each collection"" (#185) (#195) (#196) dev-api: prepare v1.1.0 + API docs + query builder fixes - Change API version to 1.1.0 - Add OpenAPI spec so /api-docs works locally - Document stac-api-validator usage - Update api/.env.example - Query builder: select required fields for collections across db_tables; adjust tests (alias `c.`) Commits included: - 34bf962 Changed API-Version name to 1.1.0 instead of 1.0.0 - b047389 Added description on how to use `stac-api-validator` (currently only valid for `core`) - b811288 Added `openapi.yaml` (so http://localhost:3000/api-docs/ works); modified app.js accordingly - 6e5ab3e Merge branch 'dev-api-robin' of github.com:SpatioCore/STAC-Atlas into dev-api-robin - 70dc043 Updated Query-Builder to get all necessary fields from all db_tables for collections. Added tests and fixed existing tests (table names now start with alias `c.`) - d83eeb4 Update api/.env.example - 5a7af5b Updated Query-Builder to get all necessary fields from all db_tables for collections. Added tests and fixed existing tests (table names now start with alias `c.`) Co-authored-by: Robin Tammo Gummels <github@gummels.eu> * Implement more Queryable-Fields and add the keywords-field to q fulltext search (#200) * Add provider and license filters to collection search API - Updated buildCollectionSearchQuery to include provider and license parameters for filtering collections. - Enhanced validateCollectionSearchParams middleware to validate provider and license query parameters. - Modified collections route to handle new provider and license filters in search queries. - Implemented validation functions for provider and license parameters in collectionSearchParams. * Add validation tests for provider and license * Enhance full-text search by including keywords in the tsvector expression and update related tests * Add provider and license to query parameter extraction in collection search validation * Revert "Enhance full-text search by including keywords in the tsvector expression and update related tests" This reverts commit 872443d. * Implement GET /collections/{id} endpoint with validation and QueryBuilder integration (#186) * added SQLQuery-builder with these parameters: q,bbox,datetime,sortby,limit and token * finalised bbox and datetime * adapted to DB, QueryBuilder and added helperfunction runQuery * added question-TODOs * added bbox+datetime to the Query-Builder from Jonas * added tests for Query-Builder from Jonas * added tests from George * added falsely deleted TODOs again * fixed collumn names to match our DB and adjusted full text search to match 05_indexes.sql correctly * Used a formatter and linter on `buildCollectionSearchQuery.js * Did some major and minor fixes to the collection search. - Updated `buildCollectionSearchQuery` to support pagination and improved text search with English language settings. - Modified tests in `buildCollectionsSearchQuery.basic.test.js`, `collections-pagination.test.js`, and `collections-sort.test.js` to reflect new query behavior and validation logic. - Enhanced sort validation in `validators.test.js` and `collectionSearchParams.js` to map API fields to database column names. - Implemented total count retrieval for matched results in `collections.js`. * Added a internal .env creation in the CI/CD Pipeline. It utilzes GitHub Repository Secrets to not publish any private Logins and stuff. * Forgot that the second Job of the CI/CD pipeline runs seperatly and needs a internal .env file too. * Enhance documentation for buildCollectionSearchQuery Updated the documentation for: - the buildCollectionSearchQuery function - the fulltextsearch * Refactor buildCollectionSearchQuery and updated SELECT part Changed the SELECT part to match our bid and the database shema. Updated comments and for clarity. Changed full-text search to use 'simple' configuration instead of 'english'. * Update api/routes/collections.js small typo Co-authored-by: Robin Tammo Gummels <github@gummels.eu> * Remove sorting TODO from collections route Removed TODO comment about sorting based on sortby parameter. * Explicitly return undefined for normalized in validateSortby Update validateSortby function to explicitly return undefined for normalized when sortby is not provided. * small fix in buildCollectionSearch.fulltext.test.js Change plainto_tsquery language from 'english' to 'simple' * Fix duplicate SELECT keyword in query Remove duplicate 'SELECT' keyword in SQL query. * Fix missing newline at end of collectionSearchParams.js * Fixed missing bracket in collectionSearchParams.js * Refactor validateSortby for optional parameter handling Refactor validateSortby function to handle optional sortby parameter and improve validation logic. * Stabilize API test pipeline by running Jest in-band with extended timeout Run Jest in CI with --runInBand and a higher default --testTimeout to stabilize database-backed integration tests. Multiple Jest workers were competing for the same PostgreSQL connection pool and some long-running /collections queries exceeded the default 5s timeout, causing failures in existing test suites (e.g. collectionSearch and DBconnection). * Fixed leaking tests that blocked CI/CD-Pipeline. - Added a global Teardown for jest and force-exited the tests to prevent leaking. - Made a change to db_APIconnection to only log the pool-(dis)connection if it isn't run in a test enviroment. * Did a minimum amount of Formatting to the discription * Used `npm audit fix --force` to fix all vulnerabilties in our used packages. * Fixed curious doublechecking for empty Strings for the sortby-Parameter. - Now we only check once for a empty sortby - And added a test which distinguish between `sortby=""` and `sortby="+"` * Update api/routes/collections.js Removed the TODO about switching from mock-data to the real db * Removed globalTeardown as i brought up some problems corresponding to long db-queries (for example BBOX). Instead i increased the maximal testTimeout. * added validator for collections{id} and correctly implemented collections{id} * added test for collections{id} * removed unnecessary parameter * added id parameter to the Query (temporary fix) * test-fixes to match our current tests and a fix to the baseURL for collection{id} * test fix * fixed problem with tests in api.test.js and adjusted the "invalid-id-test" in the validator. * Update api/routes/collections.js - Renamed `collection.id` to `c.collection.id` * added test for negative ids * deleted the whole "existing links" part and build base Links * fixed bug in validateCollectionId.js * Refactor negative ID test i encoded the "-1" value in the negative ID test instead of directly putting it into the path. * Removed a german comment in `api/routes/collections.js` --------- Co-authored-by: Robin Tammo Gummels <github@gummels.eu> --------- Co-authored-by: Sönke Hoffmann <hoffmann.sonke@gmx.de> Co-authored-by: JonasK <156602337+BrokeJ@users.noreply.github.com> Co-authored-by: Vincent Kühn <vkuehn@uni-muenster.de> * chore: switched to the apis collections/:id in the ui * style: calender icon response to dark & light mode * chroe: removed svg for language button style: slightly thicker border for nav-buttons * style: added a slight highlight to input/ interaction buttons * style: update dark mode colors for text and adjust tag background * style: smaller vertical padding for select options * feat: source and contact are shown when clicked * feat: search functionality * style: centered the paging icon chore: view css folder to seperate style and content * feat: removed the unnecessary Filter Header * feat: add bounding box drawing functionality and integrate with filter section --------- Co-authored-by: RobinGummels <github@gummels.eu> Co-authored-by: Jakob <vertrox78@gmail.com> Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de> Co-authored-by: jklaer <jklaer@uni-muenster.de> Co-authored-by: VincentKuehn <vkuehn@uni-muenster.de> Co-authored-by: mammutor <mammutor@gmail.com> Co-authored-by: Humam <44206081+Mammutor@users.noreply.github.com> Co-authored-by: Georgios Voulgaris <gvoulgar@uni-muenster.de> Co-authored-by: Sönke Hoffmann <hoffmann.sonke@gmx.de> Co-authored-by: JonasK <156602337+BrokeJ@users.noreply.github.com> * style: bigger metadata section, hover state for items feat: item thumbnail * feat: routing the home when clicking on the logo and app title * feat: enhance button states and update provider button in CollectionDetail * style: scrollbar styling for dark mode * feat: docker * feat: enhance collection API integration and UI components with queryables support * feat: docker dev container hot reload, removed "Page" text in front of pagination-info * feat: add .vite to .gitignore * fix: update scrollbar track background to transparent * feat: add feedback for copy to clipboard action with visual indication * fix: change justify-content to start for better alignment in collection detail layout * fix: update favicon * feat: items in collection page * feat: add CQL2 filter support in filter section and API integration * chore: markdown lint, removed vue.svg * refactor: remove development Dockerfile and associated docker-compose configuration * feat: add active and API status filters to FilterSection and update filter store * style: update spacing in filter section and set height for right section in collection detail * fix: docker start / build * fix: add missing closing div in FilterSection and clean up imports in CollectionDetail * Refactor code structure for improved readability and maintainability * style: bigger search cards * feat: licences & providers queryables * fix: update API status labels for clarity * style: gap in metadata-item for seperation * fix: update active filter default value and placeholder for clarity * feat: enhance pagination with input jump and update items per page * feat: enhance loading and no-results UI with spinner and messages * style: add ellipsis for overflowing text in source and provider URLs * style: comment out information button in navbar for future implementation * feat: i18n support across components and views, documentation * feat: enhance pagination with input jump, update items per page, and improve item count display * docs: update README with comprehensive structure, deployment instructions, and environment variable details * feat: add loading indicator for items count and additional properties section in CollectionDetail * feat: implement collapsible additional properties section in CollectionDetail * feat: update German translations for consistency in Collection terminology * feat: add selfUrl property to ItemCard and CollectionDetail for external link support * feat: enhance item fetching to set selfUrl based on resolved URLs * fix: update German translations for consistency in terminology * fix: remove max-width constraint from search result card styles * feat: enhance source link handling to prioritize source_root and open in STAC Browser * Delete api/data/collections.js * Delete api/examples/README.md * Readded some Changes to runQuery() in collections.js Add error handling to runQuery function * Added one `TAB` to much... * feat: add bounding box modal styles for improved UI * refactor: remove unused styles from BoundingBoxModal.vue * feat: add example.env for environment configuration and update README with testing instructions * feat: add Playwright E2E tests for accessibility, collection detail, map view, and search functionality * feat: add empty page message for paginated item lists in CollectionDetail * fix: change overflow property to enable vertical scrolling for paginated item list * fix: reset activeFilter to empty string for no filter by default --------- Co-authored-by: Simon Benjamin Imfeld <simfeld@uni-muenster.de> Co-authored-by: RobinGummels <github@gummels.eu> Co-authored-by: Jakob <vertrox78@gmail.com> Co-authored-by: Sönke Hoffmann <shoffma5@uni.muenster.de> Co-authored-by: jklaer <jklaer@uni-muenster.de> Co-authored-by: VincentKuehn <vkuehn@uni-muenster.de> Co-authored-by: mammutor <mammutor@gmail.com> Co-authored-by: Humam <44206081+Mammutor@users.noreply.github.com> Co-authored-by: Georgios Voulgaris <gvoulgar@uni-muenster.de> Co-authored-by: Sönke Hoffmann <hoffmann.sonke@gmx.de> Co-authored-by: JonasK <156602337+BrokeJ@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR introduces the GET /collections/:id endpoint, enabling retrieval of a single STAC Collection by its unique identifier.
The new endpoint follows the same data-selection logic as GET /collections by reusing the shared QueryBuilder, ensuring consistent output across both endpoints.
A new middleware validateCollectionId ensures that the :id parameter is:
The new route:
uses the QueryBuilder with:
retrieves exactly one collection (or none)
returns:
a new test suite was added: