Upgrade to latest stac-fastapi-pgstac #195

pantierra · 2025-04-01T21:15:48Z

No description provided.

pantierra · 2025-04-01T22:13:37Z

Debugging here. Locally it works, but the CI tests starts failing once we upgrade stac-fastapi-pgstac from version 3.0.1 to 4.0.0.

vincentsarago · 2025-04-01T22:22:10Z

Debugging here. Locally it works, but the CI tests starts failing once we upgrade stac-fastapi-pgstac from version 3.0.1 to 4.0.0.

did you try 5.0.1 ?

pantierra · 2025-04-01T22:51:06Z

Yes, I tried the latest 5.0.1 first, and then downgraded to find the point that everything after 4.0.0 fails, but 3.0.1 and below still works.

It might be related to this line which is not being called locally...

vincentsarago · 2025-04-02T08:08:30Z

🤔 I don't see anything that changed between 3.x and 5.x in the docker image itself

there might be some logic change in the log but we don't see them in the actions

while [[ -z "$(kubectl get pod  | grep "^stac-$RELEASE_NAME-.*$" | cut -d' ' -f1 | xargs -I{} kubectl logs pod/{} | grep "GET /_mgmt/ping" | head -n 1)" ]]; do

Are we mounting the app with a root-path or prefix? maybe GET /_mgmt/ping is GET {root-path}/_mgmt/ping 🤷

pantierra · 2025-04-03T08:57:14Z

Checked the logs, again we come back to the problem:

ERROR:    Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/starlette/routing.py", line 692, in lifespan
    async with self.lifespan_context(app) as maybe_state:
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/fastapi/routing.py", line 133, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/fastapi/routing.py", line 133, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/fastapi/routing.py", line 133, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/fastapi/routing.py", line 133, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/fastapi/routing.py", line 133, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/fastapi/routing.py", line 133, in merged_lifespan
    async with original_context(app) as maybe_original_state:
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/contextlib.py", line 210, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/app/stac_fastapi/pgstac/app.py", line 158, in lifespan
    await connect_to_db(app)
  File "/app/stac_fastapi/pgstac/db.py", line 69, in connect_to_db
    app.state.writepool = await db.create_pool(writepool, postgres_settings)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/stac_fastapi/pgstac/db.py", line 146, in create_pool
    pool = await asyncpg.create_pool(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/asyncpg/pool.py", line 418, in _async__init__
    await self._initialize()
  File "/usr/local/lib/python3.12/site-packages/asyncpg/pool.py", line 445, in _initialize
    await first_ch.connect()
  File "/usr/local/lib/python3.12/site-packages/asyncpg/pool.py", line 132, in connect
    self._con = await self._pool._get_new_connection()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/asyncpg/pool.py", line 517, in _get_new_connection
    con = await self._connect(
          ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/asyncpg/connection.py", line 2421, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
    conn = await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
    return await __connect_addr(params, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/asyncpg/connect_utils.py", line 934, in __connect_addr
    await connected
asyncpg.exceptions.TooManyConnectionsError: remaining connection slots are reserved for roles with the SUPERUSER attribute

pantierra · 2025-04-03T08:57:48Z

After over 10 minutes we it works, but then the pipeline already failed.

sunu · 2025-04-03T09:43:12Z

@pantierra might be worth setting WEB_CONCURRENCY to 1 in all the services to limit the worker counts while running the tests

Otherwise the number of connections become conn_size x WEB_CONCURRENCY x no. of services which is probably greater than the default max_connections of 100 in postgres

vincentsarago · 2025-04-03T09:47:31Z

After talking on meet with @pantierra we think this is linked to ROOT_PATH being happened to the endpoint root and thus not matching the grep command grep "GET /_mgmt/ping"

The fix would be to relax the grep command ;-)

pantierra · 2025-04-03T12:30:14Z

This works now. I also checked that the deployment has no /stac/stac route... Ready for review.

vincentsarago · 2025-04-03T12:50:23Z

.github/workflows/helm-tests.yml

          done
          echo "vector service has started, moving on..."
-          while [[ -z "$(kubectl get pod  | grep "^stac-$RELEASE_NAME-.*$" | cut -d' ' -f1 | xargs -I{} kubectl logs pod/{} | grep "GET /_mgmt/ping" | head -n 1)" ]]; do
+          while [[ -z "$(kubectl get pod  | grep "^stac-$RELEASE_NAME-.*$" | cut -d' ' -f1 | xargs -I{} kubectl logs pod/{} | grep "GET /stac/_mgmt/ping" | head -n 1)" ]]; do


I would love to understand why the other don't need this... but I'll just put that on my infinite todo list

The other also need it, after updates...! They now all run on:

/raster

/vector

/stac

pantierra force-pushed the feature/upgrade-dependencies-2025-04 branch 3 times, most recently from c8535b1 to 6667d37 Compare April 1, 2025 22:12

pantierra changed the title ~~Upgrade database~~ Upgrade dependencies - April 2025 Apr 1, 2025

pantierra changed the title ~~Upgrade dependencies - April 2025~~ Upgrade to latest stac-fastapi-pgstac Apr 3, 2025

pantierra mentioned this pull request Apr 3, 2025

Upgrade dependencies #196

Merged

pantierra force-pushed the feature/upgrade-dependencies-2025-04 branch 2 times, most recently from d7b1ec3 to 77e2b58 Compare April 3, 2025 08:36

pantierra self-assigned this Apr 3, 2025

pantierra force-pushed the feature/upgrade-dependencies-2025-04 branch 6 times, most recently from ae8a5c0 to a692a5c Compare April 3, 2025 12:05

Upgraded stac-fastapi-pgstac to 5.0.1.

909356d

pantierra force-pushed the feature/upgrade-dependencies-2025-04 branch from a692a5c to 909356d Compare April 3, 2025 12:29

pantierra marked this pull request as ready for review April 3, 2025 12:29

pantierra requested review from ciaransweet, ividito and sunu April 3, 2025 12:30

vincentsarago self-requested a review April 3, 2025 12:49

vincentsarago approved these changes Apr 3, 2025

View reviewed changes

vincentsarago reviewed Apr 3, 2025

View reviewed changes

pantierra merged commit 7bf6609 into main Apr 3, 2025
2 checks passed

pantierra deleted the feature/upgrade-dependencies-2025-04 branch April 3, 2025 13:08

sunu mentioned this pull request Apr 3, 2025

Malformed links when setting root-path through uvicorn stac-utils/stac-fastapi-pgstac#220

Closed

j08lue mentioned this pull request Apr 8, 2025

STAC API maintenance eoAPI (Q4) EOEPCA/resource-discovery#125

Closed

2 tasks

Uh oh!

Upgrade to latest stac-fastapi-pgstac #195

Upgrade to latest stac-fastapi-pgstac #195

Uh oh!

Conversation

pantierra commented Apr 1, 2025

Uh oh!

pantierra commented Apr 1, 2025

Uh oh!

vincentsarago commented Apr 1, 2025

Uh oh!

pantierra commented Apr 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vincentsarago commented Apr 2, 2025

Uh oh!

pantierra commented Apr 3, 2025

Uh oh!

pantierra commented Apr 3, 2025

Uh oh!

sunu commented Apr 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vincentsarago commented Apr 3, 2025

Uh oh!

pantierra commented Apr 3, 2025

Uh oh!

vincentsarago Apr 3, 2025

Choose a reason for hiding this comment

Uh oh!

pantierra Apr 3, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

pantierra commented Apr 1, 2025 •

edited

Loading

sunu commented Apr 3, 2025 •

edited

Loading