Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

postgresql{,TestHook}: avoid running postgresql during builds on darwin #392430

Merged
merged 4 commits into from
Mar 24, 2025

Conversation

wolfgangwalther
Copy link
Contributor

@wolfgangwalther wolfgangwalther commented Mar 23, 2025

This is related to #371242. TLDR: Because of a nix bug, we disabled postgresql's tests again in d0f3296 (thanks @vcunat, that makes total sense!).

This cleans up the situation a bit more:

  • Expands the comments for why we disabled the checks, according to what we know now.
  • Marks postgresqlTestHook as broken on darwin, because it leads to exactly the same problems again and again.
  • Re-enables tests for some packages, which were accidentally disabled in 9869d7f, where we moved from checkPhase to installCheckPhase.

I'll probably target the last commit at haskell-updates after a test run here to avoid merge conflicts.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@ofborg ofborg bot added the 6.topic: darwin Running or building packages on Darwin label Mar 23, 2025
@github-actions github-actions bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 11-100 labels Mar 23, 2025
@nix-owners nix-owners bot requested review from eadwu and gador March 23, 2025 16:16
@wolfgangwalther
Copy link
Contributor Author

Currently, python3Packages.asyncpg fails after re-enabling the tests:

============================= test session starts ==============================
platform linux -- Python 3.12.9, pytest-8.3.5, pluggy-1.5.0
rootdir: /build/asyncpg-0.30.0
configfile: pyproject.toml
testpaths: tests
plugins: xdist-3.6.1
80 workers [313 items]   m
s..ss...s.s..s.F...................................s.s.........................................................................................................>
=================================== FAILURES ===================================
____________________ TestConnectParams.test_connect_params _____________________
[gw38] linux -- Python 3.12.9 /nix/store/cxhrrsf7spcgdkrxmlbfzc1bh46rzf7w-python3-3.12.9/bin/python3.12
Traceback (most recent call last):
  File "/nix/store/cxhrrsf7spcgdkrxmlbfzc1bh46rzf7w-python3-3.12.9/lib/python3.12/unittest/case.py", line 58, in testPartExecutor
    yield
  File "/nix/store/cxhrrsf7spcgdkrxmlbfzc1bh46rzf7w-python3-3.12.9/lib/python3.12/unittest/case.py", line 634, in run
    self._callTestMethod(testMethod)
  File "/nix/store/cxhrrsf7spcgdkrxmlbfzc1bh46rzf7w-python3-3.12.9/lib/python3.12/unittest/case.py", line 589, in _callTestMethod
    if method() is not None:
       ^^^^^^^^
  File "/build/asyncpg-0.30.0/tests/test_connect.py", line 1237, in test_connect_params
    self.run_testcase(testcase)
  File "/build/asyncpg-0.30.0/tests/test_connect.py", line 1154, in run_testcase
    addrs, params = connect_utils._parse_connect_dsn_and_args(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/n33chcqvmsygqhm87b1swidp9q26lgqf-python3.12-asyncpg-0.30.0/lib/python3.12/site-packages/asyncpg/connect_utils.py", line 280, in _parse_conne>
    parsed = urllib.parse.urlparse(dsn)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/cxhrrsf7spcgdkrxmlbfzc1bh46rzf7w-python3-3.12.9/lib/python3.12/urllib/parse.py", line 395, in urlparse
    splitresult = urlsplit(url, scheme, allow_fragments)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/cxhrrsf7spcgdkrxmlbfzc1bh46rzf7w-python3-3.12.9/lib/python3.12/urllib/parse.py", line 516, in urlsplit
    _check_bracketed_netloc(netloc)
  File "/nix/store/cxhrrsf7spcgdkrxmlbfzc1bh46rzf7w-python3-3.12.9/lib/python3.12/urllib/parse.py", line 451, in _check_bracketed_netloc
    raise ValueError("Invalid IPv6 URL")
ValueError: Invalid IPv6 URL
=========================== short test summary info ============================
FAILED tests/test_connect.py::TestConnectParams::test_connect_params - ValueError: Invalid IPv6 URL
================== 1 failed, 304 passed, 8 skipped in 22.72s ===================

@wolfgangwalther
Copy link
Contributor Author

According to MagicStack/asyncpg#1236 (comment), we had seen this failure already on the 24.11 branch, where the tests were still enabled.

To fix, we can apply 2b588bf on master as well. @vcunat this should also answer your question why you didn't observe that on master.

@wolfgangwalther
Copy link
Contributor Author

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 392430


x86_64-linux

⏩ 18 packages marked as broken and skipped:
  • chatgpt-retrieval-plugin
  • chatgpt-retrieval-plugin.dist
  • haskellPackages.rel8
  • haskellPackages.rel8.doc
  • khoj
  • khoj.dist
  • maubot
  • maubot.dist
  • mautrix-googlechat
  • mautrix-googlechat.dist
  • mautrix-telegram
  • mautrix-telegram.dist
  • python312Packages.maubot
  • python312Packages.maubot.dist
  • python312Packages.ormar
  • python312Packages.ormar.dist
  • python313Packages.ormar
  • python313Packages.ormar.dist
❌ 10 packages failed to build:
  • open-webui
  • open-webui.dist
  • private-gpt
  • private-gpt.dist
  • python312Packages.langgraph
  • python312Packages.langgraph.dist
  • python312Packages.private-gpt
  • python312Packages.private-gpt.dist
  • sourcehut.listssrht
  • sourcehut.listssrht.dist
✅ 61 packages built:
  • haskellPackages.bcp47-orphans
  • haskellPackages.bcp47-orphans.doc
  • haskellPackages.espial
  • haskellPackages.espial.doc
  • haskellPackages.esqueleto
  • haskellPackages.esqueleto-compat
  • haskellPackages.esqueleto-compat.doc
  • haskellPackages.esqueleto.doc
  • haskellPackages.persistent-pagination
  • haskellPackages.persistent-pagination.doc
  • haskellPackages.persistent-sql-lifted
  • haskellPackages.persistent-sql-lifted.doc
  • haskellPackages.persistent-typed-db
  • haskellPackages.persistent-typed-db.doc
  • haskellPackages.type-level-kv-list-esqueleto
  • haskellPackages.type-level-kv-list-esqueleto.doc
  • heisenbridge
  • heisenbridge.dist
  • litestar (python312Packages.litestar)
  • litestar.dist (python312Packages.litestar.dist)
  • matrix-zulip-bridge
  • matrix-zulip-bridge.dist
  • nominatim
  • pgadmin4
  • pgadmin4-desktopmode
  • pgadmin4-desktopmode.dist
  • pgadmin4.dist
  • prefect
  • prefect.dist
  • python312Packages.asyncpg
  • python312Packages.asyncpg.dist
  • python312Packages.langgraph-checkpoint-postgres
  • python312Packages.langgraph-checkpoint-postgres.dist
  • python312Packages.llama-index-vector-stores-postgres
  • python312Packages.llama-index-vector-stores-postgres.dist
  • python312Packages.mautrix (python312Packages.mautrix-appservice)
  • python312Packages.mautrix.dist (python312Packages.mautrix-appservice.dist)
  • python312Packages.orm
  • python312Packages.orm.dist
  • python312Packages.pgvector
  • python312Packages.pgvector.dist
  • python312Packages.txtai
  • python312Packages.txtai.dist
  • python312Packages.weaviate-client
  • python312Packages.weaviate-client.dist
  • python313Packages.asyncpg
  • python313Packages.asyncpg.dist
  • python313Packages.langgraph-checkpoint-postgres
  • python313Packages.langgraph-checkpoint-postgres.dist
  • python313Packages.litestar
  • python313Packages.litestar.dist
  • python313Packages.mautrix (python313Packages.mautrix-appservice)
  • python313Packages.mautrix.dist (python313Packages.mautrix-appservice.dist)
  • python313Packages.orm
  • python313Packages.orm.dist
  • python313Packages.pgvector
  • python313Packages.pgvector.dist
  • python313Packages.txtai
  • python313Packages.txtai.dist
  • python313Packages.weaviate-client
  • python313Packages.weaviate-client.dist

aarch64-linux

⏩ 22 packages marked as broken and skipped:
  • chatgpt-retrieval-plugin
  • chatgpt-retrieval-plugin.dist
  • haskellPackages.rel8
  • haskellPackages.rel8.doc
  • khoj
  • khoj.dist
  • maubot
  • maubot.dist
  • mautrix-googlechat
  • mautrix-googlechat.dist
  • mautrix-telegram
  • mautrix-telegram.dist
  • private-gpt
  • private-gpt.dist
  • python312Packages.maubot
  • python312Packages.maubot.dist
  • python312Packages.ormar
  • python312Packages.ormar.dist
  • python312Packages.private-gpt
  • python312Packages.private-gpt.dist
  • python313Packages.ormar
  • python313Packages.ormar.dist
❌ 4 packages failed to build:
  • python312Packages.langgraph
  • python312Packages.langgraph.dist
  • sourcehut.listssrht
  • sourcehut.listssrht.dist
✅ 61 packages built:
  • haskellPackages.bcp47-orphans
  • haskellPackages.bcp47-orphans.doc
  • haskellPackages.espial
  • haskellPackages.espial.doc
  • haskellPackages.esqueleto
  • haskellPackages.esqueleto-compat
  • haskellPackages.esqueleto-compat.doc
  • haskellPackages.esqueleto.doc
  • haskellPackages.persistent-pagination
  • haskellPackages.persistent-pagination.doc
  • haskellPackages.persistent-sql-lifted
  • haskellPackages.persistent-sql-lifted.doc
  • haskellPackages.persistent-typed-db
  • haskellPackages.persistent-typed-db.doc
  • haskellPackages.type-level-kv-list-esqueleto
  • haskellPackages.type-level-kv-list-esqueleto.doc
  • heisenbridge
  • heisenbridge.dist
  • litestar (python312Packages.litestar)
  • litestar.dist (python312Packages.litestar.dist)
  • matrix-zulip-bridge
  • matrix-zulip-bridge.dist
  • nominatim
  • pgadmin4
  • pgadmin4-desktopmode
  • pgadmin4-desktopmode.dist
  • pgadmin4.dist
  • prefect
  • prefect.dist
  • python312Packages.asyncpg
  • python312Packages.asyncpg.dist
  • python312Packages.langgraph-checkpoint-postgres
  • python312Packages.langgraph-checkpoint-postgres.dist
  • python312Packages.llama-index-vector-stores-postgres
  • python312Packages.llama-index-vector-stores-postgres.dist
  • python312Packages.mautrix (python312Packages.mautrix-appservice)
  • python312Packages.mautrix.dist (python312Packages.mautrix-appservice.dist)
  • python312Packages.orm
  • python312Packages.orm.dist
  • python312Packages.pgvector
  • python312Packages.pgvector.dist
  • python312Packages.txtai
  • python312Packages.txtai.dist
  • python312Packages.weaviate-client
  • python312Packages.weaviate-client.dist
  • python313Packages.asyncpg
  • python313Packages.asyncpg.dist
  • python313Packages.langgraph-checkpoint-postgres
  • python313Packages.langgraph-checkpoint-postgres.dist
  • python313Packages.litestar
  • python313Packages.litestar.dist
  • python313Packages.mautrix (python313Packages.mautrix-appservice)
  • python313Packages.mautrix.dist (python313Packages.mautrix-appservice.dist)
  • python313Packages.orm
  • python313Packages.orm.dist
  • python313Packages.pgvector
  • python313Packages.pgvector.dist
  • python313Packages.txtai
  • python313Packages.txtai.dist
  • python313Packages.weaviate-client
  • python313Packages.weaviate-client.dist

none of the failures seem related to me.

wolfgangwalther and others added 3 commits March 24, 2025 21:40
https://hydra.nixos.org/build/291045308/nixlog/1/tail
Looks like triggered by 06aecd3
but curiously on nixpkgs master we do not suffer from the issue.

(cherry picked from commit 2b588bf)
Those were accidentally disabled a while ago, when postgresql.doCheck
was set to false unconditionally. Instead the checks are now done as
part of the installCheck phase.
@wolfgangwalther
Copy link
Contributor Author

I'll probably target the last commit at haskell-updates after a test run here to avoid merge conflicts.

Split the haskell commit off and will bring it back once this PR has reached haskell-updates.

Will merge this once eval is through.

@wolfgangwalther wolfgangwalther merged commit caaf0ad into NixOS:master Mar 24, 2025
23 of 27 checks passed
@wolfgangwalther wolfgangwalther deleted the postgresql-do-check branch March 24, 2025 20:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: darwin Running or building packages on Darwin 6.topic: python 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 11-100
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants