Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bkyml: init at 1.4.3 #122958

Merged
merged 2 commits into from May 29, 2021
Merged

Conversation

olebedev
Copy link
Member

@olebedev olebedev commented May 14, 2021

Motivation for this change

This PR closes #122953 by adding the bkyml python application to pkgs/tools/misc/bkyml. bkyml is a CLI tool to generate a pipeline in yaml format to be consumed by a Buildkite agent on the fly.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@legendofmiracles
Copy link
Contributor

All package related commits should be squashed into one, the first commit has the right message.

It's good that you made a separate commit for the maintainer, but it should be in the format maintainers: add <name>

pkgs/tools/misc/bkyml/default.nix Outdated Show resolved Hide resolved
pkgs/tools/misc/bkyml/default.nix Outdated Show resolved Hide resolved
pkgs/tools/misc/bkyml/default.nix Outdated Show resolved Hide resolved
Comment on lines +28 to +29
# Don't run tests because they are broken when run within
# buildPythonApplication for reasons I don't quite understand.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have an error log?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, click to expand
$ nix-shell --pure -p bkyml -I nixpkgs=./default.nix

these derivations will be built:
  /nix/store/64nanv8kyhabpdi82y2kdngmzf0n46f8-bkyml-1.4.3.drv
building '/nix/store/64nanv8kyhabpdi82y2kdngmzf0n46f8-bkyml-1.4.3.drv'...
Sourcing python-remove-tests-dir-hook
Sourcing python-catch-conflicts-hook.sh
Sourcing python-remove-bin-bytecode-hook.sh
Sourcing setuptools-build-hook
Using setuptoolsBuildPhase
Using setuptoolsShellHook
Sourcing pip-install-hook
Using pipInstallPhase
Sourcing python-imports-check-hook.sh
Using pythonImportsCheckPhase
Sourcing python-namespaces-hook
Sourcing setuptools-check-hook
Using setuptoolsCheckPhase
unpacking sources
unpacking source archive /nix/store/qy9g2a4bls6blg07kvvd23kdn6jdsm9w-bkyml-1.4.3.tar.gz
source root is bkyml-1.4.3
setting SOURCE_DATE_EPOCH to timestamp 1545195659 of file bkyml-1.4.3/src/bkyml.egg-info/top_level.txt
patching sources
configuring
no configure script, doing nothing
building
Executing setuptoolsBuildPhase
/nix/store/8lxqamalgdip0jk2nkm3r56i5qjmi16c-python3.8-setuptools-54.2.0/lib/python3.8/site-packages/setuptools/dist.py:642: UserWarning: Usage of dash-separated 'author-email' will not be supported in future versions. Please use the underscore name 'author_email' instead
  warnings.warn(
/nix/store/8lxqamalgdip0jk2nkm3r56i5qjmi16c-python3.8-setuptools-54.2.0/lib/python3.8/site-packages/setuptools/dist.py:642: UserWarning: Usage of dash-separated 'long-description' will not be supported in future versions. Please use the underscore name 'long_description' instead
  warnings.warn(
running bdist_wheel
running build
running build_py
creating build
creating build/lib
creating build/lib/bkyml
copying src/bkyml/skeleton.py -> build/lib/bkyml
copying src/bkyml/__init__.py -> build/lib/bkyml
running egg_info
writing src/bkyml.egg-info/PKG-INFO
writing dependency_links to src/bkyml.egg-info/dependency_links.txt
writing entry points to src/bkyml.egg-info/entry_points.txt
writing requirements to src/bkyml.egg-info/requires.txt
writing top-level names to src/bkyml.egg-info/top_level.txt
reading manifest file 'src/bkyml.egg-info/SOURCES.txt'
writing manifest file 'src/bkyml.egg-info/SOURCES.txt'
installing to build/bdist.macosx-10.6-x86_64/wheel
running install
running install_lib
creating build/bdist.macosx-10.6-x86_64
creating build/bdist.macosx-10.6-x86_64/wheel
creating build/bdist.macosx-10.6-x86_64/wheel/bkyml
copying build/lib/bkyml/skeleton.py -> build/bdist.macosx-10.6-x86_64/wheel/bkyml
copying build/lib/bkyml/__init__.py -> build/bdist.macosx-10.6-x86_64/wheel/bkyml
running install_egg_info
Copying src/bkyml.egg-info to build/bdist.macosx-10.6-x86_64/wheel/bkyml-0.0.0-py3.8.egg-info
running install_scripts
adding license file "LICENSE.txt" (matched pattern "LICEN[CS]E*")
adding license file "AUTHORS.rst" (matched pattern "AUTHORS*")
creating build/bdist.macosx-10.6-x86_64/wheel/bkyml-0.0.0.dist-info/WHEEL
creating 'dist/bkyml-0.0.0-py2.py3-none-any.whl' and adding 'build/bdist.macosx-10.6-x86_64/wheel' to it
adding 'bkyml/__init__.py'
adding 'bkyml/skeleton.py'
adding 'bkyml-0.0.0.dist-info/AUTHORS.rst'
adding 'bkyml-0.0.0.dist-info/LICENSE.txt'
adding 'bkyml-0.0.0.dist-info/METADATA'
adding 'bkyml-0.0.0.dist-info/WHEEL'
adding 'bkyml-0.0.0.dist-info/entry_points.txt'
adding 'bkyml-0.0.0.dist-info/top_level.txt'
adding 'bkyml-0.0.0.dist-info/RECORD'
removing build/bdist.macosx-10.6-x86_64/wheel
Finished executing setuptoolsBuildPhase
installing
Executing pipInstallPhase
/private/var/folders/2k/hl3c_3g13x790m4mpkhx4dkr0000gn/T/nix-build-bkyml-1.4.3.drv-0/bkyml-1.4.3/dist /private/var/folders/2k/hl3c_3g13x790m4mpkhx4dkr0000gn/T/nix-build-bkyml-1.4.3.drv-0/bkyml-1.4.3
Processing ./bkyml-0.0.0-py2.py3-none-any.whl
Requirement already satisfied: ruamel.yaml in /nix/store/bgsxv838m57s278rmlmpgc6hcwjzlyjb-python3.8-ruamel.yaml-0.16.13/lib/python3.8/site-packages (from bkyml==0.0.0) (0.16.13)
Requirement already satisfied: ruamel.yaml.clib>=0.1.2 in /nix/store/713xci3bnipmf5rgwrl7xq2xmlhx8c4w-python3.8-ruamel.yaml.clib-0.2.0/lib/python3.8/site-packages (from ruamel.yaml->bkyml==0.0.0) (0.2.0)
Installing collected packages: bkyml
Successfully installed bkyml-0.0.0
/private/var/folders/2k/hl3c_3g13x790m4mpkhx4dkr0000gn/T/nix-build-bkyml-1.4.3.drv-0/bkyml-1.4.3
Finished executing pipInstallPhase
post-installation fixup
strip is /nix/store/pvms38jd72iqdrn3j376yfwcik0wvsiv-cctools-binutils-darwin-949.0.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/vaqwbzk8ck59mq45qhlp178mwdipa1h3-bkyml-1.4.3/lib  /nix/store/vaqwbzk8ck59mq45qhlp178mwdipa1h3-bkyml-1.4.3/bin 
patching script interpreter paths in /nix/store/vaqwbzk8ck59mq45qhlp178mwdipa1h3-bkyml-1.4.3
Rewriting #!/nix/store/l4q9nfrq3wz4qgk5ikbzfcs7iqdb32y1-python3-3.8.9/bin/python3.8 to #!/nix/store/l4q9nfrq3wz4qgk5ikbzfcs7iqdb32y1-python3-3.8.9
wrapping `/nix/store/vaqwbzk8ck59mq45qhlp178mwdipa1h3-bkyml-1.4.3/bin/bkyml'...
Rewriting #!/nix/store/l4q9nfrq3wz4qgk5ikbzfcs7iqdb32y1-python3-3.8.9/bin/python3.8 to #!/nix/store/l4q9nfrq3wz4qgk5ikbzfcs7iqdb32y1-python3-3.8.9
wrapping `/nix/store/vaqwbzk8ck59mq45qhlp178mwdipa1h3-bkyml-1.4.3/bin/bkyaml'...
Executing pythonRemoveTestsDir
Finished executing pythonRemoveTestsDir
running install tests
no Makefile or custom installCheckPhase, doing nothing
pythonCatchConflictsPhase
pythonRemoveBinBytecodePhase
pythonImportsCheckPhase
Executing pythonImportsCheckPhase
setuptoolsCheckPhase
Executing setuptoolsCheckPhase
/nix/store/8lxqamalgdip0jk2nkm3r56i5qjmi16c-python3.8-setuptools-54.2.0/lib/python3.8/site-packages/setuptools/dist.py:642: UserWarning: Usage of dash-separated 'author-email' will not be supported in future versions. Please use the underscore name 'author_email' instead
  warnings.warn(
/nix/store/8lxqamalgdip0jk2nkm3r56i5qjmi16c-python3.8-setuptools-54.2.0/lib/python3.8/site-packages/setuptools/dist.py:642: UserWarning: Usage of dash-separated 'long-description' will not be supported in future versions. Please use the underscore name 'long_description' instead
  warnings.warn(
running test
error: error in setup.cfg: command 'test' has no such option 'addopts'
builder for '/nix/store/64nanv8kyhabpdi82y2kdngmzf0n46f8-bkyml-1.4.3.drv' failed with exit code 1
error: build of '/nix/store/64nanv8kyhabpdi82y2kdngmzf0n46f8-bkyml-1.4.3.drv' failed

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to a quick google adding pytestCheckHook to checkInputs should fix that error.

Copy link
Member Author

@olebedev olebedev May 23, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have tried:

diff --git a/pkgs/tools/misc/bkyml/default.nix b/pkgs/tools/misc/bkyml/default.nix
index aea83e7c99f..1733f017d76 100644
--- a/pkgs/tools/misc/bkyml/default.nix
+++ b/pkgs/tools/misc/bkyml/default.nix
@@ -25,9 +25,11 @@ buildPythonApplication rec {
         "__version__ = \"${version}\""
   '';
 
-  # Don't run tests because they are broken when run within
-  # buildPythonApplication for reasons I don't quite understand.
-  doCheck = false;
+  checkInputs = [
+    pytestCheckHook
+    snapshottest
+    pytest-cov
+  ];
 
   pythonImportsCheck = [ "bkyml" ];

but got another error:

Click here to expand
...
Check whether the following modules can be imported: bkyml
pytestCheckPhase
Executing pytestCheckPhase
============================= test session starts ==============================
platform darwin -- Python 3.8.9, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 -- /nix/store/l4q9nfrq3wz4qgk5ikbzfcs7iqdb32y1-python3-3.8.9/bin/python3.8
cachedir: .pytest_cache
rootdir: /private/var/folders/2k/hl3c_3g13x790m4mpkhx4dkr0000gn/T/nix-build-bkyml-1.4.3.drv-0/bkyml-1.4.3, configfile: setup.cfg
plugins: snapshottest-0.6.0, cov-2.11.1
collected 0 items                                                              


---------- coverage: platform darwin, python 3.8.9-final-0 -----------
Name                                                                                                    Stmts   Miss Branch BrPart  Cover   Missing
---------------------------------------------------------------------------------------------------------------------------------------------------
/nix/store/zqyd916jpql9iraab1vdig3mr4q8hw7j-bkyml-1.4.3/lib/python3.8/site-packages/bkyml/__init__.py       4      0      0      0   100%
/nix/store/zqyd916jpql9iraab1vdig3mr4q8hw7j-bkyml-1.4.3/lib/python3.8/site-packages/bkyml/skeleton.py     375    299    146      0    15%   27-29, 40-42, 46-51, 55-59, 63-69, 73, 77-80, 84-86, 90-99, 106-153, 157-203, 207-221, 228-284, 288-330, 337-354, 358-370, 377-384, 388-390, 397-398, 403, 410-420, 424, 434-577, 581-613, 617-709, 716-722, 726-734, 746-787, 796-797, 802-804, 813-815, 821
---------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                                                                     379    299    146      0    15%

============================ no tests ran in 0.34s =============================
builder for '/nix/store/4g51c6q4kj2jxzgwhyawm862npkznpf4-bkyml-1.4.3.drv' failed with exit code 5
error: build of '/nix/store/4g51c6q4kj2jxzgwhyawm862npkznpf4-bkyml-1.4.3.drv' failed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SuperSandro2000, can we get it in as it is now or you have another idea how we can make these tests pass?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have any idea. I think upstream is just to old for our current pytest version. I think we can ignore tests here with a comment.

@olebedev
Copy link
Member Author

@legendofmiracles, is that supposed from me to do something else to get this PR merged after your approval, or I just wait?

@legendofmiracles
Copy link
Contributor

legendofmiracles commented May 20, 2021

My approval is basically worthless, I can't merge the PR, and I'm still relatively new to all this, so I often miss basic things when reviewing PR's.
So you'll have to wait for someone with write permission to look over this, and merge it

@uri-canva
Copy link
Contributor

ping @SuperSandro2000

@SuperSandro2000 SuperSandro2000 merged commit 3b87c94 into NixOS:master May 29, 2021
@olebedev olebedev deleted the add-bkyml-python-package branch May 30, 2021 05:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add bkyml CLI tool
5 participants