Implement `maturin upload` command #484

ravenexp · 2021-04-03T15:08:15Z

It behaves like twine upload, but is restricted to the python wheels and source distributions.

The upload command mostly reuses the publish command implementation.

konstin

Thank you for working on finishing this!

konstin · 2021-04-08T13:30:10Z

src/main.rs

            if files.is_empty() {
                println!("⚠  Warning: No files given, exiting.");
                return Ok(());
            }
-            let metadata: Vec<Vec<(String, String)>> = files
+
+            // FIXME: All uploaded files are expected to share the build context


I think the easiest way to check this is to read all metadata and check they are equal

I've added a sanity check ensuring that all wheels come from the same Python package.

Readme.md

konstin · 2021-04-08T13:42:05Z

Readme.md

+```
+Uploads the distribution files to the repository (package index)
+
+It behaves like `twine upload`, but is restricted to the python wheels and source distributions.


I wouldn't be so bold to claim that it has the same behaviour as twine. There are a lot of edge cases maturin doesn't (and will never) cover.

I've reworded it to look more like the original comment.

Readme.md

src/main.rs

Split out the maturin build context members from `PublishOpt`. Extract the build context handling code from `upload_ui()`. The existing functionality is not changed in this commit. The trimmed publish options structure will be reused for the new `maturin upload` command.

Uploads a set of wheels and/or source distributions that belong to a single python package. Refuse to upload files from different packages in one command invocation.

ravenexp · 2021-04-09T08:32:23Z

Applied suggestions and rebased on main.

konstin

Thanks again!

konstin · 2021-04-09T12:32:08Z

Note that currently the source distribution upload fails because the PKG-INFO is not in the root directory, but we can fix that later.

ravenexp · 2021-04-09T13:08:30Z

Note that currently the source distribution upload fails because the PKG-INFO is not in the root directory, but we can fix that later.

Does it? I don't make source distributions myself, but I've tried to upload test-data/pyo3_mixed-2.1.1.tar.gz and everything looked fine.

konstin · 2021-04-09T13:17:23Z

I just tried the following on the main branch:

$ rm -rf test-crates/pyo3-mixed/target/wheels/
$ cargo run -- build -m test-crates/pyo3-mixed/Cargo.toml 
    Finished dev [unoptimized + debuginfo] target(s) in 0.09s
     Running `target/debug/maturin build -m test-crates/pyo3-mixed/Cargo.toml`
🍹 Building a mixed python/rust project
🔗 Found pyo3 bindings
🐍 Found CPython 3.6m at python3.6, CPython 3.7m at python3.7, CPython 3.8 at python3.8, CPython 3.9 at python3.9, CPython 3.10 at python3.10
📦 Built source distribution to /home/konsti/maturin/test-crates/pyo3-mixed/target/wheels/pyo3_mixed-2.1.3.tar.gz
   Compiling pyo3 v0.13.2
   Compiling pyo3-mixed v2.1.3 (/home/konsti/maturin/test-crates/pyo3-mixed)
    Finished dev [unoptimized + debuginfo] target(s) in 4.16s
📦 Built wheel for CPython 3.6m to /home/konsti/maturin/test-crates/pyo3-mixed/target/wheels/pyo3_mixed-2.1.3-cp36-cp36m-manylinux_2_24_x86_64.whl
   Compiling pyo3 v0.13.2
   Compiling pyo3-mixed v2.1.3 (/home/konsti/maturin/test-crates/pyo3-mixed)
    Finished dev [unoptimized + debuginfo] target(s) in 4.00s
📦 Built wheel for CPython 3.7m to /home/konsti/maturin/test-crates/pyo3-mixed/target/wheels/pyo3_mixed-2.1.3-cp37-cp37m-manylinux_2_24_x86_64.whl
   Compiling pyo3 v0.13.2
   Compiling pyo3-mixed v2.1.3 (/home/konsti/maturin/test-crates/pyo3-mixed)
    Finished dev [unoptimized + debuginfo] target(s) in 3.93s
📦 Built wheel for CPython 3.8 to /home/konsti/maturin/test-crates/pyo3-mixed/target/wheels/pyo3_mixed-2.1.3-cp38-cp38-manylinux_2_24_x86_64.whl
   Compiling pyo3 v0.13.2
   Compiling pyo3-mixed v2.1.3 (/home/konsti/maturin/test-crates/pyo3-mixed)
    Finished dev [unoptimized + debuginfo] target(s) in 4.02s
📦 Built wheel for CPython 3.9 to /home/konsti/maturin/test-crates/pyo3-mixed/target/wheels/pyo3_mixed-2.1.3-cp39-cp39-manylinux_2_24_x86_64.whl
   Compiling pyo3 v0.13.2
   Compiling pyo3-mixed v2.1.3 (/home/konsti/maturin/test-crates/pyo3-mixed)
    Finished dev [unoptimized + debuginfo] target(s) in 3.99s
📦 Built wheel for CPython 3.10 to /home/konsti/maturin/test-crates/pyo3-mixed/target/wheels/pyo3_mixed-2.1.3-cp310-cp310-manylinux_2_24_x86_64.whl
(reverse-i-search)`upload ': twine ^Cload --help
$ cargo run -- upload test-crates/pyo3-mixed/target/wheels/*
    Finished dev [unoptimized + debuginfo] target(s) in 0.09s
     Running `target/debug/maturin upload test-crates/pyo3-mixed/target/wheels/pyo3_mixed-2.1.3-cp310-cp310-manylinux_2_24_x86_64.whl test-crates/pyo3-mixed/target/wheels/pyo3_mixed-2.1.3-cp36-cp36m-manylinux_2_24_x86_64.whl test-crates/pyo3-mixed/target/wheels/pyo3_mixed-2.1.3-cp37-cp37m-manylinux_2_24_x86_64.whl test-crates/pyo3-mixed/target/wheels/pyo3_mixed-2.1.3-cp38-cp38-manylinux_2_24_x86_64.whl test-crates/pyo3-mixed/target/wheels/pyo3_mixed-2.1.3-cp39-cp39-manylinux_2_24_x86_64.whl test-crates/pyo3-mixed/target/wheels/pyo3_mixed-2.1.3.tar.gz`
💥 maturin failed
  Caused by: Failed to read metadata from source distribution at "test-crates/pyo3-mixed/target/wheels/pyo3_mixed-2.1.3.tar.gz"
  Caused by: Source distribution "test-crates/pyo3-mixed/target/wheels/pyo3_mixed-2.1.3.tar.gz" does not contain a PKG-INFO, but it should

ravenexp · 2021-04-09T13:26:48Z

Hmm, that means the source distribution tarball that the current maturin version builds for test-crates/pyo3-mixed differs from test-data/pyo3_mixed-2.1.1.tar.gz which is used in the read_metadata_for_source_distribution() unit test.

konstin · 2021-04-09T13:32:42Z

I totally missed that! Initially I implemented source distributions with PKG-INFO in the root, then later found out it should be in the main folder. I fixed that in 036fff7 but forgot to update the test.

konstin reviewed Apr 8, 2021

View reviewed changes

ravenexp added 3 commits April 9, 2021 08:41

Implement the maturin upload command

0733ca7

Uploads a set of wheels and/or source distributions that belong to a single python package. Refuse to upload files from different packages in one command invocation.

Add matruin upload Readme and Changelog entries

6e06d58

ravenexp force-pushed the upload branch from aa75a0c to 6e06d58 Compare April 9, 2021 08:00

konstin approved these changes Apr 9, 2021

View reviewed changes

konstin merged commit 179d6f5 into PyO3:main Apr 9, 2021

ravenexp deleted the upload branch April 14, 2021 13:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement `maturin upload` command #484

Implement `maturin upload` command #484

ravenexp commented Apr 3, 2021

konstin left a comment

konstin Apr 8, 2021

ravenexp Apr 9, 2021

konstin Apr 8, 2021

ravenexp Apr 9, 2021

ravenexp commented Apr 9, 2021

konstin left a comment

konstin commented Apr 9, 2021

ravenexp commented Apr 9, 2021

konstin commented Apr 9, 2021

ravenexp commented Apr 9, 2021

konstin commented Apr 9, 2021

Implement maturin upload command #484

Implement maturin upload command #484

Conversation

ravenexp commented Apr 3, 2021

konstin left a comment

Choose a reason for hiding this comment

konstin Apr 8, 2021

Choose a reason for hiding this comment

ravenexp Apr 9, 2021

Choose a reason for hiding this comment

konstin Apr 8, 2021

Choose a reason for hiding this comment

ravenexp Apr 9, 2021

Choose a reason for hiding this comment

ravenexp commented Apr 9, 2021

konstin left a comment

Choose a reason for hiding this comment

konstin commented Apr 9, 2021

ravenexp commented Apr 9, 2021

konstin commented Apr 9, 2021

ravenexp commented Apr 9, 2021

konstin commented Apr 9, 2021

Implement `maturin upload` command #484

Implement `maturin upload` command #484