Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

peotry version doesn't bump the value in __version__ #144

Closed
ksamuel opened this issue May 26, 2018 · 75 comments
Closed

peotry version doesn't bump the value in __version__ #144

ksamuel opened this issue May 26, 2018 · 75 comments
Labels
area/cli Related to the command line kind/feature Feature requests/implementations

Comments

@ksamuel
Copy link

ksamuel commented May 26, 2018

$ poetry version 0.1.1
Bumping version from 0.1.0 to 0.1.1
$ grep version pyproject.toml 
version = "0.1.1"
$ grep version */__init__.py
src/__init__.py:__version__ = '0.1.0'
$ poetry --version
Poetry 0.9.0

I don't know if it's intended or not. A way to do that safely is to parse the root __init__.py, detect __version__, back it up, extract the ast, bump the version string and replace it, then extract the new ast and compare the result. If both ast are the same, except for the version, the file semantics have been preserved. Otherwise, rollback the change and display an error message stating we can't bump the version safely.

@sdispater
Copy link
Member

At the moment this is not possible no.

poetry version will only bump the version in pyproject.toml. This is not intended and that's definitely something I plan on adding.

@sdispater sdispater added enhancement area/cli Related to the command line labels May 27, 2018
@Victor-Savu
Copy link

I really wish that the version field in the .toml ended up superseding the __version__ string in __init__.py. This would reduce the burden of having to keep multiple places in the code in sync.

@ksamuel
Copy link
Author

ksamuel commented May 27, 2018 via email

@Victor-Savu
Copy link

Victor-Savu commented May 27, 2018

@ksamuel Thanks for pointing that out! I use that setuptools feature as well :) It feels a bit backwards that it needs to be referenced in two places, but I suppose that is just my ignorance with respect to the benefits of having programmatic access to the version of a package. I can kind of imagine a scenario, but I have never seen a real-world example of where that was used (again, it's just my inexperience, I am not criticizing such a practice :) )

@jgirardet
Copy link
Contributor

you may use that

import toml
from pathlib import Path

def get_version():
   path = Path(__file__).resolve().parents[1] / 'pyproject.toml'
   pyproject = toml.loads(open(str(path)).read())
   return pyproject['tool']['poetry']['version']

__version__ = get_version()

@ksamuel
Copy link
Author

ksamuel commented May 28, 2018 via email

@digitalresistor
Copy link
Contributor

Personally I think storing a __version__ in __init__.py seems duplicative. You can use pkg_resources to retrieve the version for the installed package using:

pkg_resources.get_distribution("packagename").version

If used with:

poetry develop

This will work even when developing locally (it requires the egg-info).

@ksamuel
Copy link
Author

ksamuel commented May 30, 2018 via email

@madig
Copy link

madig commented Jun 22, 2018

Using pkg_resources also introduces an install-time dependency on setuptools, no?

@vlcinsky
Copy link
Contributor

Currently workaround can be to use bump2version package (maintained fork of bumpversion).

For my package poetry_demo with default poetry_demo/__init__.py and tests/test_poetry_demo.py content I have following .bumpversion.cfg file (use of setup.cfg file name is also possible):

[bumpversion]
current_version = 0.1.0
commit = True
tag = True

[bumpversion:file:pyproject.toml]
search = version = "{current_version}"
replace = version = "{new_version}"

[bumpversion:file:poetry_demo/__init__.py]
search = __version__ = '{current_version}'
replace = __version__ = '{new_version}'

[bumpversion:file:tests/test_poetry_demo.py]
search = __version__ == '{current_version}'
replace = __version__ == '{new_version}'

To bump the version:
$ bump2version patch
and all version instances will become 0.1.1.

Provided configuration will also commit the modification and create tag. This (and many more features) are configurable.

A little problem is, that currently it requires extra configuration file .bumpversion.cfg, but there is an issue c4urself/bump2version#42 planning to use pyproject.toml

The tool seems rather mature, has many features and is used for quite some time.

To implement bumping versions in poetry, it may serve at least as nice inspiration.

@pmav99
Copy link
Contributor

pmav99 commented Dec 24, 2018

Apart from poetry itself, __version__ bumping should be implemented in the scaffold project that is being created by poetry new too. In that particular case, I think it would make sense to stick to the standard library instead of adding a toml depenedency, so something like this could be used:

import configparser

parser = configparser.ConfigParser()
parser.read("pyproject.toml")
__version__ = parser["tool.poetry"]["version"]

__all__ = ["__version__"]

@madig
Copy link

madig commented Dec 24, 2018

One thing to keep in mind with the automatisms like that is that they will access the disk on importing the library. Code execution on import is not that uncommon in Python, but not desireable to keep down import times and minimize side effects.

Does the pyproject.toml file even ship with wheels?

@digitalresistor
Copy link
Contributor

@pmav99 you can't do that unless you also ship pyproject.toml with your WHEEL for instance.

@pmav99 pmav99 mentioned this issue Apr 13, 2019
2 tasks
@MRigal
Copy link

MRigal commented Apr 16, 2019

I also think this is something very important.

Another example is when you integrate it with a tool like Sentry, you are willing to know for which version it applies. Currently, I couldn't find a proper way to include it except writing a __version__ string in the __init__.py

@willingham
Copy link

Depending on your build process, one solution is to use a shell script for bumping the version instead of running poetry version explicitly. I prefer using makefiles, so I added these two rules to mine to solve this problem.

poetryversion:
	poetry version $(version) 
	
version: poetryversion
	$(eval NEW_VERS := $(shell cat pyproject.toml | grep "^version = \"*\"" | cut -d'"' -f2))
	sed -i "" "s/__version__ = .*/__version__ = \"$(NEW_VERS)\"/g" x12_utils/__init__.py
	

This command can be run as follows: make version=1.2.3 version

If you dislike specifying the version(type) that way, this could just as easily be implemented as a shell script such as:

poetry version $1
new_vers=$(cat pyproject.toml | grep "^version = \"*\"" | cut -d'"' -f2)
sed -i "" "s/__version__ = .*/__version__ = \"${new_vers}\"/g" x12_utils/__init__.py

debugtalk added a commit to httprunner/httprunner that referenced this issue Jul 18, 2019
this will be improved after poetry fixed issue python-poetry/poetry#144
@NargiT
Copy link

NargiT commented Jul 30, 2019

In my case reading from pyproject.toml does not make sense as I ship a CLI and myapp version has to be able to answer to this question without pyproject.toml.

@brycedrennan brycedrennan added kind/feature Feature requests/implementations and removed enhancement labels Aug 9, 2019
@JnyJny
Copy link
Contributor

JnyJny commented Aug 27, 2019

I would also like to see poetry version bump the version in:

  • pyproject.toml
  • <package_name>/__init__.py
  • tests/*.py # any tests which have a version literal in them

Since pyproject.toml is already handled, it seems something like using astor might be a good choice for safely modifying python source.

JnyJny added a commit to JnyJny/poetry that referenced this issue Sep 5, 2019
Added a dry-run option to version command and logic to
replace current version strings with next version in:
- pyproject.toml
- python source ( assignments & comparisons )
- markdown | restructured text | plain text
@JnyJny
Copy link
Contributor

JnyJny commented Sep 5, 2019

I've prototyped this feature and would like some feedback. I consider this an 80% solution that will make most people happy.

In addition to updating pyproject.toml, the new command will update version strings in specific python contexts; specifically assignments and comparisons. It will also do a global replacement on text files with specific suffixes:

  • *.[mM]*[dD]* for Markdown
  • *.[rR][sS][tT] for ReStructured Text
  • *.[tT]*[xX][tT] for plain text files

The updated command syntax is:

poetry version <rule> [--dry-run]

@dmontagu
Copy link

dmontagu commented Sep 5, 2019

@JnyJny If I understand correctly, your current approach for text files is just replacing the version string wherever it occurs in any file with a matching extension. I feel like that has a relatively high potential for false positives, e.g. if you list required versions for dependencies anywhere in your documentation.

I haven't dug into your implementation for python files at all but in principle I think it would be less likely to cause problems there (given you can analyze code more deeply than documentation, and code is less likely to include unrelated version strings).

Also, I think this would be a nice quality-of-life improvement even if it was only applied to __init__.pys (and given the potential for false positives, that may be preferable anyway).

@ttamg
Copy link
Contributor

ttamg commented Jul 11, 2021

Reading this thread, it seems to me that for now it's best to keep both the pyproject.toml version and the package.__version__.

So my process is just to stop me accidentally forgetting to update one.

I've got a simple test that checks that those two are in alignment and fails if not.

import toml
from pathlib import Path
import my_package

def test_versions_are_in_sync():
    """Checks if the pyproject.toml and package.__init__.py __version__ are in sync."""

    path = Path(__file__).resolve().parents[2] / "pyproject.toml"
    pyproject = toml.loads(open(str(path)).read())
    pyproject_version = pyproject["tool"]["poetry"]["version"]

    package_init_version = my_package.__version__
    
    assert package_init_version == pyproject_version

Seems to do the job without having to resort to some of the convoluted solutions above which may be hard to maintain

@nikhiljohn10
Copy link

Here is my suggestion.

First, poetry new package create __init__.py as an empty file. In addition, create __version__.py next to it. But the content is:

#v0.1.0

( its a hashtag. pun intended. xD )
Anyway,

  1. Since it's a python file, it will be packed while building the package.
  2. It is safe to edit by poetry version along with pyproject.toml
  3. Even a simple script like VERSION="0.1.0"; echo "#v$VERSION" > package/__version__.py can bump version
  4. The #v will be the marker for the regular expression to search and find it even if the file contains any other content.

The following code can be used inside setup.py to obtain the version

#!/usr/bin/env python3

import re
from pathlib import Path

HERE = Path(__file__).parent
version_file = HERE / "CloudflareAPI/__version__.py"
version_file = version_file.resolve(strict=True).read_text()
version = re.search(r'#v(.+)', version_file).group(1)
print(version)

It can also be used in test files where the version is required.
(I do not feel like using importlib.metadata)

Any thoughts?

@deftio
Copy link

deftio commented Jan 23, 2022

I'm coming in here in Jan 2022, and I'm actually just looking for the canonical answer. Where is the single source of truth spot to put the version string (or where in the docs as poetry version doesn't seem to be the place).

I ran across this:
https://pythonrepo.com/repo/tiangolo-poetry-version-plugin-python-package-dependency-management

which seems to show that others have this issue and are solving it with plugins but if I missed this a pointer in the right direction is appreciated.

TanakitIntR added a commit to TanakitIntR/httprunner4 that referenced this issue Feb 2, 2022
this will be improved after poetry fixed issue python-poetry/poetry#144
@adam-grant-hendry
Copy link

I'm coming in here in Jan 2022, and I'm actually just looking for the canonical answer. Where is the single source of truth spot to put the version string (or where in the docs as poetry version doesn't seem to be the place).

I ran across this: https://pythonrepo.com/repo/tiangolo-poetry-version-plugin-python-package-dependency-management

which seems to show that others have this issue and are solving it with plugins but if I missed this a pointer in the right direction is appreciated.

I'd also like some clarification. I'm not sure what to follow. There are still several different plugins floating around:

  1. https://github.com/bulv1ne/poetry-bumpversion
  2. https://github.com/tiangolo/poetry-version-plugin
  3. https://github.com/mtkennerly/poetry-dynamic-versioning

I'm fine to use importlib.metadata, so long as I can also use it for local packages I am developing (develop = true). Has anyone tested this?

@adam-grant-hendry
Copy link

adam-grant-hendry commented Feb 27, 2022

If you package your project, the version number is integrated into it. Then use importlib_metadata (since python 3.8 as build-in importlib.metadata) to access the version number during run time if needed.

Does this work for packages in development mode? Can you point me to tests?

@deftio
Copy link

deftio commented Mar 23, 2022

Well FWIW in jsonvice I solved the problem by adding:

and this in the master source code file:

import pkg_resources # part of setuptools
version = pkg_resources.require("jsonvice")[0].version

Now there is only one place the version is only specified in pyproject.toml and the string is available at runtime so the program can self-report its version number when queried.

@monim67
Copy link

monim67 commented May 19, 2022

I have published poetry-bumpversion plugin to update version in other files when poetry upgrades the package version. To update __version__ just install the plugin and add the following to your pyproject.toml file.

[tool.poetry_bumpversion.file."your_package/__init__.py"]

@svaningelgem
Copy link

FWIW I'm using this:

import importlib.metadata
__version__= importlib.metadata.version('<package name>')

Works fine 👍

@NargiT
Copy link

NargiT commented Mar 29, 2023

I also switched to importlib.metadata
But I had to change my release workflow, before it was fully automated as I was able to choose in advance the version number during development.

My Take Away for importlib.metadata

Pros:
No more version number in files
No more required to bump/commit/push a new version during development or release workflow
Version is only available in git tags and inside package
relatively easy to automate for patches releases even with branch protection

Cons:
hard to automated for minor and major release

Bonus

Here is the github action snippet code to trigger a package publish for version 1.0.0
My Workflow

  • develop
  • create pr
  • merge
  • release

I use manually gh cli gh release create 1.0.0 --generate-notes or directly from github.com (it generates the same event)

name: publish package

on:
  push:
    tags:
    - '**'

jobs:
  publish:
    runs-on: ubuntu-20.04
    steps:
    - uses: actions/checkout@v3
      name: Checkout code
    - uses: actions/setup-python@v4
      name: Set up Python
      with:
        python-version: '3.10'
    - name: Install tools
      run: |
        pip install --upgrade pip
        pip install poetry
    - name: Publish distribution 📦 to PyPI
      run: |
        poetry config repositories.pypi ${{ secrets.PYPI_REPOSITORY }}
        poetry version ${{ github.ref_name }}
        poetry publish --build -r pypi -u ${{ secrets.PYPI_PUBLISH_USERNAME }} -p ${{ secrets.PYPI_PUBLISH_TOKEN }}

susodapop added a commit to databricks/dbt-databricks that referenced this issue Aug 24, 2023
We now specify the package version in a single place: pyproject.toml so
it's not necessary to dynamically figure it out in setup.py.

Copped the new contents of __version__.py from the Poetry github issue
tracker here:

python-poetry/poetry#144 (comment)

Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com>
benc-db pushed a commit to databricks/dbt-databricks that referenced this issue Sep 14, 2023
Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com>

(2/x) Move package description from setup.py

Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com>

(3/x) Move package author from setup.py to pyproject.toml

Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com>

(4/x) Move classifiers from setup.py to pyproject.toml

Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com>

squash with 56005dc

Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com>

(5/x) Move long description from setup.py to pyproject.toml

Poetry will take the configured readme and set the long_description to
its contents and the long_description_content_type to "text/markdown"
automatically.

Ref: python-poetry/poetry#1979

Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com>

(6/x) Move python dependency spec from setup.py to pyproject.toml

Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com>

(7/x) Move package version from setup.py to pyproject.toml

We now specify the package version in a single place: pyproject.toml so
it's not necessary to dynamically figure it out in setup.py.

Copped the new contents of __version__.py from the Poetry github issue
tracker here:

python-poetry/poetry#144 (comment)

Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com>

(8/x) Move namespace package inclusion from setup.py to pyproject.toml

Note that per poetry docs [^1]

> Poetry is clever enough to detect Python subpackages.
>
> Thus, you only have to specify the directory where your root package
  resides.

[1]: https://python-poetry.org/docs/pyproject/

Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com>

(9/x) Move homepage from setup.py to pyproject.toml

Also add the repository metadata which appears in the Pypi sidebar

Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com>

(10/x) Entirely remove zip_safe config

It's obsolete:
https://setuptools.pypa.io/en/latest/deprecated/zip_safe.html and not
required by poetry
python-poetry/poetry#928 (comment)

Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com>

(11/x) Move project dependencies from setup.py and requirements.txt
(where they were duplicated) into pyproject.toml

This adds the locked dependencies in poetry.lock

Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com>

(12/x) Entirely remove setup.py

Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com>

(13/x) Move dev dependencies from dev-requirements.txt

Re-lock the dependencies

Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com>

(14/x) Completely remove MANIFEST.in

It's not needed by poetry

Ref:
https: //stackoverflow.com/questions/64654860/replacing-manifest-in-with-pyproject-toml
Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com>

(15/x) Entirely remove /scripts

This script is to be replaced with `poetry build`

Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com>

I compared a build from `poetry build` to the output from `python
setup.py sdist bdist_wheel` and then ran a diff against the generated
files. Here's what I found. I added line numbers that I can reference
below.

```
1 Only in dbt-databricks-1.6.1-pypi: MANIFEST.in
2 Files dbt-databricks-1.6.1-pypi/PKG-INFO and dbt_databricks-1.6.1-poetry/PKG-INFO differ
3 Files dbt-databricks-1.6.1-pypi/dbt/adapters/databricks/__version__.py and dbt_databricks-1.6.1-poetry/dbt/adapters/databricks/__version__.py differ
4 Only in dbt-databricks-1.6.1-pypi: dbt_databricks.egg-info
5 Only in dbt_databricks-1.6.1-poetry: pyproject.toml
6 Only in dbt-databricks-1.6.1-pypi: setup.cfg
7 Only in dbt-databricks-1.6.1-pypi: setup.py
```

1: OK see e55e261

2: OK differences are because poetry writes an extra classifier and
includes the new homepage

3: OK difference is because we now automatically fetch the package version set by pyproject.toml

4: OK we're not uploading eggs anymore. They're about to be deprecated
by pypi anyway
https://blog.pypi.org/posts/2023-06-26-deprecate-egg-uploads/

5: OK to add pyproject.toml to the output distribution. This is standard for poetry.

6: OK this is egg-related. See diff 4.

7: OK since we don't use setup.py anymore

Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com>

moving linting to nox (and eventually everything from tox)

unit tests to nox

migrating integration tests

typo

use cluster http path

Replace build_dist.sh to perform build checks

Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com>

Preliminary GitHub Action to publish to test-pypi with poetry

Signed-off-by: Jesse Whitehouse <jesse.whitehouse@databricks.com>
Copy link

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 29, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area/cli Related to the command line kind/feature Feature requests/implementations
Projects
None yet
Development

No branches or pull requests