Preserve line breaks in poetry.lock (regression)

[extrwi]

• Poetry version: 1.3.2
• Python version: 3.11.1
• OS version and name: Windows 10

• I am on the latest stable Poetry version, installed using a recommended method.
• I have searched the issues of this repo and believe that this is not a duplicate.
• I have consulted the FAQ and blog for any relevant entries or release notes.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

When generating poetry.lock in mixed environments (e.g. Windows and Linux), CRLF gets replaced by LF and vice versa. This had already been reported and supposedly fixed before (#1488), but probably regressed during the introduction of tomli (#6562). I couldn't find any special treatment for line endings of poetry.lock in poetry's code.

[dimbleby]

there has been no recent change here, poetry continues to use tomlkit to write its lockfiles.

The tomlkit API offers no control so far as I can see over this, so if there's something important to you here you'll need to raise it at that project.

python-poetry/tomlkit#200 and python-poetry/tomlkit#201 look like the last time anyone worried about this

[dimbleby]

Maybe this has changed actually, perhaps reading the lockfile via tomlkit allowed preservation of the existing line endings where reading it through tomli discards that information.

IMO this would be a reason to move further away from tomlkit and write the lockfile with tomli-w (which uses the same line ending on all platforms).

[radoering]

IMO this would be a reason to move further away from tomlkit and write the lockfile with tomli-w (which uses the same line ending on all platforms).

I don't think that's reason enough. We discussed that we don't want to introduce two libraries for writing toml files if there is not a really good reason. And we are tied to tomlkit for modifying pyproject.toml via poetry add. IMO the more promising approach is to extend the tomlkit API to force specific line endings so we can keep the existing line endings (if they are consistent) even if we use tomllib for reading.

When generating poetry.lock in mixed environments (e.g. Windows and Linux), CRLF gets replaced by LF and vice versa.

I suppose core.autocrlf=true on Windows is not an option?

[dimbleby]

I think the right tomlkit-based-direction would be to add an option to force the line endings - so that poetry could always write the same line endings regardless of platform. ie not even trying to preserve existing line endings.

Of course we already discussed the pros and cons of using tomli-w for writing in #6562, eg #6201 is another example of tomlkit being too clever for its own good.

But sure: although this issue would seem to count in tomli-w's favour that's not necessarily enough to tip the balance.

[dimbleby]

re "two toml-writing libraries": I definitely acknowledge that it would look a little bit weird.

On the other hand

• tomli-w is a very small additional dependency - essentially one file, 200 lines
• poetry really does have two different kinds of requirement for writing TOML files
  • pyproject.toml must be comment-preserving and order-preserving and human-friendly,
  • poetry.lock is primarily aimed at machine readers, and we are gathering bugs which show that consistent / repeatable output is important

No doubt it's possible to work through those bugs while sticking with tomlkit, but for me: I would be comfortable with the above as justification enough to change.

[dimbleby]

Those who are developing on both windows and other platforms likely ought to have a suitable .gitattributes in their project anyway; and that ought to deal with this.

[bmitc]

Those who are developing on both windows and other platforms likely ought to have a suitable .gitattributes in their project anyway; and that ought to deal with this.

I develop on all three major operating systems, and I never have to deal with this because I simply only create files with LF endings (configured via my editor which syncs its settings across operating systems) and tell Git to stop messing with line endings (i.e., never change them) such that my files only every have LF. However, this is until I started using Poetry, as any file it creates or updates (apparently) uses a platform specific toggle.