Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cargo add: Nicer TOML Files #15

Closed
3 tasks done
killercup opened this issue Oct 11, 2015 · 16 comments
Closed
3 tasks done

cargo add: Nicer TOML Files #15

killercup opened this issue Oct 11, 2015 · 16 comments

Comments

@killercup
Copy link
Owner

killercup commented Oct 11, 2015

  • Don't mess up Cargo.toml structure too much, e.g. keep the ordering
  • Don't remove comments.
  • Use inline tables whenever possible, e.g. clippy = {version = "0.0.19", optional = true}
@euclio
Copy link
Contributor

euclio commented Oct 21, 2015

+1 to this. Running using cargo add on https://github.com/Hoverbear/rust-rosetta/blob/master/Cargo.toml was a minor disaster (comments removed, binaries reordered, etc.). The toml linked above might make an interesting test case.

@flying-sheep
Copy link

another annoyance:

  • my single quoted strings get converted to double quoted ones (ideally cargo-add would detect the predominate string style and use it)

we should really parse this into an extensive AST and operate on that level instead of using hashmaps without stylistic and meta information.

i didn’t need that yet, but i can also imagine a complex Cargo.toml having some sections divided by blank lines that should be preserved

@killercup
Copy link
Owner Author

Like @rgardner mentioned in #57, there was some effort by @vosen to implement some of this in toml-rs/toml-rs#64.

It might be a good idea to use that code as a starting point for a new crate that parses toml into an AST that maintains more stylistic information but that also offers a "high level" interface that can be used like a nested hashmap to easily insert/update values.

@flying-sheep
Copy link

toml-rs was written when the rust ecosystem wasn’t as far so maybe someone would have fun just implementing a new TOML parser in nom or peresil or so.

toml-rs is fine for reading TOML, but not for editing it.

@killercup
Copy link
Owner Author

Good idea, @flying-sheep. There are always people asking for suggestions on what to implement to learn Rust. A TOML parser that can write nice files is probably both easy to start and enough of a challenge to spend some time with. I think I'm going to post this in a few places 😄

@flying-sheep
Copy link

yup, toml is really easy to parse. no whitespace, no ambiguities. therefore no state and no backtracking.

whatever you choose (parser combinator or classic tokenizer + AST parser), it shouldn’t be much of a problem

@vosen
Copy link

vosen commented Nov 27, 2015

Hi guys. This discussion got a little bit fragmented. I've summed up the current state of toml-rs/toml-rs#64 on Discourse (tl;dr It's not dead). But I think this place is slightly more fitting for planning what to actually do. Since my goal is the same as yours (having TOML parsing that doesn't mangle Cargo.toml, part of PistonDevelopers/VisualRust#196), I want to invite you to join me in toml-rs/toml-rs#64 (or take over). From what I can tell this code is furthest ahead when it comes to handling TOML gracefully: ordering, whitrespace, comments are all preserved. What's missing is a public interface for querying/editing/adding/removing elements.

@joelself
Copy link

tomllib (I renamed toml_parser) is out now at version 0.1.1. It is currently limited to parsing documents, getting values, setting/changing values and getting subkeys of a key. This should be enough to get cargo-bump and the cargo list command for cargo-edit (by using the get_children method on the "dependencies" key and then get_value on each of the child keys). For the next set of features I'm targeting the cargo add and cargo rm commands.

@killercup
Copy link
Owner Author

That sounds awesome, @joelself! I'll have a look at tomllib (← cool name, btw) when I'm home next week. Maybe I'll even have some time to help out! 😃

@steveklabnik
Copy link

steveklabnik commented May 18, 2017

I would still like to see this happen someday...

bors bot added a commit that referenced this issue Dec 17, 2017
181: Start world domination r=killercup a=ordian

cc #15, #69.

This is an experiment with [toml_edit](https://github.com/ordian/toml_edit) to figure out what the right API (LeopoldArkham/Molten#8, cc @LeopoldArkham) is for the upcoming integration with Molten, but also can be useful by itself while Molten is not yet ready.

Basically, all we need is an `Item`, which can represent anything (`None`, `Table`, `InlineTable`, `String`, ...), `IndexMut<&str, Output=Item>` implementation for Item and a bunch of downcasting methods (`is_table`, `as_table`, ...).

Deleting an item is just replacing it with `None`.
@flying-sheep
Copy link

I think the three checkboxes can be ticked, but cargo-edit still adds double-quoted version specifiers to my single-quoted .toml files. Should I file a new issue and let this be closed or wat do? 😄

@berkus
Copy link

berkus commented Jul 14, 2018

@flying-sheep best you open a new one for that!

@ordian
Copy link
Collaborator

ordian commented Oct 27, 2019

Fixes by #181 and #312.

@flying-sheep feel free to open an issue for respecting single-quoted style.

@ordian ordian closed this as completed Oct 27, 2019
@flying-sheep
Copy link

I did! #217

@eugenesvk
Copy link

It seems that cargo upgrade still removes some comments, e.g. in this file the commant of a specific package was removed

[dependencies]
# very useful comment remains
wasm-bindgen	= "0.2.80" # very useful comment REMOVED!!!

Is there a way for the upgrade to just change the version number and leave everything as is?

@epage
Copy link
Collaborator

epage commented Jul 11, 2022

Just double checked that cargos cargo add doesn't have this problem

See rust-lang/cargo#10849

epage added a commit to epage/cargo that referenced this issue Jul 13, 2022
A comment on killercup/cargo-edit#15 had me worried that `cargo add` was
deleting comments now.  It appears that isn't the case for inline
tables.

Standard tables however do delete comments.  The work to make sure they
don't conflicts with another need.  When changing the source, we delete
the old source fields and append the new which can cause some formatting
to be carried over unnecessarily.

For example, what would normally look like
```toml
cargo-list-test-fixture-dependency = { optional = true, path = "../dependency", version = "0.0.0" }
```
When fixed to preserve comments with my naive solution looks like
```toml
cargo-list-test-fixture-dependency = { optional = true , path = "../dependency", version = "0.0.0" }
```
Note that `optional = true` used to be last, so space separating it and
`}` was kept, now separating it and `,`.

More work will be needed to get this into an ideal state but we can at
least have confidence with inline tables for now.
bors added a commit to rust-lang/cargo that referenced this issue Jul 13, 2022
test(add): Ensure comments are preserved

A comment on killercup/cargo-edit#15 had me worried that `cargo add` was
deleting comments now.  It appears that isn't the case for inline
tables.

Standard tables however do delete comments.  The work to make sure they
don't conflicts with another need.  When changing the source, we delete
the old source fields and append the new which can cause some formatting
to be carried over unnecessarily.

For example, what would normally look like
```toml
cargo-list-test-fixture-dependency = { optional = true, path = "../dependency", version = "0.0.0" }
```
When fixed to preserve comments with my naive solution looks like
```toml
cargo-list-test-fixture-dependency = { optional = true , path = "../dependency", version = "0.0.0" }
```
Note that `optional = true` used to be last, so space separating it and
`}` was kept, now separating it and `,`.

More work will be needed to get this into an ideal state but we can at
least have confidence with inline tables for now.
Hezuikn pushed a commit to Hezuikn/cargo that referenced this issue Sep 22, 2022
A comment on killercup/cargo-edit#15 had me worried that `cargo add` was
deleting comments now.  It appears that isn't the case for inline
tables.

Standard tables however do delete comments.  The work to make sure they
don't conflicts with another need.  When changing the source, we delete
the old source fields and append the new which can cause some formatting
to be carried over unnecessarily.

For example, what would normally look like
```toml
cargo-list-test-fixture-dependency = { optional = true, path = "../dependency", version = "0.0.0" }
```
When fixed to preserve comments with my naive solution looks like
```toml
cargo-list-test-fixture-dependency = { optional = true , path = "../dependency", version = "0.0.0" }
```
Note that `optional = true` used to be last, so space separating it and
`}` was kept, now separating it and `,`.

More work will be needed to get this into an ideal state but we can at
least have confidence with inline tables for now.
Hezuikn pushed a commit to Hezuikn/cargo that referenced this issue Sep 22, 2022
A comment on killercup/cargo-edit#15 had me worried that `cargo add` was
deleting comments now.  It appears that isn't the case for inline
tables.

Standard tables however do delete comments.  The work to make sure they
don't conflicts with another need.  When changing the source, we delete
the old source fields and append the new which can cause some formatting
to be carried over unnecessarily.

For example, what would normally look like
```toml
cargo-list-test-fixture-dependency = { optional = true, path = "../dependency", version = "0.0.0" }
```
When fixed to preserve comments with my naive solution looks like
```toml
cargo-list-test-fixture-dependency = { optional = true , path = "../dependency", version = "0.0.0" }
```
Note that `optional = true` used to be last, so space separating it and
`}` was kept, now separating it and `,`.

More work will be needed to get this into an ideal state but we can at
least have confidence with inline tables for now.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

10 participants