Adds 'try_parsing' option for Environment #137

joelgallant · 2020-04-29T02:00:20Z

This can be particularly helpful for MY_SERVER_PORT=4334 cargo run

I know that serde's hints should do this because of the deserialize_* functions, but there are cases where it uses deserialize_any (for me this happened when refactoring to a nested struct in an enum). Maybe there's a better serde pattern for that/this?

szarykott · 2020-09-06T13:54:13Z

tests/env.rs

+
+    let values = environment.collect().unwrap();
+
+    assert_eq!(


Maybe a beginner question, as I did not do much dev in this crate, but what is added value of this PR if you still have to use into_int().ok(), so parse it again to retrieve final value?

Yeah, on the surface it looks like that for sure. Only reason I had to do this is that tests in the crate don't expose the int directly, at least as far as I understand it. Seems to be because the Value type exposed doesn't allow access to it's ValueKind. It's possible I could make a test that does deserialization with struct using serde instead?

Not that it's any consolation, but I have used this patch for projects of my own with success.

I think I need to dig deeper in the code to fully grasp the added value then ;) Hope I'll find time to do it, this crate seems really nice and it would be good to ressurect it.

matthiasbeyer · 2021-03-13T10:34:09Z

Are you still interested in this?

If yes, I created a maintenance fork (read here).
Feel free to submit your patches! 😄

joelgallant · 2021-03-13T18:13:56Z

I think @johnb8 would be interested.

johnb8 · 2021-03-17T02:33:34Z

Yeah I'm still interested in getting this merged. @matthiasbeyer looks like you're a maintainer here now so your fork isn't needed anymore?

matthiasbeyer · 2021-03-17T06:23:42Z

Yep, that's right. I'd like to wait for #175 and then ask to rebase this, and go from there.

matthiasbeyer · 2021-03-17T16:35:32Z

Please rebase.

johnb8 · 2021-03-17T18:24:29Z

Rebased on latest master

matthiasbeyer

Overall, this looks good. But, I would like to see more tests for this kind of functionality.

First of all: Could you please split the added test into three test cases? This would be the first step. Don't worry about code duplication - I consider a certain level of code duplication in tests as necessity!
Second: Could you add more tests where the parsing is configured differently or fails entirely? Some where parsing fails, some where types do not match, some where types match and parsing is disabled... and so on!

Either way: Thanks for contributing to this crate!

johnb8 · 2021-03-23T02:07:56Z

Added some more environment variable parsing tests. Let me know if there's any more you want to see, I'm happy to add them. Thanks for stepping in to maintain this crate!

matthiasbeyer

Okay, so another review for you here. I hope you don't mind my concerns.

matthiasbeyer · 2021-03-26T16:06:17Z

src/env.rs

+                if let (true, Ok(parsed)) = (
+                    value.to_lowercase() == "true" || value.to_lowercase() == "false",
+                    string_value.clone().into_bool(),
+                ) {


This if condition is rather hard to parse IMO. Can you rewrite it, so that the first part of the condition is a variable that is used in the if? This way, this could fit into one line, which would make it way easier to parse for humans.

matthiasbeyer · 2021-03-26T16:07:46Z

src/env.rs

-                Value::new(Some(&uri), ValueKind::String(value)),
-            );
+            let value = if self.try_parsing {
+                let string_value = Value::new(Some(&uri), ValueKind::String(value.clone()));


There's a lot of clone()ing involved here. Can we somehow reduce it?

This line clones, and later we clone the string_value variable again in each if-let ... so in the worst-case, there are 5 clone() calls. I bet this can be improved!

johnb8 · 2021-04-12T02:37:45Z

@matthiasbeyer I've implemented both of your suggestions. I don't mind your concerns, definitely agree with the excessive cloning

matthiasbeyer · 2021-04-17T09:50:39Z

Overall I like these changes here and I am looking forward on merging them. Two things, though:

Please rebase to master, because this has merge conflicts, but also because I'd like to see the history cleaned up (don't forget to give attribution to all authors using git trailers).
I am not entirely sure, but maybe we can work and improve the situation about the try-to-parse loop in the Source for Environment implementation.
Right now it tries to parse the string value into bool, i64 and f64, trying the next one if the previous one fails.
I'm not sure how this can be improved, but I guess there is a way with some type inference from the users site.
Because, after all, we know to what we deserialize because the user gives a type to the API, right?

I guess the Source trait has to be altered quite a bit for this change, and I am willing to let the changes from this PR be merged without such an improvement, because it is a good start. And also because altering the Source trait is (IMO) out of scope for this PR. But I'd really like to see some work with type inference here, which would greatly improve runtime of the parsing (three try-parsing steps in worst case vs. none).

Maybe you could add a # Note comment about the runtime of the impl Source for Environment on the try_parsing() function?

Also, if we think this PR one step further, we might give the user the option to add own parsing functions - consider they want to pass something like FOO=bar=42, which is not that uncommon IMO (consider the normal rust debug env vars: RUST_LOG=module=debug. Also lists are possible, or even other things. I don't want to define the parsing for those for the user, but give them the option to provide own parsers.

This can be particularly helpful for `MY_SERVER_PORT=4334 cargo run`

johnb8 · 2021-05-04T16:57:56Z

Rebased to latest master and added performance note to try_parsing.

Yeah I don't think we can do type inference with the way the Source trait is setup now, since collect doesn't know anything about what type the user is expecting. I think fixing that's out of scope for this too.

I'll look into list parsing and try and get something working before we merge this

matthiasbeyer · 2021-05-08T08:18:39Z

Finally got around to review this. Looks good IMO, will merge now.

joelgallant changed the title ~~Adds 'parse_numbers' options for Environment~~ Adds 'try_parsing' option for Environment May 3, 2020

szarykott reviewed Sep 6, 2020

View reviewed changes

johnb8 force-pushed the parse-env-numbers branch from 4289856 to 4c5cc21 Compare March 17, 2021 17:44

matthiasbeyer requested changes Mar 18, 2021

View reviewed changes

matthiasbeyer requested changes Mar 26, 2021

View reviewed changes

joelgallant and others added 9 commits May 4, 2021 10:30

Adds 'parse_numbers' options for Environment

a608884

This can be particularly helpful for `MY_SERVER_PORT=4334 cargo run`

Uses into_* for value conversions and adds bool as an option

a5c3fab

Renames parse_numbers to try_parsing

98b7804

If env value is true or false parse as bool

320b841

more environment variable parsing tests

ab8d006

clearer env bool parse if

2048c03

more predictable env parsing and reduced clones

fc439d0

reduce env parsing repetition

801567b

Add perfromance note to try_parsing

c06915e

johnb8 force-pushed the parse-env-numbers branch from 38f5143 to c06915e Compare May 4, 2021 16:57

Fix env tests, remove unused import

cfd385e

johnb8 force-pushed the parse-env-numbers branch from 8f0eca7 to cfd385e Compare May 4, 2021 17:19

matthiasbeyer approved these changes May 8, 2021

View reviewed changes

matthiasbeyer merged commit 79883ff into mehcode:master May 8, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds 'try_parsing' option for Environment #137

Adds 'try_parsing' option for Environment #137

joelgallant commented Apr 29, 2020 •

edited

szarykott Sep 6, 2020

joelgallant Sep 7, 2020

szarykott Sep 7, 2020

matthiasbeyer commented Mar 13, 2021

joelgallant commented Mar 13, 2021

johnb8 commented Mar 17, 2021

matthiasbeyer commented Mar 17, 2021 •

edited

matthiasbeyer commented Mar 17, 2021

johnb8 commented Mar 17, 2021

matthiasbeyer left a comment

johnb8 commented Mar 23, 2021

matthiasbeyer left a comment

matthiasbeyer Mar 26, 2021

matthiasbeyer Mar 26, 2021

johnb8 commented Apr 12, 2021

matthiasbeyer commented Apr 17, 2021

johnb8 commented May 4, 2021

matthiasbeyer commented May 8, 2021

Adds 'try_parsing' option for Environment #137

Adds 'try_parsing' option for Environment #137

Conversation

joelgallant commented Apr 29, 2020 • edited

szarykott Sep 6, 2020

Choose a reason for hiding this comment

joelgallant Sep 7, 2020

Choose a reason for hiding this comment

szarykott Sep 7, 2020

Choose a reason for hiding this comment

matthiasbeyer commented Mar 13, 2021

joelgallant commented Mar 13, 2021

johnb8 commented Mar 17, 2021

matthiasbeyer commented Mar 17, 2021 • edited

matthiasbeyer commented Mar 17, 2021

johnb8 commented Mar 17, 2021

matthiasbeyer left a comment

Choose a reason for hiding this comment

johnb8 commented Mar 23, 2021

matthiasbeyer left a comment

Choose a reason for hiding this comment

matthiasbeyer Mar 26, 2021

Choose a reason for hiding this comment

matthiasbeyer Mar 26, 2021

Choose a reason for hiding this comment

johnb8 commented Apr 12, 2021

matthiasbeyer commented Apr 17, 2021

johnb8 commented May 4, 2021

matthiasbeyer commented May 8, 2021

joelgallant commented Apr 29, 2020 •

edited

matthiasbeyer commented Mar 17, 2021 •

edited