Tracking Issue for str_split_once

[matklad]

Feature gate: #![feature(str_split_once)]

Public API

impl str {
    pub fn split_once<'a, P: Pattern<'a>>(&'a self, delimiter: P) -> Option<(&'a str, &'a str)>;
    pub fn rsplit_once<'a, P: Pattern<'a>>(&'a self, delimiter: P) -> Option<(&'a str, &'a str)>;
}

Steps / History

• Implementation: Add str::[r]split_once #74707
• Final commenting period (FCP): Tracking Issue for str_split_once #74773 (comment)
• Stabilization PR: Stabilize str_split_once #81940

Unresolved Questions

• Should the method return the separator as well ((&str, &str, &str))? (str::split_once is underpowered #76512)
  • Tracking Issue for str_split_once #74773 (comment)

[KamilaBorowska]

I wonder if it shouldn't return (&str, Option<&str>), I don't think the first element when using split method could ever be None.

[jsdw]

The way I am interpreting the current signature of Option<(&str,&str)> is that either the separator is matched, and so you get a pair of &strs (they might both be empty, but that's ok), or alternately the separator is not matched, in which case the splitting failed and so you get None. This seems consistent with other functions.

Returning (&str, Option<&str>) IMO leaves the meaning of the Option more open to interpretation (do you get None back if we failed to split, or if the second string is zero length?). Having the Option at the top level feels clearer to me (you get your two strings back if it worked, and nothing back if it didnt; I have difficulty working out how else I'd interpret it (which is good; don't want ambiguity)).

[Niedzwiedzw]

I've just used it in one of my projects and it felt very natural, I agree with your reasoning James

[yoshuawuyts]

Tried it today for a quick string matching script, and it felt really good to use. Not returning the separator felt consistent with str::split, which I've used a lot in the past. I think the API as-is is probably what it should be.

edit: probably two things to consider: vec::split_first also exists, and feels similar. Should this use the same name?

[clemenswasser]

I have used it here.

[CryZe]

I seems like the current implementation compiles in a bunch of panic checks, which shouldn't be necessary: https://rust.godbolt.org/z/WTTn6K

[matklad]

Yeah, the impl should be changed to use unchecked indexing (I believe there's no safe API we can use here), but it probably makes sense to do this after stabilization.

[matklad]

I think we should stabilize the API as is:

• many folks find this API useful
• it have been backing in the nightly for some (not too long) time
• the API surface is small enough that it feels unlikely we can improve this further

I think we should not complicate the API to return the ranged matched by the delimiter pattern: this makes the common case more difficult to use, and it goes inline with other pattern-using APIs. We can always add split_once_with_delim later, if we deem it necessary.

I don't think we should change the split_once name -- it feels like it fits the purpose well. split_first for &str would return a (char, &str) I think.

Is anyone willing to submit a stabilization PR? Or are there some concerns I am missing here.

[internetionals]

The only thing that I miss is the more generic:

 pub fn split_n<'a, P: Pattern<'a>>(&'a self, delimiter: P, n: usize) -> Option<[&'a str; n]>;

But that would need const generics and the split into two parts is by far the most common subcase, so a dedicated and easily recognizable function for that wouldn't hurt.

[internetionals]

That is different from the current splitn in that it could be used through destructuring, which helps when parsing simple delimited formats with a freeform last part.

[matklad]

@internetionals this is possible today using itertools:

use itertools::Itertools;

let (a, b, c) = "foo bar baz".split_whitespace().collect_tupple().unwrap();

I believe we'll add an fn collect_array<const N: usize>(self) -> Option<[T; N]> to Iterator in std in some form one day (not sure if there's existing feature for that already).

EDIT: submitted #79659

[matklad]

Actually, before we stabilize this, let's dogfood this a bit! I see there are quite a few opportunities to use this function in rust-lang/rust:

λ rg -F 'splitn(2'

Could someone send a PR replacing those usages with .split_once and checking that everything works?

[Eric-Arellano]

Could someone send a PR replacing those usages with .split_once and checking that everything works?

Hey @matklad, I'd be interested in doing this, but first want to check if it's sensible for a first-time contributor to take this? With checking that it all works, would running through CI be sufficient or you'd want manual testing?

[matklad]

@Eric-Arellano yup, this would be good first issue, especially given that you can do it running only x.py check / x.py test tidy locally (which is loads faster then the whole test suite).

[Eric-Arellano]

Thanks @matklad! I got it set up and am making progress.

FYI: one thing I'm encountering a lot is a difference in API, that split_once models a missing value as "" whereas splitn(2) as None. The call sites do seem to care about the empty string case, so I'm in most places wrapping like this:

let (key, value) = option.split_once('=').unwrap();
let value = if value.is_empty() { None } else { Some(value) };

(As discussed in https://internals.rust-lang.org/t/pre-rfc-none-if-string-is-empty/6742, sounds like we don't want to add sugar to that second line.)

Compared to:

let mut iter = option.splitn(2, '=');
let key = iter.next().unwrap();
let value = iter.next();

Note that my rewrite is missing an assertion that the key is non-empty. I'm not sure if I should be adding assert!() statements; at that point, the rewrite would be 3 LoC and more complicated than before.

While I do agree with the decision to return Option<(&'a str, &'a str)> rather than Option<(&str, Option<&str>)> or Option<(Option<&str>, Option<&str>)>, it does make this less comprehensive than before.

Lmk if it would help to put up my WIP so that you can see this diff.

[KamilaBorowska]

The way I am interpreting the current signature of Option<(&str,&str)> is that either the separator is matched, and so you get a pair of &strs (they might both be empty, but that's ok), or alternately the separator is not matched, in which case the splitting failed and so you get None. This seems consistent with other functions.

Returning (&str, Option<&str>) IMO leaves the meaning of the Option more open to interpretation (do you get None back if we failed to split, or if the second string is zero length?). Having the Option at the top level feels clearer to me (you get your two strings back if it worked, and nothing back if it didnt; I have difficulty working out how else I'd interpret it (which is good; don't want ambiguity)).

The following program:

fn main() {
    println!("{:?}", "".splitn(2, ":").collect::<Vec<_>>());
}

Returns the following string:

[""]

Note how the first element is always here when splitting.

If a string is :: then we get the following.

["", ""]

[matklad]

@Eric-Arellano I think both versions treat the missing value ("foo=") as "". The difference is how they treat missing separator ("foo", without =). One will panic, the other would return (Some("foo"), None).

I don't think we should use split_once API for the case where the separator is optional. It will be useful to survey how many cases in rustc require support for optional values, and how many are fine without it. Posting the links to the source code into this issue would be helpful.

[Eric-Arellano]

Thank you for pointing that out, @matklad and @xfix. See #79809 - there were several cases where the nuance that splitn(2)'s first element is always Some wasn't being used, e.g. unnecessary checks for the first element. That nuance was not obvious to me, and I like that split_once() makes the situation more clear imo.

[rvolgers]

This function is very useful for writing simple ad-hoc parsers, and I find its current signature perfect for that.

I looked at the other suggestions but returning a single enum which shows whether or not the separator was found seems clearly the right choice to me.

I briefly considered the minor variation Result<(&str, &str), &str> (in other words, like the current implementation, but it gives you the input back as the Err variant when the separator is not found), not because I think it's actually a nicer signature, but because I thought it might lead to more ergonomical chaining. After playing with it for a bit though, I don''t think it's worth it.

(I vaguely remember there being some precedent for this kind of API where the Err variant gives you back the rejected input, but I can't for the life of me remember which std API that might have been. I remember noticing it because it felt out of place to me, so perhaps not the greatest precedent, but still.)

TLDR: I think the current signature is the right one and would very much like this to be stabilized.

[rvolgers]

Please do! I missed this somehow, yikes.

[madsmtm]

No sure if this is the right place to ask, but shouldn't we have slice::split_once as well?

[jhpratt]

PR is up.

[matklad]

TOCTOU: stabilization rules changed a bit in the meantime, see https://github.com/rust-lang/rustc-dev-guide/pull/1036/files#diff-c41e6c5c33d6960efeff23f2f54c1d948ea73f64d604deda521e93ae0b4bd79bR75?

@m-ou-se could you kick an FCP here based on this excellent summary?

[m-ou-se]

Unresolved Questions

• Should the method return the separator as well ((&str, &str, &str))? (str::split_once is underpowered #76512)

Has this question been resolved yet?

[matklad]

I would think so (by #74773 (comment)):

• it complicates the common case.
• other pattren based API ('find, split`) don't return the range matched by the pattern.

[m-ou-se]

@rfcbot merge

[rfcbot]

Team member @m-ou-se has proposed to merge this. The next step is review by the rest of the tagged team members:

• @Amanieu
• @BurntSushi
• @KodrAus
• @dtolnay
• @m-ou-se
• @sfackler

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

The RFC will be merged soon.

[nalply]

@rvolgers wrote on 11 Dec 2020:

(I vaguely remember there being some precedent for this kind of API where the Err variant gives you back the rejected input, but I can't for the life of me remember which std API that might have been. I remember noticing it because it felt out of place to me, so perhaps not the greatest precedent, but still.)

Perhaps you meant OsString::into_string?