Tracking Issue for ASCII trim functions on byte slices

[dbrgn]

Feature gate: #![feature(byte_slice_trim_ascii)]

This is a tracking issue for ASCII trim functions on byte slices.

Public API

The feature adds three new methods to byte slices ([u8]):

• const fn trim_ascii_start(&self) -> &[u8]: Remove leading ASCII whitespace
• const fn trim_ascii_end(&self) -> &[u8]: Remove trailing ASCII whitespace
• const fn trim_ascii(&self) -> &[u8]: Remove leading and trailing ASCII whitespace

For deciding what bytes to treat as whitespace, u8::is_ascii_whitespace is used. See the linked docs for more details.

Examples:

assert_eq!(b" \t hello world\n".trim_ascii_start(), b"hello world\n");
assert_eq!(b"\r hello world\n ".trim_ascii_end(), b"\r hello world");
assert_eq!(b"\r hello world\n ".trim_ascii(), b"hello world");

Steps / History

• Implementation: core: Implement ASCII trim functions on byte slices #93686
• Final comment period (FCP)
• Stabilization PR

Unresolved Questions

• Should the methods be made const by using a pattern matching approach instead of indexing?
• Should corresponding methods be added to str as well?

[nagisa]

If we're adding ascii variants to [u8], we should also add equivalent operations to str at the same time. I feel this is pretty important because otherwise the APIs made available by libstd can come across like a grab bag of random stuff.

It seems reasonable enough to want to trim ASCII whitespace from a &str too (there's already split_ascii_whitespace), and so today one would resort to trim_matches or with this implementation they could convert between [u8] and str. The need to do latter can come across as a particularly frustrating API surface omission.

---

naming bikeshed: might trim_ascii_start and trim_ascii_end be more consistent with other functions already in libstd? There's some prior art in naming of str::split_ascii_whitespace vs str::split_whitespace, {char,u8}::is_ascii_* vs {char,u8}::is_* etc.

[dbrgn]

If we're adding ascii variants to [u8], we should also add equivalent operations to str at the same time. I feel this is pretty important because otherwise the APIs made available by libstd can come across like a grab bag of random stuff.

Since str is validated unicode, trim_start and trim_end could be used, which uses the unicode definition of whitespace. On the other hand, we have char::is_ascii_whitespace, so it should be fairly easy to add these functions to string slices as well. However, I don't really see a use case for this. Why would someone use trim_start_ascii over trim_start?

naming bikeshed: might trim_ascii_start and trim_ascii_end be more consistent with other functions already in libstd? There's some prior art in naming of str::split_ascii_whitespace vs str::split_whitespace, {char,u8}::is_ascii_* vs {char,u8}::is_* etc.

Yes, you have a point there! There's also str::to_lowercase and str::to_ascii_lowercase. I will update the proposal and implementation PR.

[Kage-Yami]

On the other hand, we have char::is_ascii_whitespace, so it should be fairly easy to add these functions to string slices as well. However, I don't really see a use case for this. Why would someone use trim_start_ascii over trim_start?

Naively (without knowing the internals at all), I could think of "performance" as a reason.

That being said, does this unresolved question really need to block this from being stabilised? After all, it's specifically about &[u8], not slices in general. If we can agree to that, then what's stopping this from proceeding to FCP? (I'm having trouble finding documentation for the process for tracking issue FCPs, could only find something for RFCs...)

[BurntSushi]

Both byte slices and string slices can have both Unicode-aware trimming and ASCII-aware trimming. Unicode support needn't be limited to str and ASCII support needn't be limited to [u8].

With that said, I think adding these methods starts us down the path of treating byte slices as byte strings. I think I am in general in favor of that, but I do think we want to proceed deliberately here. It might make sense to block this on a higher level policy question of 1) do we want byte slices to be byte strings? and 2) if so, what do we want to add? (2) is particularly relevant to make sure we end up with good names I think.

[golddranks]

I have a use case where I have to trim a bunch of b'\0's from the end of a byte slice. To me, trim_matches-like methods for slices in all generality would be warranted. I wonder whether people agree with that?

[kornelski]

I think these would be useful (including also on str). I often find myself writing mini-parsers that deal with identifiers and syntaxes that are definitely ASCII-only. I don't want to pay code size increase for proper Unicode processing. I also don't want to create APIs or data formats that unnecessarily allow complexity of all of Unicode whitespace.

[c410-f3r]

What is the status of this feature?

[dbrgn]

With that said, I think adding these methods starts us down the path of treating byte slices as byte strings. I think I am in general in favor of that, but I do think we want to proceed deliberately here.
(...)
do we want byte slices to be byte strings?

I don't think that this a policy decision like this is needed. Byte slices may contain byte strings, or something else. Of course trimming ASCII whitespace results in nonsensical results, if the slices aren't actually ASCII strings, but that's all.

Note that there are already other slice methods that treat bytes as potential strings, for example:

• escape_ascii
• is_ascii
• eq_ignore_ascii_case
• make_ascii_uppercase
• make_ascii_lowercase

What would be needed to move this proposal into FCP?

[okaneco]

I filed a libs ACP, which was accepted, for extending these functions to &str rust-lang/libs-team#313

Implementation PR #118523

• Add trim_ascii_start, trim_ascii_end, and trim_ascii functions to &str for trimming ASCII whitespace
• Add #[inline] to [u8] trim_ascii functions

I was instructed to use this tracking issue.

[okaneco]

Stabilization Report

Implementation History

• Original PR is implemented core: Implement ASCII trim functions on byte slices #93686
• Libs ACP is proposed and accepted for adding the trimming functions to &str ACP: Add ASCII whitespace trimming functions to &str libs-team#313
• PR that extends the trimming functions to &str is merged Add ASCII whitespace trimming functions to &str #118523

API Summary

The following six functions would be stabilized: three functions on &[u8] and three functions on &str.
u8::is_ascii_whitespace is used to determine whitespace.

• Remove leading and trailing ASCII whitespace
  • const fn trim_ascii(&self) -> &[u8]
  • const fn trim_ascii(&self) -> &str
• Remove trailing ASCII whitespace
  • const fn trim_ascii_end(&self) -> &[u8]
  • const fn trim_ascii_end(&self) -> &str
• Remove leading ASCII whitespace
  • const fn trim_ascii_start(&self) -> &[u8]
  • const fn trim_ascii_start(&self) -> &str

Examples:

// &[u8]
assert_eq!(b"\r hello world\n ".trim_ascii(), b"hello world");
assert_eq!(b"\r hello world\n ".trim_ascii_end(), b"\r hello world");
assert_eq!(b" \t hello world\n".trim_ascii_start(), b"hello world\n");
// &str
assert_eq!("\r hello world\n ".trim_ascii(), "hello world");
assert_eq!("\r hello world\n ".trim_ascii_end(), "\r hello world");
assert_eq!(" \t hello world\n".trim_ascii_start(), "hello world\n");

Possibly Unresolved Questions

Question: Does a policy decision need to be made about treating byte slices as byte strings? What methods should be added?

With that said, I think adding these methods starts us down the path of treating byte slices as byte strings. I think I am in general in favor of that, but I do think we want to proceed deliberately here. It might make sense to block this on a higher level policy question of 1) do we want byte slices to be byte strings? and 2) if so, what do we want to add? (2) is particularly relevant to make sure we end up with good names I think.

#94035 (comment)

Response:

I don't think that this a policy decision like this is needed. Byte slices may contain byte strings, or something else. Of course trimming ASCII whitespace results in nonsensical results, if the slices aren't actually ASCII strings, but that's all.

Note that there are already other slice methods that treat bytes as potential strings, for example:

• escape_ascii
• is_ascii
• eq_ignore_ascii_case
• make_ascii_uppercase
• make_ascii_lowercase

#94035 (comment)

For more context, escape_ascii was stabilized in 1.60 and the other four methods were stabilized in 1.23.

---

If this is adequate to start FCP, I can make the stabilization PR.

[vacuus]

Seems like this is more than ready for stabilization.

[jhpratt]

Nominating for T-libs-api, as there seem to be no blockers to stabilization, with a report already being written.

[BurntSushi]

This overall seems reasonable to me. We've clearly already started down the path of "&[u8] is sometimes a byte string." I also think we should continue to that path.

@rfcbot fcp merge

[rfcbot]

Team member @BurntSushi has proposed to merge this. The next step is review by the rest of the tagged team members:

• @Amanieu
• @BurntSushi
• @dtolnay
• @joshtriplett
• @m-ou-se

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

This will be merged soon.

[jonhoo]

🎉 Is this worth an explicit relnotes tag?

[jhpratt]

@jonhoo Definitely; I've added it.