Skip to content
This repository has been archived by the owner on Jun 8, 2021. It is now read-only.

lib: Introduce GString #389

Merged
merged 4 commits into from Dec 16, 2018
Merged

lib: Introduce GString #389

merged 4 commits into from Dec 16, 2018

Conversation

philn
Copy link
Contributor

@philn philn commented Nov 16, 2018

This new module keeps track of CStrings to help avoid allocating new Strings in
gir's generated code.

@philn philn force-pushed the cstringholder branch 2 times, most recently from 6dc40d0 to 7541346 Compare November 16, 2018 14:02
@philn
Copy link
Contributor Author

philn commented Nov 16, 2018

@sdroege I think this is ready for review finally :)

@GuillaumeGomez
Copy link
Member

Awesome! 👍

src/cstringholder.rs Outdated Show resolved Hide resolved
pub struct CStringHolder {
slice: Box<CStr>,
ptr: *const c_char,
owned: bool,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would only do this for owned strings, otherwise it doesn't not really make sense or does it? Do you have an example usage for a non-owned string?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the unit-tests of the gstreamer subcrate I had to manually create CStringHolders.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have a link?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


pub fn as_str(&self) -> &str {
let bytes = self.slice.to_bytes();
match std::str::from_utf8(bytes) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here you could do

  let bytes = slice::from_raw_parts(ptr, len); // calculate the length via strlen in the constructor
  let cstr = CString::from_bytes_with_nul_unchecked(bytes);
  cstr.to_str().unwrap()

Copy link
Contributor Author

@philn philn Nov 16, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That doesn't work, CStr::from_bytes_with_nul_unchecked expects an &[u8] but bytes is &[i8]

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cast the *const c_char to a *const u8 before from_raw_parts. It's i8 or u8 depending on architecture

}
}

impl From<String> for CStringHolder {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one would involve copying, not sure where it would be useful to use?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah let's remove it, doesn't seem useful indeed :)

src/cstringholder.rs Outdated Show resolved Hide resolved
src/cstringholder.rs Outdated Show resolved Hide resolved
src/cstringholder.rs Outdated Show resolved Hide resolved
src/translate.rs Outdated Show resolved Hide resolved
src/translate.rs Outdated Show resolved Hide resolved
src/lib.rs Show resolved Hide resolved
@EPashkin
Copy link
Member

Note: building with rust 1.28.0 produce error:

error[E0658]: access to extern crates through prelude is experimental (see issue #44660)
  --> D:/eap/rust/0/glib\src\cstringholder.rs:44:15
   |
44 |         match std::str::from_utf8(bytes) {
   |               ^^^

@philn
Copy link
Contributor Author

philn commented Nov 16, 2018

Hopefully this code won't be part of the next patch iteration :)

@philn philn force-pushed the cstringholder branch 3 times, most recently from d6d3a81 to 54b0486 Compare November 18, 2018 08:50
@philn
Copy link
Contributor Author

philn commented Nov 18, 2018

Here's my WIP gstreamer fork adding support for CStringHolder too, https://gitlab.freedesktop.org/philn/gstreamer-rs/commits/master

@philn
Copy link
Contributor Author

philn commented Nov 18, 2018

@cgwalters
Copy link

I only glanced at this but it looks like there's some overlap with https://crates.io/crates/c_utf8 ? I started using that crate in coreos/rpm-ostree#1588

@philn
Copy link
Contributor Author

philn commented Nov 20, 2018

This was briefly discussed on IRC, the outcome was that for this specific task we would prefer to avoid adding a new external dependency.

@philn
Copy link
Contributor Author

philn commented Nov 25, 2018

PR updated. I suspect CI might fail again though, will keep an eye on it :)

Copy link
Member

@EPashkin EPashkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't fully understand safe use this class or not 😢

};
let ptr = slice.as_ptr();
assert!(!ptr.is_null());
Self { ptr, length: s.len() - 1, owned: false }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With impl FromGlibPtrNone<*const c_char> for CStringHolder { this function worried me,
what happened if C string freed?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CStringHolder can only really be implemented in a useful way of from_glib_full

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO from_glib_borrow also can be safely implemented for signal string parameters.
Problem with string, returned from function as from_glib_none, I don't see safe way to remove copy.

src/cstringholder.rs Outdated Show resolved Hide resolved
src/cstringholder.rs Outdated Show resolved Hide resolved
let bytes = std::slice::from_raw_parts(self.ptr as *const u8, self.length + 1);
let cstr = CStr::from_bytes_with_nul_unchecked(bytes);
cstr.to_str().unwrap()
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about safety here too.
Also not too many operation for "as_xx" functions? As I remember it preferred be almost 0-cost

src/cstringholder.rs Outdated Show resolved Hide resolved
fn to_glib_none(&'a self) -> Stash<'a, *const c_char, Self> {
unsafe {
let tmp = CString::from_raw(self.ptr as *mut i8);
Stash(tmp.as_ptr(), tmp)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CString created for all call to_glib_none?
On owned=true you can just use CStringHolder as storage,
on owned=false depend of lifetime.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand what you mean there :) The stash would expect a CStringHolder then, not a reference. Also I'm not sure how the storage can be changed depending on owned value.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You right, storage can't changed depending on owned value, sorry.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually it can be something like Result<&CStringHolder, CString> or tupple with 2 options, but not sure about &CStringHoder part

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems to work with a tuple

src/cstringholder.rs Outdated Show resolved Hide resolved
@EPashkin
Copy link
Member

Maybe in not_owned case use CString field as internal storage?

@EPashkin
Copy link
Member

@philn Reason to add CStringHolder is minimize memory copy when receiving C string?

@sdroege
Copy link
Member

sdroege commented Nov 25, 2018

Maybe in not_owned case use CString field as internal storage?

CString would involve a copy, CStr would be unsafe. Strings that are transfer none we must always directly copy to new memory.

@EPashkin
Copy link
Member

Assuming that CStringHolder need to minimize memory copy when receiving C string
IMHO it need implemented like AnyBox: https://github.com/gtk-rs/glib/blob/master/src/boxed.rs#L259-L263 with 3 separated cases:
for from_glib_full it store pointer and drop it,
for from_glib_borrow it store pointer and not drop it,
for from_glib_none it store String or CString with copy data (this is bad because too many string returned that way).

@EPashkin
Copy link
Member

Name CStringHolder IMHO misleading, something like GStr(ing)?Holder is better.

src/cstringholder.rs Outdated Show resolved Hide resolved
@sdroege
Copy link
Member

sdroege commented Nov 26, 2018

What @EPashkin said in #389 (comment) seems like the best way forward here. I also agree that the name is a bit suboptimal. I would've called it GString.

src/gstring.rs Outdated Show resolved Hide resolved
src/gstring.rs Outdated Show resolved Hide resolved
src/gstring.rs Outdated Show resolved Hide resolved
src/gstring.rs Outdated Show resolved Hide resolved
src/gstring.rs Outdated
impl From<String> for GString {
#[inline]
fn from(s: String) -> Self {
GString::from(s.as_bytes().to_vec())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This copies, which we maybe don't want?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure we can avoid the copy here.

impl From<Box<str>> for GString {
#[inline]
fn from(s: Box<str>) -> Self {
s.as_bytes().to_vec().into()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This copies, which we maybe don't want?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same :(

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it has to be copied unfortunately as we need to get the \0 at the end. Nothing we can do about that :(

src/gstring.rs Outdated Show resolved Hide resolved
src/gstring.rs Outdated Show resolved Hide resolved
src/gstring.rs Outdated Show resolved Hide resolved
@philn
Copy link
Contributor Author

philn commented Dec 15, 2018

PR updated!

Copy link
Member

@EPashkin EPashkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, now we have borrows.

src/gstring.rs Outdated
impl From<GString> for Box<str> {
#[inline]
fn from(s: GString) -> Self {
Box::from(s.as_str())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I worry about result for GString::Owned case, as it will be g_freed.
cc @sdroege

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now I convert to a String and then Boxed str

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

src/gstring.rs Outdated
impl From<String> for GString {
#[inline]
fn from(s: String) -> Self {
GString::Borrowed(s.as_ptr() as *const c_char, s.len())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also worry about use dropped String data.

src/gstring.rs Outdated
impl From<Box<str>> for GString {
#[inline]
fn from(s: Box<str>) -> Self {
GString::Borrowed(s.as_ptr() as *const c_char, s.len())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also worry about use dropped Box data.

src/gstring.rs Outdated Show resolved Hide resolved
@sdroege
Copy link
Member

sdroege commented Dec 15, 2018

I'll give this another review tomorrow :) Thanks a lot for your patience, @philn


#[derive(Debug)]
pub enum GString {
ForeignOwned(CString),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where foreign means Rust-allocated? Ok :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe Native?

src/gstring.rs Outdated Show resolved Hide resolved
@sdroege
Copy link
Member

sdroege commented Dec 16, 2018

Looks good to me now, just some cosmetic things above.

@EPashkin
Copy link
Member

@philn All nightly gtk3.4 fails due missing update gir submodule,
also gir version in version.txt better be "c5498ac"

There 2 legit error in 1.28.0 build:

error[E0658]: access to extern crates through prelude is experimental (see issue #44660)
  --> src/gstring.rs:72:6
   |
72 | impl std::borrow::Borrow<str> for GString {
   |      ^^^
error[E0106]: missing lifetime specifier
   --> src/gstring.rs:229:11
    |
229 | impl From<&CStr> for GString {
    |           ^ expected lifetime parameter

@philn
Copy link
Contributor Author

philn commented Dec 16, 2018

@EPashkin OK! I had to force a clean gir build to have the right version.txt :)

@philn
Copy link
Contributor Author

philn commented Dec 16, 2018

OK I think CI will pass this time!

@GuillaumeGomez
Copy link
Member

Thanks a lot!

@GuillaumeGomez GuillaumeGomez merged commit 6b42b9d into gtk-rs:master Dec 16, 2018
@philn philn deleted the cstringholder branch December 16, 2018 14:19
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants