Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lib: Introduce GString #389

Merged
merged 4 commits into from Dec 16, 2018
Merged

lib: Introduce GString #389

merged 4 commits into from Dec 16, 2018

Conversation

@philn
Copy link
Contributor

@philn philn commented Nov 16, 2018

This new module keeps track of CStrings to help avoid allocating new Strings in
gir's generated code.

@philn philn force-pushed the philn:cstringholder branch 2 times, most recently from 6dc40d0 to 7541346 Nov 16, 2018
@philn
Copy link
Contributor Author

@philn philn commented Nov 16, 2018

@sdroege I think this is ready for review finally :)

src/cstringholder.rs Outdated Show resolved Hide resolved
src/cstringholder.rs Outdated Show resolved Hide resolved
@philn philn force-pushed the philn:cstringholder branch from 7541346 to dcba9dc Nov 16, 2018
@GuillaumeGomez
Copy link
Member

@GuillaumeGomez GuillaumeGomez commented Nov 16, 2018

Awesome! 👍

src/cstringholder.rs Outdated Show resolved Hide resolved
pub struct CStringHolder {
slice: Box<CStr>,
ptr: *const c_char,
owned: bool,

This comment has been minimized.

@sdroege

sdroege Nov 16, 2018
Member

I would only do this for owned strings, otherwise it doesn't not really make sense or does it? Do you have an example usage for a non-owned string?

This comment has been minimized.

@philn

philn Nov 16, 2018
Author Contributor

In the unit-tests of the gstreamer subcrate I had to manually create CStringHolders.

This comment has been minimized.

@sdroege

sdroege Nov 16, 2018
Member

You have a link?


pub fn as_str(&self) -> &str {
let bytes = self.slice.to_bytes();
match std::str::from_utf8(bytes) {

This comment has been minimized.

@sdroege

sdroege Nov 16, 2018
Member

Here you could do

  let bytes = slice::from_raw_parts(ptr, len); // calculate the length via strlen in the constructor
  let cstr = CString::from_bytes_with_nul_unchecked(bytes);
  cstr.to_str().unwrap()

This comment has been minimized.

@philn

philn Nov 16, 2018
Author Contributor

That doesn't work, CStr::from_bytes_with_nul_unchecked expects an &[u8] but bytes is &[i8]

This comment has been minimized.

@sdroege

sdroege Nov 16, 2018
Member

Cast the *const c_char to a *const u8 before from_raw_parts. It's i8 or u8 depending on architecture

}
}

impl From<String> for CStringHolder {

This comment has been minimized.

@sdroege

sdroege Nov 16, 2018
Member

This one would involve copying, not sure where it would be useful to use?

This comment has been minimized.

@philn

philn Nov 16, 2018
Author Contributor

Yeah let's remove it, doesn't seem useful indeed :)

src/cstringholder.rs Outdated Show resolved Hide resolved
src/cstringholder.rs Outdated Show resolved Hide resolved
src/cstringholder.rs Outdated Show resolved Hide resolved
src/translate.rs Outdated Show resolved Hide resolved
src/translate.rs Outdated Show resolved Hide resolved
src/lib.rs Show resolved Hide resolved
@EPashkin
Copy link
Member

@EPashkin EPashkin commented Nov 16, 2018

Note: building with rust 1.28.0 produce error:

error[E0658]: access to extern crates through prelude is experimental (see issue #44660)
  --> D:/eap/rust/0/glib\src\cstringholder.rs:44:15
   |
44 |         match std::str::from_utf8(bytes) {
   |               ^^^
@philn
Copy link
Contributor Author

@philn philn commented Nov 16, 2018

Hopefully this code won't be part of the next patch iteration :)

@philn philn force-pushed the philn:cstringholder branch 3 times, most recently from d6d3a81 to 54b0486 Nov 18, 2018
@philn
Copy link
Contributor Author

@philn philn commented Nov 18, 2018

Here's my WIP gstreamer fork adding support for CStringHolder too, https://gitlab.freedesktop.org/philn/gstreamer-rs/commits/master

@cgwalters
Copy link

@cgwalters cgwalters commented Nov 19, 2018

I only glanced at this but it looks like there's some overlap with https://crates.io/crates/c_utf8 ? I started using that crate in coreos/rpm-ostree#1588

@philn
Copy link
Contributor Author

@philn philn commented Nov 20, 2018

This was briefly discussed on IRC, the outcome was that for this specific task we would prefer to avoid adding a new external dependency.

@philn philn force-pushed the philn:cstringholder branch from 54b0486 to e68fc03 Nov 25, 2018
@philn
Copy link
Contributor Author

@philn philn commented Nov 25, 2018

PR updated. I suspect CI might fail again though, will keep an eye on it :)

@philn philn force-pushed the philn:cstringholder branch from e68fc03 to e896713 Nov 25, 2018
Copy link
Member

@EPashkin EPashkin left a comment

Don't fully understand safe use this class or not 😢

};
let ptr = slice.as_ptr();
assert!(!ptr.is_null());
Self { ptr, length: s.len() - 1, owned: false }

This comment has been minimized.

@EPashkin

EPashkin Nov 25, 2018
Member

With impl FromGlibPtrNone<*const c_char> for CStringHolder { this function worried me,
what happened if C string freed?

This comment has been minimized.

@sdroege

sdroege Nov 25, 2018
Member

CStringHolder can only really be implemented in a useful way of from_glib_full

This comment has been minimized.

@EPashkin

EPashkin Nov 25, 2018
Member

IMHO from_glib_borrow also can be safely implemented for signal string parameters.
Problem with string, returned from function as from_glib_none, I don't see safe way to remove copy.

src/cstringholder.rs Outdated Show resolved Hide resolved
src/cstringholder.rs Outdated Show resolved Hide resolved
let bytes = std::slice::from_raw_parts(self.ptr as *const u8, self.length + 1);
let cstr = CStr::from_bytes_with_nul_unchecked(bytes);
cstr.to_str().unwrap()
}

This comment has been minimized.

@EPashkin

EPashkin Nov 25, 2018
Member

Not sure about safety here too.
Also not too many operation for "as_xx" functions? As I remember it preferred be almost 0-cost

src/cstringholder.rs Outdated Show resolved Hide resolved
fn to_glib_none(&'a self) -> Stash<'a, *const c_char, Self> {
unsafe {
let tmp = CString::from_raw(self.ptr as *mut i8);
Stash(tmp.as_ptr(), tmp)

This comment has been minimized.

@EPashkin

EPashkin Nov 25, 2018
Member

CString created for all call to_glib_none?
On owned=true you can just use CStringHolder as storage,
on owned=false depend of lifetime.

This comment has been minimized.

@philn

philn Dec 1, 2018
Author Contributor

I don't understand what you mean there :) The stash would expect a CStringHolder then, not a reference. Also I'm not sure how the storage can be changed depending on owned value.

This comment has been minimized.

@EPashkin

EPashkin Dec 1, 2018
Member

You right, storage can't changed depending on owned value, sorry.

This comment has been minimized.

@EPashkin

EPashkin Dec 1, 2018
Member

Actually it can be something like Result<&CStringHolder, CString> or tupple with 2 options, but not sure about &CStringHoder part

This comment has been minimized.

@philn

philn Dec 1, 2018
Author Contributor

Seems to work with a tuple

src/cstringholder.rs Outdated Show resolved Hide resolved
@EPashkin
Copy link
Member

@EPashkin EPashkin commented Nov 25, 2018

Maybe in not_owned case use CString field as internal storage?

@EPashkin
Copy link
Member

@EPashkin EPashkin commented Nov 25, 2018

@philn Reason to add CStringHolder is minimize memory copy when receiving C string?

@sdroege
Copy link
Member

@sdroege sdroege commented Nov 25, 2018

Maybe in not_owned case use CString field as internal storage?

CString would involve a copy, CStr would be unsafe. Strings that are transfer none we must always directly copy to new memory.

@EPashkin
Copy link
Member

@EPashkin EPashkin commented Nov 25, 2018

Assuming that CStringHolder need to minimize memory copy when receiving C string
IMHO it need implemented like AnyBox: https://github.com/gtk-rs/glib/blob/master/src/boxed.rs#L259-L263 with 3 separated cases:
for from_glib_full it store pointer and drop it,
for from_glib_borrow it store pointer and not drop it,
for from_glib_none it store String or CString with copy data (this is bad because too many string returned that way).

@EPashkin
Copy link
Member

@EPashkin EPashkin commented Nov 25, 2018

Name CStringHolder IMHO misleading, something like GStr(ing)?Holder is better.

src/cstringholder.rs Outdated Show resolved Hide resolved
@sdroege
Copy link
Member

@sdroege sdroege commented Nov 26, 2018

What @EPashkin said in #389 (comment) seems like the best way forward here. I also agree that the name is a bit suboptimal. I would've called it GString.

src/gstring.rs Show resolved Hide resolved
src/gstring.rs Outdated Show resolved Hide resolved
src/gstring.rs Outdated Show resolved Hide resolved
src/gstring.rs Outdated Show resolved Hide resolved
impl From<String> for GString {
#[inline]
fn from(s: String) -> Self {
GString::from(s.as_bytes().to_vec())

This comment has been minimized.

@sdroege

sdroege Dec 10, 2018
Member

This copies, which we maybe don't want?

This comment has been minimized.

@philn

philn Dec 15, 2018
Author Contributor

Not sure we can avoid the copy here.

impl From<Box<str>> for GString {
#[inline]
fn from(s: Box<str>) -> Self {
s.as_bytes().to_vec().into()

This comment has been minimized.

@sdroege

sdroege Dec 10, 2018
Member

This copies, which we maybe don't want?

This comment has been minimized.

@philn

philn Dec 15, 2018
Author Contributor

Same :(

This comment has been minimized.

@sdroege

sdroege Dec 16, 2018
Member

Yes it has to be copied unfortunately as we need to get the \0 at the end. Nothing we can do about that :(

src/gstring.rs Outdated Show resolved Hide resolved
src/gstring.rs Outdated Show resolved Hide resolved
src/gstring.rs Outdated Show resolved Hide resolved
@philn philn force-pushed the philn:cstringholder branch from c084ce4 to a551006 Dec 15, 2018
@philn
Copy link
Contributor Author

@philn philn commented Dec 15, 2018

PR updated!

Copy link
Member

@EPashkin EPashkin left a comment

Thanks, now we have borrows.

impl From<GString> for Box<str> {
#[inline]
fn from(s: GString) -> Self {
Box::from(s.as_str())

This comment has been minimized.

@EPashkin

EPashkin Dec 15, 2018
Member

I worry about result for GString::Owned case, as it will be g_freed.
cc @sdroege

This comment has been minimized.

@philn

philn Dec 15, 2018
Author Contributor

Now I convert to a String and then Boxed str

This comment has been minimized.

impl From<String> for GString {
#[inline]
fn from(s: String) -> Self {
GString::Borrowed(s.as_ptr() as *const c_char, s.len())

This comment has been minimized.

@EPashkin

EPashkin Dec 15, 2018
Member

Also worry about use dropped String data.

impl From<Box<str>> for GString {
#[inline]
fn from(s: Box<str>) -> Self {
GString::Borrowed(s.as_ptr() as *const c_char, s.len())

This comment has been minimized.

@EPashkin

EPashkin Dec 15, 2018
Member

Also worry about use dropped Box data.

src/gstring.rs Outdated Show resolved Hide resolved
@philn philn force-pushed the philn:cstringholder branch from a551006 to 6c90de8 Dec 15, 2018
@sdroege
Copy link
Member

@sdroege sdroege commented Dec 15, 2018

I'll give this another review tomorrow :) Thanks a lot for your patience, @philn


#[derive(Debug)]
pub enum GString {
ForeignOwned(CString),

This comment has been minimized.

@sdroege

sdroege Dec 16, 2018
Member

Where foreign means Rust-allocated? Ok :)

This comment has been minimized.

@philn

philn Dec 16, 2018
Author Contributor

Maybe Native?

src/gstring.rs Show resolved Hide resolved
src/gstring.rs Outdated Show resolved Hide resolved
@sdroege
Copy link
Member

@sdroege sdroege commented Dec 16, 2018

Looks good to me now, just some cosmetic things above.

@EPashkin
Copy link
Member

@EPashkin EPashkin commented Dec 16, 2018

@philn All nightly gtk3.4 fails due missing update gir submodule,
also gir version in version.txt better be "c5498ac"

There 2 legit error in 1.28.0 build:

error[E0658]: access to extern crates through prelude is experimental (see issue #44660)
  --> src/gstring.rs:72:6
   |
72 | impl std::borrow::Borrow<str> for GString {
   |      ^^^
error[E0106]: missing lifetime specifier
   --> src/gstring.rs:229:11
    |
229 | impl From<&CStr> for GString {
    |           ^ expected lifetime parameter
@philn
Copy link
Contributor Author

@philn philn commented Dec 16, 2018

@EPashkin OK! I had to force a clean gir build to have the right version.txt :)

philn added 4 commits Dec 16, 2018
This new module keeps track of C strings to help avoid allocating new Strings in
gir's generated code.
@philn philn force-pushed the philn:cstringholder branch from 6c90de8 to af0c4d6 Dec 16, 2018
@philn
Copy link
Contributor Author

@philn philn commented Dec 16, 2018

OK I think CI will pass this time!

@GuillaumeGomez
Copy link
Member

@GuillaumeGomez GuillaumeGomez commented Dec 16, 2018

Thanks a lot!

@GuillaumeGomez GuillaumeGomez merged commit 6b42b9d into gtk-rs:master Dec 16, 2018
2 checks passed
2 checks passed
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@philn philn deleted the philn:cstringholder branch Dec 16, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

5 participants