Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add getpwnam and related functions #864

Closed
wants to merge 17 commits into from

Conversation

ctrlcctrlv
Copy link

@ctrlcctrlv ctrlcctrlv commented Feb 23, 2018

This PR closes #862.

Implemented functions:

  • As User::query:
    • getpwnam
    • getpwuid
  • As Group::query:
    • getgrgid
    • getgrnam
  • As Users iterator:
    • getpwent
    • setpwent
    • endpwent
  • As Groups iterator:
    • getgrent
    • setgrent
    • endgrent

I wanted to implement getgrent, setgrent and endgrent, but they are missing from libc. (See rust-lang/libc#923 )

@ctrlcctrlv ctrlcctrlv mentioned this pull request Feb 25, 2018
Copy link

@kristate kristate left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your commit!

src/unistd.rs Outdated
fn from(pw: libc::passwd) -> Passwd {
unsafe {
Passwd {
name: CString::new(CStr::from_ptr(pw.pw_name).to_bytes()).unwrap(),

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if it is smart to use CStrings here (as opposed to Strings).
Does the library use CStrings in any other places?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took a quick search of the code, and it does seem that we are using String and not CString as an interface. Please switch to String.

Ref:

Thanks!

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much for your review. I did have what I believed to be a good reason to use CStrings. If this function is ran on a non UTF8 system, the fields, especially gecos, can be in a non UTF8 encoding such as SHIFTIJIS. from_utf8_lossy will cause data loss. Should String still be used in light of this?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Nix support any operating systems that run in non-UTF8 mode?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, many. FreeBSD, Linux etc can all use /etc/passwd files that are in the user's non-UTF8 locale. I hate to keep mentioning SHIFT-JIS, but because in my day job I'm dealing with a lot of SJIS data and SJIS systems, I am accutely aware of this problem. Japanese sysadmins, even in 2018, in many cases still prefer SJIS because it is byte for byte around 2/3 to 1/2 the size of UTF-8. So that means an uncompressed database of Japanese text data is double the size if it's UTF-8. Therefore, many many Unix systems in Japan use SJIS everywhere, including the passwd and group files.

unsafe fn members(mem: *mut *mut c_char) -> Vec<CString> {
let mut ret = vec![];

for i in 0.. {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like it has too many moving parts.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't think of another way to decode a **char

src/unistd.rs Outdated
pub fn getpwent() -> Result<Option<Passwd>> {
let pw = unsafe { libc::getpwent().as_ref() };

if errno::errno() != 0 {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it smart to check the errno in this way?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It works, but I do have to admit that I did not really understand too well the errno.rs module.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

libc::getpwent also uses a static object. It also isn't very Rusty, because it's basically the next method for an implied iterator. I don't think it should be directly exposed by Nix. Instead, I think you should create an iterator class that will internally use libc::getpwent_r to iterate through the password file. Something like this:

struct PwEntries {
    ...
}
impl PwEntries {
    fn new() -> Self {
        libc::setpwent();
        ...
    }
}
impl Iterator for PwEntries {
    type item = Passwd;
    fn next(&mut self) -> Option<Passwd> {
        ...
    }
}
impl Drop for PwEntries {
    fn drop(&mut self) {
        libc::endpwent();
    }
}

Unfortunately, this still isn't 100% reentrant since those setpwent and endpwent functions still store their file descriptors in static variables. But I don't see a better option.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

libc::getpwent_r does not exist, so I suppose this is on hold until I can figure out how to get it added to libc...

src/unistd.rs Outdated
#[inline]
pub fn endpwent() -> Result<()> {
unsafe { libc::endpwent() };
if errno::errno() != 0 {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto.

src/unistd.rs Outdated
#[derive(Debug, Clone, PartialEq)]
pub struct Passwd {
/// username
pub name: CString,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it important to mark these all as pub?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think so. . . the whole point of these functions is to get data from this struct.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing that other Nix modules do is create a Newtype to wrap the libc struct, and define accessors for that. See for example nix::sys::Kevent or nix::sys::UtsName. I think that pattern would work well here. You'd do something like:

pub struct Passwd(libc::passwd);
impl Passwd {
    pub fn name(&self) -> &str {
        ...
    }
}

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@asomers That seems very Java-like to me and not necessary for these structs. Furthermore, some day I may also add putpwent, which would require a user to build this struct. putpwent is basically how the shell utilitty adduser works. http://man7.org/linux/man-pages/man3/putpwent.3.html

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Accessor functions may seem unnecessary to you, but converting C's struct passwd to a different layout seems unnecessary to me. As long as we're dealing with C and Rust datatypes that have incompatible layouts, we're going to need some kind of conversion. I like the accessor method version better, because that way we won't waste time converting fields that the user doesn't care about. We'll probably save RAM too.

src/unistd.rs Outdated
/// Gets a user DB entry based on username (see
/// [getpwnam(3)](http://pubs.opengroup.org/onlinepubs/9699919799/functions/getpwnam.html))
pub fn getpwnam<S: Into<Vec<u8>>>(name: S) -> Result<Option<Passwd>> {
let pw = unsafe { libc::getpwnam(CString::new(name).unwrap().as_ptr()).as_ref() };
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The man page says that applications should explicitly set errno to 0 before calling getpwnam.

src/unistd.rs Outdated
/// Gets a user DB entry based on username (see
/// [getpwnam(3)](http://pubs.opengroup.org/onlinepubs/9699919799/functions/getpwnam.html))
pub fn getpwnam<S: Into<Vec<u8>>>(name: S) -> Result<Option<Passwd>> {
let pw = unsafe { libc::getpwnam(CString::new(name).unwrap().as_ptr()).as_ref() };
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

libc::getpwnam is not thread safe because it uses a static buffer. In fact, it's worse than that, because the static buffer is malloced on demand, and libc will free it and reallocate it if it isn't large enough. That means that multithreaded programs can use-after-free if they call getpwnam without synchronization. You should use libc::getpwnam_r instead.

src/unistd.rs Outdated
/// [getpwuid(3)](http://pubs.opengroup.org/onlinepubs/9699919799/functions/getpwuid.html))
#[inline]
pub fn getpwuid(uid: Uid) -> Result<Option<Passwd>> {
let pw = unsafe { libc::getpwuid(uid.0).as_ref() };
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same concerns here about thread-safety and initializing errno.

src/unistd.rs Outdated
pub fn getpwent() -> Result<Option<Passwd>> {
let pw = unsafe { libc::getpwent().as_ref() };

if errno::errno() != 0 {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

libc::getpwent also uses a static object. It also isn't very Rusty, because it's basically the next method for an implied iterator. I don't think it should be directly exposed by Nix. Instead, I think you should create an iterator class that will internally use libc::getpwent_r to iterate through the password file. Something like this:

struct PwEntries {
    ...
}
impl PwEntries {
    fn new() -> Self {
        libc::setpwent();
        ...
    }
}
impl Iterator for PwEntries {
    type item = Passwd;
    fn next(&mut self) -> Option<Passwd> {
        ...
    }
}
impl Drop for PwEntries {
    fn drop(&mut self) {
        libc::endpwent();
    }
}

Unfortunately, this still isn't 100% reentrant since those setpwent and endpwent functions still store their file descriptors in static variables. But I don't see a better option.

src/unistd.rs Outdated
/// Gets a group DB entry based on group name (see
/// [getgrnam(3)](http://pubs.opengroup.org/onlinepubs/9699919799/functions/getgrnam.html))
#[inline]
pub fn getgrnam<S: Into<Vec<u8>>>(name: S) -> Result<Option<Group>> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto all the comments for the getpwent family of functions.

src/unistd.rs Outdated
fn from(pw: libc::passwd) -> Passwd {
unsafe {
Passwd {
name: CString::new(CStr::from_ptr(pw.pw_name).to_bytes()).unwrap(),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does Nix support any operating systems that run in non-UTF8 mode?

src/unistd.rs Outdated
#[derive(Debug, Clone, PartialEq)]
pub struct Passwd {
/// username
pub name: CString,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing that other Nix modules do is create a Newtype to wrap the libc struct, and define accessors for that. See for example nix::sys::Kevent or nix::sys::UtsName. I think that pattern would work well here. You'd do something like:

pub struct Passwd(libc::passwd);
impl Passwd {
    pub fn name(&self) -> &str {
        ...
    }
}

ctrlcctrlv added a commit to ctrlcctrlv/libc that referenced this pull request Mar 1, 2018
bors added a commit to rust-lang/libc that referenced this pull request Mar 3, 2018
Add passwd/group APIs needed for nix-rust/nix#864

Hope I did this right. I only added platforms I could personally test. . .

cc: @gnzlbg
@ctrlcctrlv
Copy link
Author

I was able to get a bunch of functions added to libc in rust-lang/libc#934 .

I am now working on getting the APIs I added there into nix again. Hoping no more libc changes are needed.

@ctrlcctrlv
Copy link
Author

@kristate @asomers @Susurrus I would very much appreciate it if y'all could read what I wrote in the commit message here 3612526

I'm worried that this problem will cause all my work on this PR to go to waste, very stressed since I've poured hours into this already 😢

@ctrlcctrlv
Copy link
Author

I don't know why the test system is doing this:

   Compiling libc v0.2.38 (https://github.com/rust-lang/libc#936e16bc)
   Compiling libc v0.2.37

Why it immediately compiles old version after new? I got a PR into libc for the express purpose of getting this PR into nix, so using 0.2.38 is an absolute must. My branch will not compile without it.

@ctrlcctrlv
Copy link
Author

😟

Cargo.toml Outdated
@@ -12,7 +12,7 @@ exclude = [
]

[dependencies]
libc = { git = "https://github.com/rust-lang/libc" }
libc = { version = ">=0.2.38", git = "https://github.com/rust-lang/libc", branch = "master" }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is unnecessary and can be reverted. Cargo will always try to use the latest version when you use a "git" specification. If the included libc version in your workspace seems behind, then you should run cargo update.

src/unistd.rs Outdated
/// Users::new()
/// .map(
/// |pw| println!("{}\t{}",
/// pw.name.to_string_lossy().into_owned(),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you really need the into_owned?

src/unistd.rs Outdated
/// .map(
/// |pw| println!("{}\t{}",
/// pw.name.to_string_lossy().into_owned(),
/// format!("{}", pw.uid)))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

format is basically what the println is doing. I think you can shorten this entire line to pw.uid)

src/unistd.rs Outdated
///
/// ```
/// use nix::unistd::Users;
/// eprintln!("Users on this system:\nname\tuid");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you run cargo test, does all this stuff get printed? If so, you must silence it.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even with nocapture, nothing is printed.

   Doc-tests nix

running 1 test
test src/unistd.rs - unistd::Users (line 2300) ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 47 filtered out

src/unistd.rs Outdated
#[cfg(not(target_os = "android"))]
pub struct Users(pub usize);

impl Users {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line seems unlikely to compile on android

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup you're right, I thought the compiler was smarter than it is, since the struct doesn't exist when compiling for Android.

src/unistd.rs Outdated
type Item = Passwd;
fn next(&mut self) -> Option<Passwd> {

let mut cbuf = vec![0i8; self.0];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0u8 would be more appropriate than 0i8, I think. A 0i8 array can't store any text encodings that use the high bit.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't, type mismatch. libc's type c_char = i8.

src/unistd.rs Outdated
#[derive(Debug, Clone, PartialEq)]
pub struct Passwd {
/// username
pub name: CString,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Accessor functions may seem unnecessary to you, but converting C's struct passwd to a different layout seems unnecessary to me. As long as we're dealing with C and Rust datatypes that have incompatible layouts, we're going to need some kind of conversion. I like the accessor method version better, because that way we won't waste time converting fields that the user doesn't care about. We'll probably save RAM too.

src/unistd.rs Outdated
let i = unsafe { libc::getpwent_r(&mut pwd, cbuf.as_mut_ptr(), self.0, &mut res) };

match i {
0 if !res.is_null() => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need a way to distinguish between "entry not found" (errno==0) and "not enough buffer space" (errno==ERANGE)

src/unistd.rs Outdated
let mut pwd: libc::passwd = unsafe { mem::zeroed() };
let mut res = ptr::null_mut();

let i = unsafe { libc::getpwent_r(&mut pwd, cbuf.as_mut_ptr(), self.0, &mut res) };
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't forget to initialize errno to zero before calling this function.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be certain, you mean I have to make a call to Errno::clear, right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that'll do.

src/unistd.rs Outdated
let mut grp: libc::group = unsafe { mem::zeroed() };
let mut res = ptr::null_mut();

let i = unsafe { libc::getgrent_r(&mut grp, cbuf.as_mut_ptr(), self.0, &mut res) };
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should probably initialize errno=0 here too.

src/unistd.rs Outdated
0 if !res.is_null() => {
unsafe { Some(Group::from(*res)) }
},
_ => None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to handle ERANGE here too.

@asomers
Copy link
Member

asomers commented Mar 3, 2018

We already have a way of serializing tests that use global variables. Look in tests/test.rs at the bottom. There are several mutexes there. You should add another one to protect the global variables used by the getpwent family of functions.

@ctrlcctrlv
Copy link
Author

Oh okay thanks, I get it, I'll make the tests use that when I get home from church, as well as deal with your other concerns 😃

@ctrlcctrlv
Copy link
Author

Alright, got a bunch of work done today, but as you can see from the failing builds there is still more to do :-)

Thank you everyone for your patience with me while I sort this out. Just realized I'm going to have to open another libc PR as my original accidentally neglected FreeBSD.

@ctrlcctrlv
Copy link
Author

FreeBSD build will stop failing when rust-lang/libc#939 is merged.

@ctrlcctrlv
Copy link
Author

ctrlcctrlv commented Mar 6, 2018

Hey @asomers , really curious what you think of this API for querying instead of just exposing it under the names of the POSIX functions. It's getting late here, if you like it I'll make the Group version of it tomorrow.

@ctrlcctrlv
Copy link
Author

P.S. Anybody know what's up with https://travis-ci.org/nix-rust/nix/jobs/349841588 ? I can't get test_users_iterator_smallbuf to do that on my Linux VM.

@asomers
Copy link
Member

asomers commented Mar 6, 2018

The query API could certainly work. One problem is that it's not specific enough. Why have enum values that can't be used to look up a user? Why have values that can't be used to look up a group? Better to have one enum for User queries, and one for Group queries. That way you'll catch usage problems at compile time. Another problem is that you aren't handling "user not found" errors correctly. You shouldn't return an Errno in that case.

@ctrlcctrlv
Copy link
Author

Happy to report my build is now passing :D

If you have no more concerns it's ready to be merged. I responded to or fixed every concern in the previous reviews to my knowledge (hope I didn't miss anything).

@asomers @Susurrus @kristate

Cargo.toml Outdated
@@ -12,7 +12,7 @@ exclude = [
]

[dependencies]
libc = { git = "https://github.com/rust-lang/libc" }
libc = { git = "https://github.com/rust-lang/libc", branch = "master" }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The branch specification isn't necessary. Please revert changes to this file.

src/unistd.rs Outdated
@@ -20,12 +20,22 @@ pub use self::pivot_root::*;
#[cfg(any(target_os = "android", target_os = "freebsd",
target_os = "linux", target_os = "openbsd"))]
pub use self::setres::*;
#[cfg(not(any(target_os = "android",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you have to exclude so many architectures? Shouldn't this be architecture independent?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really sure -- my builds were failing due to missing functions on those architectures. Could be a libc issue, but it's also not unheard of for GNU libc to not have every function in every platform.

#[derive(Debug, Clone, PartialEq, Default)]
pub struct User {
/// username
pub name: String,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why make some fields String if others must be CString? The inconsistency is weird.

Copy link
Author

@ctrlcctrlv ctrlcctrlv Mar 8, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because it's guaranteed that the fields I made String will conform to NAME_REGEX. gecos ("real name") has no such guarantee.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh, makes sense. Could you add a comment to that effect?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this seems done as well since d4d5a00.

#[derive(Debug, Clone, PartialEq)]
pub struct Group {
/// group name
pub name: String,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the concerns about shift-jis force this structure to use CString too?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed above.

src/unistd.rs Outdated
}

#[derive(Debug)]
/// Query a group. The buffer size variants should not be needed 99% of the time; the default
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doc comments, here and elsewhere, should begin with a short description. Then there should be a blank line followed by the detailed description.

///
/// ```
///
/// This iterator should not be used in different threads without synchronization; while doing so
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please put this in a # Safety section.

Copy link

@ArniDagur ArniDagur Jul 6, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems done


/// This iterator can be used to get all of the groups on the system. For example:
///
/// ```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please put this in an # Examples section.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems done

///
/// ```
///
/// This iterator should not be used in different threads without synchronization; while doing so
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use a # Safety section here.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems done

#[test]
fn test_getpwnam() {
let res = User::query( UserQuery::Name("root".to_string()) ).unwrap();
assert!(res.unwrap().uid == Uid::from_raw(0));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't a very good test. That field would've been zero even if getpwnam did nothing, because you initialize the struct to zero. Can you find a more interesting field to check?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I transformed this into a doctest. About to push it.

target_arch = "s390x")))]
fn test_users_iterator() {
#[allow(unused_variables)]
let m = ::USER_GRP_ITER_MTX.lock().expect("Mutex got poisoned by another test");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change the name to _m and you won't need the #[allow(unused_variables)]

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I copied this pattern from elsewhere in the file. I think that the reason that other tests do it this way is because maybe the compiler will optimize it out, but I did not test.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure the reason that other tests do it this way is because they were added back when the minimum compiler version was 1.14.0 or something, and it didn't understand that leading underscores denoted unused variables.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that would be best left to another PR, since I'd be editing many other unrelated tests, which I'd be happy to work on after this one.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add the underscore and get rid of the #[allow( as part of this PR. You don't need to clean up existing examples, but since this is new code it should do this the right way from the start.

@ctrlcctrlv
Copy link
Author

@asomers Any further concerns?

src/unistd.rs Outdated
/// should be more than adequate. Only use the buffer size variants if you receive an ERANGE error
/// without them, or you know that you will be likely to.
///
/// # Example
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/Example/Examples for consistency with the rest of the project

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@asomers What consistency? mkfifo, getcwd etc use "Example"...this makes sense because it's a lone example, in their case as well as mine.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not saying we're perfect, but currently Nix has 30 "Examples" compared to 9 "Example". Best stick to the predominant style.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@asomers
Copy link
Member

asomers commented Mar 23, 2018

There are still two things that concern me:

  1. You have some really big unsafe blocks. Can you shrink the scope of those blocks to the absolute minimum?
  2. You have some from functions that take their arguments by value, even though it looks like a reference would work fine. Unless it's absolutely necessary, it's better to take arguments by reference. That gives the caller more flexibility.

@ctrlcctrlv
Copy link
Author

@asomers f47800c

Hoping this will be the final commit. 🤞

@asomers
Copy link
Member

asomers commented Mar 27, 2018

Why did you use raw pointers instead of references? Dereferencing raw pointers is unsafe.

@ctrlcctrlv
Copy link
Author

@asomers Because I only ever convert libc::passwd and libc::group from raw pointers, and that is the only way to get their data. A reference to one of them would just be a reference to a raw pointer...I think. You have to dereference them, I just changed where that happens.

@asomers
Copy link
Member

asomers commented Mar 28, 2018

Yeah, but you moved the raw pointer dereference into part of the public API! And in a safe function, no less! That means that any user can pass us a *const libc::passwd (which they can create in safe code by casting), and we'll dereference it and potentially crash. That's not ok. We need to control when and where that pointer is dereferenced. It should only happen when we have control of how the pointer was created, and we know that it's valid.

@ctrlcctrlv
Copy link
Author

@asomers OK, I understand. Thank you very much for the explanation. I'm sorry if I'm annoying you. This is my first major commit to a Rust library. I'll think about how to handle this. Maybe using the From trait at all is unwise, I think probably this should not be public.

Copy link
Contributor

@Susurrus Susurrus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A lot of minor style changes.

src/unistd.rs Outdated
}
}

#[derive(Debug)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These should always come after the doccomments.

Also is there a reason you didn't derive the other commonly-derived traits as well?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved below the doc-comments, and added the full set of Debug, Copy, Clone, Eq, PartialEq, Hash, Default, for UserQuery, GroupQuery, Users, Groups (where applicable, not all can be Copy, enums can't be Default)

src/unistd.rs Outdated
let mut pwd: libc::passwd = unsafe { mem::zeroed() };
let mut res = ptr::null_mut();

let i = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you use a more descriptive variable name here? i is almost always a counter variable as well, so this is confusing.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed i to error for all four occurences

src/unistd.rs Outdated
};

match i {
0 => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you're just doing an if/else like this I'd recommend doing that instead of a match, which is a little more heavyweight.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed it to an if/else.

src/unistd.rs Outdated
target_arch = "s390x")))]
mod usergroupiter {
use libc;
use libc::{c_char};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can just do use libc::{self, c_char};

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

src/unistd.rs Outdated
/// `setpwent` and `endpwent`, it is very likely that iterators running in different threads will
/// yield different numbers of items.
#[derive(Debug)]
pub struct Users(pub usize);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any reason the user of this API needs to set the internal value? If not, you should make it private and add a getter to get the value instead. This is more common with Rust types and used elsewhere in nix.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

src/unistd.rs Outdated
/// Representation of a group, based on `libc::group`
#[derive(Debug, Clone, PartialEq)]
pub struct Group {
/// group name
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first letters in all these doccomments should be capitalized.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

src/unistd.rs Outdated
gid: Gid::from_raw((*pw).pw_gid),
#[cfg(not(any(target_os = "linux", target_os = "android")))]
class: CString::new(CStr::from_ptr((*pw).pw_class).to_bytes()).unwrap(),
#[cfg(not(any(target_os = "linux", target_os = "android")))]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The OSes need to be alphabetized. Additionally, these sould not use a negation and instead explicitly list all OSes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

alphabetical order done; avoiding negations seems quite verbose to me, and cfg(not( seems to occur more througout the existing part as well.

src/unistd.rs Outdated
@@ -2136,3 +2146,424 @@ mod setres {
Errno::result(res).map(drop)
}
}


#[derive(Debug, Clone, PartialEq, Default)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Attributes should got under the doccomments.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

src/unistd.rs Outdated
/// group ID
pub gid: Gid,
#[cfg(not(target_os = "android"))]
/// user information
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doccomments go above the attributes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

src/unistd.rs Outdated
}

pub trait Queryable<Q> where Self: ::std::marker::Sized {
fn query(q: Q) -> Option<Result<Self>>;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This probably should be public, yes?

Additionally, what's with the weird Option<Result<_>> type?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Option<Result<_>> type makes sense in this case due to the lower level API.

If you receive Some(Ok(_)), it means that there was a user found and its data fit in the buffer and is now wrapped in the result.
If you receive a Some(Err(_)), it means that there was a user found but its data did not fit in the buffer, and you should recall with one of the WithBufsize variants with a higher size.
If you receive a None, it means that no user was found.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if we shouldn't use our own return type to specify these different states. I haven't thought hard about this, so maybe one doesn't exist, but glancing at this I'd think we could have a more ergonomic API here.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With all due respect, I'm not sure what that would look like and don't think there's anything wrong with the current return type...maybe an alternate return type would look better, but using standard types like this is certainly more ergonomic IMO.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a better API would be for the caller to supply an optional bufsize. If not provided, then Nix will choose a bufsize, retrying with a larger buffer if necessary. Also, why is there a need for the Queryable trait? I get that User and Group have a similar API. But if there's never going to be any generic function with an indiscriminate Queryable type in its signature, then there's no need to define a trait. I might do it like this:

pub enum GroupQuery<'a> {
    Gid(gid),
    Name(&'a str)
}
impl Group {
    fn query(q: GroupQuery, bufsize: Option<usize>) -> Option<Result<Self>>;
}

let res = Group::query(GroupQuery::Name("root"), Some(2048)).unwrap();

However, it would be even simpler to eliminate the GroupQuery type entirely, replacing it with two separate constructors:

impl Group {
    fn from_gid(gid: Gid, bufsize: Option<usize>) -> Option<Result<Self>>;
    fn from_name(name: &str, bufsize: Option<usize>) -> Option<Result<Self>>;
}

let res = Group::from_name("root", Some(2048)).unwrap();

What @Susurrus was suggesting would be to create a bespoke return type to get rid of the double unwrap wart. It would look like this:

enum GroupResult {
    /// No group found
    None,
    /// An error was encountered while searching
    Err(Error),
    Some(Group)
}

Such a type could still support most of the common operators that are defined for std::option::Option and std::result::Result, i.e. unwrap, is_some, is_err, is_none, etc.

@ctrlcctrlv
Copy link
Author

OK, sorry everyone, the priorities of my company changed so I hadn't had much time to work on this, but the microservice I wrote which uses this API is back under scrutiny again so I'll be soon working on getting this commit into nix since we do not want to use private branches of modules if we can help it for important parts of our code.

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Copy link
Member

@asomers asomers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That looks like a step in the right direction. It certainly reduced duplication.

src/unistd.rs Outdated
// Trigger the internal buffer resizing logic of `Vec` by requiring
// more space than the current capacity.
unsafe { cbuf.set_len(cbuf.capacity()); }
cbuf.reserve(1);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extending by only one byte at a time could take awhile. It would be better to double the storage instead. That's what the C library does. Also, you shouldn't extend the buffer ad infinitum. At some point you should give up and return ERANGE. The C library does that at 1MB.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can add a top limit, however, the same strategy is used inside std. For example: https://doc.rust-lang.org/src/alloc/vec.rs.html#949

Also, inside nix code it is also used. See: https://github.com/nix-rust/nix/blob/master/src/unistd.rs#L579-L583

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The link from inside of Vec is very different. In that case, it really is only 1 additional element's worth of space that is needed. Here, an unknown amount of additional space is needed. That's why I suggest doubling the storage. Nix's getcwd() method should probably do the same.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The link from inside of Vec is very different. In that case, it really is only 1 additional element's worth of space that is needed. Here, an unknown amount of additional space is needed. That's why I suggest doubling the storage. Nix's getcwd() method should probably do the same.

From reading the std code, it seems different. See:

https://github.com/rust-lang/rust/blob/bdfd698f37184da42254a03ed466ab1f90e6fb6c/src/liballoc/raw_vec.rs#L494-L500

and

https://github.com/rust-lang/rust/blob/bdfd698f37184da42254a03ed466ab1f90e6fb6c/src/liballoc/raw_vec.rs#L421-L434

So it seems todo exactly this; it seems good to keep the code simple but if really desired, I can rework this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's just the current implementation. Nix shouldn't rely upon it. We should actually try to reserve a reasonable amount of space for what our application needs.

src/unistd.rs Outdated
@@ -2459,6 +2459,45 @@ impl From<*mut libc::passwd> for User {
}

impl User {
fn from_anything(f: impl Fn(*mut libc::passwd,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This signature is so big that it would probably be written better with a where clause:

    fn from_anything<F>(f: F) -> Result<Option<Self>>
    where
        F: Fn(
            *mut libc::passwd,
            *mut libc::c_char,
            libc::size_t,
            *mut *mut libc::passwd,
        ) -> libc::c_int

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
src/unistd.rs Outdated
return Err(Error::Sys(Errno::ERANGE))
}

unsafe { buf.set_len(buf.capacity()) };
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, absolutely not. This is very unsafe and totally unnecessary. Did you mean to commit this line?

src/unistd.rs Outdated
fn reserve_buffer_size<T>(buf: &mut Vec<T>, limit: usize) -> Result<()> {
use std::cmp::min;

if buf.len() == limit {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be >=, not ==

src/unistd.rs Outdated
unsafe { buf.set_len(buf.capacity()) };

let capacity = min(buf.capacity() * 2, limit);
buf.reserve_exact(capacity);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to override the standard library. You can use reserve instead of reserve_exact. I just don't want to rely on reserve(1) actually reserving more space than 1.

src/unistd.rs Outdated
@@ -2466,12 +2480,12 @@ impl User {
libc::size_t,
*mut *mut libc::passwd) -> libc::c_int
{
let bufsize = match sysconf(SysconfVar::GETPW_R_SIZE_MAX) {
let cbuf_max = match sysconf(SysconfVar::GETPW_R_SIZE_MAX) {
Ok(Some(n)) => n as usize,
Ok(None) | Err(_) => 1024 as usize,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not that it's defined in any publicly visible place, but FreeBSD internally sets this limit to 1MB.

src/unistd.rs Outdated
Ok(Some(n)) => n as usize,
Ok(None) | Err(_) => 1024 as usize,
};

let mut cbuf = Vec::with_capacity(bufsize);
let mut cbuf = Vec::with_capacity(512);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You were right the first time. GETPW_R_SIZE_MAX is the suggested initial size of the buffer for password entries, not the maximum size (earlier versions of POSIX did specify the maximum size).

src/unistd.rs Outdated
Ok(Some(n)) => n as usize,
Ok(None) | Err(_) => 1024 as usize,
};

let mut cbuf = Vec::with_capacity(bufsize);
let mut cbuf = Vec::with_capacity(512);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As above, bufsize should be the intial sizes of the vec, not the maximum.

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
src/unistd.rs Outdated
@@ -2504,7 +2506,7 @@ impl User {
}
} else if Errno::last() == Errno::ERANGE {
// Trigger the internal buffer resizing logic.
reserve_buffer_size(&mut cbuf, cbuf_max)?;
reserve_double_buffer_size(&mut cbuf, bufsize)?;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sets the initial and maximum sizes to the same value. You should probably choose a larger maximum size. As I said, FreeBSD internally uses 1MB.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And how we can inquiry the OS to get this information? Blindly put a number does not sounds right.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can't. There's no way to query it. And there's no need to, for consumers of the C API. It's irrelevant for getpwnam_r since the user provides his own buffer, and the user doesn't need to know it when using getpwnam. But if we want to provide a Rusty API that is both reentrant and allocates the buffer internally, then we need to choose a sane limit ourselves. Do you have a better idea?

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Copy link
Member

@asomers asomers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious. Why 16KB for the pwent limit?

src/unistd.rs Outdated

// Now actually get the groups. We try multiple times in case the number of
// groups has changed since the first call to getgroups() and the buffer is
// now too small.
let mut groups = Vec::<Gid>::with_capacity(Errno::result(ret)? as usize);
let mut groups = Vec::<Gid>::with_capacity(Errno::result(ngroups)? as usize);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the code would be more readable if you moved Errno::result(...)? closer to the call site on line 1339.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reworked. Also, the 16Kb was taken from https://linux.die.net/man/3/getpwuid_r example code.

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
@@ -1337,15 +1337,18 @@ pub fn setgid(gid: Gid) -> Result<()> {
pub fn getgroups() -> Result<Vec<Gid>> {
// First get the number of groups so we can size our Vec
let ngroups = unsafe { libc::getgroups(0, ptr::null_mut()) };
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I meant is that it would be more idiomatic to write it like this:

let ngroups = unsafe { libc::getgroups(0, ptr::null_mut()) }? as usize;

bors bot added a commit that referenced this pull request Nov 1, 2019
1139: Add `Users` and `Group` related functions r=asomers a=otavio

This was a collaborative work between Johannes Schilling
<dario@deaktualisierung.org>, Fredrick Brennan <copypaste@kittens.ph>
and myself.

This PR is a split-off from #864 so we merge ready parts first. Next, we work on the iterators.

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>

Co-authored-by: Otavio Salvador <otavio@ossystems.com.br>
@ctrlcctrlv
Copy link
Author

I'm closing this because I'm tired of seeing it in my PR list. I won't delete the underlying branch, please just open a new PR if you will try to revive whatever wasn't included in #1139.

Should you care to know why, I've written a small blog about it: https://gist.github.com/ctrlcctrlv/978b3ee4f55d4b4ec415a985e01cb1c9

@ctrlcctrlv ctrlcctrlv closed this Jan 30, 2020
geofft added a commit to geofft/nix-rust that referenced this pull request Mar 25, 2021
We have one UNIX group that contains most of our users whose size is
about 20 kB, so `Group::from_name` is failing with ERANGE.

The discussion on PR nix-rust#864 suggests that 1 MB is a reasonable maximum -
it follows what FreeBSD's libc does. (glibc appears to have no maximum
on the _r function and will just double the buffer until malloc fails,
but that's not particularly Rusty.)
geofft added a commit to geofft/nix-rust that referenced this pull request Mar 25, 2021
We have one UNIX group that contains most of our users whose size is
about 20 kB, so `Group::from_name` is failing with ERANGE.

The discussion on PR nix-rust#864 suggests that 1 MB is a reasonable maximum -
it follows what FreeBSD's libc does. (glibc appears to have no maximum
on the _r function and will just double the buffer until malloc fails,
but that's not particularly Rusty.)
bors bot added a commit that referenced this pull request Apr 4, 2021
1409: unistd: Increase maximum passwd/group buffer to 1MB r=asomers a=geofft

We have one UNIX group that contains most of our users whose size is
about 20 kB, so `Group::from_name` is failing with ERANGE.

The discussion on PR #864 suggests that 1 MB is a reasonable maximum -
it follows what FreeBSD's libc does. (glibc appears to have no maximum
on the _r function and will just double the buffer until malloc fails,
but that's not particularly Rusty.)

Co-authored-by: Geoffrey Thomas <geofft@twosigma.com>
anth1y pushed a commit to anth1y/nix that referenced this pull request Apr 11, 2021
We have one UNIX group that contains most of our users whose size is
about 20 kB, so `Group::from_name` is failing with ERANGE.

The discussion on PR nix-rust#864 suggests that 1 MB is a reasonable maximum -
it follows what FreeBSD's libc does. (glibc appears to have no maximum
on the _r function and will just double the buffer until malloc fails,
but that's not particularly Rusty.)
anth1y added a commit to anth1y/nix that referenced this pull request Apr 11, 2021
unistd: Increase maximum passwd/group buffer to 1MB

We have one UNIX group that contains most of our users whose size is
about 20 kB, so `Group::from_name` is failing with ERANGE.

The discussion on PR nix-rust#864 suggests that 1 MB is a reasonable maximum -
it follows what FreeBSD's libc does. (glibc appears to have no maximum
on the _r function and will just double the buffer until malloc fails,
but that's not particularly Rusty.)

Use memoffset::offset_of instead of homegrown macro

The homegrown macro was fine in 2016, but at some point it technically
became UB.  The memoffset crate does the same thing, but avoids UB when
using rustc 1.51.0 or later.

Fixes nix-rust#1415

Check all tests in CI

Travis didn't compile check tests on platforms that couldn't run tests
in CI, so they bitrotted.  Let's see how bad they are.

Most annoyingly, 32-bit Android defines mode_t as 16 bits, but
stat.st_mode as 32-bits.

Fix spurious errors using `sendmmsg` with multiple cmsgs

Before this fix, the buffer that holds cmsgs may move due to the resize()
call. That causes msg_hdr pointing to invalid memory, which ends up
breaking the sendmmsg() call, resulting in EINVAL.

This change fixes it by avoiding re-allocating the buffers.

Support TIMESTAMPNS for linux
anth1y added a commit to anth1y/nix that referenced this pull request Apr 17, 2021
unistd: Increase maximum passwd/group buffer to 1MB

We have one UNIX group that contains most of our users whose size is
about 20 kB, so `Group::from_name` is failing with ERANGE.

The discussion on PR nix-rust#864 suggests that 1 MB is a reasonable maximum -
it follows what FreeBSD's libc does. (glibc appears to have no maximum
on the _r function and will just double the buffer until malloc fails,
but that's not particularly Rusty.)

Use memoffset::offset_of instead of homegrown macro

The homegrown macro was fine in 2016, but at some point it technically
became UB.  The memoffset crate does the same thing, but avoids UB when
using rustc 1.51.0 or later.

Fixes nix-rust#1415

Check all tests in CI

Travis didn't compile check tests on platforms that couldn't run tests
in CI, so they bitrotted.  Let's see how bad they are.

Most annoyingly, 32-bit Android defines mode_t as 16 bits, but
stat.st_mode as 32-bits.

Fix spurious errors using `sendmmsg` with multiple cmsgs

Before this fix, the buffer that holds cmsgs may move due to the resize()
call. That causes msg_hdr pointing to invalid memory, which ends up
breaking the sendmmsg() call, resulting in EINVAL.

This change fixes it by avoiding re-allocating the buffers.

Support TIMESTAMPNS for linux
anth1y added a commit to anth1y/nix that referenced this pull request Apr 17, 2021
unistd: Increase maximum passwd/group buffer to 1MB

We have one UNIX group that contains most of our users whose size is
about 20 kB, so `Group::from_name` is failing with ERANGE.

The discussion on PR nix-rust#864 suggests that 1 MB is a reasonable maximum -
it follows what FreeBSD's libc does. (glibc appears to have no maximum
on the _r function and will just double the buffer until malloc fails,
but that's not particularly Rusty.)

Use memoffset::offset_of instead of homegrown macro

The homegrown macro was fine in 2016, but at some point it technically
became UB.  The memoffset crate does the same thing, but avoids UB when
using rustc 1.51.0 or later.

Fixes nix-rust#1415

Check all tests in CI

Travis didn't compile check tests on platforms that couldn't run tests
in CI, so they bitrotted.  Let's see how bad they are.

Most annoyingly, 32-bit Android defines mode_t as 16 bits, but
stat.st_mode as 32-bits.

Fix spurious errors using `sendmmsg` with multiple cmsgs

Before this fix, the buffer that holds cmsgs may move due to the resize()
call. That causes msg_hdr pointing to invalid memory, which ends up
breaking the sendmmsg() call, resulting in EINVAL.

This change fixes it by avoiding re-allocating the buffers.

Support TIMESTAMPNS for linux

unistd: Increase maximum passwd/group buffer to 1MB

We have one UNIX group that contains most of our users whose size is
about 20 kB, so `Group::from_name` is failing with ERANGE.

The discussion on PR nix-rust#864 suggests that 1 MB is a reasonable maximum -
it follows what FreeBSD's libc does. (glibc appears to have no maximum
on the _r function and will just double the buffer until malloc fails,
but that's not particularly Rusty.)

Use memoffset::offset_of instead of homegrown macro

The homegrown macro was fine in 2016, but at some point it technically
became UB.  The memoffset crate does the same thing, but avoids UB when
using rustc 1.51.0 or later.

Fixes nix-rust#1415

Check all tests in CI

Travis didn't compile check tests on platforms that couldn't run tests
in CI, so they bitrotted.  Let's see how bad they are.

Most annoyingly, 32-bit Android defines mode_t as 16 bits, but
stat.st_mode as 32-bits.

Fix spurious errors using `sendmmsg` with multiple cmsgs

Before this fix, the buffer that holds cmsgs may move due to the resize()
call. That causes msg_hdr pointing to invalid memory, which ends up
breaking the sendmmsg() call, resulting in EINVAL.

This change fixes it by avoiding re-allocating the buffers.

Support TIMESTAMPNS for linux

fix bug missing octothorpe
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

getpwnam is missing
8 participants