-
-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use crypt_ra to allocate scratch area for password hashing #16981
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd really hide the buffer allocation in the inner-most code block. The reusing of the allocated buffers seems a lot complexity in the interface for pretty much no gain, since we typically deal with a single password to hash only, and in the worst case maybe a handful, but this still shouldn't matter at all performance-wise.
Or to say this differently: the actual hashing of the pw is likely a lot more expensive than the malloc() we might do too much, hence it shouldn't matter
132e357
to
7b884b5
Compare
I made the cached allocation mostly hidden, but I left it in places where we loop over passwords, e.g. in user_record_quality_check_password(). PTAL. |
Hmm, the build fails because crypt_ra is not available, strange. |
CI is not happy... I still wouldn't bother even for the pw quality check stuff. Yeah, it looks like O(n^2), but given that n is almost always == 1, the malloc() shouldn't matter... |
oh, uh. i think i remember now: crypt_ra() is libxcrypt-only. glibc didn't have it. our CIs still use glibcs that did not do the libxcrypt split-out. I guess this means there needs to be some fallback for a while. crypt + crypt_r are available on glibc, and crypt_rn/crypt_ra are openwall/libxcrypt additions. |
7b884b5
to
9d74764
Compare
I removed the caching of the buffer now. It is only exposed in the low-level hash_password_full() function and used in tests. "Normal" callers use the wrapper which always allocates. |
09a2c13
to
365711b
Compare
meson.build
Outdated
crypt_header = conf.get('HAVE_CRYPT_H') == 1 ? \ | ||
'''#include <crypt.h>''' : '''#include <unistd.h>''' | ||
foreach ident : [ | ||
['crypt_r', crypt_header], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
crypt_r is glibc already, we should not need to test for it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The impetus for this PR came from centos ci which reports:
20:36:36 archlinux_systemd_ci: Checking for function "crypt_r" with dependency -lcrypt: YES
20:36:36 archlinux_systemd_ci: Checking for function "crypt_rn" with dependency -lcrypt: NO
20:36:36 archlinux_systemd_ci: Checking for function "crypt_ra" with dependency -lcrypt: NO
If we had crypt_rn
, there'd be hope to improve things. But since there is neither _ra
nor _rn
, most likely this will not resolve #16965, unless the glibc version there is updated. I think it's still worthwhile to pursue this PR, since the _ra
variant seems to be a much better API. WDYT?
(Am I missing some magic option to "enable" xcrypt? At least in Fedora libxcrypt.rpm only provides libcrypt.so.2, so using -lcrypt
is the only way to consume xcrypt.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hah, now I see why downgrading glibc on the current Arch Linux fails:
error: failed to commit transaction (conflicting files)
libxcrypt: /usr/include/crypt.h exists in filesystem (owned by glibc)
libxcrypt: /usr/lib/libcrypt.so exists in filesystem (owned by glibc)
The thing is that in recent glibc updates Arch transitioned from libcrypt in glibc to libcrypt in libxcrypt, see [0][1]. And, of course, the libxcrypt contains the functions you seek:
libxcrypt /usr/share/man/man3/crypt_r.3.gz
libxcrypt /usr/share/man/man3/crypt_ra.3.gz
libxcrypt /usr/share/man/man3/crypt_rn.3.gz
But right now the upgrade is blocked by the sanitizer issue, but to make it go away we need to upgrade the image, so we're in a chicken-egg situation. I guess I could temporarily disable the test which triggers the issue (TEST-46) and go ahead with the upgrade.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
all i wanted to say: there's no point in checking for crypt_r, since our baseline, i.e. old glibc had that since time began.
checking for crypt_ra() definitely makes sense since glibc never had that, and onl libxcrypt has.
crypt_rn is a call we probably don#t care about, we shouldn't test for it hence
365711b
to
46f500a
Compare
As discussed above, I think this is still worth pursuing even if it doesn't fix the immediate issue when libxcrypt is not available. Hopefully the tests will pass now. |
maybe add a minimal local fallback implementation for crypt_ra() here, that just wraps crypt_r() and does malloc() itself. i.e. call it missing_crypt_ra() and (I guess the whole issue here is just fix what this the comment in crypt(3) suggests: "struct crypt_data may be quite large (32kB in this implementation of libcrypt; over 128kB in some other implementations). This is large enough that it may be unwise to allocate it on the stack.") |
using "Some recently designed hashing methods need even more scratch memory, but the crypt_r interface makes it impossible to change the size of struct crypt_data without breaking binary compatibility. The crypt_rn interface could So yeah, unless we use crypt_ra() if it is available we would not be compatible with contemporary hashes, and I think we should |
ah, now i see you just pushed such a fallback... |
@@ -75,21 +75,23 @@ int make_salt(char **ret) { | |||
#endif | |||
} | |||
|
|||
int hash_password(const char *password, char **ret) { | |||
int hash_password_full(const char *password, void **cd_data, int *cd_size, char **ret) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i still wouldn't expose the cd_data, sounds like needless optimization. i.e it's the kind of optimization you might want to do ina hash breaker or so, but otherwise I really wouldn't bother...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is exposed here, but is only used in the test. I think it is worthwhile to show that in tests: it should make it much easier to debug issues like #16965.
meson.build
Outdated
crypt_header = conf.get('HAVE_CRYPT_H') == 1 ? \ | ||
'''#include <crypt.h>''' : '''#include <unistd.h>''' | ||
foreach ident : [ | ||
['crypt_r', crypt_header], |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
all i wanted to say: there's no point in checking for crypt_r, since our baseline, i.e. old glibc had that since time began.
checking for crypt_ra() definitely makes sense since glibc never had that, and onl libxcrypt has.
crypt_rn is a call we probably don#t care about, we shouldn't test for it hence
src/shared/libcrypt-util.c
Outdated
/* Provide a poor man's fallback that uses a fixed size buffer. */ | ||
|
||
static char* systemd_crypt_ra(const char *phrase, const char *setting, void **data, int *size) { | ||
struct crypt_data cd = {}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i think we should malloc() this, given that the structure might be too large for the stack according to the man page:
"struct crypt_data may be quite large (32kB in this implementation of libcrypt; over 128kB in some other implementations). This is large enough that it may be unwise to allocate it on the stack".
As the issue is reproducible only with libxcrypt (which is not in the current images), the tests should definitely pass. However, once the outstanding reviews in this PR are resolved, I'll run it manually against the upgraded image to see if the issue is indeed gone. |
centos7:
That machine only supports original descrypt hash, how is that even possible? @mrc0mmand maybe it's missing some library. |
They are only used under src/home/, but I want to add tests in test-libcrypt-util.c. And the functions are almost trivial, so I think it is OK to move them to shared.
Now that we wrap crypt_r/ra uses, we can include the header only in libcrypt-util.c.
We always seem to have either libcrypt_r and not the other two, or all three. So the fallback for libcrypt_ra needs to be based on libcrypt_r.
Since the loop to check various xcrypt functions is already in place, adding one more is cheap. And it is nicer to check for the function directly. People like to backport things, so we might get lucky even without having libxcrypt.
Following the style in missing_syscall.h, we use a non-conflicting name for the function and use a macro to map to the real name to the replacement.
I'm tired of trying to figure this out.
…lable On centos7 ci: --- test-libcrypt-util begin --- Found container virtualization none. /* test_hash_password */ ew3bU1.hoKk4o: yes $1$gc5rWpTB$wK1aul1PyBn9AX1z93stk1: no $2b$12$BlqcGkB/7BFvNMXKGxDea.5/8D6FTny.cbNcHW/tqcrcyo6ZJd8u2: no $5$lGhDrcrao9zb5oIK$05KlOVG3ocknx/ThreqXE/gk.XzFFBMTksc4t2CPDUD: no $6$c7wB/3GiRk0VHf7e$zXJ7hN0aLZapE.iO4mn/oHu6.prsXTUG/5k1AxpgR85ELolyAcaIGRgzfwJs3isTChMDBjnthZyaMCfCNxo9I.: no $y$j9T$$9cKOWsAm4m97WiYk61lPPibZpy3oaGPIbsL4koRe/XD: no
961afd4
to
5d3fe6f
Compare
I suspect this might be caused by non-descrypt methods being available only in the glibc extension or something like that? From
When I tried #define _GNU_SOURCE
#include <crypt.h>
#include <stdio.h>
#include <string.h>
int main(int argc, char *argv[])
{
struct crypt_data cc = {};
const char *hash = "$6$c7wB/3GiRk0VHf7e$zXJ7hN0aLZapE.iO4mn/oHu6.prsXTUG/5k1AxpgR85ELolyAcaIGRgzfwJs3isTChMDBjnthZyaMCfCNxo9I.";
const char *k;
k = crypt_r("ppp", hash, &cc);
if (!k) {
perror("crypt_r()");
return 1;
}
printf("%s\n", (strcmp(k, hash) == 0) ? "pass" : "fail");
return 0;
}
|
I assume that this is on the host itself. The failure happens in nested image in TEST-24-UNITTESTS. |
Ah, my bad. Will look into that. |
@keszybz so, after comparing straces from inside/outside of the VM and playing around with installing various missing libraries, installing E.g.: diff --git a/test/test-functions b/test/test-functions
index 9893864..c3204a8 100644
--- a/test/test-functions
+++ b/test/test-functions
@@ -446,6 +446,10 @@ setup_basic_environment() {
if [[ "$IS_BUILT_WITH_ASAN" = "yes" ]]; then
create_asan_wrapper
fi
+
+ inst_binary /root/systemd/main /main
+ inst_library /usr/lib64/libfreeblpriv3.so
+ inst /lib64/libfreeblpriv3.chk
} (ignore the Before:
After:
|
@mrc0mmand wow, thanks for tracking this down. The failing test was caused by missing deps. But I think we do want to test such systems too: this is not something that we verify during the configuration step, and it is possible that users might build on such systems. Also, in principle in the future systemd might be built on a system which only has newer hashes (or just different, e.g. |
Hmm, so what's the state of this? I guess we can just merge this? PR looks good to me. Any PPC work to delay this for? If not, let's merge! |
I think this is good to go. |
#include "missing_stdlib.h" | ||
#include "random-util.h" | ||
#include "string-util.h" | ||
#include "strv.h" | ||
|
||
int make_salt(char **ret) { | ||
|
||
#ifdef XCRYPT_VERSION_MAJOR | ||
#if HAVE_CRYPT_GENSALT_RA |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@keszybz this caused a regression for me - I have crypt_gensalt_ra but not crypt_preferred_method which is used in the same ifdef. I'll send a PR to check for both in meson.build and here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After systemd#16981 only the presence of crypt_gensalt_ra is checked, but there are cases where that function is available but crypt_preferred_method is not, and they are used in the same ifdef. Add a check for the latter as well.
After systemd#16981 only the presence of crypt_gensalt_ra is checked, but there are cases where that function is available but crypt_preferred_method is not, and they are used in the same ifdef. Add a check for the latter as well.
After #16981 only the presence of crypt_gensalt_ra is checked, but there are cases where that function is available but crypt_preferred_method is not, and they are used in the same ifdef. Add a check for the latter as well.
After systemd#16981 only the presence of crypt_gensalt_ra is checked, but there are cases where that function is available but crypt_preferred_method is not, and they are used in the same ifdef. Add a check for the latter as well.
After systemd#16981 only the presence of crypt_gensalt_ra is checked, but there are cases where that function is available but crypt_preferred_method is not, and they are used in the same ifdef. Add a check for the latter as well.
After systemd/systemd#16981 only the presence of crypt_gensalt_ra is checked, but there are cases where that function is available but crypt_preferred_method is not, and they are used in the same ifdef. Add a check for the latter as well.
No description provided.