New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Build environment updates #145
Conversation
|
||
# Compatible with -j... | ||
$str =~ s/^(\s+@?)/$1cd \$(TEST_D) && /; | ||
return $str; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suspect removing this will break FreeBSD.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Quite the opposite; ./GitMake test was broken for me everywhere until I removed these lines. I just did a full 'make test' and 'GitMake test' on FreeBSD 9.1 to make certain that it works.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On 9 July 2014 21:44, Mike Bland notifications@github.com wrote:
In util/pl/unix.pl:
@@ -226,12 +228,6 @@ sub fixdeps
sub fixrules
{my ($str) = @_;
Compatible with -j...
- $str =~ s/^(\s+@?)/$1cd $(TEST_D) && /;
- return $str;
Quite the opposite; ./GitMake test was broken for me everywhere until I
removed these lines. I just did a full 'make test' and 'GitMake test' on
FreeBSD 9.1 to make certain that it works.
Did you do GitMake -j 20 test?
—
Reply to this email directly or view it on GitHub
https://github.com/openssl/openssl/pull/145/files#r14734751.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did a 'GitMake -j 12' followed by a 'GitMake test' before writing.
Just now, I checked out a clean branch based on the upstream master. First, I did a 'GitMake -j 20' followed by 'GitMake test'. The 'GitMake test' step broke.
Then I ran 'GitMake -j 20' test. It also broke.
Back in my branch containing these changes, 'GitMake -j 20' passed. This is all on FreeBSD 9.1.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well. I pulled these changes and did GitMake -j 1 -P test (which works for me) and I get:
test SSL protocol
cd out.mbland-build && /usr/home/ben/git-work/openssl/out.mbland-build/shlib_wrap.sh ./ssltest -test_cipherlist
cd: out.mbland-build: No such file or directory
cd: out.mbland-build: No such file or directory
cd: out.mbland-build: No such file or directory
cd: out.mbland-build: No such file or directory
test_ssl test done
Remaking `test_req'
Results of making test_req:
test_req test started
testing req conversions
p -> d
*** [test_req] Error code 1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On 10 July 2014 19:38, Mike Bland notifications@github.com wrote:
In util/pl/unix.pl:
@@ -226,12 +228,6 @@ sub fixdeps
sub fixrules
{my ($str) = @_;
Compatible with -j...
- $str =~ s/^(\s+@?)/$1cd $(TEST_D) && /;
- return $str;
I said it worked because originally I wasn't using '-j', so it was working
for me. When I used '-j 20', I didn't see the "cd: out.build: No such file
or directory" errors and the shell return code was 0.
That's what's puzzling me - I get you could miss the text-only errors, but
I got an actual make-stopping error, and I'm wondering why you didn't?
Perhaps try that on your end to verify; with the (now obsolete) change, it
works, no errors. But then, it defeats the purpose of using '-j' to
parallelize the build by breaking that case.Yeah, I know -P doesn't change behavior; it's just interesting that it's
not present in all version of bsdmake anymore, not even between versions of
FreeBSD. :-)—
Reply to this email directly or view it on GitHub
https://github.com/openssl/openssl/pull/145/files#r14784424.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On Thu, Jul 10, 2014 at 3:42 PM, Ben Laurie notifications@github.com wrote:
In util/pl/unix.pl:
@@ -226,12 +228,6 @@ sub fixdeps
sub fixrules
{my ($str) = @_;
Compatible with -j...
- $str =~ s/^(\s+@?)/$1cd $(TEST_D) && /;
- return $str;
On 10 July 2014 19:38, Mike Bland notifications@github.com wrote: In util/pl/unix.pl: > @@ -226,12 +228,6 @@ sub fixdeps > sub fixrules > { > my ($str) = @_; > - > - # Compatible with -j... > - $str =~ s/^(\s+@?)/$1cd $(TEST_D) && /; > - return $str; I said it worked because originally I wasn't using '-j', so it was working for me. When I used '-j 20', I didn't see the "cd: out.build: No such file or directory" errors and the shell return code was 0.
That's what's puzzling me - I get you could miss the text-only errors, but I got an actual make-stopping error, and I'm wondering why you didn't?
There is other GitMake weirdness going on that I'm investigating,
which I was going to raise in a separate thread later, but since it
seems to be germane here...
While beginning my investigation into your earlier report this
morning, I noticed that in a brand new working directory identical to
the latest OpenSSL master branch, if I do a 'GitMake -j 1',
test_verify fails. If I then do a regular make, the subsequent
'GitMake -j 1 test_verify' will pass.
It seems something similar is happening with 'GitMake -j 1 test' in
general. I use git-new-workdir to have a private dir per branch. In my
original FreeBSD 9.1 working dir for this 'build' branch, running
'GitMake -j 1 test' even now (after re-introducing the obsolete
change), I'll see the 'cd' errors, but the make will return status
zero. I have previously done a regular 'make' in this directory.
Using the same, identical branch in a different workdir, without first
running 'make', I see the failures and make returns an error.
I'm spending this afternoon digging in to try to understand what the
difference is between the two directories that accounts for this
behavior. It's not obvious to me by diffing the output of 'find . >
files.txt' for each dir.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On Thu, Jul 10, 2014 at 4:09 PM, Mike Bland mbland@acm.org wrote:
On Thu, Jul 10, 2014 at 3:42 PM, Ben Laurie notifications@github.com wrote:
In util/pl/unix.pl:
@@ -226,12 +228,6 @@ sub fixdeps
sub fixrules
{my ($str) = @_;
Compatible with -j...
- $str =~ s/^(\s+@?)/$1cd $(TEST_D) && /;
- return $str;
On 10 July 2014 19:38, Mike Bland notifications@github.com wrote: In util/pl/unix.pl: > @@ -226,12 +228,6 @@ sub fixdeps > sub fixrules > { > my ($str) = @_; > - > - # Compatible with -j... > - $str =~ s/^(\s+@?)/$1cd $(TEST_D) && /; > - return $str; I said it worked because originally I wasn't using '-j', so it was working for me. When I used '-j 20', I didn't see the "cd: out.build: No such file or directory" errors and the shell return code was 0.
That's what's puzzling me - I get you could miss the text-only errors, but I got an actual make-stopping error, and I'm wondering why you didn't?
There is other GitMake weirdness going on that I'm investigating,
which I was going to raise in a separate thread later, but since it
seems to be germane here...While beginning my investigation into your earlier report this
morning, I noticed that in a brand new working directory identical to
the latest OpenSSL master branch, if I do a 'GitMake -j 1',
test_verify fails. If I then do a regular make, the subsequent
'GitMake -j 1 test_verify' will pass.It seems something similar is happening with 'GitMake -j 1 test' in
general. I use git-new-workdir to have a private dir per branch. In my
original FreeBSD 9.1 working dir for this 'build' branch, running
'GitMake -j 1 test' even now (after re-introducing the obsolete
change), I'll see the 'cd' errors, but the make will return status
zero. I have previously done a regular 'make' in this directory.Using the same, identical branch in a different workdir, without first
running 'make', I see the failures and make returns an error.I'm spending this afternoon digging in to try to understand what the
difference is between the two directories that accounts for this
behavior. It's not obvious to me by diffing the output of 'find . >
files.txt' for each dir.
FWIW, here's the broken test_verify output I'm getting:
test_verify test started
The following command should have some OK's and some failures
There are definitly a few expired certificates
/Users/msb/src/openssl-fresh/out.build/shlib_wrap.sh
/Users/msb/src/openssl-fresh/out.build/openssl verify -CApath
/Users/msb/src/openssl-fresh/certs/demo
/Users/msb/src/openssl-fresh/certs/demo/.pem
WARNING: can't open config file: /usr/local/ssl/openssl.cnf
140735297401696:error:0B06E06B:x509 certificate
routines:X509_get_pubkey_parameters:unable to find parameters in
chain:/Users/msb/src/openssl-fresh/crypto/x509/x509_vfy.c:2005:
/Users/msb/src/openssl-fresh/certs/demo/ca-cert.pem: C = AU, ST =
Queensland, O = CryptSoft Pty Ltd, CN = Test CA (1024 bit)
error 20 at 0 depth lookup:unable to get local issuer certificate
/Users/msb/src/openssl-fresh/certs/demo/dsa-ca.pem: C = AU, ST =
Some-State, O = Internet Widgits Pty Ltd, CN = CA
error 20 at 0 depth lookup:unable to get local issuer certificate
/Users/msb/src/openssl-fresh/certs/demo/dsa-pca.pem: C = AU, ST =
Some-State, O = Internet Widgits Pty Ltd, CN = PCA
error 18 at 0 depth lookup:self signed certificate
C = AU, ST = Some-State, O = Internet Widgits Pty Ltd, CN = PCA
error 10 at 0 depth lookup:certificate has expired
OK
/Users/msb/src/openssl-fresh/certs/demo/pca-cert.pem: C = AU, ST =
Queensland, O = CryptSoft Pty Ltd, CN = Test PCA (1024 bit)
error 18 at 0 depth lookup:self signed certificate
C = AU, ST = Queensland, O = CryptSoft Pty Ltd, CN = Test PCA (1024 bit)
error 10 at 0 depth lookup:certificate has expired
OK
** Error code 2
1 error
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On Thu, Jul 10, 2014 at 4:15 PM, Mike Bland mbland@acm.org wrote:
On Thu, Jul 10, 2014 at 4:09 PM, Mike Bland mbland@acm.org wrote:
On Thu, Jul 10, 2014 at 3:42 PM, Ben Laurie notifications@github.com wrote:
In util/pl/unix.pl:
@@ -226,12 +228,6 @@ sub fixdeps
sub fixrules
{my ($str) = @_;
Compatible with -j...
- $str =~ s/^(\s+@?)/$1cd $(TEST_D) && /;
- return $str;
On 10 July 2014 19:38, Mike Bland notifications@github.com wrote: In util/pl/unix.pl: > @@ -226,12 +228,6 @@ sub fixdeps > sub fixrules > { > my ($str) = @_; > - > - # Compatible with -j... > - $str =~ s/^(\s+@?)/$1cd $(TEST_D) && /; > - return $str; I said it worked because originally I wasn't using '-j', so it was working for me. When I used '-j 20', I didn't see the "cd: out.build: No such file or directory" errors and the shell return code was 0.
That's what's puzzling me - I get you could miss the text-only errors, but I got an actual make-stopping error, and I'm wondering why you didn't?
There is other GitMake weirdness going on that I'm investigating,
which I was going to raise in a separate thread later, but since it
seems to be germane here...While beginning my investigation into your earlier report this
morning, I noticed that in a brand new working directory identical to
the latest OpenSSL master branch, if I do a 'GitMake -j 1',
test_verify fails. If I then do a regular make, the subsequent
'GitMake -j 1 test_verify' will pass.It seems something similar is happening with 'GitMake -j 1 test' in
general. I use git-new-workdir to have a private dir per branch. In my
original FreeBSD 9.1 working dir for this 'build' branch, running
'GitMake -j 1 test' even now (after re-introducing the obsolete
change), I'll see the 'cd' errors, but the make will return status
zero. I have previously done a regular 'make' in this directory.Using the same, identical branch in a different workdir, without first
running 'make', I see the failures and make returns an error.I'm spending this afternoon digging in to try to understand what the
difference is between the two directories that accounts for this
behavior. It's not obvious to me by diffing the output of 'find . >
files.txt' for each dir.FWIW, here's the broken test_verify output I'm getting:
test_verify test started
The following command should have some OK's and some failures
There are definitly a few expired certificates
/Users/msb/src/openssl-fresh/out.build/shlib_wrap.sh
/Users/msb/src/openssl-fresh/out.build/openssl verify -CApath
/Users/msb/src/openssl-fresh/certs/demo
/Users/msb/src/openssl-fresh/certs/demo/.pem
WARNING: can't open config file: /usr/local/ssl/openssl.cnf
140735297401696:error:0B06E06B:x509 certificate
routines:X509_get_pubkey_parameters:unable to find parameters in
chain:/Users/msb/src/openssl-fresh/crypto/x509/x509_vfy.c:2005:
/Users/msb/src/openssl-fresh/certs/demo/ca-cert.pem: C = AU, ST =
Queensland, O = CryptSoft Pty Ltd, CN = Test CA (1024 bit)
error 20 at 0 depth lookup:unable to get local issuer certificate
/Users/msb/src/openssl-fresh/certs/demo/dsa-ca.pem: C = AU, ST =
Some-State, O = Internet Widgits Pty Ltd, CN = CA
error 20 at 0 depth lookup:unable to get local issuer certificate
/Users/msb/src/openssl-fresh/certs/demo/dsa-pca.pem: C = AU, ST =
Some-State, O = Internet Widgits Pty Ltd, CN = PCA
error 18 at 0 depth lookup:self signed certificate
C = AU, ST = Some-State, O = Internet Widgits Pty Ltd, CN = PCA
error 10 at 0 depth lookup:certificate has expired
OK
/Users/msb/src/openssl-fresh/certs/demo/pca-cert.pem: C = AU, ST =
Queensland, O = CryptSoft Pty Ltd, CN = Test PCA (1024 bit)
error 18 at 0 depth lookup:self signed certificate
C = AU, ST = Queensland, O = CryptSoft Pty Ltd, CN = Test PCA (1024 bit)
error 10 at 0 depth lookup:certificate has expired
OK
** Error code 2
1 error
Figured out the case of test_verify. 'make rehash' needs to be invoked
to create symlinks in certs/demo, via 'make build_apps'. Hand-jamming
this into a previously-failing GitMake-only working dir causes
test_verify to pass. So it looks like a dependency disconnect I'll
need to patch up somehow...
Still working on finding out why the entire test set doesn't return an
error code, though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On Thu, Jul 10, 2014 at 5:19 PM, Mike Bland mbland@acm.org wrote:
On Thu, Jul 10, 2014 at 4:15 PM, Mike Bland mbland@acm.org wrote:
On Thu, Jul 10, 2014 at 4:09 PM, Mike Bland mbland@acm.org wrote:
On Thu, Jul 10, 2014 at 3:42 PM, Ben Laurie notifications@github.com wrote:
In util/pl/unix.pl:
@@ -226,12 +228,6 @@ sub fixdeps
sub fixrules
{my ($str) = @_;
Compatible with -j...
- $str =~ s/^(\s+@?)/$1cd $(TEST_D) && /;
- return $str;
On 10 July 2014 19:38, Mike Bland notifications@github.com wrote: In util/pl/unix.pl: > @@ -226,12 +228,6 @@ sub fixdeps > sub fixrules > { > my ($str) = @_; > - > - # Compatible with -j... > - $str =~ s/^(\s+@?)/$1cd $(TEST_D) && /; > - return $str; I said it worked because originally I wasn't using '-j', so it was working for me. When I used '-j 20', I didn't see the "cd: out.build: No such file or directory" errors and the shell return code was 0.
That's what's puzzling me - I get you could miss the text-only errors, but I got an actual make-stopping error, and I'm wondering why you didn't?
There is other GitMake weirdness going on that I'm investigating,
which I was going to raise in a separate thread later, but since it
seems to be germane here...While beginning my investigation into your earlier report this
morning, I noticed that in a brand new working directory identical to
the latest OpenSSL master branch, if I do a 'GitMake -j 1',
test_verify fails. If I then do a regular make, the subsequent
'GitMake -j 1 test_verify' will pass.It seems something similar is happening with 'GitMake -j 1 test' in
general. I use git-new-workdir to have a private dir per branch. In my
original FreeBSD 9.1 working dir for this 'build' branch, running
'GitMake -j 1 test' even now (after re-introducing the obsolete
change), I'll see the 'cd' errors, but the make will return status
zero. I have previously done a regular 'make' in this directory.Using the same, identical branch in a different workdir, without first
running 'make', I see the failures and make returns an error.I'm spending this afternoon digging in to try to understand what the
difference is between the two directories that accounts for this
behavior. It's not obvious to me by diffing the output of 'find . >
files.txt' for each dir.FWIW, here's the broken test_verify output I'm getting:
test_verify test started
The following command should have some OK's and some failures
There are definitly a few expired certificates
/Users/msb/src/openssl-fresh/out.build/shlib_wrap.sh
/Users/msb/src/openssl-fresh/out.build/openssl verify -CApath
/Users/msb/src/openssl-fresh/certs/demo
/Users/msb/src/openssl-fresh/certs/demo/.pem
WARNING: can't open config file: /usr/local/ssl/openssl.cnf
140735297401696:error:0B06E06B:x509 certificate
routines:X509_get_pubkey_parameters:unable to find parameters in
chain:/Users/msb/src/openssl-fresh/crypto/x509/x509_vfy.c:2005:
/Users/msb/src/openssl-fresh/certs/demo/ca-cert.pem: C = AU, ST =
Queensland, O = CryptSoft Pty Ltd, CN = Test CA (1024 bit)
error 20 at 0 depth lookup:unable to get local issuer certificate
/Users/msb/src/openssl-fresh/certs/demo/dsa-ca.pem: C = AU, ST =
Some-State, O = Internet Widgits Pty Ltd, CN = CA
error 20 at 0 depth lookup:unable to get local issuer certificate
/Users/msb/src/openssl-fresh/certs/demo/dsa-pca.pem: C = AU, ST =
Some-State, O = Internet Widgits Pty Ltd, CN = PCA
error 18 at 0 depth lookup:self signed certificate
C = AU, ST = Some-State, O = Internet Widgits Pty Ltd, CN = PCA
error 10 at 0 depth lookup:certificate has expired
OK
/Users/msb/src/openssl-fresh/certs/demo/pca-cert.pem: C = AU, ST =
Queensland, O = CryptSoft Pty Ltd, CN = Test PCA (1024 bit)
error 18 at 0 depth lookup:self signed certificate
C = AU, ST = Queensland, O = CryptSoft Pty Ltd, CN = Test PCA (1024 bit)
error 10 at 0 depth lookup:certificate has expired
OK
** Error code 2
1 errorFigured out the case of test_verify. 'make rehash' needs to be invoked
to create symlinks in certs/demo, via 'make build_apps'. Hand-jamming
this into a previously-failing GitMake-only working dir causes
test_verify to pass. So it looks like a dependency disconnect I'll
need to patch up somehow...Still working on finding out why the entire test set doesn't return an
error code, though.
Got it. In a clean working dir, with this change, test{req,key}.pem
aren't being generated into out.build. If you do 'GitMake -j 1 -P
test_req' specifically, you'll see the prerequisite _fake1 target
(which generates the pem files) produce nothing yet still indicate a
successful result, which then causes test_req to break.
In my original working dir, the files were there from a previous build
before this change, so the test_req target continued to "pass" in
'GitMake -j' mode. Removing them caused the exact same failure and
output you described earlier.
5f80276
to
c66b14d
Compare
b73a958
to
b9c2516
Compare
cscope.out is generated by cscope as described in: http://wiki.openssl.org/index.php/Testing_and_Development_Tools_and_Tips .d files are compiler-generated Makefile dependency files (e.g. using 'gcc -MMD -MP foo.c'). Signed-off-by: Mike Bland <mbland@acm.org>
This fixes the errors when trying to assemble .s files using GitMake on OS X. Signed-off-by: Mike Bland <mbland@acm.org>
These are based on debug-ben-debug-64-clang and is intended to produce consistent settings for folks involved in the unit testing effort detailed at: http://wiki.openssl.org/index.php/Unit_Testing -fsanitize has been removed from the set of clang flags for now. Apparently clang 3.1, which ships with FreeBSD 9.1, completely ignores -fsanitize. Clang 3.3, which ships with FreeBSD 9.2, compiles with it, but fails to link due to the absence of libasan: http://lists.freebsd.org/pipermail/freebsd-hackers/2013-December/043995.html https://www.mail-archive.com/cfe-commits@cs.uiuc.edu/msg92260.html http://reviews.llvm.org/D2644 We need -Wno-error=unused-const-variable because of this error: .../crypto/ec/ec_lib.c:74:19: error: unused variable 'EC_version' [-Werror,-Wunused-const-variable] static const char EC_version[] = "EC" OPENSSL_VERSION_PTEXT; Signed-off-by: Mike Bland <mbland@acm.org>
Before this change, variables for which a '=' appeared in the assignment would be parsed as the entire string up until the final '='. For example: BUILD_CMD=shlib_target=; if [ -n "$(SHARED_LIBS)" ]; then \ would result in the variable name "BUILD_CMD=shlib_target". This doesn't appear to harm the current generation of MINFO, but creates problems for other Makefile-related work I'm attempting. Signed-off-by: Mike Bland <mbland@acm.org>
These correspond to targets of the same name in test/Makefile that clash when using the single-makefile build method using GitConfigure and GitMake. Signed-off-by: Mike Bland <mbland@acm.org>
This is the only Makefile without SRC defined. This change enables a standard Makefile include directive to cover crypto/jpake/*.d files. This was automatically applied by AddSrcVarIfNeeded() in: https://code.google.com/p/mike-bland/source/browse/openssl/update_makefiles.py Signed-off-by: Mike Bland <mbland@acm.org>
Signed-off-by: Mike Bland <mbland@acm.org>
This should have been $(CC) all along; removing $(BUILDENV) from the make invocation causes ${{CC}} to be empty, triggering a build failure as LDCMD (in Makefile.shared) receives no value. Signed-off-by: Mike Bland <mbland@acm.org>
It appears all of the changes are in master, so I'm closing this. |
* Change to build with CMake * Fix problems during shared library test * Try to find and install libraries correctly * Don't delete OQS symlinks
This pull contains commits to fix the OS X build and allow GitMake test to pass.