New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: fix ninja build failure #12484

Closed
wants to merge 6 commits into
base: master
from

Conversation

Projects
None yet
9 participants
@danbev
Member

danbev commented Apr 18, 2017

When working on commit 6a09a69
("build: enable cctest to use generated objects") I did not take into
account building with ninja:

$ ./configure
$ tools/gyp_node.py -f ninja
$ ninja -C out/Release
$ ln -fs out/Release/node node

When ninja generated the ninja build files, src files that are
relative to the src directory will be named with a dot instead of a
path separator, for example:

out/Release/obj/src/node/node.o

would instead become:

out/Release/obj/src/node.node.o

This commit adds an additional variable for the typ of object separator
used for this case.

Also, the LIBS specified in the 'libraries' section are not
being included in the --start-group --end-group section which
means that these libraries will not be searched causing issue
with linkers where the order matters.

Fixes: #12448

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • commit message follows commit guidelines
Affected core subsystem(s)

build

@danbev

This comment has been minimized.

Show comment
Hide comment
@danbev

danbev Apr 18, 2017

Member

I'm not really sure if this is a good solution but want to see if this passes all the CI runs before looking into something better (perhaps making a change to ninja.py).

CI: https://ci.nodejs.org/job/node-test-pull-request/7469/

Member

danbev commented Apr 18, 2017

I'm not really sure if this is a good solution but want to see if this passes all the CI runs before looking into something better (perhaps making a change to ninja.py).

CI: https://ci.nodejs.org/job/node-test-pull-request/7469/

@sam-github

This comment has been minimized.

Show comment
Hide comment
@sam-github

sam-github Apr 18, 2017

Member

Nice commit message, but " typ of object separator" has a typo ("type").

Member

sam-github commented Apr 18, 2017

Nice commit message, but " typ of object separator" has a typo ("type").

@benjamingr

LGTM, I don't think the fix is that ugly fwiw and it fixes the problem locally for me and in the CI - but I'm not a very skilled ninja user by any means.

'<(OBJ_TRACING_PATH)<(OBJ_SEPARATOR)agent.<(OBJ_SUFFIX)',
'<(OBJ_TRACING_PATH)<(OBJ_SEPARATOR)node_trace_buffer.<(OBJ_SUFFIX)',
'<(OBJ_TRACING_PATH)<(OBJ_SEPARATOR)node_trace_writer.<(OBJ_SUFFIX)',
'<(OBJ_TRACING_PATH)<(OBJ_SEPARATOR)trace_event.<(OBJ_SUFFIX)',

This comment has been minimized.

@refack

refack Apr 18, 2017

Member

Is this necessary? GYP usually knows how to handle slashes?

@refack

refack Apr 18, 2017

Member

Is this necessary? GYP usually knows how to handle slashes?

This comment has been minimized.

@gibfahn

gibfahn Apr 19, 2017

Member

??? The object separator is .node on ninja or / on gyp, not / or \. This is the bit that actually fixes building with ninja, why wouldn't it be necessary?

@gibfahn

gibfahn Apr 19, 2017

Member

??? The object separator is .node on ninja or / on gyp, not / or \. This is the bit that actually fixes building with ninja, why wouldn't it be necessary?

This comment has been minimized.

@refack

refack Apr 19, 2017

Member

Sorry I missed the 'OBJ_SEPARATOR': 'node.',

@refack

refack Apr 19, 2017

Member

Sorry I missed the 'OBJ_SEPARATOR': 'node.',

This comment has been minimized.

@refack

refack Apr 19, 2017

Member

Still it probably should be 'OBJ_SEPARATOR': '/node.', with no trailing slashes in other variables.

@refack

refack Apr 19, 2017

Member

Still it probably should be 'OBJ_SEPARATOR': '/node.', with no trailing slashes in other variables.

Show outdated Hide outdated node.gyp
@refack

This comment has been minimized.

Show comment
Hide comment
@refack

refack Apr 18, 2017

Member

I have a feeling the slash manipulation change in node.gyp is not necessary (or might even degrade ninja output on Windows). Is there a way for you to gist a before and after outputs? for us to compare?

Member

refack commented Apr 18, 2017

I have a feeling the slash manipulation change in node.gyp is not necessary (or might even degrade ninja output on Windows). Is there a way for you to gist a before and after outputs? for us to compare?

@danbev

This comment has been minimized.

Show comment
Hide comment
@danbev

danbev Apr 18, 2017

Member

I have a feeling the slash manipulation change in node.gyp is not necessary (or might even degrade ninja output on Windows).

I'll run the ninja build on windows to verify that it works there too.

Member

danbev commented Apr 18, 2017

I have a feeling the slash manipulation change in node.gyp is not necessary (or might even degrade ninja output on Windows).

I'll run the ninja build on windows to verify that it works there too.

@Fishrock123

This comment has been minimized.

Show comment
Hide comment
@Fishrock123

Fishrock123 Apr 18, 2017

Member

FWIW building with ninja on master works with or without this for me. macOS 10.12.4, ninja 1.6.0.

Member

Fishrock123 commented Apr 18, 2017

FWIW building with ninja on master works with or without this for me. macOS 10.12.4, ninja 1.6.0.

@danbev

This comment has been minimized.

Show comment
Hide comment
@danbev

danbev Apr 19, 2017

Member

FWIW building with ninja on master works with or without this for me. macOS 10.12.4, ninja 1.6.0.

Is it possible that you already have built the project, which would mean that you have existing object files in out/Release/obj.target/node/src and the ninja build would in that case succeed?
I think doing a clean will show this error.

Member

danbev commented Apr 19, 2017

FWIW building with ninja on master works with or without this for me. macOS 10.12.4, ninja 1.6.0.

Is it possible that you already have built the project, which would mean that you have existing object files in out/Release/obj.target/node/src and the ninja build would in that case succeed?
I think doing a clean will show this error.

@Fishrock123

This comment has been minimized.

Show comment
Hide comment
@Fishrock123

Fishrock123 Apr 19, 2017

Member

Ah, I thought I did make clean yesterday but apparently not. Yes, I also have the issue and this fixes it.

Member

Fishrock123 commented Apr 19, 2017

Ah, I thought I did make clean yesterday but apparently not. Yes, I also have the issue and this fixes it.

@danbev

This comment has been minimized.

Show comment
Hide comment
@danbev

danbev Apr 19, 2017

Member

I've managed to run this on windows now and I've identified two issues. One is a simple fix where the win settings were overriding the ones set by the ninja condition.

The second issue I found was that the libraries in the cctest target in node.gyp, which in our case are object files, are getting a .lib extension in out/Release/obj/cctest.ninja. This causes a failure as there are no such files.
I've added a condition to tools/gyp/pylib/gyp/msvs_emulation.py to check for this situation. Not sure if this is a good solution or not but perhaps others can chime in now that they know the issue.

These are the commands I used to build:

> python configure --debug --ninja --dest-cpu=x86 --without-intl
> tools\gyp_node.py -f ninja
> ninja -C out\Release
> out\Release\cctest.exe
Member

danbev commented Apr 19, 2017

I've managed to run this on windows now and I've identified two issues. One is a simple fix where the win settings were overriding the ones set by the ninja condition.

The second issue I found was that the libraries in the cctest target in node.gyp, which in our case are object files, are getting a .lib extension in out/Release/obj/cctest.ninja. This causes a failure as there are no such files.
I've added a condition to tools/gyp/pylib/gyp/msvs_emulation.py to check for this situation. Not sure if this is a good solution or not but perhaps others can chime in now that they know the issue.

These are the commands I used to build:

> python configure --debug --ninja --dest-cpu=x86 --without-intl
> tools\gyp_node.py -f ninja
> ninja -C out\Release
> out\Release\cctest.exe
@danbev

This comment has been minimized.

Show comment
Hide comment
Member

danbev commented Apr 19, 2017

@refack

Try to remove trailing slashes, and use prefixed slashes instead.

'<(OBJ_TRACING_PATH)<(OBJ_SEPARATOR)agent.<(OBJ_SUFFIX)',
'<(OBJ_TRACING_PATH)<(OBJ_SEPARATOR)node_trace_buffer.<(OBJ_SUFFIX)',
'<(OBJ_TRACING_PATH)<(OBJ_SEPARATOR)node_trace_writer.<(OBJ_SUFFIX)',
'<(OBJ_TRACING_PATH)<(OBJ_SEPARATOR)trace_event.<(OBJ_SUFFIX)',

This comment has been minimized.

@refack

refack Apr 19, 2017

Member

Still it probably should be 'OBJ_SEPARATOR': '/node.', with no trailing slashes in other variables.

@refack

refack Apr 19, 2017

Member

Still it probably should be 'OBJ_SEPARATOR': '/node.', with no trailing slashes in other variables.

@refack

This comment has been minimized.

Show comment
Hide comment
@refack

refack Apr 19, 2017

Member

@danbev sorry you got into GYP hell. If When you get frustrated, my best advice, do like @bnoordhuis in #12231, move the crazy logic into a python file. You can then call it with <! or <!@ https://gyp.gsrc.io/docs/InputFormatReference.md#Command-Expansions

Member

refack commented Apr 19, 2017

@danbev sorry you got into GYP hell. If When you get frustrated, my best advice, do like @bnoordhuis in #12231, move the crazy logic into a python file. You can then call it with <! or <!@ https://gyp.gsrc.io/docs/InputFormatReference.md#Command-Expansions

@refack

This comment has been minimized.

Show comment
Hide comment
@refack

refack Apr 19, 2017

Member

@danbev another tip: remember that GYP is just the Makefile generator. Running configure is cheap. Then rename output dir (out or Release) change .gyp files, rerun and text compare.

Member

refack commented Apr 19, 2017

@danbev another tip: remember that GYP is just the Makefile generator. Running configure is cheap. Then rename output dir (out or Release) change .gyp files, rerun and text compare.

@refack

This comment has been minimized.

Show comment
Hide comment
@refack

refack Apr 19, 2017

Member

@gibfahn @danbev @Fishrock123 I'm knee deep in the GYP code (they want the ninja generator to be VS2017 compatible), any thing I can fix while I'm there?

Member

refack commented Apr 19, 2017

@gibfahn @danbev @Fishrock123 I'm knee deep in the GYP code (they want the ninja generator to be VS2017 compatible), any thing I can fix while I'm there?

@danbev

This comment has been minimized.

Show comment
Hide comment
@danbev

This comment has been minimized.

Show comment
Hide comment
@danbev

danbev Apr 20, 2017

Member

Try to remove trailing slashes, and use prefixed slashes instead.

@refack I've used a prefixed slash now, let me know what you think.

Member

danbev commented Apr 20, 2017

Try to remove trailing slashes, and use prefixed slashes instead.

@refack I've used a prefixed slash now, let me know what you think.

@gibfahn

This comment has been minimized.

Show comment
Hide comment
@gibfahn

gibfahn Apr 20, 2017

Member

any thing I can fix while I'm there?

@refack the thing that irritates me with ninja is that if you do a ninja build and then a make test, make doesn't understand that the ninja build happened and rebuilds everything. I'm not sure how easy that would be to fix though, I suspect that might be beyond what make can do.

Also there could be a/some ninja based make target(s), but I don't know if collaborators would be okay with that.

The other thing is nodejs/node-gyp#1057, but that's mac specific, and I think upstream GYP aren't doing the xcodebuild stuff any more, so IDK if there's a way we can update gyp and avoid the warnings. In any case, warnings every time you configure are a pain.

Member

gibfahn commented Apr 20, 2017

any thing I can fix while I'm there?

@refack the thing that irritates me with ninja is that if you do a ninja build and then a make test, make doesn't understand that the ninja build happened and rebuilds everything. I'm not sure how easy that would be to fix though, I suspect that might be beyond what make can do.

Also there could be a/some ninja based make target(s), but I don't know if collaborators would be okay with that.

The other thing is nodejs/node-gyp#1057, but that's mac specific, and I think upstream GYP aren't doing the xcodebuild stuff any more, so IDK if there's a way we can update gyp and avoid the warnings. In any case, warnings every time you configure are a pain.

@bnoordhuis

This comment has been minimized.

Show comment
Hide comment
@bnoordhuis

bnoordhuis Apr 20, 2017

Member

Also there could be a/some ninja based make target(s), but I don't know if collaborators would be okay with that.

The Makefile used to have them but I removed it in de224d6. I don't have anything against ninja but approximately no one was using it at the time and it was slowing down the make build.

Member

bnoordhuis commented Apr 20, 2017

Also there could be a/some ninja based make target(s), but I don't know if collaborators would be okay with that.

The Makefile used to have them but I removed it in de224d6. I don't have anything against ninja but approximately no one was using it at the time and it was slowing down the make build.

@gibfahn

This comment has been minimized.

Show comment
Hide comment
@gibfahn

gibfahn Apr 20, 2017

Member

approximately no one was using it at the time and it was slowing down the make build.

How much does it slow down the build? It's interesting that people don't use it, I find it faster and cleaner to use, the only issue is that it doesn't play well with other make targets.

Member

gibfahn commented Apr 20, 2017

approximately no one was using it at the time and it was slowing down the make build.

How much does it slow down the build? It's interesting that people don't use it, I find it faster and cleaner to use, the only issue is that it doesn't play well with other make targets.

@gibfahn

This comment has been minimized.

Show comment
Hide comment
@gibfahn

gibfahn Apr 20, 2017

Member

@refack see #12531 regarding GYP errors on macOS.

Member

gibfahn commented Apr 20, 2017

@refack see #12531 regarding GYP errors on macOS.

@bnoordhuis

This comment has been minimized.

Show comment
Hide comment
@bnoordhuis

bnoordhuis Apr 20, 2017

Member

It's interesting that people don't use it, I find it faster and cleaner to use

I speculate that most people invoke ninja directly rather than through make (at least that is what I did - you use ninja because it's faster whereas the Makefile is a lumbering behemoth.)

Member

bnoordhuis commented Apr 20, 2017

It's interesting that people don't use it, I find it faster and cleaner to use

I speculate that most people invoke ninja directly rather than through make (at least that is what I did - you use ninja because it's faster whereas the Makefile is a lumbering behemoth.)

@refack

This comment has been minimized.

Show comment
Hide comment
@refack

refack Apr 20, 2017

Member

In #12425 I'm suggesting replacing make with ninja in order to eliminate the win/posix build disparity. Also it's faster (even than make).

Member

refack commented Apr 20, 2017

In #12425 I'm suggesting replacing make with ninja in order to eliminate the win/posix build disparity. Also it's faster (even than make).

@refack

This comment has been minimized.

Show comment
Hide comment
@refack

refack Apr 20, 2017

Member

Code looks better, but ninja on windows fails:

D:\code\node\out.ninja\out\Debug$ ninja -j 3 -v -f build.ninja
[1/3] C:\bin\dev\python27\python.exe gyp-win-tool link-with-manifests environment.x64 True node.exe "C:\bin\dev\python27\python.exe gyp-win-tool link-wrapper environment.x64 False link.exe /nologo /OUT:node.exe @node.exe.rsp" 1 mt.exe rc.exe "obj\node.node.exe.intermediate.manifest" obj\node.node.exe.generated.manifest ..\..\..\src\res\node.exe.extra.manifest
[2/3] C:\bin\dev\python27\python.exe gyp-win-tool stamp obj\cctest.actions_depends.stamp
[3/3] C:\bin\dev\python27\python.exe gyp-win-tool link-with-manifests environment.x64 True cctest.exe "C:\bin\dev\python27\python.exe gyp-win-tool link-wrapper environment.x64 False link.exe /nologo /OUT:cctest.exe @cctest.exe.rsp" 1 mt.exe rc.exe "obj\cctest.cctest.exe.intermediate.manifest" obj\cctest.cctest.exe.generated.manifest ..\..\..\src\res\node.exe.extra.manifest
FAILED: cctest.exe cctest.exe.pdb
C:\bin\dev\python27\python.exe gyp-win-tool link-with-manifests environment.x64 True cctest.exe "C:\bin\dev\python27\python.exe gyp-win-tool link-wrapper environment.x64 False link.exe /nologo /OUT:cctest.exe @cctest.exe.rsp" 1 mt.exe rc.exe "obj\cctest.cctest.exe.intermediate.manifest" obj\cctest.cctest.exe.generated.manifest ..\..\..\src\res\node.exe.extra.manifest
LINK : fatal error LNK1104: cannot open file 'obj/gen/node.node_javascript.obj.lib'

But since I have my knees deep in GYP I'm following up aswell.

Member

refack commented Apr 20, 2017

Code looks better, but ninja on windows fails:

D:\code\node\out.ninja\out\Debug$ ninja -j 3 -v -f build.ninja
[1/3] C:\bin\dev\python27\python.exe gyp-win-tool link-with-manifests environment.x64 True node.exe "C:\bin\dev\python27\python.exe gyp-win-tool link-wrapper environment.x64 False link.exe /nologo /OUT:node.exe @node.exe.rsp" 1 mt.exe rc.exe "obj\node.node.exe.intermediate.manifest" obj\node.node.exe.generated.manifest ..\..\..\src\res\node.exe.extra.manifest
[2/3] C:\bin\dev\python27\python.exe gyp-win-tool stamp obj\cctest.actions_depends.stamp
[3/3] C:\bin\dev\python27\python.exe gyp-win-tool link-with-manifests environment.x64 True cctest.exe "C:\bin\dev\python27\python.exe gyp-win-tool link-wrapper environment.x64 False link.exe /nologo /OUT:cctest.exe @cctest.exe.rsp" 1 mt.exe rc.exe "obj\cctest.cctest.exe.intermediate.manifest" obj\cctest.cctest.exe.generated.manifest ..\..\..\src\res\node.exe.extra.manifest
FAILED: cctest.exe cctest.exe.pdb
C:\bin\dev\python27\python.exe gyp-win-tool link-with-manifests environment.x64 True cctest.exe "C:\bin\dev\python27\python.exe gyp-win-tool link-wrapper environment.x64 False link.exe /nologo /OUT:cctest.exe @cctest.exe.rsp" 1 mt.exe rc.exe "obj\cctest.cctest.exe.intermediate.manifest" obj\cctest.cctest.exe.generated.manifest ..\..\..\src\res\node.exe.extra.manifest
LINK : fatal error LNK1104: cannot open file 'obj/gen/node.node_javascript.obj.lib'

But since I have my knees deep in GYP I'm following up aswell.

@danbev

This comment has been minimized.

Show comment
Hide comment
@danbev

danbev Apr 20, 2017

Member

Code looks better, but ninja on windows fails:

LINK : fatal error LNK1104: cannot open file 'obj/gen/node.node_javascript.obj.lib'

This does not look like you have the the changes in 5ddca82. Could you double check?

Member

danbev commented Apr 20, 2017

Code looks better, but ninja on windows fails:

LINK : fatal error LNK1104: cannot open file 'obj/gen/node.node_javascript.obj.lib'

This does not look like you have the the changes in 5ddca82. Could you double check?

@refack

This comment has been minimized.

Show comment
Hide comment
@refack

refack Apr 20, 2017

Member

This does not look like you have the the changes in 5ddca82. Could you double check?

Clashing GYP patches. I have patched that line to fix a different bug.
I feel like if you have to patch GYP in order to enable a feature (#11956), you might be doing something wrong 😉

Really no disrespect intended, GYP is super fragile, and it's baby .gyp files are super convoluted.
What do you say we roll back #11956, and work on it from a wider perspective together?

Member

refack commented Apr 20, 2017

This does not look like you have the the changes in 5ddca82. Could you double check?

Clashing GYP patches. I have patched that line to fix a different bug.
I feel like if you have to patch GYP in order to enable a feature (#11956), you might be doing something wrong 😉

Really no disrespect intended, GYP is super fragile, and it's baby .gyp files are super convoluted.
What do you say we roll back #11956, and work on it from a wider perspective together?

@danbev

This comment has been minimized.

Show comment
Hide comment
@danbev

danbev Apr 20, 2017

Member

Clashing GYP patches. I have patched that line to fix a different bug.
I feel like if you have to patch GYP in order to enable a feature (#11956), you might be doing something wrong 😉

It sounds great if this can be improved upon but I think that might be better for a different PR. Since I broke this functionality I wanted to fix it and it seems that it would be just as easy to merge this and then make the changes you are working on instead of leaving it broken. But I'm fine either way.

Member

danbev commented Apr 20, 2017

Clashing GYP patches. I have patched that line to fix a different bug.
I feel like if you have to patch GYP in order to enable a feature (#11956), you might be doing something wrong 😉

It sounds great if this can be improved upon but I think that might be better for a different PR. Since I broke this functionality I wanted to fix it and it seems that it would be just as easy to merge this and then make the changes you are working on instead of leaving it broken. But I'm fine either way.

@refack

This comment has been minimized.

Show comment
Hide comment
@refack

refack Apr 20, 2017

Member

It sounds great if this can be improved upon but I think that might be better for a different PR. Since I broke this functionality I wanted to fix it and it seems that it would be just as easy to merge this and then make the changes you are working on instead of leaving it broken. But I'm fine either way.

I'm happy you're open to the idea. I'll think about it some more, whether we can get #11956 in while minimizing impact on GYP.

Member

refack commented Apr 20, 2017

It sounds great if this can be improved upon but I think that might be better for a different PR. Since I broke this functionality I wanted to fix it and it seems that it would be just as easy to merge this and then make the changes you are working on instead of leaving it broken. But I'm fine either way.

I'm happy you're open to the idea. I'll think about it some more, whether we can get #11956 in while minimizing impact on GYP.

@refack

This comment has been minimized.

Show comment
Hide comment
@refack

refack Apr 20, 2017

Member

(context) I'm pushing stuff upstream GYP that's why I'd rather parallelize then serialize the patches. If you've hit a real bug, I wanna push it upstream as well.

Member

refack commented Apr 20, 2017

(context) I'm pushing stuff upstream GYP that's why I'd rather parallelize then serialize the patches. If you've hit a real bug, I wanna push it upstream as well.

danbev added some commits Apr 18, 2017

build: fix ninja build failure
When working on commit 6a09a69
("build: enable cctest to use generated objects") I did not take into
account building with ninja:

$ ./configure
$ tools/gyp_node.py -f ninja
$ ninja -C out/Release
$ ln -fs out/Release/node node

When ninja generated the ninja build files, src files that are
relative to the src directory will be named with a dot instead of a
path separator, for example:

out/Release/obj/src/node/node.o
would instead become:
out/Release/obj/src/node.node.o

This commit adds an additional variable for the typ of object separator
used for this case.

Also, the LIBS specified in the 'libraries' section are not
being included in the --start-group --end-group section which
means that these libraries will not be searched causing issue
with linkers where the order matters.

Fixes: #12448
add check for .obj file extensions
Currently the files specified in libraries in node.gyp cctest target are
getting a '.lib' extension on windows when generated with ninja.
This commit adds a check to see if a file has a '.obj' extension and in
that case no '.lib' extension will be added.
make if/else out of the generator statement
Currently the check for if ninja is being used is a normal if statement
as are the following os checks (win and aix). But the win and aix ones
should only be evaluated if the build is not generated by ninja.
This commit turns this logic into an if ninja else statement.
remove debug-agent from cctest target
This file has been removed in upstream/master but was still included in
the cctest target causing an error.
@danbev

This comment has been minimized.

Show comment
Hide comment
@danbev

danbev May 11, 2017

Member

I've rebased this with upstream master and checked that it works locally.

The instructions we have about building with ninja will currently fail and I'm suggesting that we merge this until we have a better solution. If not I'd like to close this pull request.

@refack What do you think about this?

Member

danbev commented May 11, 2017

I've rebased this with upstream master and checked that it works locally.

The instructions we have about building with ninja will currently fail and I'm suggesting that we merge this until we have a better solution. If not I'd like to close this pull request.

@refack What do you think about this?

@danbev

This comment has been minimized.

Show comment
Hide comment
Member

danbev commented May 11, 2017

@gibfahn

This comment has been minimized.

Show comment
Hide comment
@gibfahn

gibfahn May 11, 2017

Member

If this doesn't break anything I'd like to get this landed so we can fix master. I'm all for working towards a better solution, but I don't want to leave master broken while we do that.

@refack LGTY?

Member

gibfahn commented May 11, 2017

If this doesn't break anything I'd like to get this landed so we can fix master. I'm all for working towards a better solution, but I don't want to leave master broken while we do that.

@refack LGTY?

@refack

This comment has been minimized.

Show comment
Hide comment
@refack

refack May 11, 2017

Member

I'm really weary of patching GYP, if we have to let's make sure it's in a different commit that isn't squashed.
Give me a few hours to look this over.

P.S. there might be another bug with building cctest: #12952

Member

refack commented May 11, 2017

I'm really weary of patching GYP, if we have to let's make sure it's in a different commit that isn't squashed.
Give me a few hours to look this over.

P.S. there might be another bug with building cctest: #12952

@refack

refack approved these changes May 11, 2017

@refack

This comment has been minimized.

Show comment
Hide comment
@refack

refack May 11, 2017

Member

Let me land this...

Member

refack commented May 11, 2017

Let me land this...

@refack

This comment has been minimized.

Show comment
Hide comment
@refack

refack May 11, 2017

Member

Landed in: 9fd22bc

Member

refack commented May 11, 2017

Landed in: 9fd22bc

@refack refack closed this May 11, 2017

refack added a commit that referenced this pull request May 11, 2017

build: fix ninja build failure
When working on commit 6a09a69
("build: enable cctest to use generated objects") I did not take into
account building with ninja:

$ ./configure
$ tools/gyp_node.py -f ninja
$ ninja -C out/Release
$ ln -fs out/Release/node node

When ninja generated the ninja build files, src files that are
relative to the src directory will be named with a dot instead of a
path separator, for example:

out/Release/obj/src/node/node.o
would instead become:
out/Release/obj/src/node.node.o

This commit adds an additional variable for the type of object separator
used for this case.

Currently the check for if ninja is being used is a normal if statement
as are the following os checks (win and aix). But the win and aix ones
should only be evaluated if the build is not generated by ninja.
This commit turns this logic into an if ninja else statement.

PR-URL: #12484
Fixes: #12448
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>

refack added a commit that referenced this pull request May 11, 2017

build: fix ninja build failure (GYP patch)
Currently the files specified in libraries in node.gyp `cctest` target are
getting a '.lib' extension on windows when generated with ninja.
This commit adds a check to see if a file has a '.obj' extension and in
that case no '.lib' extension will be added.

Also, the LIBS specified in the 'libraries' section are not
being included in the --start-group --end-group section which
means that these libraries will not be searched causing issue
with linkers where the order matters.

PR-URL: #12484
Fixes: #12448
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
@danbev

This comment has been minimized.

Show comment
Hide comment
@danbev

danbev May 11, 2017

Member

@refack Thanks!

Member

danbev commented May 11, 2017

@refack Thanks!

@refack

This comment has been minimized.

Show comment
Hide comment
@refack
Member

refack commented May 11, 2017

refack added a commit to refack/node that referenced this pull request May 16, 2017

build: fix ninja build failure (GYP patch)
Currently the files specified in libraries in node.gyp `cctest` target are
getting a '.lib' extension on windows when generated with ninja.
This commit adds a check to see if a file has a '.obj' extension and in
that case no '.lib' extension will be added.

Also, the LIBS specified in the 'libraries' section are not
being included in the --start-group --end-group section which
means that these libraries will not be searched causing issue
with linkers where the order matters.

PR-URL: nodejs#12484
Fixes: nodejs#12448
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>

anchnk added a commit to anchnk/node that referenced this pull request May 19, 2017

build: fix ninja build failure
When working on commit 6a09a69
("build: enable cctest to use generated objects") I did not take into
account building with ninja:

$ ./configure
$ tools/gyp_node.py -f ninja
$ ninja -C out/Release
$ ln -fs out/Release/node node

When ninja generated the ninja build files, src files that are
relative to the src directory will be named with a dot instead of a
path separator, for example:

out/Release/obj/src/node/node.o
would instead become:
out/Release/obj/src/node.node.o

This commit adds an additional variable for the type of object separator
used for this case.

Currently the check for if ninja is being used is a normal if statement
as are the following os checks (win and aix). But the win and aix ones
should only be evaluated if the build is not generated by ninja.
This commit turns this logic into an if ninja else statement.

PR-URL: nodejs#12484
Fixes: nodejs#12448
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>

anchnk added a commit to anchnk/node that referenced this pull request May 19, 2017

build: fix ninja build failure (GYP patch)
Currently the files specified in libraries in node.gyp `cctest` target are
getting a '.lib' extension on windows when generated with ninja.
This commit adds a check to see if a file has a '.obj' extension and in
that case no '.lib' extension will be added.

Also, the LIBS specified in the 'libraries' section are not
being included in the --start-group --end-group section which
means that these libraries will not be searched causing issue
with linkers where the order matters.

PR-URL: nodejs#12484
Fixes: nodejs#12448
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>

@jasnell jasnell referenced this pull request May 28, 2017

Closed

8.0.0 Release Proposal #12220

@gibfahn gibfahn referenced this pull request Jun 15, 2017

Closed

Auditing for 6.11.1 #230

2 of 3 tasks complete
@gibfahn

This comment has been minimized.

Show comment
Hide comment
@gibfahn

gibfahn Jun 20, 2017

Member

Fix for #11956, so should be backported with that.

Member

gibfahn commented Jun 20, 2017

Fix for #11956, so should be backported with that.

@gibfahn gibfahn referenced this pull request Jun 20, 2017

Closed

build: enable cctest to use generated objects #11956

2 of 2 tasks complete

@danbev danbev deleted the danbev:ninja-build branch Jun 28, 2017

sam-github pushed a commit to sam-github/node that referenced this pull request Jul 12, 2017

build: fix ninja build failure (GYP patch)
Currently the files specified in libraries in node.gyp `cctest` target are
getting a '.lib' extension on windows when generated with ninja.
This commit adds a check to see if a file has a '.obj' extension and in
that case no '.lib' extension will be added.

Also, the LIBS specified in the 'libraries' section are not
being included in the --start-group --end-group section which
means that these libraries will not be searched causing issue
with linkers where the order matters.

PR-URL: nodejs#12484
Fixes: nodejs#12448
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>

sam-github pushed a commit to sam-github/node that referenced this pull request Jul 18, 2017

build: fix ninja build failure (GYP patch)
Currently the files specified in libraries in node.gyp `cctest` target are
getting a '.lib' extension on windows when generated with ninja.
This commit adds a check to see if a file has a '.obj' extension and in
that case no '.lib' extension will be added.

Also, the LIBS specified in the 'libraries' section are not
being included in the --start-group --end-group section which
means that these libraries will not be searched causing issue
with linkers where the order matters.

PR-URL: nodejs#12484
Fixes: nodejs#12448
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>

refack added a commit to refack/node that referenced this pull request Jul 18, 2017

build: fix ninja build failure (GYP patch)
Currently the files specified in libraries in node.gyp `cctest` target are
getting a '.lib' extension on windows when generated with ninja.
This commit adds a check to see if a file has a '.obj' extension and in
that case no '.lib' extension will be added.

Also, the LIBS specified in the 'libraries' section are not
being included in the --start-group --end-group section which
means that these libraries will not be searched causing issue
with linkers where the order matters.

PR-URL: nodejs#12484
Fixes: nodejs#12448
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>

refack added a commit to refack/node that referenced this pull request Aug 9, 2017

gyp: fix ninja build failure (GYP patch)
Currently the files specified in libraries in node.gyp `cctest` target are
getting a '.lib' extension on windows when generated with ninja.
This commit adds a check to see if a file has a '.obj' extension and in
that case no '.lib' extension will be added.

Also, the LIBS specified in the 'libraries' section are not
being included in the --start-group --end-group section which
means that these libraries will not be searched causing issue
with linkers where the order matters.

PR-URL: nodejs#12484
Fixes: nodejs#12448
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>

@refack refack added the embedding label Aug 17, 2017

refack added a commit to refack/node that referenced this pull request Aug 19, 2017

gyp: fix ninja build failure (GYP patch)
Currently the files specified in libraries in node.gyp `cctest` target are
getting a '.lib' extension on windows when generated with ninja.
This commit adds a check to see if a file has a '.obj' extension and in
that case no '.lib' extension will be added.

Also, the LIBS specified in the 'libraries' section are not
being included in the --start-group --end-group section which
means that these libraries will not be searched causing issue
with linkers where the order matters.

PR-URL: nodejs#12484
Fixes: nodejs#12448
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>

refack added a commit to refack/node that referenced this pull request Aug 23, 2017

gyp: fix ninja build failure (GYP patch)
Currently the files specified in libraries in node.gyp `cctest` target are
getting a '.lib' extension on windows when generated with ninja.
This commit adds a check to see if a file has a '.obj' extension and in
that case no '.lib' extension will be added.

Also, the LIBS specified in the 'libraries' section are not
being included in the --start-group --end-group section which
means that these libraries will not be searched causing issue
with linkers where the order matters.

PR-URL: nodejs#12484
Fixes: nodejs#12448
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>

addaleax added a commit to addaleax/ayo that referenced this pull request Aug 25, 2017

gyp: fix ninja build failure (GYP patch)
Currently the files specified in libraries in node.gyp `cctest` target are
getting a '.lib' extension on windows when generated with ninja.
This commit adds a check to see if a file has a '.obj' extension and in
that case no '.lib' extension will be added.

Also, the LIBS specified in the 'libraries' section are not
being included in the --start-group --end-group section which
means that these libraries will not be searched causing issue
with linkers where the order matters.

PR-URL: nodejs/node#12484
Fixes: nodejs/node#12448
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>

addaleax added a commit to ayojs/ayo that referenced this pull request Aug 28, 2017

gyp: fix ninja build failure (GYP patch)
Currently the files specified in libraries in node.gyp `cctest` target are
getting a '.lib' extension on windows when generated with ninja.
This commit adds a check to see if a file has a '.obj' extension and in
that case no '.lib' extension will be added.

Also, the LIBS specified in the 'libraries' section are not
being included in the --start-group --end-group section which
means that these libraries will not be searched causing issue
with linkers where the order matters.

PR-URL: nodejs/node#12484
Fixes: nodejs/node#12448
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>

MylesBorins added a commit that referenced this pull request Sep 10, 2017

gyp: fix ninja build failure (GYP patch)
Currently the files specified in libraries in node.gyp `cctest` target are
getting a '.lib' extension on windows when generated with ninja.
This commit adds a check to see if a file has a '.obj' extension and in
that case no '.lib' extension will be added.

Also, the LIBS specified in the 'libraries' section are not
being included in the --start-group --end-group section which
means that these libraries will not be searched causing issue
with linkers where the order matters.

PR-URL: #12484
Fixes: #12448
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>

@MylesBorins MylesBorins referenced this pull request Sep 10, 2017

Merged

v8.5.0 proposal #15308

MylesBorins added a commit that referenced this pull request Sep 12, 2017

gyp: fix ninja build failure (GYP patch)
Currently the files specified in libraries in node.gyp `cctest` target are
getting a '.lib' extension on windows when generated with ninja.
This commit adds a check to see if a file has a '.obj' extension and in
that case no '.lib' extension will be added.

Also, the LIBS specified in the 'libraries' section are not
being included in the --start-group --end-group section which
means that these libraries will not be searched causing issue
with linkers where the order matters.

PR-URL: #12484
Fixes: #12448
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>

maclover7 added a commit to maclover7/node-gyp that referenced this pull request Jul 31, 2018

gyp: fix ninja build failure (GYP patch)
Currently the files specified in libraries in node.gyp `cctest` target are
getting a '.lib' extension on windows when generated with ninja.
This commit adds a check to see if a file has a '.obj' extension and in
that case no '.lib' extension will be added.

Also, the LIBS specified in the 'libraries' section are not
being included in the --start-group --end-group section which
means that these libraries will not be searched causing issue
with linkers where the order matters.

PR-URL: nodejs/node#12484
Fixes: nodejs/node#12448
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>

rvagg added a commit to nodejs/node-gyp that referenced this pull request Aug 9, 2018

gyp: fix ninja build failure (GYP patch)
Currently the files specified in libraries in node.gyp `cctest` target are
getting a '.lib' extension on windows when generated with ninja.
This commit adds a check to see if a file has a '.obj' extension and in
that case no '.lib' extension will be added.

Also, the LIBS specified in the 'libraries' section are not
being included in the --start-group --end-group section which
means that these libraries will not be searched causing issue
with linkers where the order matters.

PR-URL: nodejs/node#12484
Fixes: nodejs/node#12448
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>

rvagg added a commit to nodejs/node-gyp that referenced this pull request Aug 9, 2018

gyp: fix ninja build failure (GYP patch)
Currently the files specified in libraries in node.gyp `cctest` target are
getting a '.lib' extension on windows when generated with ninja.
This commit adds a check to see if a file has a '.obj' extension and in
that case no '.lib' extension will be added.

Also, the LIBS specified in the 'libraries' section are not
being included in the --start-group --end-group section which
means that these libraries will not be searched causing issue
with linkers where the order matters.

PR-URL: nodejs/node#12484
Fixes: nodejs/node#12448
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment