zos: support platform #1276

jBarz · 2017-08-24T21:58:13Z

recognize a zos system
use correct cp arguments for zos
add zos compile options
add zos makedep arguments
use export file in link step on zos

Checklist

npm install && npm test passes
tests are included
documentation is changed or added
commit message follows commit guidelines

Description of change

jasnell · 2017-08-24T22:04:16Z

@nodejs/build ... thoughts on this?

richardlau · 2017-08-25T17:33:12Z

Are the gyp changes being upstreamed?

jBarz · 2017-08-25T17:34:07Z

Yes. I am in the process of doing that.

refack · 2017-08-25T18:17:40Z

Are the gyp changes being upstreamed?

Yes. I am in the process of doing that.

I don't think that upstream GYP has a zos try-bot, but at least we could get some regression testing.

refack · 2017-08-25T18:19:11Z

gyp/pylib/gyp/common.py

@@ -429,6 +429,8 @@ def GetFlavor(params):
    return 'netbsd'
  if sys.platform.startswith('aix'):
    return 'aix'
+  if sys.platform.startswith('os390'):
+    return 'os390'


if the platform is named zos why use os390?

Pretty sure that's what uname -m prints on z/OS machines.

Ack. So that's the input (L432), I'm wondering is we should keep it as the internal alias?
We could do something similar to:

node-gyp/gyp/pylib/gyp/common.py

Lines 422 to 423 in 2ed26fb

if sys.platform.startswith('sunos'):

return 'solaris'

if that's more congruent with the IBM's future plans.

A lot of z/OS projects built using gyp already expect 'os390'. So that would require a major change across multiple z/OS projects.

I meant the other way around, i.e.:

if sys.platform.startswith('os390'): return 'zos'

since the return os390 (a.k.a. flavor) is only used as an internal alias to map to

node-gyp/gyp/pylib/gyp/generator/make.py

Line 2060 in 2ed26fb

elif flavor == 'os390':

But in anycase, you IBM people know best. I'm just asking...

@jBarz we could just alias it inside the node build scripts I guess.

However I think os390 is the standard "technical" name for the ARCH, like x86_64 is for Intel, so it's probably be less surprising to stick with that.

@refack apologies, you're right that this is only internal alias.
@gibfahn is right though, 'os390' is a standard technical name for z/OS arch. So I would prefer to leave it as 'os390' if that is ok with you

refack

Approving as a floating patch since IMHO it's not going to be a trivial to upstream.

refack · 2017-08-28T16:34:17Z

CI: https://ci.nodejs.org/view/Node.js/job/nodegyp-test-pull-request/35/

gibfahn · 2017-08-28T19:03:52Z

Approving as a floating path since IMHO it's not going to be a trivial to upstream.

@jBarz you mentioned that other z/OS projects are using GYP, are there plans to upstream the gyp related changes?

refack

Need to fix some python syntax

refack · 2017-08-28T22:37:29Z

gyp/pylib/gyp/generator/ninja.py

@@ -2250,7 +2250,10 @@ def GenerateOutputForConfig(target_list, target_dicts, data, params,
    master_ninja.rule(
      'copy',
      description='COPY $in $out',
-      command='rm -rf $out && cp -af $in $out')
+      if sys.platform in ('os390'):


This is a python syntax error:

gyp info spawn args '-Goutput_dir=.' ] Traceback (most recent call last): File "c:\workspace\nodegyp-test-commit\nodes\win2012r2\gyp\gyp_main.py", line 16, in <module> sys.exit(gyp.script_main()) File "c:\workspace\nodegyp-test-commit\nodes\win2012r2\gyp\pylib\gyp\__init__.py", line 545, in script_main return main(sys.argv[1:]) File "c:\workspace\nodegyp-test-commit\nodes\win2012r2\gyp\pylib\gyp\__init__.py", line 538, in main return gyp_main(args) File "c:\workspace\nodegyp-test-commit\nodes\win2012r2\gyp\pylib\gyp\__init__.py", line 514, in gyp_main options.duplicate_basename_check) File "c:\workspace\nodegyp-test-commit\nodes\win2012r2\gyp\pylib\gyp\__init__.py", line 91, in Load generator = __import__(generator_name, globals(), locals(), generator_name) File "c:\workspace\nodegyp-test-commit\nodes\win2012r2\gyp\pylib\gyp\generator\msvs.py", line 15, in <module> import gyp.generator.ninja as ninja_generator File "c:\workspace\nodegyp-test-commit\nodes\win2012r2\gyp\pylib\gyp\generator\ninja.py", line 2253 if sys.platform in ('os390'): ^ SyntaxError: invalid syntax

You could do the if before hand near L2167, or use the python ternary

refack · 2017-08-28T22:38:55Z

lib/configure.js

@@ -270,6 +270,23 @@ function configure (gyp, argv, callback) {
        return callback(new Error(msg))
      }
    }
+    else if (process.platform === 'os390') {


usually else if should be on the same line as the previous }

refack · 2017-08-28T22:40:24Z

lib/configure.js

+           fs.accessSync(node_exp_file, fs.R_OK)
+           // exp file found, stop looking
+           break
+         } catch (exception) {


Should rename exception to _ to indicate it's intentionally ignored

refack · 2017-08-28T22:46:42Z

test/fixtures/test-charmap.py

@@ -4,6 +4,9 @@
 reload(sys)

 def main():
+  if sys.platform.startswith('os390'):


Could you replace this with

if not encoding: return False

after L10

jBarz · 2017-08-29T00:04:31Z

Fixed issues.

jBarz · 2017-08-29T00:06:19Z

@jBarz you mentioned that other z/OS projects are using GYP, are there plans to upstream the gyp related changes?

Yes, these changes will go upstream.

refack · 2017-08-29T02:13:45Z

Fixed issues.

Thanks for following up.
Let's see what the CI sais: https://ci.nodejs.org/job/nodegyp-test-pull-request/36/ ✔️

refack

LGTM

refack · 2017-08-29T02:17:08Z

@jBarz @gibfahn is what time frame would you like to see this out? Since Ben is on vacation, and I'd like for him to take a look.

jBarz · 2017-08-29T02:19:34Z

No urgency :-). We can wait

bnoordhuis

LGTM with a question.

bnoordhuis · 2017-08-29T06:17:57Z

lib/configure.js

+      for (var next = 0; next < candidates.length; next++) {
+         node_exp_file = path.resolve(node_root_dir, candidates[next])
+         try {
+           fs.accessSync(node_exp_file, fs.R_OK)


fs.accessSync() does not exist in v0.10 and v0.12 (see also #955) but since this is a z/OS-only code path, I guess that's alright. That said, can't this be merged with the AIX code path above?

Yes it can be merged with AIX code. Will do

jBarz · 2017-08-29T16:49:11Z

Merged the AIX and z/OS code paths to find the export file

jBarz · 2017-08-31T01:28:17Z

gyp changes have been pushed upstream.
https://chromium.googlesource.com/external/gyp/+/dd0eafbde598e1b7848a3d25f826bbcc34d6b3be

refack · 2017-08-31T01:44:54Z

Wow Dirk was quick :)
I'll incorporate into #1183

saper · 2017-09-02T18:15:09Z

@jBarz Is there any way a mere mortal can try this? For example using Hercules? (I only have ancient MVS running there).

bnoordhuis · 2017-09-04T14:43:17Z

lib/configure.js

+      var candidates = ['include/node/node',
+                        'out/Release/node',
+                        'out/Debug/node',
+                        'node'].map((file) => {


You can't use arrow functions just yet, they don't parse in old node.js versions.

bnoordhuis · 2017-09-04T14:43:39Z

lib/configure.js

      var logprefix = 'find exports file'
      node_exp_file = findAccessibleSync(logprefix, node_root_dir, candidates)
      if (node_exp_file !== undefined) {
        log.verbose(logprefix, 'Found exports file: %s', node_exp_file)
      } else {
-        var msg = msgFormat('Could not find node.exp file in %s', node_root_dir)
+        var msg = msgFormat(`Could not find node.${ext} file in %s`, node_root_dir)


Ditto: no backticks just yet.

bnoordhuis · 2017-09-05T20:16:04Z

lib/configure.js

+                        'out/Release/node',
+                        'out/Debug/node',
+                        'node'].map(function(file) {
+                          return `${file}.${ext}`


No backticks.

Sorry, i missed that one!

Why no backticks? node-gyp@4 is node>=4.

node-gyp/package.json

Lines 38 to 40 in 7245415

"engines": {

"node": ">= 4.0.0"

},

Well, that was a surreptitious change. I wouldn't have hidden that in an otherwise unrelated commit. (I'm talking about 5f924ce.)

At any rate, this way we can do another 3.x release and include this change.

Sure, no harm in keeping compatibility.

bnoordhuis · 2017-09-06T12:38:01Z

lib/configure.js

+                        'out/Release/node',
+                        'out/Debug/node',
+                        'node'].map(function(file) {
+                          return '' + file + '.' + ext


First (empty) concatenation isn't necessary but LGTM apart from that.

* recognize a zos system * use correct cp arguments for zos * add zos compile options * add zos makedep arguments * use export file in link step on zos

refack · 2017-09-07T18:28:24Z

Pre-land CI: https://ci.nodejs.org/job/nodegyp-test-pull-request/38/

jBarz · 2017-09-12T16:43:13Z

@refack Do you know approx when this could be landed and released?
I just wanted an estimate of the date for node-gyp to support z/OS.
z/OS support for nodejs was released this month so I suspect some of our users will run into this issue :-).
And the next release version is 4.0.0, correct?

gibfahn · 2017-09-26T18:03:40Z

I think this is good to go, given that the GYP changes have been upstreamed.

CI: https://ci.nodejs.org/view/Node.js/job/nodegyp-test-pull-request/40/

EDIT: CI is green.

jasnell requested a review from bnoordhuis August 24, 2017 22:04

jBarz force-pushed the v3.6.2.zos branch from 3bccc94 to 2ed26fb Compare August 24, 2017 22:54

refack reviewed Aug 25, 2017

View reviewed changes

refack approved these changes Aug 28, 2017

View reviewed changes

refack suggested changes Aug 28, 2017

View reviewed changes

jBarz force-pushed the v3.6.2.zos branch from 2ed26fb to a2e5bcc Compare August 28, 2017 23:59

refack approved these changes Aug 29, 2017

View reviewed changes

bnoordhuis approved these changes Aug 29, 2017

View reviewed changes

jBarz force-pushed the v3.6.2.zos branch from 6f9b1fd to ba07663 Compare August 31, 2017 01:30

bnoordhuis requested changes Sep 4, 2017

View reviewed changes

jBarz force-pushed the v3.6.2.zos branch from ba07663 to 1f29a35 Compare September 5, 2017 17:22

bnoordhuis requested changes Sep 5, 2017

View reviewed changes

jBarz force-pushed the v3.6.2.zos branch from 1f29a35 to a273968 Compare September 5, 2017 20:56

bnoordhuis approved these changes Sep 6, 2017

View reviewed changes

zos: support platform

cbedb1c

* recognize a zos system * use correct cp arguments for zos * add zos compile options * add zos makedep arguments * use export file in link step on zos

jBarz force-pushed the v3.6.2.zos branch from a273968 to cbedb1c Compare September 7, 2017 01:52

sync up with latest gyp z/OS changes

2011850

gibfahn approved these changes Sep 26, 2017

View reviewed changes

gibfahn merged commit a8ba528 into nodejs:master Sep 26, 2017

jBarz deleted the v3.6.2.zos branch September 26, 2017 18:40

richardlau mentioned this pull request Feb 10, 2020

ci,libuv: http://ci.nodejs.org/downloads/zos/gyp.tar.gz? nodejs/build#2172

Closed

DeeDeeG mentioned this pull request Sep 4, 2022

Upstream gyp has actually been updated somewhat recently. Review those changes and sync (some of) them? nodejs/gyp-next#108

Closed

zos: support platform #1276

zos: support platform #1276

Conversation

jBarz commented Aug 24, 2017 • edited Loading

Checklist

Description of change

jasnell commented Aug 24, 2017

richardlau commented Aug 25, 2017

jBarz commented Aug 25, 2017

refack commented Aug 25, 2017

refack Aug 25, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

refack Aug 26, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

refack Aug 28, 2017 • edited Loading

Choose a reason for hiding this comment

gibfahn Aug 28, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

refack left a comment • edited by gibfahn Loading

Choose a reason for hiding this comment

refack commented Aug 28, 2017

gibfahn commented Aug 28, 2017

refack left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jBarz commented Aug 29, 2017

jBarz commented Aug 29, 2017

refack commented Aug 29, 2017 • edited Loading

refack left a comment

Choose a reason for hiding this comment

refack commented Aug 29, 2017

jBarz commented Aug 29, 2017

bnoordhuis left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jBarz commented Aug 29, 2017

jBarz commented Aug 31, 2017 • edited Loading

refack commented Aug 31, 2017

saper commented Sep 2, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

refack commented Sep 7, 2017

jBarz commented Sep 12, 2017 • edited Loading

gibfahn commented Sep 26, 2017 • edited Loading

jBarz commented Aug 24, 2017 •

edited

Loading

refack Aug 25, 2017 •

edited

Loading

refack Aug 26, 2017 •

edited

Loading

refack Aug 28, 2017 •

edited

Loading

gibfahn Aug 28, 2017 •

edited

Loading

refack left a comment •

edited by gibfahn

Loading

refack commented Aug 29, 2017 •

edited

Loading

jBarz commented Aug 31, 2017 •

edited

Loading

jBarz commented Sep 12, 2017 •

edited

Loading

gibfahn commented Sep 26, 2017 •

edited

Loading