Updated gulp build (added rpm build and fixed linux build on macOS) #105

basdelfos · 2018-02-18T10:41:15Z

Fixed linux build on macOS (check if deb package exists)
Added RPM build for linux

Both fixes are as per BFC gulpfile.js

@mikeller: can you test if gulp successfully create the rpm package, i could't test that on my mac.

AndersHoglund · 2018-02-18T11:34:46Z

Seems to work fine on OSx (but no linux deb nor rpm packages produced by gulp, only the zip files).
On Linux I get this:

$ npm install
npm WARN registry Unexpected warning for https://registry.npmjs.org/: Miscellaneous Warning EINTEGRITY: sha512-WIr7iDkdmdbxu/Gh6eKEZJL6KPE74/5MEsf2whTOFNxbIoIixogroLdKYqB6FDav4Wavh/lZdzzd3b2KxIXC5Q== integrity checksum failed when using sha512: wanted sha512-WIr7iDkdmdbxu/Gh6eKEZJL6KPE74/5MEsf2whTOFNxbIoIixogroLdKYqB6FDav4Wavh/lZdzzd3b2KxIXC5Q== but got sha1-EfgjGPX+e7LNIpZaEI6TBiCCFtg=. (1837200 bytes)
npm WARN registry Using stale package data from https://registry.npmjs.org/ due to a request error during revalidation.
npm ERR! code EBADPLATFORM
npm ERR! notsup Unsupported platform for appdmg@0.4.5: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm ERR! notsup Valid OS:    darwin
npm ERR! notsup Valid Arch:  any
npm ERR! notsup Actual OS:   linux
npm ERR! notsup Actual Arch: x64

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/andho/.npm/_logs/2018-02-18T11_30_09_759Z-debug.log

AndersHoglund · 2018-02-18T11:53:36Z

Removed that hard dmg dependency in package.json, and re-ran on Linux.
Then there are still a a few warnings from npm install :

$ npm install
npm WARN registry Unexpected warning for https://registry.npmjs.org/: Miscellaneous Warning EINTEGRITY: sha512-WIr7iDkdmdbxu/Gh6eKEZJL6KPE74/5MEsf2whTOFNxbIoIixogroLdKYqB6FDav4Wavh/lZdzzd3b2KxIXC5Q== integrity checksum failed when using sha512: wanted sha512-WIr7iDkdmdbxu/Gh6eKEZJL6KPE74/5MEsf2whTOFNxbIoIixogroLdKYqB6FDav4Wavh/lZdzzd3b2KxIXC5Q== but got sha1-EfgjGPX+e7LNIpZaEI6TBiCCFtg=. (1837200 bytes)
npm WARN registry Using stale package data from https://registry.npmjs.org/ due to a request error during revalidation.

> betaflight-blackbox-explorer@3.0.0-rc1 postinstall /home/andho/projects/cleanflight/andwho/blackbox-log-viewer
> npm run _postinstall


> betaflight-blackbox-explorer@3.0.0-rc1 _postinstall /home/andho/projects/cleanflight/andwho/blackbox-log-viewer
> node ./node_modules/platform-dependent-modules/cli.js

* [platform-dependent-modules] process.cwd(): /home/andho/projects/cleanflight/andwho/blackbox-log-viewer __dirname: /home/andho/projects/cleanflight/andwho/blackbox-log-viewer/node_modules/platform-dependent-modules/lib
* [platform-dependent-modules] No packages to install, skipping.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.1.3 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.1.3: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

up to date in 6.692s

And gulp release --linux64 --linux32 ends up with this:

[12:44:44] 'release_linux64_rpm' errored after 274 ms
[12:44:44] ReferenceError: getLinuxPackageArch is not defined
    at release_rpm (/home/andho/projects/cleanflight/andwho/blackbox-log-viewer/gulpfile.js:529:27)
    at release_linux64_rpm (/home/andho/projects/cleanflight/andwho/blackbox-log-viewer/gulpfile.js:614:70)
    at bound (domain.js:301:14)
    at runBound (domain.js:314:12)
    at asyncRunner (/home/andho/projects/cleanflight/andwho/blackbox-log-viewer/node_modules/async-done/index.js:55:18)
    at _combinedTickCallback (internal/process/next_tick.js:131:7)
    at process._tickDomainCallback (internal/process/next_tick.js:218:9)
[12:44:44] 'release' errored after 33 s
[12:44:44] The following tasks did not complete: release_linux64_zip, release_linux64_deb, release_linux32_zip, release_linux32_deb, release_linux32_rpm
[12:44:44] Did you forget to signal async completion?

McGiverGim · 2018-02-18T11:55:51Z

Yes, the method getLinuxPackageArch exists in bfc. Maybe has not been ported.

mikeller

I've commented on the required changes. After this the .rpm package builds fine. I haven't tested installing it, since I am running ubuntu.

mikeller · 2018-02-18T13:09:03Z

package.json

  "license": "GPL-3.0",
-  "dependencies": {},
+  "dependencies": {
+    "gulp-appdmg": "^1.0.3"


We can't have this dependency in here - this will keep npm install from working on non-macosx. (Also, it is not a runtime dependency at all.)

The way that platform-dependent-modules works, and the consequence that package.json is modified every time a platform dependent module is installed, makes it hard and tedious to work with. I have tried some alternative, but wasn't able to find one that actually works better in this respect.

That's odd, didn't move that dependency as I know it belongs in the platform dependent modules. I think the clean npm install did this automatically. I will check if I can reproduce it, if this is the case then this can happen more often when someone does a clean modules install.

Yes, unfortunately platform-dependent-modules does this on npm install, making it a pain to develop with it.

OK, saves me time to reproduce this. I will change it back and take extra care with future npm install clean installs

mikeller · 2018-02-18T13:16:47Z

gulpfile.js

+    var options = {
+             name: pkg.name,
+             version: pkg.version,
+             buildArch: getLinuxPackageArch('rpm', arch),


getLinuxPackageArch() needs to be defined (grab it from gulpfile.js of the betaflight-configurator.

Oh, that one slipped through.

mikeller · 2018-02-18T13:18:26Z

gulpfile.js

+             rpmDest: RELEASE_DIR
+    };
+
+    buildRpm(options, function(err, rpm) {


buildRpm needs to be defined: const buildRpm = require('rpm-builder').

Also, turns out that buildRpm does not like the -rc1 in the package version. We'll have to sanitise this for buildRpm, or stop using SemVer prereleases.

basdelfos · 2018-02-19T21:22:48Z

Made the changes as in the review comments, can someone try again if the rpm build works now. Thanks.

AndersHoglund · 2018-02-19T23:23:27Z

Building on Linux and on OSx seems to produce packages OK. Not tried any actual install from any of the packages though, just built them.
There are however the two new warnings when building on Linux that are not seen when building on master.:

$ npm install
npm WARN registry Unexpected warning for https://registry.npmjs.org/: Miscellaneous Warning EINTEGRITY: sha512-WIr7iDkdmdbxu/Gh6eKEZJL6KPE74/5MEsf2whTOFNxbIoIixogroLdKYqB6FDav4Wavh/lZdzzd3b2KxIXC5Q== integrity checksum failed when using sha512: wanted sha512-WIr7iDkdmdbxu/Gh6eKEZJL6KPE74/5MEsf2whTOFNxbIoIixogroLdKYqB6FDav4Wavh/lZdzzd3b2KxIXC5Q== but got sha1-EfgjGPX+e7LNIpZaEI6TBiCCFtg=. (1837200 bytes)
npm WARN registry Using stale package data from https://registry.npmjs.org/ due to a request error during revalidation.
..........

Also package-lock.json becomes modified, probably due to the warnings above:

On OSx both package.json and package-lock.json becomes modified by npm install
Nothing tested on Windows.

AndersHoglund · 2018-02-19T23:34:49Z

On Linux.

$ git diff
diff --git a/package-lock.json b/package-lock.json
index 698a638..544999c 100644
--- a/package-lock.json
+++ b/package-lock.json
@@ -1,6 +1,6 @@
 {
   "name": "betaflight-blackbox-explorer",
-  "version": "3.0.0-rc1",
+  "version": "3.0.0",
   "lockfileVersion": 1,
   "requires": true,
   "dependencies": {
@@ -26,7 +26,7 @@
     "ansi-colors": {
       "version": "1.0.1",
       "resolved": "https://registry.npmjs.org/ansi-colors/-/ansi-colors-1.0.1.tgz",
-      "integrity": "sha1-6UxsMGAFr4tIIkAkHi896kuFX/M=",
+      "integrity": "sha512-yopkAU0ZD/WQ56Tms3xLn6jRuX3SyUMAVi0FdmDIbmmnHW3jHiI1sQFdUl3gfVddjnrsP3Y6ywFKvCRopvoVIA==",
       "dev": true,
       "requires": {
         "ansi-wrap": "0.1.0"
@@ -45,6 +45,7 @@
       "version": "0.1.1",
       "resolved": "https://registry.npmjs.org/ansi-gray/-/ansi-gray-0.1.1.tgz",
       "integrity": "sha1-KWLPVOyXksSFEKPetSRDaGHvclE=",
.......

AndersHoglund · 2018-02-19T23:39:32Z

On OSx

$ git diff package.json 
diff --git a/package.json b/package.json
index b6c3756..c433674 100644
--- a/package.json
+++ b/package.json
@@ -22,7 +22,9 @@
   },
   "author": "The Betaflight open source project.",
   "license": "GPL-3.0",
-  "dependencies": {},
+  "dependencies": {
+    "gulp-appdmg": "^1.0.3"
+  },
   "devDependencies": {
     "command-exists": "^1.2.2",
     "del": "^3.0.0",

And

$ git diff package-lock.json 
diff --git a/package-lock.json b/package-lock.json
index 698a638..fec6567 100644
--- a/package-lock.json
+++ b/package-lock.json
@@ -1,6 +1,6 @@
 {
   "name": "betaflight-blackbox-explorer",
-  "version": "3.0.0-rc1",
+  "version": "3.0.0",
   "lockfileVersion": 1,
   "requires": true,
   "dependencies": {
@@ -45,6 +45,7 @@
       "version": "0.1.1",
       "resolved": "https://registry.npmjs.org/ansi-gray/-/ansi-gray-0.1.1.tgz",
       "integrity": "sha1-KWLPVOyXksSFEKPetSRDaGHvclE=",
+      "dev": true,
       "requires": {
         "ansi-wrap": "0.1.0"
       }
@@ -61,17 +62,20 @@
     "ansi-regex": {
       "version": "2.1.1",
       "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-2.1.1.tgz",
-      "integrity": "sha1-w7M6te42DYbg5ijwRorn7yfWVN8="
+      "integrity": "sha1-w7M6te42DYbg5ijwRorn7yfWVN8=",
+      "dev": true
     },
.............

mikeller · 2018-02-20T01:05:18Z

package-lock.json

@@ -26,7 +26,7 @@
    "ansi-colors": {
      "version": "1.0.1",
      "resolved": "https://registry.npmjs.org/ansi-colors/-/ansi-colors-1.0.1.tgz",
-      "integrity": "sha512-yopkAU0ZD/WQ56Tms3xLn6jRuX3SyUMAVi0FdmDIbmmnHW3jHiI1sQFdUl3gfVddjnrsP3Y6ywFKvCRopvoVIA==",
+      "integrity": "sha1-6UxsMGAFr4tIIkAkHi896kuFX/M=",


This file comes up as changed after the npm install for me too, and needs updating to the latest version (posting the linux version of the file here since I suspect getting a vanilla package-lock.json on macosx is pretty much impossible due to the platform dependent module being installed):

package-lock.zip

basdelfos · 2018-02-20T22:52:33Z

package-lock.json has been replaced with the one that @mikeller supplied. Please test again on Linux.

mikeller · 2018-02-20T23:21:18Z

Confirmed working without side effects on checked in files in linux.

AndersHoglund · 2018-02-20T23:22:59Z

Strange, when I test it is still integrity warnings and some package-lock.json diff. But OK, lets see what Jenkins says.

AndersHoglund · 2018-02-21T00:03:42Z

Integrity warnings on all platforms, and package-lock.json diffs, on Jenkins too:
http://andwho.sytes.net:8080/job/Borisb_Blackbox_Log_Viewer/
O, I added the "git diff package-lock.json" test now, to be able to see it easally. The green balls in earlier version may not be so green, no such test done.
It all works on BFC, but not here. What is so different?

Also noticed that gulp errors out on linux32.rpm build, but does not return any error for Jenkins to detect. Same here and in BFC builds. Old problem (or is this expected?) I have not noticed. Error slipping by. I assumed/expected tools to return non-zero on errors. This npm-gulp-node-js-package-lockfile-world is strange...

mikeller · 2018-02-21T00:21:24Z

Interesting, the builds for Windows and linux work just fine here, and the artefacts seem to be working. Maybe your tests are broken? (I don't seem to be able to find the logs with build errors on your platform.)

AndersHoglund · 2018-02-21T07:47:03Z

You have to drill down a bit on each platform to find the console outputs per build:
http://andwho.sytes.net:8080/job/Borisb_Blackbox_Log_Viewer/Platform=MacOSx/20/console
http://andwho.sytes.net:8080/job/Borisb_Blackbox_Log_Viewer/Platform=linux/20/console
http://andwho.sytes.net:8080/job/Borisb_Blackbox_Log_Viewer/Platform=windows/20/console

mikeller · 2018-02-21T11:25:56Z

@AndersHoglund: I see now. I think the warning up top is caused by some problem in the npm registry, and probably not something we can do anything about.

Turns out the differences in package-lock.json were caused by me having outdated (but still valid) packages in node_modules. Deleting the directory and re-running npm install gave a different result. Opened #110 with new package-lock.json.

AndersHoglund · 2018-02-21T11:45:16Z

OK, lets just merge #110 and see what Jenkins has to say about it.

AndersHoglund · 2018-02-21T13:27:10Z

Seems Jenkins also need to purge node_modules to build OK. Added that as the first build step, and now it looks OK after #110.
Also skipped the check for package-lock file diffs on Mac, there will always be a diff due to additional dmg package. Only logging the diffs on Mac, so it can be manually inspected.

mikeller · 2018-02-21T22:17:30Z

@AndersHoglund: Yes, I think purging node_modules before automated builds is a good move. This should help detect it if a packet-lock.json that is based on out of date packet information that is cached in a user's node_modules is pushed.

We should look into setting up travis (at least for one of the platforms), so we get integrated warnings.

Updated gulp build (added rpm build and fixed linux build on macOS)

7fb9c52

basdelfos requested a review from mikeller February 18, 2018 10:41

mikeller requested changes Feb 18, 2018

View reviewed changes

basdelfos added 2 commits February 19, 2018 21:49

Fixes as per review

2669d55

Changed version to 3.0.0

d59be4c

mikeller approved these changes Feb 20, 2018

View reviewed changes

mikeller requested changes Feb 20, 2018

View reviewed changes

Replaced package-lock.json with the one provided by mikeller

137b832

mikeller approved these changes Feb 20, 2018

View reviewed changes

mikeller merged commit 6c9fdca into betaflight:master Feb 20, 2018

basdelfos deleted the gulp-build branch February 21, 2018 09:09

Uh oh!

Updated gulp build (added rpm build and fixed linux build on macOS) #105

Updated gulp build (added rpm build and fixed linux build on macOS) #105

Uh oh!

Conversation

basdelfos commented Feb 18, 2018

Uh oh!

AndersHoglund commented Feb 18, 2018

Uh oh!

AndersHoglund commented Feb 18, 2018

Uh oh!

McGiverGim commented Feb 18, 2018

Uh oh!

mikeller left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

basdelfos commented Feb 19, 2018

Uh oh!

AndersHoglund commented Feb 19, 2018

Uh oh!

AndersHoglund commented Feb 19, 2018

Uh oh!

AndersHoglund commented Feb 19, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

basdelfos commented Feb 20, 2018

Uh oh!

mikeller commented Feb 20, 2018

Uh oh!

AndersHoglund commented Feb 20, 2018

Uh oh!

AndersHoglund commented Feb 21, 2018

Uh oh!

mikeller commented Feb 21, 2018

Uh oh!

AndersHoglund commented Feb 21, 2018

Uh oh!

mikeller commented Feb 21, 2018

Uh oh!

AndersHoglund commented Feb 21, 2018

Uh oh!

AndersHoglund commented Feb 21, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mikeller commented Feb 21, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

AndersHoglund commented Feb 21, 2018 •

edited

Loading

mikeller commented Feb 21, 2018 •

edited

Loading