[ZEPPELIN-2094] Decrease npm install retry time (for branch-0.7) #2095
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
SehoNoh
force-pushed
the
ZEPPELIN-2094-for-0.7
branch
from
a9565ba
to
4382286
Compare
SehoNoh
force-pushed
the
ZEPPELIN-2094-for-0.7
branch
from
4382286
to
4388ff8
Compare
|
I will update the pr description soon. |
|
Thanks @NohSeho. It would be nice if step-by-step description for reproducing this issue in 0.7.0 |
|
LGTM and merge to branch-0.7 if no further discussions. |
asfgit
pushed a commit
that referenced
this pull request
Mar 6, 2017
### What is this PR for? **This pr is for branch-0.7** It’s too delayed for npm install when computer do not connected any networks. Beacause when npm install, it has too long retry timeout. This PR is to decrease retry timeout when npm install. [pr for mater](#2060) #2095 ### What type of PR is it? Improvement ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-2094 ### How should this be tested? you must enable any one helium before test Line 197 In zeppelin-zengine org.apache.zeppelin.helium.HeliumBundleFactory.java First set with `String npmCommand = "install —loglevel=error”;` and You don’t connect any ethernet or wireless internet. build & run and set with `String npmCommand = "install —fetch-retries=2 —fetch-retry-factor=1 —fetch-retry-mintimeout=5000 —loglevel=error”;` also don’t connect any networks, build & run. WHY retries = 2 factor = 1 mintimeout = 5(sec)? npm use [retry](https://github.com/tim-kos/node-retry) module to retry. It refers [this article](http://dthain.blogspot.kr/2009/02/exponential-backoff-in-distributed.html) for retry algorithms. It is a math which structured _Math.min(Math.round(random * minTimeout * Math.pow(factor, attempt)), maxTimeout)_. In retry source code, between two retries. First retry doesn't care _Math.min()_, just _Math.round(random * minTimeout * Math.pow(factor, attempt))_) Description | Before | After ------- | ------- | ------- Condition | npm's default setting<br>random = False = 1<br>retry = 2<br>minTimeout = 10 (sec)<br>maxTimeout = 60 (sec)<br>factor = 10 | custom setting<br>random = False = 1<br>retry = 2<br>minTimeout = 5 (sec)<br>maxTimeout = 60 (sec)<br>factor = 1<br> First retry | Math.round(1 * 10 (sec) * 10^1)) | Math.round(1 * 5 (sec) * 1^1)) First retry result (Approximately) | 100 (sec) | 5 (sec) Second retry | Math.min(Math.round(1 * 10 (sec) * 10^2), 60 (sec)) | Math.min(Math.round(1 * 5 (sec) * 1^2), 60 (sec)) Second retry result (Approximately) | 60 (sec) | 5 (sec) Total waiting time (Approximately) | 160 (sec) | 10 (sec) You can check like this below Screenshots. ### Screenshots Before | After -------|------- <img width="1077" alt="2017-02-24 12 32 06" src="https://cloud.githubusercontent.com/assets/1144643/23267951/9deaec6e-fa2f-11e6-9171-5792f24de76d.png"> | <img width="1081" alt="2017-02-24 12 37 10" src="https://cloud.githubusercontent.com/assets/1144643/23267954/a12c0c0a-fa2f-11e6-99cd-335deef607ac.png"> ### Questions: * Does the licenses files need update? N/A * Is there breaking changes for older versions? N/A * Does this needs documentation? N/A Author: NohSeho <iam@sehonoh.kr> Closes #2095 from NohSeho/ZEPPELIN-2094-for-0.7 and squashes the following commits: 4388ff8 [NohSeho] [ZEPPELIN-2094] Decrease npm install retry time
|
This PR is just merged to branch-0.7 but @asfgit does not close this PR automatically. @NohSeho Can you manually close this PR? |
|
@NohSeho And please let me know your apache JIRA id, so i can resolve https://issues.apache.org/jira/browse/ZEPPELIN-2094 with your name in assignee. |
|
@Leemoonsoo username is NohSeho |
|
Thanks @NohSeho. ZEPPELIN-2094 is marked as resolved. |
prabhjyotsingh
pushed a commit
to prabhjyotsingh/zeppelin
that referenced
this pull request
Mar 7, 2017
### What is this PR for? **This pr is for branch-0.7** It’s too delayed for npm install when computer do not connected any networks. Beacause when npm install, it has too long retry timeout. This PR is to decrease retry timeout when npm install. [pr for mater](apache#2060) apache#2095 ### What type of PR is it? Improvement ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-2094 ### How should this be tested? you must enable any one helium before test Line 197 In zeppelin-zengine org.apache.zeppelin.helium.HeliumBundleFactory.java First set with `String npmCommand = "install —loglevel=error”;` and You don’t connect any ethernet or wireless internet. build & run and set with `String npmCommand = "install —fetch-retries=2 —fetch-retry-factor=1 —fetch-retry-mintimeout=5000 —loglevel=error”;` also don’t connect any networks, build & run. WHY retries = 2 factor = 1 mintimeout = 5(sec)? npm use [retry](https://github.com/tim-kos/node-retry) module to retry. It refers [this article](http://dthain.blogspot.kr/2009/02/exponential-backoff-in-distributed.html) for retry algorithms. It is a math which structured _Math.min(Math.round(random * minTimeout * Math.pow(factor, attempt)), maxTimeout)_. In retry source code, between two retries. First retry doesn't care _Math.min()_, just _Math.round(random * minTimeout * Math.pow(factor, attempt))_) Description | Before | After ------- | ------- | ------- Condition | npm's default setting<br>random = False = 1<br>retry = 2<br>minTimeout = 10 (sec)<br>maxTimeout = 60 (sec)<br>factor = 10 | custom setting<br>random = False = 1<br>retry = 2<br>minTimeout = 5 (sec)<br>maxTimeout = 60 (sec)<br>factor = 1<br> First retry | Math.round(1 * 10 (sec) * 10^1)) | Math.round(1 * 5 (sec) * 1^1)) First retry result (Approximately) | 100 (sec) | 5 (sec) Second retry | Math.min(Math.round(1 * 10 (sec) * 10^2), 60 (sec)) | Math.min(Math.round(1 * 5 (sec) * 1^2), 60 (sec)) Second retry result (Approximately) | 60 (sec) | 5 (sec) Total waiting time (Approximately) | 160 (sec) | 10 (sec) You can check like this below Screenshots. ### Screenshots Before | After -------|------- <img width="1077" alt="2017-02-24 12 32 06" src="https://cloud.githubusercontent.com/assets/1144643/23267951/9deaec6e-fa2f-11e6-9171-5792f24de76d.png"> | <img width="1081" alt="2017-02-24 12 37 10" src="https://cloud.githubusercontent.com/assets/1144643/23267954/a12c0c0a-fa2f-11e6-99cd-335deef607ac.png"> ### Questions: * Does the licenses files need update? N/A * Is there breaking changes for older versions? N/A * Does this needs documentation? N/A Author: NohSeho <iam@sehonoh.kr> Closes apache#2095 from NohSeho/ZEPPELIN-2094-for-0.7 and squashes the following commits: 4388ff8 [NohSeho] [ZEPPELIN-2094] Decrease npm install retry time
FRosner
added a commit
to FRosner/docker-zeppelin
that referenced
this pull request
Mar 7, 2017
because we want to get apache/zeppelin#2095
|
Hi @NohSeho I built Zeppelin after your fix and checked again. Now it is still hanging for two minutes before actually moving to the part that was changed. We have a proxy but it cannot be used because of https://issues.apache.org/jira/browse/ZEPPELIN-2152. Why is this time so long still? Any clues? |
|
Hi @FRosner Could you specify which log is the last line (the last printed) before hang? |
|
@1ambda you can see it at the time (2 minutes difference): |
|
Then, What @NohSeho fixed is not related with that part. It's the delay caused by maven-frontend-plugin while downloading nodejs, npm. IMO,
|
|
Thanks for clarifying @1ambda. Is there a Jira issue for the first point already? Can you point me to the part of the code where this is configured? |
|
@FRosner hey there, I think that You're in trouble with installing local node.js and npm, isn't right? |
|
@NohSeho I have the problem that we are behind a proxy and the proxy is not configured. I don't want npm and I don't want node.js but Zeppelin tries to download it for 2 minutes. I thought your fix was going to help but apparently it's not. @1ambda pointed me to the maven-frontend-plugin being the source of my problems but I'd like to know where exactly I need go to change this behaviour. Does it make sense? |
|
How about make Zeppelin skip install npm and node.js when no visualization/spell is enabled? |
|
That would be amazing. :)
--
Sent from my phone. Please excuse my brevity.
|
This was referenced Mar 15, 2017
asfgit
pushed a commit
that referenced
this pull request
Mar 17, 2017
…ge is selected ### What is this PR for? Zeppelin 0.7.0 installs node and npm when it first starts for Helium package. To be installed, or to failed to be installed due to network timeout, it takes some times. We can just create empty file when no Helium package is enabled, instead of install npm and build bundle. See discussion #2095 (comment) ### What type of PR is it? Improvement ### Todos * [x] - skip node,npm install and bundle package when no package is selected ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-2260 ### How should this be tested? When no package is selected (e.g. right after clean install Zeppelin), npm and node is no longer installed on startup. ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Lee moon soo <moon@apache.org> Closes #2137 from Leemoonsoo/ZEPPELIN-2260 and squashes the following commits: 520ad3f [Lee moon soo] Skip node,npm install and bundle when no helium package is selected
asfgit
pushed a commit
that referenced
this pull request
Mar 17, 2017
… helium package is selected ### What is this PR for? Apply #2137 to `branch-0.7` Zeppelin 0.7.0 installs node and npm when it first starts for Helium package. To be installed, or to failed to be installed due to network timeout, it takes some times. We can just create empty file when no Helium package is enabled, instead of install npm and build bundle. See discussion #2095 (comment) ### What type of PR is it? Improvement ### Todos * [x] - skip node,npm install and bundle package when no package is selected ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-2260 ### How should this be tested? When no package is selected (e.g. right after clean install Zeppelin), npm and node is no longer installed on startup. ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Lee moon soo <moon@apache.org> Closes #2138 from Leemoonsoo/ZEPPELIN-2260_branch-0.7 and squashes the following commits: e8f0686 [Lee moon soo] Skip node,npm install and bundle when no helium package is selected
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What is this PR for?
This pr is for branch-0.7
It’s too delayed for npm install when computer do not connected any networks.
Beacause when npm install, it has too long retry timeout.
This PR is to decrease retry timeout when npm install.
pr for mater #2095
What type of PR is it?
Improvement
What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-2094
How should this be tested?
you must enable any one helium before test
Line 221 In zeppelin-zengine org.apache.zeppelin.helium.HeliumVisualizationFactory.java
First set with
String commandForNpmInstall = "install —loglevel=error”;and You don’t connect any ethernet or wireless internet.
build & run
and set with
also don’t connect any networks, build & run.
and also line 345. do it
npm install some artifactagain like above.WHY
retries = 2
factor = 1
mintimeout = 5(sec)?
npm use retry module to retry.
It refers this article for retry algorithms.
It is a math which structured Math.min(Math.round(random * minTimeout * Math.pow(factor, attempt)), maxTimeout).
In retry source code, between two retries. First retry doesn't care Math.min(), just Math.round(random * minTimeout * Math.pow(factor, attempt)))
random = False = 1
retry = 2
minTimeout = 10 (sec)
maxTimeout = 60 (sec)
factor = 10
random = False = 1
retry = 2
minTimeout = 5 (sec)
maxTimeout = 60 (sec)
factor = 1
You can check like this below Screenshots.
Screenshots
Questions: