Error message from WS

[itaimalek]

Hi, after a while I get this from WS

Terminal still works, but I cannot see my files.
This happens right after I run npm -g install

This is what installed:

├── jit-grunt@0.9.1
├── extend@3.0.0
├── ncp@2.0.0
├── async@1.5.2
├── dot@1.0.3
├── grunt-contrib-copy@0.8.0 (file-sync-cmp@0.1.1, chalk@0.5.1)
├── promise@7.1.1 (asap@2.0.3)
├── glob@6.0.4 (path-is-absolute@1.0.0, inherits@2.0.1, once@1.3.3, inflight@1.0                                                                              .4, minimatch@3.0.0)
├── grunt-newer@1.1.0 (rimraf@2.2.8, async@0.9.0)
├── mv@2.1.1 (rimraf@2.4.5, mkdirp@0.5.1)
├── time-grunt@1.2.1 (date-time@1.0.0, figures@1.3.5, number-is-nan@1.0.0, text-                                                                              table@0.2.0, hooker@0.2.3, chalk@1.1.0, pretty-ms@1.4.0)
├── jade@0.26.3 (commander@0.6.1, mkdirp@0.3.0)
├── grunt-contrib-concat@0.5.1 (chalk@0.5.1, source-map@0.3.0)
├── postcss@5.0.4 (js-base64@2.1.9, supports-color@3.1.2, source-map@0.4.4)
├── grunt-contrib-connect@0.10.1 (opn@1.0.2, connect-livereload@0.5.3, async@0.9                                                                              .0, portscanner@1.0.0, connect@2.29.1)
├── grunt-mocha@0.4.12 (grunt-lib-phantomjs@0.4.0, lodash@2.3.0, mocha@1.21.5)
├── grunt-contrib-requirejs@0.4.4 (requirejs@2.1.20)
├── grunt-postcss@0.7.1 (es6-promise@3.0.2, chalk@1.1.1, diff@2.2.1, postcss@5.0                                                                              .12)
├── lodash@4.0.1
├── grunt@0.4.5 (which@1.0.9, dateformat@1.0.2-1.2.3, eventemitter2@0.4.14, geto                                                                              bject@0.1.0, rimraf@2.2.8, colors@0.6.2, async@0.1.22, hooker@0.2.3, grunt-legac                                                                              y-util@0.2.0, exit@0.1.2, nopt@1.0.10, glob@3.1.21, lodash@0.9.2, minimatch@0.2.                                                                              14, coffee-script@1.3.3, underscore.string@2.2.1, iconv-lite@0.2.11, js-yaml@2.0                                                                              .5, findup-sync@0.1.3, grunt-legacy-log@0.1.2)
├── phantomjs@1.9.17 (which@1.0.9, progress@1.1.8, kew@0.4.0, request-progress@0                                                                              .3.1, adm-zip@0.4.4, npmconf@2.1.1, fs-extra@0.18.4, request@2.42.0)
├── grunt-contrib-jshint@0.11.1 (hooker@0.2.3, jshint@2.7.0)
├── grunt-autoprefixer@3.0.0 (diff@1.3.2, chalk@1.0.0, autoprefixer-core@5.1.11)
├── grunt-csso@0.7.1 (chalk@1.1.1, maxmin@1.1.0, csso@1.3.11)
├── grunt-sass@1.1.0 (object-assign@4.0.1, each-async@1.1.1, node-sass@3.4.2)
├── bower@1.4.1 (is-root@1.0.0, junk@1.0.2, stringify-object@1.0.1, user-home@1.                                                                              1.1, chmodr@0.1.0, rimraf@2.4.1, abbrev@1.0.7, archy@1.0.0, opn@1.0.2, graceful-                                                                              fs@3.0.8, bower-logger@0.2.2, bower-endpoint-parser@0.2.2, lockfile@1.0.1, lru-c                                                                              ache@2.6.5, nopt@3.0.3, retry@0.6.1, tmp@0.0.24, q@1.4.1, request-progress@0.3.1                                                                              , semver@2.3.2, chalk@1.1.0, shell-quote@1.4.3, which@1.1.1, bower-json@0.4.0, f                                                                              stream@1.0.7, mkdirp@0.5.0, promptly@0.2.0, p-throttler@0.1.1, glob@4.5.3, fstre                                                                              am-ignore@1.0.2, tar-fs@1.7.0, insight@0.5.3, decompress-zip@0.1.0, update-notif                                                                              ier@0.3.2, github@0.2.4, bower-registry-client@0.3.0, request@2.53.0, mout@0.11.                                                                              0, cardinal@0.4.4, handlebars@2.0.0, bower-config@0.6.1, configstore@0.3.2, inqu                                                                              irer@0.8.0)
└── grunt-babel@5.0.1 (babel-core@5.8.24)

[ghost]

Can yuu share the project?

[itaimalek]

sorry, what do you mean by "share" ?
give you access?

[ghost]

Is it publicly available so that I can take a look and try to reproduce?

[itaimalek]

can I send it to you privately somehow?

[ghost]

support(at)codenvy.com

[itaimalek]

sorry, I cannot..the environment is whitelisted to our org. IPs

[ghost]

I cannot reproduce it with any projects on my side. I need the exact copy of your env with the project and command.

[ghost]

@itaimalek any hints on how I might try reproducing this one?

[itaimalek]

I'm sorry. but I cannot expose business data outside our network, you can try to do :
npm install with nodeJS 12.0.4 and the list of installed item above.

[ghost]

@itaimalek no need to.. I have just reproduced it with a meteor app. Looks like a process inside a container interferes with ws agent somehow.

[itaimalek]

Cool !
Sorry for not being able to help,
Happy you could find a way to reproduce this :)
On Apr 24, 2016 7:33 PM, "Eugene Ivantsov" notifications@github.com wrote:

@itaimalek https://github.com/itaimalek no need to.. I have just
reproduced it with a meteor app. Looks like a process inside a container
interferes with ws agent somehow.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#1088 (comment)

[ghost]

I think I have found out what happens. Inside every dev machine container, Che is running a workspace agent which is a Tomcat server. It consumes RAM. When a meteor app is starting is needs RAM as well. Workspace container is capped at 1GB ram by default. When out of memory issue occurs a workspace agent either malfunctions or the kernel just kills it a the most ram consuming process.

I have started a ws with 2GB and cannot reproduce the issue anymore.

[itaimalek]

I'll try with more RAM,
Is there anyway I can scale up a workspace container after creation?
On Apr 24, 2016 7:46 PM, "Eugene Ivantsov" notifications@github.com wrote:

I think I have found out what happens. Inside every dev machine container,
Che is running a workspace agent which is a Tomcat server. It consumes RAM.
When a meteor app is starting is needs RAM as well. Workspace container is
capped at 1GB ram by default. When out of memory issue occurs a workspace
agent either malfunctions or the kernel just kills it a the most ram
consuming process.

I have started a ws with 2GB and cannot reproduce the issue anymore.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#1088 (comment)

[TylerJewell]

There is not, but I think it is due to a docker limitation. In studying the docker run docks there is an infinite option which lets the container grow its memory consumption in demand. Perhaps we should make this option available in Che?

[TylerJewell]

Actually in thinking about it, we could expose all of the memory management components directly into the dashboard for Che. We would not permit this in Codenvy as we would have no way to manage memory from aggressive user containers.

https://docs.docker.com/engine/reference/run/#runtime-constraints-on-resources

[ghost]

@itaimalek you cannot interfere with configuration of a running container. Docker won't let you do that. @TylerJewell No limitation is a default behavior for docker run.

[TylerJewell]

But we force users to apply a memory constraint in the dashboard. So it is a question of enforcement.

[ghost]

@TylerJewell I think the problem here is for Che to gracefully fail when the workspace is out of resources and this is exactly what happens here.

[TylerJewell]

@eivantsov Well there are two separate and distinct issues:

1. For containers that are defined with a fixed memory size, then we will need to provide the end user with information about internal resource consumption and fail gracefully if upper limits are reached.
2. For the broader scenario, I believe we need to expose the range of docker memory configuration items within the che dashboard. A user can create a container with the parameters that allow the container to continue growing in size. This would be something that we allow in Che, but something that would have to be activated by an administrator in Codenvy, otherwise one user could bully other users around for hogging resources.

I want to leave this issue open for a long period of time as there are numerous development tasks we can get from this.

[james10174]

@TylerJewell Can you for the defined fixed memory size copy a container workspace, then destroy that container workspace and then recreate a copy of workspace with more ram?

[ghost]

@james10174 we do plans to better detect issues with the workspace agent. Ideally, when it's down or unreachable, we will display a message and try to re-start it or offer a user to launch this workspace with more ram.

[TylerJewell]

@eivantsov - can you please cross link any Jira issues on this topic? Let's add them in here as cross-links until we can rewrite them into GitHub properly.

[ghost]

@TylerJewell

https://jira.codenvycorp.com/browse/CHE-14

[ghost]

@itaimalek I am closing this issue as a duplicate. You may follow this one - #1988

[itaimalek]

Thanks