Provide machine logs volume and configuration of logs root path #8535

akorneta · 2018-01-31T10:36:30Z

What does this PR do?

Adds configuration of workspace logs root directory;
Provides environment variable for each OpenShift machine that contains value of logs root directory;
Adds logs volume into all OpenShift machines and make distinct subpaths for logs with machine name.
Modifies installation scripts of ws-agent and exec-agent with a logic that defines the logs folder for this agents and redirects bootstrapper logs to workspace logs root.

file structure on host for different strategies:

One PV will be used for storing logs of machines that are placed in the same pod.
unique:

One PV will be used for storing logs and projects data of workspace with two machines.
common:

What issues does this PR fix or reference?

#7807
partially solves this issue(for OpenShift only): #7386

Docs PR

n/a

akorneta · 2018-01-31T13:11:59Z

ci-build

akorneta · 2018-01-31T13:12:07Z

ci-test-ocp

riuvshin · 2018-01-31T13:17:54Z

will we have ws logs on docker?

akorneta · 2018-01-31T13:24:23Z

@riuvshin maybe, but not in the scope of this pr

riuvshin · 2018-01-31T13:25:43Z

@akorneta sure, but we need ws logs on docker same as on openshift, so that should be considered I think in order to close ws logs issue

akorneta · 2018-01-31T13:40:55Z

@riuvshin I understand that.
This PR will resolve this issue: #7807
and for OpenShift only, this one: #7386 (updated the pr summary)

dmytro-ndp · 2018-01-31T13:45:01Z

@akorneta: isn't #7386 for logs of ws run on docker?

akorneta · 2018-01-31T14:07:42Z

@dmytro-ndp no(see comment above), I'll not close that issue, just will comment that for OS infra it works.

dmytro-ndp · 2018-01-31T14:12:04Z

@akorneta: I see, thank you.

riuvshin · 2018-01-31T14:38:58Z

dockerfiles/init/manifests/che.env

+#     Defines the directory inside the machine where all the workspace logs are placed.
+#     The value of this folder should be provided into machine e.g. like environment variable
+#     so agents developers can use this directory for backup agents logs.
+#CHE_WORKSPACE_LOGS_ROOT_DIR=/workspace_logs


maybe default value should be set right here in che.env instead of defining in each agent installer script?
This way it will be more obvious to end user where are logs stored by default

By default value from che.properties will be used, and it equals to /workspace_logs(we won't define it in each script). This value is commented to be the same as all properties in this file.

codenvy-ci · 2018-01-31T19:51:24Z

ci-test build report:
Build details
Test report
selenium tests report data
docker image: eclipseche/che-server:8535
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.

codenvy-ci · 2018-01-31T22:23:16Z

Build success. https://ci.codenvycorp.com/job/che-pullrequests-build/4441/

sleshchenko

LGTM.

Only one question: Did you consider configuring log dir for the terminal agent?

sleshchenko · 2018-02-01T08:43:12Z

...clipse/che/workspace/infrastructure/openshift/provision/env/LogsRootEnvVariableProvider.java

+public class LogsRootEnvVariableProvider implements EnvVarProvider {
+
+  /** Environment variable that points to root folder of projects inside machine */
+  public static final String WORKSPACE_LOGS_ROOT_VARIABLE = "CHE_WORKSPACE_LOGS_ROOT_DIR";


I'd used ENV_VAR instead of VARIABLE

akorneta · 2018-02-01T09:16:42Z

@sleshchenko
Yes, I thought about that. Actually, to be able to store terminal logs into the file we need to do some work on that because currently they are streamed to stdout and by the way, terminal logs do not contain any useful information(just start/end of the session) so, that's why I did not work on that.

garagatyi · 2018-02-01T14:24:47Z

assembly/assembly-wsmaster-war/src/main/webapp/WEB-INF/classes/che/che.properties

+#Defines the directory inside the machine where all the workspace logs are placed.
+#The value of this folder should be provided into machine e.g. like environment variable
+#so agents developers can use this directory for backup agents logs.
+che.workspace.logs.root.dir=/workspace_logs


Consider usage of underscore between root and dir words

garagatyi

Looks good, nice job!

…tegy

l0rd · 2018-02-01T16:58:11Z

👍 thanks @akorneta

l0rd · 2018-02-01T17:00:13Z

I have updated redhat-developer/rh-che#448

akorneta added status/in-progress This issue has been taken by an engineer and is under active development. kind/task Internal things, technical debt, and to-do tasks to be performed. labels Jan 31, 2018

akorneta self-assigned this Jan 31, 2018

akorneta requested review from skabashnyuk, garagatyi and sleshchenko January 31, 2018 10:36

akorneta requested review from benoitf, riuvshin and tolusha as code owners January 31, 2018 10:36

akorneta changed the title ~~[WIP]: Provide machine logs volume and configuration of logs root path~~ Provide machine logs volume and configuration of logs root path Jan 31, 2018

akorneta added status/code-review This issue has a pull request posted for it and is awaiting code review completion by the community. and removed status/in-progress This issue has been taken by an engineer and is under active development. labels Jan 31, 2018

akorneta force-pushed the CHE-7807 branch from 88c6715 to 8872f79 Compare January 31, 2018 12:08

riuvshin reviewed Jan 31, 2018

View reviewed changes

sleshchenko approved these changes Feb 1, 2018

View reviewed changes

akorneta requested a review from l0rd February 1, 2018 09:18

riuvshin approved these changes Feb 1, 2018

View reviewed changes

akorneta force-pushed the CHE-7807 branch from 1962d22 to 749fad4 Compare February 1, 2018 12:06

akorneta force-pushed the CHE-7807 branch 2 times, most recently from 74cbcbe to 87792d3 Compare February 1, 2018 14:03

benoitf approved these changes Feb 1, 2018

View reviewed changes

garagatyi reviewed Feb 1, 2018

View reviewed changes

skabashnyuk approved these changes Feb 1, 2018

View reviewed changes

garagatyi approved these changes Feb 1, 2018

View reviewed changes

akorneta force-pushed the CHE-7807 branch 2 times, most recently from eb179d3 to fdb0dcc Compare February 1, 2018 15:51

akorneta added 3 commits February 1, 2018 18:06

Add the provider of logs root path as env variable to each machine

f664673

Add logs volume provisioner and subpaths prefixes in each volume stra…

6efa963

…tegy

Change root folder for agents logs

d2ad07c

akorneta force-pushed the CHE-7807 branch from fdb0dcc to d2ad07c Compare February 1, 2018 16:06

akorneta merged commit c21e9d8 into eclipse-che:master Feb 1, 2018

akorneta deleted the CHE-7807 branch February 1, 2018 16:30

akorneta mentioned this pull request Feb 1, 2018

CHE-6: Persist workspace logs #7386

Closed

benoitf removed the status/code-review This issue has a pull request posted for it and is awaiting code review completion by the community. label Feb 5, 2018

benoitf added this to the 6.1.0 milestone Feb 5, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Provide machine logs volume and configuration of logs root path #8535

Provide machine logs volume and configuration of logs root path #8535

akorneta commented Jan 31, 2018 •

edited

Loading

akorneta commented Jan 31, 2018

akorneta commented Jan 31, 2018

riuvshin commented Jan 31, 2018

akorneta commented Jan 31, 2018 •

edited

Loading

riuvshin commented Jan 31, 2018 •

edited

Loading

akorneta commented Jan 31, 2018 •

edited

Loading

dmytro-ndp commented Jan 31, 2018 •

edited

Loading

akorneta commented Jan 31, 2018

dmytro-ndp commented Jan 31, 2018

riuvshin Jan 31, 2018

akorneta Jan 31, 2018

riuvshin Feb 1, 2018

codenvy-ci commented Jan 31, 2018

codenvy-ci commented Jan 31, 2018

sleshchenko left a comment

sleshchenko Feb 1, 2018

akorneta commented Feb 1, 2018 •

edited

Loading

garagatyi Feb 1, 2018

garagatyi left a comment

l0rd commented Feb 1, 2018

l0rd commented Feb 1, 2018

Provide machine logs volume and configuration of logs root path #8535

Provide machine logs volume and configuration of logs root path #8535

Conversation

akorneta commented Jan 31, 2018 • edited Loading

What does this PR do?

What issues does this PR fix or reference?

Docs PR

akorneta commented Jan 31, 2018

akorneta commented Jan 31, 2018

riuvshin commented Jan 31, 2018

akorneta commented Jan 31, 2018 • edited Loading

riuvshin commented Jan 31, 2018 • edited Loading

akorneta commented Jan 31, 2018 • edited Loading

dmytro-ndp commented Jan 31, 2018 • edited Loading

akorneta commented Jan 31, 2018

dmytro-ndp commented Jan 31, 2018

riuvshin Jan 31, 2018

Choose a reason for hiding this comment

akorneta Jan 31, 2018

Choose a reason for hiding this comment

riuvshin Feb 1, 2018

Choose a reason for hiding this comment

codenvy-ci commented Jan 31, 2018

codenvy-ci commented Jan 31, 2018

sleshchenko left a comment

Choose a reason for hiding this comment

sleshchenko Feb 1, 2018

Choose a reason for hiding this comment

akorneta commented Feb 1, 2018 • edited Loading

garagatyi Feb 1, 2018

Choose a reason for hiding this comment

garagatyi left a comment

Choose a reason for hiding this comment

l0rd commented Feb 1, 2018

l0rd commented Feb 1, 2018

akorneta commented Jan 31, 2018 •

edited

Loading

akorneta commented Jan 31, 2018 •

edited

Loading

riuvshin commented Jan 31, 2018 •

edited

Loading

akorneta commented Jan 31, 2018 •

edited

Loading

dmytro-ndp commented Jan 31, 2018 •

edited

Loading

akorneta commented Feb 1, 2018 •

edited

Loading