Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use linux path separator for commands executed on pod #3524

Conversation

girishramnani
Copy link
Contributor

@girishramnani girishramnani commented Jul 8, 2020

What type of PR is this?

Uncomment only one /kind line, and delete the rest.
For example, > /kind bug would simply become: /kind bug

What does does this PR do / why we need it:
use linux path separator for commands executed on pod instead of platform independent filepath.Join as the pod will always be linux but host machine can be windows. Would would join the paths in the commands incorrectly for linux.

Which issue(s) this PR fixes:

Fixes #3328

How to test changes / Special notes to the reviewer:
failing tests mentioned in the above issue should pass on windows

@@ -286,7 +288,7 @@ var _ = Describe("odo push command tests", func() {
cmpName,
appName,
project,
[]string{"stat", filepath.Join(dir, "src", "server.js")},
[]string{"stat", strings.TrimSuffix(dir, "/") + "/src/server.js"}, // this doesn't use filepath.Join for a reason as this commands needs to run on linux, but the host machine can be windows
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will ToSlash() work after the join operation?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn’t get u? can u elaborate?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It won’t because the host system would still be Windows which would give different slash the pod os - being linux

Copy link
Member

@kadel kadel Jul 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It won’t because the host system would still be Windows which would give different slash the pod os - being linux

ToSlash returns the path that always uses / even on Windows

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you could use filepath.ToSlash(filepath.Join(.....)) this should be much safer

Copy link
Contributor

@mik-dass mik-dass Jul 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are already using ToSlash() in our code base for converting all paths to linux based paths https://github.com/openshift/odo/blob/7c0fef571f2ee1b0e3ad8927759b3efbb14f8616/pkg/sync/sync.go#L31

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@kadel
Copy link
Member

kadel commented Jul 9, 2020

[odo]  •  Waiting for component to start  ...
[odo] I0709 03:20:14.632726   25570 occlient.go:1926] Status of javaee-war-test-app-1-lbxx2 pod is Pending
[odo] I0709 03:20:20.638972   25570 occlient.go:1926] Status of javaee-war-test-app-1-lbxx2 pod is Pending
[odo] I0709 03:20:20.708444   25570 occlient.go:1926] Status of javaee-war-test-app-1-lbxx2 pod is Pending
[odo] I0709 03:20:31.457000   25570 occlient.go:1926] Status of javaee-war-test-app-1-lbxx2 pod is Pending
[odo] I0709 03:20:31.893413   25570 occlient.go:1926] Status of javaee-war-test-app-1-lbxx2 pod is Pending
[odo] I0709 03:24:12.050497   25570 occlient.go:1857] Quitting collect events
[odo]  ✗  waited 4m0s but couldn't find running pod matching selector: 'deploymentconfig=javaee-war-test-app'
[odo]  ✗  Waiting for component to start [4m]

/retest

@codecov
Copy link

codecov bot commented Jul 9, 2020

Codecov Report

Merging #3524 into master will decrease coverage by 0.20%.
The diff coverage is 0.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3524      +/-   ##
==========================================
- Coverage   46.27%   46.06%   -0.21%     
==========================================
  Files         112      113       +1     
  Lines       11386    11437      +51     
==========================================
  Hits         5269     5269              
- Misses       5608     5659      +51     
  Partials      509      509              
Impacted Files Coverage Δ
pkg/preference/machine_output.go 0.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fdea76f...136b0e8. Read the comment docs.

@dharmit
Copy link
Member

dharmit commented Jul 10, 2020

/approve
/lgtm

Applying both since the PR is small and issues look addressed.

@openshift-ci-robot openshift-ci-robot added lgtm Indicates that a PR is ready to be merged. Required by Prow. approved Indicates a PR has been approved by an approver from all required OWNERS files. Required by Prow. labels Jul 10, 2020
Copy link
Contributor

@mik-dass mik-dass left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

@openshift-ci-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dharmit, mik-dass

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@girishramnani
Copy link
Contributor Author

@dharmit lets get a confirm from @amitkrout on windows. removing your lgtm so that he can do that.
/lgtm cancel

@openshift-ci-robot openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. Required by Prow. label Jul 10, 2020
@prietyc123
Copy link
Contributor

I am testing it on windows. Till then I am holding the pr.

/hold

@openshift-ci-robot openshift-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. Required by Prow. label Jul 10, 2020
@prietyc123
Copy link
Contributor

I tested the pr on windows but the change doesn't solve the problem. Still I can see failure at the same point.

Test logs:

> make test-cmd-push
ginkgo  -randomizeAllSpecs -slowSpecThreshold=120 -timeout 7200s -nodes=1 -focus="odo push command tests" tests/integration/
Running Suite: Integration Suite
================================
Random Seed: 1594396762 - Will randomize all specs
Will run 15 of 177 specs

SSSSSSSSSSS
[...]
Running oc.exe with args [oc get dc backend-app --namespace yvcskaksif -o jsonpath='{range .spec.template.spec.containers[0].env[*]}{.name}:{.value}{"\n"}{end}']
[oc] 'DEBUG_PORT:5858
[oc] ODO_S2I_SCRIPTS_URL:/usr/local/s2i
[oc] ODO_S2I_SCRIPTS_PROTOCOL:image://
[oc] ODO_S2I_SRC_BIN_PATH:/tmp
[oc] ODO_S2I_DEPLOYMENT_DIR:/deployments
[oc] ODO_S2I_WORKING_DIR:/home/jboss
[oc] ODO_S2I_BUILDER_IMG:redhat-openjdk-18/openjdk18-openshift
[oc] ODO_SRC_BACKUP_DIR:/opt/app-root/src-backup
[oc] 'Running oc.exe with args [oc get pods --namespace yvcskaksif --selector=deploymentconfig=backend-app -o jsonpath='{.items[0].metadata.name}']
[oc] 'backend-app-1-flmpv'Running oc.exe with args [oc exec backend-app-1-flmpv --namespace yvcskaksif -c backend-app -- stat /tmp/src/src/main/java/AnotherMessageProducer.java]
[oc]   File: '/tmp/src/src/main/java/AnotherMessageProducer.java'
[oc]   Size: 413        Blocks: 8          IO Block: 4096   regular file
[oc] Device: 200080h/2097280d   Inode: 234945549   Links: 1
[oc] Access: (0666/-rw-rw-rw-)  Uid: (1001170000/   jboss)   Gid: (    0/    root)
[oc] Access: 2020-07-10 15:32:21.727901168 +0000
[oc] Modify: 2020-07-10 15:32:21.727901168 +0000
[oc] Change: 2020-07-10 15:32:21.727901168 +0000
[oc]  Birth: -
Running odo.exe with args [odo push --context C:\Users\Admin\AppData\Local\Temp\959273752]
[odo] I0710 15:32:24.727940    1516 preference.go:182] The path for preference file is C:\Users\Admin\AppData\Local\Temp\959273752\config.yaml
[odo] I0710 15:32:24.850178    1516 common_push.go:164] SourceLocation: ./
[odo] I0710 15:32:24.850178    1516 common_push.go:172] Source Path: C:\Users\Admin\AppData\Local\Temp\959273752
[odo] I0710 15:32:24.878532    1516 preference.go:182] The path for preference file is C:\Users\Admin\AppData\Local\Temp\959273752\config.yaml
[odo] Validation
[...]
[odo] Applying URL changes
[odo]  V  URLs are synced with the cluster, no changes are required.
[odo]
[odo] Pushing to component backend of type local
I0710 15:32:25.145944    1516 file_indexer.go:196] .odo or .git directory detected, skipping it
[odo] I0710 15:32:25.147404    1516 file_indexer.go:215] last modified date changed: C:\Users\Admin\AppData\Local\Temp\959273752\src\main\java
 V  Checking file changes for pushing [4ms]
[odo] I0710 15:32:25.147404    1516 file_indexer.go:236] Deleting file: src/main/java/AnotherMessageProducer.java
[odo] I0710 15:32:25.147404    1516 common_push.go:244] List of files to be deleted: +[src\main\java\AnotherMessageProducer.java]
[odo] I0710 15:32:25.147404    1516 common_push.go:267] Copying directory C:\Users\Admin\AppData\Local\Temp\959273752 to pod
[odo] I0710 15:32:25.147404    1516 component.go:608] PushLocal: componentName: backend, applicationName: app, path: C:\Users\Admin\AppData\Local\Temp\959273752, files: [C:\Users\Admin\AppData\Local\Temp\959273752\src\main\java], delFiles: [src\main\java\AnotherMessageProducer.java], isForcePush: false
[odo] I0710 15:32:25.168122    1516 preference.go:182] The path for preference file is C:\Users\Admin\AppData\Local\Temp\959273752\config.yaml
[odo] I0710 15:32:25.168122    1516 occlient.go:1901] Waiting for deploymentconfig=backend-app pod
[odo]  -  Waiting for component to start  ...
[odo] I0710 15:32:25.184903    1516 occlient.go:1925] Status of backend-app-1-flmpv pod is Running
[odo] I0710 15:32:25.184903    1516 occlient.go:1928] Pod backend-app-1-flmpv is Running
[odo] I0710 15:32:25.184903    1516 component.go:647] propogating deletion of files src\main\java\AnotherMessageProducer.java to pod
[odo] I0710 15:32:25.184903    1516 occlient.go:3192] s2ipaths marked for deletion are [/deployments/src/src/main/java/AnotherMessageProducer.java /tmp/src/src/main/java/AnotherMessageProducer.java /home/jboss/src/src/main/java/AnotherMessageProducer.java /opt/app-root/src-backup/src/src/main/java/AnotherMessageProducer.java]
 V  Waiting for component to start [16ms]
[odo]  -  Syncing files to the component  ...
[...]
 V  Building component [1m]
[odo]  V  Changes successfully pushed to component
[odo] I0710 15:33:36.367222    1516 odo.go:72] Could not get the latest release information in time. Never mind, exiting gracefully :)
Running oc.exe with args [oc get pods --namespace yvcskaksif --selector=deploymentconfig=backend-app -o jsonpath='{.items[0].metadata.name}']
[oc] 'backend-app-1-flmpv'Running oc.exe with args [oc exec backend-app-1-flmpv --namespace yvcskaksif -c backend-app -- stat /tmp/src/src/main/java/AnotherMessageProducer.java]
[oc] stat: cannot stat '/tmp/src/src/main/java/AnotherMessageProducer.java': No such file or directory
[oc] command terminated with exit code 1
Deleting project: yvcskaksif
Running odo.exe with args [odo project delete yvcskaksif -f]
[odo] I0710 15:33:38.463414    1072 application.go:49] Unable to list Service Catalog instances: unable to list ServiceInstances: the server could not find 
the requested resource (get serviceinstances.servicecatalog.k8s.io)
[odo] This project contains the following applications, which will be deleted
[odo] Application app
[odo] I0710 15:33:38.557411    1072 occlient.go:2795] Getting DeploymentConfig: backend-app
[odo] I0710 15:33:38.592456    1072 component.go:987] Source for component backend is  (local)
[odo] I0710 15:33:38.592456    1072 url.go:379] Listing routes with label selector: app.kubernetes.io/part-of=app,app.kubernetes.io/instance=backend        
[odo] I0710 15:33:38.592456    1072 occlient.go:2654] Listing routes with label selector: app.kubernetes.io/part-of=app,app.kubernetes.io/instance=backend  
[odo] I0710 15:33:38.655152    1072 occlient.go:2795] Getting DeploymentConfig: backend-app
[odo] This application has following components that will be deleted
[odo] I0710 15:33:38.707045    1072 occlient.go:2795] Getting DeploymentConfig: backend-app
[odo] I0710 15:33:38.729532    1072 component.go:987] Source for component backend is  (local)
[odo] I0710 15:33:38.729532    1072 url.go:379] Listing routes with label selector: app.kubernetes.io/part-of=app,app.kubernetes.io/instance=backend        
[odo] I0710 15:33:38.729532    1072 occlient.go:2654] Listing routes with label selector: app.kubernetes.io/part-of=app,app.kubernetes.io/instance=backend  
[odo] I0710 15:33:38.778417    1072 occlient.go:2795] Getting DeploymentConfig: backend-app
[odo] component named backend
[odo] I0710 15:33:38.842435    1072 url.go:379] Listing routes with label selector: app.kubernetes.io/part-of=app,app.kubernetes.io/instance=backend        
[odo] I0710 15:33:38.842435    1072 occlient.go:2654] Listing routes with label selector: app.kubernetes.io/part-of=app,app.kubernetes.io/instance=backend  
[odo] This component has following urls that will be deleted with component
[odo] URL named backend-8080 with host backend-8080-app-yvcskaksif.apps.viraj2233934.devcluster.openshift.com having protocol http at port 8080
[odo] No services / could not get services
[odo] I0710 15:33:38.880263    1072 project.go:136] unable to list services: unable to list ServiceInstances: the server could not find the requested resource (get serviceinstances.servicecatalog.k8s.io)
[odo]  V  Deleted project : yvcskaksif
[odo]  !  Warning! Projects are deleted from the cluster asynchronously. Odo does its best to delete the project. Due to multi-tenant clusters, the project 
may still exist on a different node.
[odo] I0710 15:33:38.908367    1072 odo.go:72] Could not get the latest release information in time. Never mind, exiting gracefully :)
Deleting dir: C:\Users\Admin\AppData\Local\Temp\959273752
Setting current dir to: C:\Users\Admin\go\src\github.com\openshift\odo\tests\integration

------------------------------
+ Failure [153.610 seconds]
odo push command tests
C:/Users/Admin/go/src/github.com/openshift/odo/tests/integration/cmd_push_test.go:16
  when push command is executed
  C:/Users/Admin/go/src/github.com/openshift/odo/tests/integration/cmd_push_test.go:112
    should delete the files from the container if its removed locally [It]
    C:/Users/Admin/go/src/github.com/openshift/odo/tests/integration/cmd_push_test.go:340

    Expected
        <string>: cmd [stat /tmp/src/src/main/java/AnotherMessageProducer.java] failed with error stat: cannot stat '/tmp/src/src/main/java/AnotherMessageProducer.java': No such file or directory
        command terminated with exit code 1
         on pod backend-app-1-flmpv
    to contain substring
        <string>: cannot stat '\tmp\src\src\main\java\AnotherMessageProducer.java': No such file or directory

    C:/Users/Admin/go/src/github.com/openshift/odo/tests/integration/cmd_push_test.go:379

@@ -367,7 +367,7 @@ var _ = Describe("odo push command tests", func() {
"backend",
appName,
project,
[]string{"stat", filepath.Join(dir, "src", "src", "main", "java", "AnotherMessageProducer.java")},
[]string{"stat", filepath.ToSlash(filepath.Join(dir, "src", "src", "main", "java", "AnotherMessageProducer.java"))},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

@prietyc123 prietyc123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@girishramnani Adding filepath.ToSlash() at line https://github.com/openshift/odo/pull/3524/files#diff-e977fd6c6dedc4d8ed7367e136764033R378 will really be helpful. I have added and tested it on windows with the current change. It works fine for me and purpose of the pr was also achieved.

Can you please add filepath.ToSlash() for the requested line also? After that we are good to go 👍

@girishramnani
Copy link
Contributor Author

girishramnani commented Jul 13, 2020

thanks, doing that now

@prietyc123
Copy link
Contributor

thanks, doing that now?

Great 🙂

@girishramnani
Copy link
Contributor Author

addressed
/hold cancel

@openshift-ci-robot openshift-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. Required by Prow. label Jul 14, 2020
@prietyc123
Copy link
Contributor

prietyc123 commented Jul 14, 2020

I ran the test change on windows and it works fine.

> git fetch origin pull/3524/head:pull_3524
remote: Enumerating objects: 26, done.
remote: Counting objects: 100% (26/26), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 29 (delta 20), reused 24 (delta 20), pack-reused 3
From https://github.com/openshift/odo
 * [new ref]         refs/pull/3524/head -> pull_3524

> git checkout pull_3524
Switched to branch 'pull_3524'

> make test-cmd-push                                                                                       
ginkgo  -randomizeAllSpecs -slowSpecThreshold=120 -timeout 7200s -nodes=1 -focus="odo push command tests" tests/integration/
Running Suite: Integration Suite
================================
Random Seed: 1594715752 - Will randomize all specs
Will run 15 of 177 specs

SSSSS+SSSSSSSSSSS+SSSSSSSSSSSSSSSSSSSSSSSSSSSSS+SSSSS+SSSSS+SS+SSSSSSSSSSSSS+Sthe foobar.txt file was not created, reason open C:\Users\Admin\AppData\Local\Temp\462106292\nodejs-ex\foobar.txt: The system cannot find the path specified.+S+SSSSSSSSSSSSSSSSSSSSS+SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS+SSSSSSSSSSSS+SSSSS+SS
------------------------------
+ [SLOW TEST:132.155 seconds]
odo push command tests
C:/Users/Admin/go/src/github.com/openshift/odo/tests/integration/cmd_push_test.go:16
  when push command is executed
  C:/Users/Admin/go/src/github.com/openshift/odo/tests/integration/cmd_push_test.go:112
    should delete the files from the container if its removed locally
    C:/Users/Admin/go/src/github.com/openshift/odo/tests/integration/cmd_push_test.go:340
------------------------------
SSSSSSSSSSSS+
Ran 15 of 177 Specs in 884.366 seconds
SUCCESS! -- 15 Passed | 0 Failed | 0 Pending | 162 Skipped
PASS

Ginkgo ran 1 suite in 15m8.1496036s
Test Suite Passed

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. Required by Prow. label Jul 14, 2020
@openshift-bot
Copy link

/retest

Please review the full test history for this PR and help us cut down flakes.

7 similar comments
@openshift-bot
Copy link

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-merge-robot openshift-merge-robot merged commit cdd76b3 into redhat-developer:master Jul 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. Required by Prow. area/Windows Issues or PRs specific to Windows lgtm Indicates that a PR is ready to be merged. Required by Prow.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

File separator error "\\tmp\\src\\server.js: No such file or directory" on windows
8 participants