New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add path filtering to build session client #33859

Merged
merged 2 commits into from Jul 6, 2017

Conversation

Projects
None yet
7 participants
@tonistiigi
Member

tonistiigi commented Jun 28, 2017

This PR enables path filtering capabilities to the file send stream so that it can be used in the future for the builder to only ask the paths that are actually needed or for transferring individual files(for example Dockerfile). This was described as follow-up in #32677 as optional but it occurred to me that if a client doesn't have support for this then future daemons would need to detect if clients have support for this or not(possible but harder to maintain). The fsutil package already had support for this but it was not enabled in session/filesync.

As there isn't a daemon feature that uses it yet I added a test helper that can be used to test the full lifecycle of the session in another commit. Hopefully this will become useful in the future for testing other similar features as well.

@thaJeztah @tiborvass

@thaJeztah

LGTM

Show outdated Hide outdated builder/dockerfile/builder.go

tonistiigi added some commits Jun 28, 2017

builder: enable path filtering for filesync session
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
add test for filesync path filtering and testutil helper
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
// HandleConn handles an incoming raw connection
func (sm *Manager) HandleConn(ctx context.Context, conn net.Conn, opts map[string][]string) error {
sm.mu.Lock()

This comment has been minimized.

@boaz1337

boaz1337 Jul 3, 2017

Member

I think sm's mu is being locked twice w/o unlocking in between:
One in line 56 and second one here.

@boaz1337

boaz1337 Jul 3, 2017

Member

I think sm's mu is being locked twice w/o unlocking in between:
One in line 56 and second one here.

This comment has been minimized.

@tonistiigi

tonistiigi Jul 3, 2017

Member

@ripcurld0 HandleHTTPRequest does not call this function but directly the one that assumes lock has already been taken.

@tonistiigi

tonistiigi Jul 3, 2017

Member

@ripcurld0 HandleHTTPRequest does not call this function but directly the one that assumes lock has already been taken.

This comment has been minimized.

@boaz1337

boaz1337 Jul 3, 2017

Member

Yes sorry, my bad. I didn't notice I am mixing between handleConn and HandleConn.

@boaz1337

boaz1337 Jul 3, 2017

Member

Yes sorry, my bad. I didn't notice I am mixing between handleConn and HandleConn.

@boaz1337

LGTM 🌵

dialer := session.Dialer(testutil.TestStream(testutil.Handler(m.HandleConn)))
g, ctx := errgroup.WithContext(context.Background())

This comment has been minimized.

@tiborvass

tiborvass Jul 4, 2017

Collaborator

TIL errgroup :P

@tiborvass

tiborvass Jul 4, 2017

Collaborator

TIL errgroup :P

@tiborvass

This comment has been minimized.

Show comment
Hide comment
@tiborvass

tiborvass Jul 4, 2017

Collaborator

LGTM

Collaborator

tiborvass commented Jul 4, 2017

LGTM

@vieux

This comment has been minimized.

Show comment
Hide comment
@vieux

vieux Jul 5, 2017

Collaborator

@tonistiigi test timed out on windows, maybe it's related ?

Collaborator

vieux commented Jul 5, 2017

@tonistiigi test timed out on windows, maybe it's related ?

@tonistiigi

This comment has been minimized.

Show comment
Hide comment
@tonistiigi

tonistiigi Jul 5, 2017

Member

@vieux I can't see how it could be as it doesn't do anything with build.

For the reference:

11:32:27 --- FAIL: Test (4018.35s)
11:32:27 panic: DockerSuite.TestStartReturnCorrectExitCode test timed out after 10m0s [recovered]
11:32:27 	panic: DockerSuite.TestStartReturnCorrectExitCode test timed out after 10m0s
11:32:27 
11:32:27 goroutine 33 [running]:
11:32:27 testing.tRunner.func1(0xc0423aeb60)
11:32:27 	d:/CI/CI-a5c90a1c6/go/src/testing/testing.go:622 +0x2a4
11:32:27 panic(0xc77760, 0xc04200e160)
11:32:27 	d:/CI/CI-a5c90a1c6/go/src/runtime/panic.go:489 +0x2dd
11:32:27 github.com/docker/docker/vendor/github.com/go-check/check.(*suiteRunner).runTest(0xc042422120, 0xc042235ce0, 0xc0423f0000)
11:32:27 	c:/gopath/src/github.com/docker/docker/vendor/github.com/go-check/check/check.go:875 +0x1aa
11:32:27 github.com/docker/docker/vendor/github.com/go-check/check.(*suiteRunner).run(0xc042422120, 0x1151a88)
11:32:27 	c:/gopath/src/github.com/docker/docker/vendor/github.com/go-check/check/check.go:621 +0xfb
11:32:27 github.com/docker/docker/vendor/github.com/go-check/check.Run(0xcf8e20, 0x1151a88, 0xc04241e050, 0xccb180)
11:32:27 	c:/gopath/src/github.com/docker/docker/vendor/github.com/go-check/check/run.go:100 +0x54
11:32:27 github.com/docker/docker/vendor/github.com/go-check/check.RunAll(0xc04241e050, 0x3)
11:32:27 	c:/gopath/src/github.com/docker/docker/vendor/github.com/go-check/check/run.go:92 +0xf4
11:32:27 github.com/docker/docker/vendor/github.com/go-check/check.TestingT(0xc0423aeb60)
11:32:27 	c:/gopath/src/github.com/docker/docker/vendor/github.com/go-check/check/run.go:80 +0x415
11:32:27 github.com/docker/docker/integration-cli.Test(0xc0423aeb60)
11:32:27 	c:/gopath/src/github.com/docker/docker/integration-cli/check_test.go:81 +0x246
11:32:27 testing.tRunner(0xc0423aeb60, 0xd45ec0)
11:32:27 	d:/CI/CI-a5c90a1c6/go/src/testing/testing.go:657 +0x9d
11:32:27 created by testing.(*T).Run
11:32:27 	d:/CI/CI-a5c90a1c6/go/src/testing/testing.go:697 +0x2d1
11:32:27 exit status 2
11:32:27 FAIL	github.com/docker/docker/integration-cli	4018.409s
11:32:27 
11:32:27 
11:32:27 ERROR: Failed 'ERROR: Integration tests failed at 07/05/2017 11:32:27. Duration:01:07:09.3901763' at 07/05/2017 11:32:27
11:32:27 
11:32:27 
11:32:27 
11:32:27 INFO: Tidying up at end of run
Member

tonistiigi commented Jul 5, 2017

@vieux I can't see how it could be as it doesn't do anything with build.

For the reference:

11:32:27 --- FAIL: Test (4018.35s)
11:32:27 panic: DockerSuite.TestStartReturnCorrectExitCode test timed out after 10m0s [recovered]
11:32:27 	panic: DockerSuite.TestStartReturnCorrectExitCode test timed out after 10m0s
11:32:27 
11:32:27 goroutine 33 [running]:
11:32:27 testing.tRunner.func1(0xc0423aeb60)
11:32:27 	d:/CI/CI-a5c90a1c6/go/src/testing/testing.go:622 +0x2a4
11:32:27 panic(0xc77760, 0xc04200e160)
11:32:27 	d:/CI/CI-a5c90a1c6/go/src/runtime/panic.go:489 +0x2dd
11:32:27 github.com/docker/docker/vendor/github.com/go-check/check.(*suiteRunner).runTest(0xc042422120, 0xc042235ce0, 0xc0423f0000)
11:32:27 	c:/gopath/src/github.com/docker/docker/vendor/github.com/go-check/check/check.go:875 +0x1aa
11:32:27 github.com/docker/docker/vendor/github.com/go-check/check.(*suiteRunner).run(0xc042422120, 0x1151a88)
11:32:27 	c:/gopath/src/github.com/docker/docker/vendor/github.com/go-check/check/check.go:621 +0xfb
11:32:27 github.com/docker/docker/vendor/github.com/go-check/check.Run(0xcf8e20, 0x1151a88, 0xc04241e050, 0xccb180)
11:32:27 	c:/gopath/src/github.com/docker/docker/vendor/github.com/go-check/check/run.go:100 +0x54
11:32:27 github.com/docker/docker/vendor/github.com/go-check/check.RunAll(0xc04241e050, 0x3)
11:32:27 	c:/gopath/src/github.com/docker/docker/vendor/github.com/go-check/check/run.go:92 +0xf4
11:32:27 github.com/docker/docker/vendor/github.com/go-check/check.TestingT(0xc0423aeb60)
11:32:27 	c:/gopath/src/github.com/docker/docker/vendor/github.com/go-check/check/run.go:80 +0x415
11:32:27 github.com/docker/docker/integration-cli.Test(0xc0423aeb60)
11:32:27 	c:/gopath/src/github.com/docker/docker/integration-cli/check_test.go:81 +0x246
11:32:27 testing.tRunner(0xc0423aeb60, 0xd45ec0)
11:32:27 	d:/CI/CI-a5c90a1c6/go/src/testing/testing.go:657 +0x9d
11:32:27 created by testing.(*T).Run
11:32:27 	d:/CI/CI-a5c90a1c6/go/src/testing/testing.go:697 +0x2d1
11:32:27 exit status 2
11:32:27 FAIL	github.com/docker/docker/integration-cli	4018.409s
11:32:27 
11:32:27 
11:32:27 ERROR: Failed 'ERROR: Integration tests failed at 07/05/2017 11:32:27. Duration:01:07:09.3901763' at 07/05/2017 11:32:27
11:32:27 
11:32:27 
11:32:27 
11:32:27 INFO: Tidying up at end of run
@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Jul 6, 2017

Member

All green, merging

Member

thaJeztah commented Jul 6, 2017

All green, merging

@thaJeztah thaJeztah merged commit 19ee873 into moby:master Jul 6, 2017

6 checks passed

dco-signed All commits are signed
experimental Jenkins build Docker-PRs-experimental 35451 has succeeded
Details
janky Jenkins build Docker-PRs 44063 has succeeded
Details
powerpc Jenkins build Docker-PRs-powerpc 4435 has succeeded
Details
windowsRS1 Jenkins build Docker-PRs-WoW-RS1 15445 has succeeded
Details
z Jenkins build Docker-PRs-s390x 4138 has succeeded
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment