Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for the Docker Python SDK #2158

Merged
merged 50 commits into from Oct 15, 2017
Merged
Changes from 1 commit
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
8a4d331
added stub for minimal docker wrapper
Feb 26, 2017
8481b8a
don't set a name by default
Mar 1, 2017
83e6b48
handle None name
Mar 1, 2017
3b67754
changed to use the docker low level api
Mar 9, 2017
3cd4d11
removed debugging print
Mar 9, 2017
849e59c
refractored names
Mar 9, 2017
55f0528
added a test for single file mount
elipapa Mar 9, 2017
bb8122c
merged with tempdir support and image download
Mar 9, 2017
f1fb534
directory and file mounting working
Mar 9, 2017
ac55d9e
fixed tests and using busybox when needed
Mar 9, 2017
9321e4a
always use busybox
Mar 10, 2017
07fcb84
fallback if exception has no message
Mar 14, 2017
09f254e
lowered pull image to debug level
Mar 14, 2017
5258346
do not declare volumes unless they are passed as extra params
Mar 15, 2017
0f1547c
remove name requirement from tmpdir
elipapa Apr 25, 2017
a4643a1
linting
elipapa Apr 28, 2017
65b1a46
fix tests for tox
elipapa Apr 28, 2017
e0a4f28
tag must be specified to get the right number of images back from the…
elipapa Apr 28, 2017
a7da033
test to trigger duplicate mount pt issue
elipapa Apr 28, 2017
7273d3d
fix pid lock in tests
elipapa Apr 28, 2017
5750452
use a wrapper task in test
elipapa Apr 28, 2017
8c8580f
another test to see if the issue is in properties
elipapa May 1, 2017
6ce5586
created an _init_ method to avoid the mutable volumes list to be shar…
elipapa May 1, 2017
8452bf4
need args on the init to allow parameters
elipapa May 2, 2017
89ce5c0
solves pid lock
elipapa May 2, 2017
a625c32
linting
elipapa May 2, 2017
caedbcd
remove bad logging lines
elipapa May 2, 2017
fa3127c
fix handling of multiple volumes
elipapa May 17, 2017
f7c4197
must be passing kwargs when using super!
elipapa May 23, 2017
caf0070
old gitignore
elipapa Jun 15, 2017
9cf4e28
add elipapa as additional author
elipapa Jun 15, 2017
a1f0f29
Merge remote-tracking branch 'upstream/master'
elipapa Jun 15, 2017
d939dce
add a blog post about open targets use and this PR contribution
elipapa Jun 15, 2017
94f7890
skip tests if no docker daemon is present
elipapa Jun 16, 2017
0ae8df1
literal should become a bytes literal in Py3
elipapa Jun 16, 2017
32b1a88
flake8 linting
elipapa Jun 22, 2017
93608c2
do not mount tmp dir by default and remove unused import
elipapa Jun 30, 2017
ca21d2d
linting
elipapa Jun 30, 2017
90591a6
don't remove tmp directory if it was not created
elipapa Jun 30, 2017
68ffb7f
volume and binds should both be there
elipapa Jul 1, 2017
8cda56d
use host_config's auto_Remove functionality
elipapa Jul 1, 2017
93ac259
docker API v>1.25 required for auto_remove
elipapa Jul 2, 2017
72a97c2
extended module docs with contribution use case
Sep 9, 2017
e997426
flake8 linting
Sep 11, 2017
c8d007f
added version to docker dependency
Sep 11, 2017
76757f4
mount tmp dir by default, and rename volumes to binds
Sep 25, 2017
6ad3094
enable docker service in travis ci to allow DockerTask tests to run
Sep 25, 2017
f877ca3
Merge remote-tracking branch 'remotes/upstream/master'
Sep 26, 2017
fe5636d
Merge branch 'master' into master
apierleoni Oct 14, 2017
4e1fb1f
fix boto problem in tests
apierleoni Oct 14, 2017
File filter...
Filter file types
Jump to…
Jump to file or symbol
Failed to load files and symbols.
+14 −2
Diff settings

Always

Just for now

test to trigger duplicate mount pt issue

  • Loading branch information...
elipapa committed Apr 28, 2017
commit a7da0333ecf29e097e3eee19abf06bdd75962709
@@ -74,6 +74,17 @@ class MountLocalFileAsVolume(DockerTask):
volumes=[local_file.name+':/tmp/local_file_test']
command = 'test -f /tmp/local_file_test'

class MountLocalFileAsVolumeWithParam(DockerTask):
dummyopt = luigi.Parameter()
image = "busybox"
name = "MountLocalFileAsVolumeWithParam"
volumes=[local_file.name+':/tmp/local_file_test']
command = 'test -f /tmp/local_file_test'

class MultipleDockerTask(DockerTask):
def requires(self):
return [MountLocalFileAsVolumeWithParam(dummyopt = opt) for opt in ['one','two']]


class TestDockerTask(unittest.TestCase):

@@ -88,7 +99,6 @@ def test_temp_dir_creation(self):
writedir = WriteToTmpDir()
writedir.run()


def test_local_file_mount(self):
localfile = MountLocalFileAsVolume()
localfile.run()
@@ -101,4 +111,6 @@ def test_fail_job_container(self):
fail = FailJobContainer()
self.assertRaises(ContainerError, fail.run)


def test_multiple_jobs(self):
worked = luigi.run(["MultipleDockerTask", "--local-scheduler"])
self.assertTrue(worked)
ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.