Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgraded clj-docker-client to 0.5.2 #72

Merged
merged 2 commits into from
Mar 23, 2020

Conversation

TimoKramer
Copy link
Member

Most of the time it was sufficient to just change the function call
to run docker/invoke with a map of options. The new client returns
a message when there was a problem, so I needed to catch that to
return a failjure. Tests had to be changed to keep up with the
changed function calls.

A new connection is made everytime invoke is called with the new
clj-docker-client. That solves the problem we had with the former
version of the client-lib. The url bob connects to is now configured
via env-var. Documentation shows handling of variables set globally.

Squashed commit of the following:

commit e6b5a8bad70bbe331dcebbf1cc4fa053bbc6167e
Author: Timo Kramer fw-git@timokramer.de
Date: Tue Mar 17 20:36:36 2020 +0100

using patched docker-lib to solve socket-problem

commit 68b967459f9100f022b924cc5242dce09280c7d9
Author: Timo Kramer fw-git@timokramer.de
Date: Tue Mar 17 15:57:32 2020 +0100

Revert "trying different things to solve socket problems"

This reverts commit 57a335e03e895f4a2835240b97780284e0090563.

commit 57a335e
Author: Timo Kramer fw-git@timokramer.de
Date: Sun Mar 8 15:36:52 2020 +0100

trying different things to solve socket problems

commit 21ad5f9
Author: Timo Kramer fw-git@timokramer.de
Date: Fri Mar 6 13:15:51 2020 +0100

Upgraded clj-docker-client to 0.5.1

- Refactored the interfacing functions to use data-driven client
- Added different connections for different tasks to multiplex
- Amended tests to adapt to new client functions
- Refactored some naming to be more precise
- Closes #68

Most of the time it was sufficient to just change the function call
to run docker/invoke with a map of options. The new client returns
a message when there was a problem, so I needed to catch that to
return a failjure. Tests had to be changed to keep up with the
changed function calls.

Still there is a multiplex-problem when pulling multiple images.

Squashed commit of the following:

commit 64b6a322078559e8ec29a94423945dc09b8d7c0a
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Thu Mar 5 20:23:18 2020 +0100

    improved some tests

commit 554fa1115972195131e171741f4f0ed1c176b426
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Tue Mar 3 15:06:08 2020 +0100

    the errors were not printed correctly

commit 0915bfe59776b2e67a41e3fb0e116d14c484a8ae
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Tue Mar 3 15:05:45 2020 +0100

    extra connection for a pull because experiencing Socket problems

commit ab297d4f29e7ed809db5605850cfa769bed0b91c
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Tue Mar 3 01:02:04 2020 +0100

    less delay because pipeline is passed and cannot be stopped

commit a6c5c3ab7aaa7f59129676e9ac904ae0d33db3ec
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Mon Mar 2 23:32:13 2020 +0100

    logging was not working because of special chars

commit 5fd3b69f3cf0972d0bce5712167d9b06ea37b3d1
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Mon Mar 2 14:15:12 2020 +0100

    TODO was superfluous

commit 47fc68a40e6918ff60c20ff53236f8b738a966d8
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Mon Mar 2 12:24:55 2020 +0100

    refactored run to start-container because more accurate

commit 04d51c03a54a333b611b4c963658a6ffcca8a412
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Mon Mar 2 12:14:33 2020 +0100

    cider-formatting

commit 50ea2fc8bedb1db06f9df9802b5c78d9230eabbd
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Mon Mar 2 12:11:08 2020 +0100

    refactored function build to create-container because that is more accurate

commit aec1d81a3347a2d7d876288bb6d9d8ddee549b5d
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Mon Mar 2 11:34:15 2020 +0100

    comment corrected

commit f7eb5782ae4912a59ed9825287d9a4d788b614bc
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Mon Mar 2 10:58:07 2020 +0100

    stream to artifact-store works now

commit 509cbc98f8a220979edb00d6a798cdf0312f61b1
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Fri Feb 28 18:32:24 2020 +0100

    bootstrap_wendy.sh fails on extracting the tar

commit aed0816dd469082e31959fd95d6ba7b221d3b240
Merge: 05e1d75 afa375b
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Fri Feb 28 14:50:09 2020 +0100

    Merge remote-tracking branch 'bob/master' into #68-upgrade-docker-client

commit 05e1d7574162977d219e5a4d036f897ee8eade62
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Wed Feb 26 16:24:10 2020 +0100

    refactored

commit e9393274c754b991126ea0e743b3b6d29eaa3148
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Wed Feb 26 10:15:39 2020 +0100

    mistake in health-check was always returning unhealthy or errored out on connection issue

commit f9577df9dbd6a41fe370e4e1a20382d1ea24a37d
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Tue Feb 25 15:38:05 2020 +0100

    fixed malformed return of commit-image function

commit 9d05e0d0ac7aafed8b4c27ba5218e15b68430477
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Tue Feb 25 12:41:14 2020 +0100

    took functions to format CMD and ENV for passing to daemon

commit 1217e313fcb2e3c909158ac660e254c9be1cb840
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Mon Feb 24 18:50:08 2020 +0100

    TODO added

commit 5c33c056b113303d6df9a52f5bd0c4452e923598
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Mon Feb 24 14:05:13 2020 +0100

    log-streaming implemented

commit bef9118a3dde1eb719409ce441135e31650e065a
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Sun Feb 23 15:26:17 2020 +0100

    updated docker-client

commit 1bfddf12dbe13a4a7e5eb4be61d1c74ea08a726b
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Sun Feb 23 15:26:03 2020 +0100

    timeouts added for docker connection

commit 180262acadd1e9a6142d5364a56fb8ef7e2fe2b2
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Sun Feb 23 15:25:42 2020 +0100

    bug filed for clj-docker on timeout on putcontainerarchive

commit 4298a4f7167ecdd25676f330b09a7dc9d278c4b4
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Tue Feb 18 17:15:36 2020 +0100

    garbage collection repaired

commit 2036b96d0cfc830ce6055259341978eaaea56290
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Tue Feb 18 17:15:12 2020 +0100

    refactored execution internals

commit 5b3b25b303fb0d091be91ed61275536e0d5afc4f
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Tue Feb 18 14:06:07 2020 +0100

    fixed some tests

commit a550949e651c77c85084f7a4ec429dd2317a7836
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Tue Feb 18 11:36:02 2020 +0100

    pipeline internals adapted

commit fbf8a98ab802230dc1258964ab90e0f7d955f88e
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Mon Feb 17 17:55:39 2020 +0100

    refactored commit-image into execution/internals and adapted pipeline/internals next-step to new clj-docker-client

commit e7084b3db88e8262421b3ea426ed8f7fc65174b2
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Sat Feb 15 18:21:15 2020 +0100

    failing test; dont know how to mock file

commit 44acdb9c70d51f67dd82a7015d835831312f3aae
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Sat Feb 15 18:20:20 2020 +0100

    made commit a state

commit d9e003f721dc59e3f792a368f00c8a7ab31c40ac
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Fri Feb 14 16:06:23 2020 +0100

    resource internals adapted

commit 74baf389b5d648a23f3c298dbe61dceff85b2ec2
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Mon Feb 10 16:01:08 2020 +0100

    artifact/core adapted

commit 6b81cce92b510b9d82930aa38a2a988c3a175030
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Mon Feb 10 12:44:52 2020 +0100

    health check adapted

commit 5c9f70b1f1f1df9dea345da112fc7eb27116fe05
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Thu Feb 6 17:50:31 2020 +0100

    run function using new clj-docker

commit 448df78fede46626272b697dc00ffbeb09921b18
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Thu Feb 6 16:56:54 2020 +0100

    improved build to use failjure return

commit 1f89b7595a7b9dfdc2ca505af437dc1f86177210
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Thu Feb 6 16:22:16 2020 +0100

    status-of function now using new clj-docker lib

commit 8de927372d9ca1aa0f033027fd5042cfc7c7ae58
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Thu Feb 6 15:06:03 2020 +0100

    validation for image name improved and container build implemented

commit b8eda9d2af00139058f2be756266e4aafe28dae3
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Tue Feb 4 15:01:02 2020 +0100

    log-and-fail to avoid repetition

commit d38ff6c8e5a674c03a12939ab0d8e8b8bb698874
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Sun Feb 2 18:36:38 2020 +0100

    pull image function refactored

commit c7a4c662b639b374ebd1158ccdd13020fc22d639
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Thu Jan 30 16:05:27 2020 +0100

    switched to states

commit 6d51bd2799eee2568522c3ac7f6e9ab0eac8625c
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Wed Jan 29 13:16:42 2020 +0100

    kill-container

commit d944ebc62827e93647e53d496c4aa7dc223e8e5d
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Wed Jan 29 10:51:55 2020 +0100

    first function transitioned to clj-docker 0.4.0

@TimoKramer
Copy link
Member Author

TimoKramer commented Mar 23, 2020

Tests run green locally but the new version of the clj-docker-client needs to be available to show green tests in github-actions.
I think #56 should be resolved now as you told me, I am not sure but you told me so.

@lispyclouds
Copy link
Member

Yay! Finally! Lemme know when I can merge the docker client PR. This is awesome 😄

@lispyclouds
Copy link
Member

We'll keep the PR open till we upload the new docker client

@lispyclouds
Copy link
Member

Released

#(is (f/failed? (status-of "id"))))))

(deftest container-runs
(testing "successful run"
;(deftest logs-live-test
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can remove this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to test the streaming of logs but failed. Should we pursue that?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can remove this. its again side effects. maybe just leave a TODO here without the comment?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the whole comment. integration tests are covering it and since it is side effects anyways we are better off with integration testing it... I think...

 - Refactored the interfacing functions to use data-driven client
 - Added different connections for different tasks to multiplex
 - Amended tests to adapt to new client functions
 - Refactored some naming to be more precise
 - Closes bob-cd#68
 - Closes bob-cd#56
 - Refers bob-cd#20 into-docker/clj-docker-client#20

 Most of the time it was sufficient to just change the function call
 to run docker/invoke with a map of options. The new client returns
 a message when there was a problem, so I needed to catch that to
 return a failjure. Tests had to be changed to keep up with the
 changed function calls.

 A new connection is made everytime invoke is called with the new
 clj-docker-client. That solves the problem we had with the former
 version of the client-lib. The url bob connects to is now configured
 via env-var. Documentation shows handling of variables set globally.

Squashed commit of the following:

commit e6b5a8bad70bbe331dcebbf1cc4fa053bbc6167e
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Tue Mar 17 20:36:36 2020 +0100

    using patched docker-lib to solve socket-problem

commit 68b967459f9100f022b924cc5242dce09280c7d9
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Tue Mar 17 15:57:32 2020 +0100

    Revert "trying different things to solve socket problems"

    This reverts commit 57a335e.

commit 57a335e
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Sun Mar 8 15:36:52 2020 +0100

    trying different things to solve socket problems

commit 21ad5f9
Author: Timo Kramer <fw-git@timokramer.de>
Date:   Fri Mar 6 13:15:51 2020 +0100

    Upgraded clj-docker-client to 0.5.1

    - Refactored the interfacing functions to use data-driven client
    - Added different connections for different tasks to multiplex
    - Amended tests to adapt to new client functions
    - Refactored some naming to be more precise
    - Closes bob-cd#68

    Most of the time it was sufficient to just change the function call
    to run docker/invoke with a map of options. The new client returns
    a message when there was a problem, so I needed to catch that to
    return a failjure. Tests had to be changed to keep up with the
    changed function calls.

    Still there is a multiplex-problem when pulling multiple images.

    Squashed commit of the following:

    commit 64b6a32
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Thu Mar 5 20:23:18 2020 +0100

        improved some tests

    commit 554fa11
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Tue Mar 3 15:06:08 2020 +0100

        the errors were not printed correctly

    commit 0915bfe
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Tue Mar 3 15:05:45 2020 +0100

        extra connection for a pull because experiencing Socket problems

    commit ab297d4
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Tue Mar 3 01:02:04 2020 +0100

        less delay because pipeline is passed and cannot be stopped

    commit a6c5c3a
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Mon Mar 2 23:32:13 2020 +0100

        logging was not working because of special chars

    commit 5fd3b69
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Mon Mar 2 14:15:12 2020 +0100

        TODO was superfluous

    commit 47fc68a
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Mon Mar 2 12:24:55 2020 +0100

        refactored run to start-container because more accurate

    commit 04d51c0
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Mon Mar 2 12:14:33 2020 +0100

        cider-formatting

    commit 50ea2fc
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Mon Mar 2 12:11:08 2020 +0100

        refactored function build to create-container because that is more accurate

    commit aec1d81
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Mon Mar 2 11:34:15 2020 +0100

        comment corrected

    commit f7eb578
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Mon Mar 2 10:58:07 2020 +0100

        stream to artifact-store works now

    commit 509cbc9
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Fri Feb 28 18:32:24 2020 +0100

        bootstrap_wendy.sh fails on extracting the tar

    commit aed0816
    Merge: 05e1d75 afa375b
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Fri Feb 28 14:50:09 2020 +0100

        Merge remote-tracking branch 'bob/master' into bob-cd#68-upgrade-docker-client

    commit 05e1d75
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Wed Feb 26 16:24:10 2020 +0100

        refactored

    commit e939327
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Wed Feb 26 10:15:39 2020 +0100

        mistake in health-check was always returning unhealthy or errored out on connection issue

    commit f9577df
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Tue Feb 25 15:38:05 2020 +0100

        fixed malformed return of commit-image function

    commit 9d05e0d
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Tue Feb 25 12:41:14 2020 +0100

        took functions to format CMD and ENV for passing to daemon

    commit 1217e31
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Mon Feb 24 18:50:08 2020 +0100

        TODO added

    commit 5c33c05
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Mon Feb 24 14:05:13 2020 +0100

        log-streaming implemented

    commit bef9118
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Sun Feb 23 15:26:17 2020 +0100

        updated docker-client

    commit 1bfddf1
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Sun Feb 23 15:26:03 2020 +0100

        timeouts added for docker connection

    commit 180262a
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Sun Feb 23 15:25:42 2020 +0100

        bug filed for clj-docker on timeout on putcontainerarchive

    commit 4298a4f
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Tue Feb 18 17:15:36 2020 +0100

        garbage collection repaired

    commit 2036b96
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Tue Feb 18 17:15:12 2020 +0100

        refactored execution internals

    commit 5b3b25b
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Tue Feb 18 14:06:07 2020 +0100

        fixed some tests

    commit a550949
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Tue Feb 18 11:36:02 2020 +0100

        pipeline internals adapted

    commit fbf8a98
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Mon Feb 17 17:55:39 2020 +0100

        refactored commit-image into execution/internals and adapted pipeline/internals next-step to new clj-docker-client

    commit e7084b3
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Sat Feb 15 18:21:15 2020 +0100

        failing test; dont know how to mock file

    commit 44acdb9
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Sat Feb 15 18:20:20 2020 +0100

        made commit a state

    commit d9e003f
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Fri Feb 14 16:06:23 2020 +0100

        resource internals adapted

    commit 74baf38
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Mon Feb 10 16:01:08 2020 +0100

        artifact/core adapted

    commit 6b81cce
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Mon Feb 10 12:44:52 2020 +0100

        health check adapted

    commit 5c9f70b
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Thu Feb 6 17:50:31 2020 +0100

        run function using new clj-docker

    commit 448df78
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Thu Feb 6 16:56:54 2020 +0100

        improved build to use failjure return

    commit 1f89b75
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Thu Feb 6 16:22:16 2020 +0100

        status-of function now using new clj-docker lib

    commit 8de9273
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Thu Feb 6 15:06:03 2020 +0100

        validation for image name improved and container build implemented

    commit b8eda9d
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Tue Feb 4 15:01:02 2020 +0100

        log-and-fail to avoid repetition

    commit d38ff6c
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Sun Feb 2 18:36:38 2020 +0100

        pull image function refactored

    commit c7a4c66
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Thu Jan 30 16:05:27 2020 +0100

        switched to states

    commit 6d51bd2
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Wed Jan 29 13:16:42 2020 +0100

        kill-container

    commit d944ebc
    Author: Timo Kramer <fw-git@timokramer.de>
    Date:   Wed Jan 29 10:51:55 2020 +0100

        first function transitioned to clj-docker 0.4.0
@TimoKramer
Copy link
Member Author

there was an update of the chime-lib a few days ago. It needed some rewriting. I fixed it.

@TimoKramer
Copy link
Member Author

seems to be working fine for me... ready

@lispyclouds
Copy link
Member

Yeah maybe the whole async revamp can reduce the need for extra libs like this.

@TimoKramer
Copy link
Member Author

integration tests are a bit flaky

@lispyclouds
Copy link
Member

yeah can be investigated separately.

@lispyclouds lispyclouds merged commit 0f61c8f into bob-cd:master Mar 23, 2020
lispyclouds pushed a commit that referenced this pull request Mar 23, 2020
* Upgraded clj-docker-client to 0.5.2

 - Refactored the interfacing functions to use data-driven client
 - Added different connections for different tasks to multiplex
 - Amended tests to adapt to new client functions
 - Refactored some naming to be more precise
 - Closes #68
 - Closes #56
 - Refers #20 into-docker/clj-docker-client#20
@TimoKramer TimoKramer deleted the #68-upgrade-docker-client branch March 30, 2020 15:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Upgrade to 0.4.0+ of clj-docker-client Make it work with JDK 13+
2 participants