docker-compose up #2699

yancyribbens · 2019-02-25T16:23:12Z

https://i.gifer.com/52an.gif

build and start both lnd and the btcd backend with a single command docker-compose up. In order to enable docker-compose to bring up the services, I removed service definitions that aren't needed for running the environment. The docs have been updated with new instructions for how to run btcctl now that it's not a separate service.

Also, this PR Moves ltc to a separate docker-compose file. A future PR can allow ltc to be brought up with a single command and inherit from docker-compose.yml.

cfromknecht · 2019-08-12T19:35:11Z

@yancyribbens might need a proper rebase?

yancyribbens · 2019-08-12T20:03:01Z

@cfromknecht I thought I did a proper rebase.. any idea why the diff between this branch and master is so... different?

cfromknecht · 2019-08-12T20:05:11Z

@yancyribbens it looks like a merge was applied instead of a rebase, judging by bfbe58b

yancyribbens · 2019-08-12T20:16:19Z

@cfromknecht think it was caused by a pull after rebase but before push.

valentinewallace

Ran it, works great! Thank you for this feature.

To reduce the diff, could you have docker-compose.ltc.yml inherit from docker-compose.yml and then add the docker litecoin instructions as docker-compose -f docker-compose.yml -f docker-compose.ltc.yml up (or something to that effect)?

yancyribbens · 2019-09-15T17:49:33Z

@valentinewallace I removed the btc specific contents from docker-compose.ltc.yml which reduced the diff a lot. Litecoin can be run now using docker-compose -f docker-compse.ltc.yml up. Does this work ok to reduce the diff size and keep Litcoin functionality?

yancyribbens · 2019-09-28T06:31:17Z

Successfully tested this on mainnet to make a small transaction. It took 4-5days to sync mainnet using BTCD on a dedicated machine with 8cores, 16gb of memory and a SSD drive. I think if this is merged the next thing would be to add the option to use bitcoind instead to improve performance.

yancyribbens · 2019-09-28T06:33:12Z

This PR closes issues: #2829, #2508, #2509

guggero · 2019-11-14T10:55:30Z

@yancyribbens: We had some changes to these files recently. Could you please rebase then I'm going to have a look. Thanks!

yancyribbens · 2019-11-17T18:40:28Z

@guggero sure i'll see if I can find some time this week to fix the merge conflicts.

guggero · 2019-12-04T12:11:10Z

Needs rebase.

yancyribbens · 2019-12-04T12:54:59Z

@guggero done

docker/docker-compose.ltc.yml

yancyribbens · 2019-12-08T09:44:01Z

@guggero my bad. I should have test the changes better. I've updated with your suggestions and confirmed docker-compose.ltc.yml builds and syncs with mainnet:

sudo NETWORK=mainnet docker-compose -f ./docker-compose.ltc.yml up
sudo docker exec -it ltcd /start-ltcctl.sh getblockchaininfo

Also, if we want the experience to be similar for both, I recommend updating the docker docs to add instructions for using LTC. #3808

guggero

We're almost there!
There are some trailing spaces that should be removed. And it would be nice if you could squash everything together into two commits, maybe one for the README and one for the rest. And use the correct docker: prefix in the commit messages.

guggero · 2019-12-11T14:00:12Z

docker/docker-compose.ltc.yml

+        - lnd:/root/.lnd
+      entrypoint: ["./start-lnd.sh"]
+      links:
+        - "ltcd:blockchain"   


nit: there are a few trailing spaces on this line

yancyribbens · 2019-12-12T14:49:45Z

@guggero I squashed the commits and prefixed docker to the message. I think it's preferable to include the readme changes in the same commit, so that if anyone is looking at the commit history can observe that the readme updates are tied together with the docker-compose file changes that go together. I'm happy to split the readme update to a different commit if you have a strong opinion to do so.

guggero

I tested the newest version and got some collisions. It would be nice if one could start both docker-compose.yml and docker-compose.ltc.yml at the same time.

docker/docker-compose.ltc.yml

guggero · 2019-12-16T13:46:52Z

docker/docker-compose.ltc.yml

+
+  # lnd volume is used for persisting lnd application data and chain state
+  # during container lifecycle.
+  lnd:


To avoid collision with the other lnd, name this volume lnd_ltc (change also in the volumes section above).

docker/ltcd/start-ltcctl.sh

yancyribbens · 2019-12-23T19:07:12Z

@guggero agreed that would be ideal. I think there's more work todo then your comments to make this happen. For example, L58 of start-lnd.sh needs to be changed so that if both docker-compose.yml and docker-compose.ltc.yml are running they don't both reference blockchain. In addition there are other changes since as renaming the shared volume. Would you be OK with merging this PR since it solves the original issue and then we could create a new issue and PR to run both compose files on the same machine? that way this won't continue to fall behind the master branch as well if we merge these changes as is.

johng · 2019-12-23T19:25:24Z

docker/docker-compose.ltc.yml

+        - shared:/rpc
+        - lnd:/root/.lnd
+      entrypoint: ["./start-lnd.sh"]
+      links:


Should we remove links, which is deprecated?

Links are a legacy option. We recommend using networks instead.

@johng removing the legacy issues could take place in a separate PR. I think there are enough changes in this PR and better to keep the PR simple and address the github issue imo.

guggero · 2020-01-03T13:44:19Z

I agree that more changes are necessary to get both bitcoin and litecoin running in parallel. In my opinion it's OK to postpone these changes (like switching to networks to get rid of the blockchain link/alias) to a separate PR.
But using the same volume for both lnds will lead to problems. So it would be nice if you could rename the ltc one. Then we can get this merged.

yancyribbens · 2020-01-03T18:27:59Z

@guggero the ltc lnd volume name has been updated.

guggero

Thanks, looks good to me!

carlaKC

Looks good to me 🐳

Only comment is that I ran into a sneaky gotcha when testing some of the block generation commands. docker-compose up only rebuilds if there is a change to the Dockerfile (which there isn't in this change), so the change to use localhost in start-btcctl.sh didn't update when I ran this the first time. Might be worth updating the readme to make a note of that?

Otherwise, awesome PR, glad this one is finally going to go in!

carlaKC · 2020-01-08T07:53:17Z

Also 😂 👏

https://i.gifer.com/52an.gif

yancyribbens · 2020-01-09T19:13:10Z

@carlaKC thanks for testing out the changes. If I understand your comment, I think if you run docker-compose build --no-cache that should work, even if the images had been previously built. Although, this should only be a temporary problem.

carlaKC · 2020-01-09T20:40:33Z

@carlaKC thanks for testing out the changes. If I understand your comment, I think if you run docker-compose build --no-cache that should work, even if the images had been previously built. Although, this should only be a temporary problem.

Yeah, I ended up rebuilding the image and it was fine :)
This is a once off problem, but I'm wondering if we should make a comment to warn people that they need to run with --node-cache so they don't run into the same issue. Very small thing though, excited to get this in!

guggero · 2020-01-10T08:19:19Z

@carlaKC I think we can just add that warning/hint to the release notes where we describe the changes of this PR. Then we can merge it for 0.9.0-beta-rc2. What do you think?

carlaKC · 2020-01-10T08:38:53Z

@carlaKC I think we can just add that warning/hint to the release notes where we describe the changes of this PR. Then we can merge it for 0.9.0-beta-rc2. What do you think?

Sure, sounds good to me! Happy to merge as is, it's just a small nit.

yancyribbens changed the title ~~Docker compose up~~ docker-compose up Feb 25, 2019

Roasbeef added docker Docker-related PRs/Issues P3 might get fixed, nice to have labels Mar 1, 2019

yancyribbens force-pushed the docker-compose-up branch 2 times, most recently from a1b78ee to 3d9cffd Compare March 15, 2019 13:31

yancyribbens force-pushed the docker-compose-up branch from 3d9cffd to 1db9326 Compare March 21, 2019 18:27

yancyribbens requested review from cfromknecht, halseth, joostjager, Roasbeef, valentinewallace and wpaulino as code owners August 12, 2019 19:05

yancyribbens force-pushed the docker-compose-up branch 2 times, most recently from 1db9326 to e8bedf3 Compare August 12, 2019 20:14

Roasbeef removed request for Roasbeef, cfromknecht, halseth, joostjager and wpaulino August 13, 2019 03:03

valentinewallace suggested changes Sep 13, 2019

View reviewed changes

guggero self-assigned this Nov 14, 2019

yancyribbens force-pushed the docker-compose-up branch from 94e8b01 to 0d70e66 Compare December 4, 2019 12:53

guggero requested changes Dec 4, 2019

View reviewed changes

docker/docker-compose.ltc.yml Show resolved Hide resolved

docker/docker-compose.ltc.yml Show resolved Hide resolved

yancyribbens force-pushed the docker-compose-up branch from 0d70e66 to 660b925 Compare December 8, 2019 09:36

guggero reviewed Dec 11, 2019

View reviewed changes

yancyribbens force-pushed the docker-compose-up branch from 660b925 to ebec6a2 Compare December 12, 2019 14:43

guggero requested changes Dec 16, 2019

View reviewed changes

yancyribbens force-pushed the docker-compose-up branch from ebec6a2 to 4eb0d44 Compare December 23, 2019 19:06

johng reviewed Dec 23, 2019

View reviewed changes

docker: enable docker-compose up command for btc and ltc compose files

1f457ac

yancyribbens force-pushed the docker-compose-up branch from 4eb0d44 to 1f457ac Compare January 3, 2020 18:26

guggero approved these changes Jan 6, 2020

View reviewed changes

guggero requested a review from carlaKC January 6, 2020 13:04

carlaKC approved these changes Jan 8, 2020

View reviewed changes

guggero merged commit 0ce2d6f into lightningnetwork:master Jan 10, 2020

guggero mentioned this pull request Jan 14, 2020

it should be simple to start setup with docker-compose up #2509

Closed

guggero added this to WIP in v0.9.0-beta via automation Jan 14, 2020

guggero moved this from WIP to Done in v0.9.0-beta Jan 14, 2020

guggero added this to the 0.9.0 milestone Jan 14, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docker-compose up #2699

docker-compose up #2699

yancyribbens commented Feb 25, 2019 •

edited

cfromknecht commented Aug 12, 2019

yancyribbens commented Aug 12, 2019

cfromknecht commented Aug 12, 2019

yancyribbens commented Aug 12, 2019

valentinewallace left a comment •

edited

yancyribbens commented Sep 15, 2019

yancyribbens commented Sep 28, 2019

yancyribbens commented Sep 28, 2019

guggero commented Nov 14, 2019

yancyribbens commented Nov 17, 2019

guggero commented Dec 4, 2019

yancyribbens commented Dec 4, 2019

yancyribbens commented Dec 8, 2019

guggero left a comment

guggero Dec 11, 2019

yancyribbens Dec 12, 2019

yancyribbens commented Dec 12, 2019

guggero left a comment

guggero Dec 16, 2019

yancyribbens commented Dec 23, 2019

johng Dec 23, 2019

yancyribbens Jan 3, 2020

guggero commented Jan 3, 2020

yancyribbens commented Jan 3, 2020

guggero left a comment

carlaKC left a comment

carlaKC commented Jan 8, 2020

yancyribbens commented Jan 9, 2020

carlaKC commented Jan 9, 2020

guggero commented Jan 10, 2020

carlaKC commented Jan 10, 2020

docker-compose up #2699

docker-compose up #2699

Conversation

yancyribbens commented Feb 25, 2019 • edited

cfromknecht commented Aug 12, 2019

yancyribbens commented Aug 12, 2019

cfromknecht commented Aug 12, 2019

yancyribbens commented Aug 12, 2019

valentinewallace left a comment • edited

Choose a reason for hiding this comment

yancyribbens commented Sep 15, 2019

yancyribbens commented Sep 28, 2019

yancyribbens commented Sep 28, 2019

guggero commented Nov 14, 2019

yancyribbens commented Nov 17, 2019

guggero commented Dec 4, 2019

yancyribbens commented Dec 4, 2019

yancyribbens commented Dec 8, 2019

guggero left a comment

Choose a reason for hiding this comment

guggero Dec 11, 2019

Choose a reason for hiding this comment

yancyribbens Dec 12, 2019

Choose a reason for hiding this comment

yancyribbens commented Dec 12, 2019

guggero left a comment

Choose a reason for hiding this comment

guggero Dec 16, 2019

Choose a reason for hiding this comment

yancyribbens commented Dec 23, 2019

johng Dec 23, 2019

Choose a reason for hiding this comment

yancyribbens Jan 3, 2020

Choose a reason for hiding this comment

guggero commented Jan 3, 2020

yancyribbens commented Jan 3, 2020

guggero left a comment

Choose a reason for hiding this comment

carlaKC left a comment

Choose a reason for hiding this comment

carlaKC commented Jan 8, 2020

yancyribbens commented Jan 9, 2020

carlaKC commented Jan 9, 2020

guggero commented Jan 10, 2020

carlaKC commented Jan 10, 2020

yancyribbens commented Feb 25, 2019 •

edited

valentinewallace left a comment •

edited