Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deploy balena build docker images via balena push to Local Mode devices #613

Open
niklasnorin opened this issue Aug 9, 2017 · 24 comments
Open

Comments

@niklasnorin
Copy link

@niklasnorin niklasnorin commented Aug 9, 2017

The title basically says it all, let me know if you need any clarifications.

Front logo Front conversations

@pimterry
Copy link
Contributor

@pimterry pimterry commented Aug 9, 2017

Good idea. Sounds great to me, and I'm not immediately aware of anything that would make this especially difficult.

We've got some work upcoming to improve the CLI experience quite a bit, particularly around builds, deploys & local push - I'll make sure we consider this too as part of that (cc @shaunmulligan, @CameronDiver)

Loading

@niklasnorin
Copy link
Author

@niklasnorin niklasnorin commented Sep 6, 2017

Any progress on this one? :)

Loading

@pimterry
Copy link
Contributor

@pimterry pimterry commented Sep 11, 2017

Nothing yet I'm afraid, but I'll keep you posted.

Loading

@pimterry
Copy link
Contributor

@pimterry pimterry commented Sep 21, 2017

@niklasnorin this is now in the current (though not definite) plan. We're aiming to remove the word local entirely, and support equivalent functionality on both resin.io and locally using resin.io appnames/UUIDs vs ips, to differentiate the two cases. For this case, that would look like:

resin build . -t myImage
resin deploy appName myImage # deploy to resin.io
resin deploy 10.0.0.10 myImage # deploy to a local device

Would that work for you? None of this is definitively confirmed, but that's the current plan we're working with, just to keep you up to date.

Loading

@niklasnorin
Copy link
Author

@niklasnorin niklasnorin commented Oct 18, 2017

Been away and just got back, but I think that sounds like an excellent idea

Loading

@niklasnorin
Copy link
Author

@niklasnorin niklasnorin commented Dec 1, 2017

Hi guys,

Any ETA on this feature?

Being able to build and push in the same way from our CI servers and locally would really help speed up our development while working with the same Resin primitives.

Best regards,
Niklas

Loading

@CameronDiver
Copy link
Contributor

@CameronDiver CameronDiver commented Dec 4, 2017

Hey @niklasnorin

So the build + deploy functionality of the CLI is going to be overhauled, starting with the mulitcontainer update but continuing on after that. Unfortuantely I don't have any dates to give you right now, but we're in the alpha testing stages of the update (which includes CLI changes).

Loading

@curtismuntz
Copy link

@curtismuntz curtismuntz commented Jul 11, 2018

Are there any updates on this feature? I create container images using a build system outside of resin build and would love to be able to use this feature.

The current resin deploy appName myImage is sufficient for deploying to production devices but proves to be too slow for my local workflow.

Loading

@pimterry
Copy link
Contributor

@pimterry pimterry commented Jul 16, 2018

I'm afraid not @curtismuntz, but keep an eye on this issue and we'll keep you posted.

Loading

@devxpy
Copy link

@devxpy devxpy commented Jul 21, 2018

@pimterry Is 10.0.0.10 the local ip address of the device in your example code?

$ resin deploy 192.168.1.127 iwatkie_main
ResinApplicationNotFound: Application not found: 192.168.1.127

If you need help, don't hesitate in contacting us at:

  GitHub: https://github.com/resin-io/resin-cli/issues/new
  Forums: https://forums.resin.io
$ resin deploy 10.0.0.10 iwatkie_main
ResinApplicationNotFound: Application not found: 10.0.0.10

If you need help, don't hesitate in contacting us at:

  GitHub: https://github.com/resin-io/resin-cli/issues/new
  Forums: https://forums.resin.io

Loading

@pimterry
Copy link
Contributor

@pimterry pimterry commented Jul 23, 2018

@devxpy that's the suggestion, yes, but those are example commands for an upcoming feature. Those commands aren't something that will work right now.

resin deploy exists, but it's designed for deploying a built image via the resin.io cloud service, not to a local device.

Loading

@devxpy
Copy link

@devxpy devxpy commented Jul 25, 2018

I did try doing a resin build on my machine, and i have to admit, it was quite slow compared to the resin servers.

The resin servers fly compared to my local machine, probably because they do the builds on fast arm servers.

Has anyone else tried a resin build on their x86 machine? How was the experience?

Resin sync is fast enough for small code-changes.

On another note, As @curtismuntz suggested, I do find it slower than the traditional "write and run" cycle. For now, I have resorted to making my applications run-able on x86.

For example, my python app uses the keyboard library to emulate GPIO when it's running on x86.

try:
    import RPi.GPIO
except (RuntimeError, ImportError):
    import keyboard

    def is_pressed():
        return keyboard.is_pressed('ctrl+alt+r')
else:
    GPIO.setmode(GPIO.BCM)
    GPIO.setup(18, GPIO.IN, pull_up_down=GPIO.PUD_UP)

    def is_pressed():
        return not GPIO.input(18)

Loading

@pimterry
Copy link
Contributor

@pimterry pimterry commented Jul 25, 2018

Hi @devxpy! That sounds interesting. The CLI issue tracker is primarily for bug reports & tracking our internal development tasks though, not discussion. Could you post this in our forums instead? That'll get a much wider audience, and I'm sure there are others there who'd be interested to discuss approaches to this.

Loading

@flippidippi
Copy link

@flippidippi flippidippi commented Oct 5, 2018

I doing something like the following

resin build . -t myApp --deviceType raspberrypi3 --arch armv7l --docker '/var/run/docker.sock'

And I would love to be able to deploy that image instead of using local push and having to rebuild the image every time.

Loading

@pdcastro
Copy link
Contributor

@pdcastro pdcastro commented Jan 16, 2019

The balena push command now supports both remote and local builds, given an app name or the IP address of a device. (Btw, resin was renamed to balena.) When given an IP address, it both builds an image and and runs a container on the device itself, which for ARM devices like the Raspberry Pi 3 can be faster than using emulators on a laptop/desktop, and sometimes faster than the balena cloud builders (I think it depends on the project, like the balance between bandwidth intensive vs CPU intensive builds, the network speed, how much of the build can be cached, and the load on the cloud ARM builders).
So potentially this issue could be closed. Comments, feedback?

Loading

@wlisac
Copy link

@wlisac wlisac commented Mar 27, 2019

@pdcastro balena push supporting local builds is very nice.

I landed on this issue hoping to find a workaround for #1032 (thinking I'd try and build an image that skips .gitignore and then deploy it locally).

Workaround aside, I can still see potential workflow benefits of building an image independently and then deploying that to a local device 🙂

Loading

@mz8i
Copy link

@mz8i mz8i commented Mar 28, 2019

@pdcastro I still feel there should be an option, when using balena push with a local IP address, to specify that the build should be carried out on the local machine (PC) and not the device.

You say that the build might be faster on the Pi itself, but have there been any tests carried out to show that that's actually true? We are now working on some Pi's (we need to test software on the device itself because it's hardware-related) and to be honest the development process is a bit painful, with builds on the device taking a while, a lot of time is lost (already tried to optimise the builds with multi-stage etc). It just seems unlikely to me that the same build on the dev PCs, which are significantly more powerful than the Pi, would be slower, even when emulation is factored in.

The ideal workflow would really be: keep building the docker image on dev PC and delta-syncing to local device.

Loading

@balena-ci
Copy link
Contributor

@balena-ci balena-ci commented Jun 27, 2019

[mikesimos] This issue has attached support thread https://jel.ly.fish/#/support-thread~4225ec64-d1a0-4a1e-8f44-0284eb7c1ddd

Loading

@pdcastro pdcastro changed the title Deploy resin builddocker images via resin local push to Local Mode devices Deploy balena build docker images via balena push to Local Mode devices Nov 21, 2019
@pdcastro
Copy link
Contributor

@pdcastro pdcastro commented Nov 21, 2019

Just a note in case this helps anyone putting together custom scripts: with a device running a development image, it's possible to "manually" transfer Docker images between a laptop (running standard Docker) and a device (running balenaOS / balenaEngine) with commands such as:

# Copy a docker image from the laptop to the device
$ docker save image_name | ssh root@192.168.2.3 -p 22222 balena-engine load

# Copy a docker image from the device to the laptop
ssh root@192.168.2.3 -p 22222 balena-engine save image_name | docker load

... replacing "192.168.2.3" with the device's IP address on the local network.

Also worth noting: as of CLI v11 (currently 11.18.2), the "live push" feature is enabled by default with the balena push <ip-address> command. Documentation around deployment was also much improved earlier this year; check Deploy to your Fleet.

Loading

@Entropy512
Copy link

@Entropy512 Entropy512 commented Dec 13, 2019

@mz8i The build, in my workflow, MUST be carried out on the development workstation, because where I am, we have two VLANs for Linux devices:

  1. Workstations - this VLAN has public Internet access, but severe pressure on its IP address space at this point, so putting embedded devices on it isn't really an option
  2. Device testing - lots of IP address space, but access to the outside world is heavily restricted

'balena push" with a device sitting on VLAN 2 fails due to the inability for the device to pull anything from the outside world

Are there any other examples of an embedded Docker approach for deploying applications to a device that you are aware of that is not as cloud-dependent as Balena? I had been hoping to find something a little less painful for development workflow for embedded Linux devices than developing everything as a Yocto layer, but so far it's looking as unusable as Ubuntu Core in our environment.

Loading

@pdcastro
Copy link
Contributor

@pdcastro pdcastro commented Dec 13, 2019

@Entropy512, a couple of thoughts:

'balena push" with a device sitting on VLAN 2 fails due to the inability for the device to pull anything from the outside world

If the restriction on pulling from the outside world is more of a technical limitation than a security policy (i.e., if it is OK to bypass this restriction), then I would suggest using workstation-based NAT, in the form of the "Internet Connection Sharing" feature of Windows / Mac / Linux (docs for Windows, docs for Mac). This way, the device would effectively connect to the outside world under VLAN 1, but without requiring an additional IP address. I sometimes use this setup myself, for devices that have an Ethernet port but no WiFi -- it effectively creates a tiny subnet where my laptop is the router (it's great to debug issues at the network package level too, running a sniffer on the laptop).

Another thought is using on-premises openBalena, so that "the outside world" is actually your local subnet. You could use openBalena for either development or production, or both. Balena also offers services to help you customise and deploy your on-prem solution. The pricing page has a "get in touch" link (Contact Sales form), in case you're interested. If you just wanted to more casually discuss openBalena or alternative solutions, or expand on you VLAN setup, I would also suggest the balena forums, which are monitored by the balena support team: https://forums.balena.io/

Loading

@Entropy512
Copy link

@Entropy512 Entropy512 commented Dec 13, 2019

For initial development/evaluation, more of a technical issue, but it generally is a red flag that indicates unsuitability for eventual production deployment. Anything that requires embedded devices to have Internet connectivity to update (as opposed to me walking out to a device with a laptop and reflashing it) generally leads to me abandoning the approach quickly.

Rough estimate is 10% of our customers use our cloud-based telematics solutions (mostly mom-and-pop shops who don't have their own IT infrastructure), 60%+ use them with self-hosted infrastructure , and 30% use our telematics solutions as a prettier display but don't have WiFi in their warehouses and won't be installing it. (For those, updates are always done by USB stick or a field service laptop.)

As I said, I'm primarily looking at less painful workflows thank Yocto's layers for embedded development (including adding additional applications to a base OS for special-purpose things, or repurposing already-developed hardware for alternative purposes by making it easier to change out the applications), a vendor recently made references to using Docker as part of their embedded solution, and Balena happens to be the first hit on Google for "embedded Docker" (And the only one I can find that isn't someone describing how to use Docker to standardize their embedded development toolchain.)

I'll try to stop in on the forums, maybe someone has a thought of how to handle that "tech with USB stick or field service" laptop use case at a not insignificant percentage of customer sites. balena push with local mode seems like it would in theory be the closest possible solution here, IF it did building on the host machine.

Loading

@kelvinlawson
Copy link

@kelvinlawson kelvinlawson commented Jun 5, 2020

Hi @pdcastro, interesting. I came here to see if docker save could be used to upgrade devices without an Internet connection.

Are you saying it needs to be a development image because that is needed to do the balena-engine load or because SSH is used to upload the image?

I'm wondering if a USB stick could be used to upgrade the services on a non-development build, for customer sites that don't have or deliberately avoid an Internet connection.

Loading

@pdcastro
Copy link
Contributor

@pdcastro pdcastro commented Jun 5, 2020

@kelvinlawson, it's only because ssh is used to upload the image. If you can get the image file to the device with a USB stick, then you could run:

balena-engine load -i local-image-file

By the way, if the device is not connected to the internet but it is connected to a local network, and it is running a production image of balenaOS, the ssh commands could still be used (instead of a USB stick) by adding a ssh key to the config.json file (and rebooting).

Having said that, I don't know if this would be sufficient to perform "offline application updates". The balena supervisor on the device is responsible for managing the application images (deleting them and replacing them with newer app releases when available), and it will be (unsuccessfully) attempting to connect to balenaCloud in order to obtain the device's "target state". I have never tested manually replacing application images on a device that is not connected to the internet.

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Linked pull requests

Successfully merging a pull request may close this issue.

None yet