Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker : Create official image #3356

Closed
3 tasks
zekth opened this issue Nov 16, 2019 · 45 comments
Closed
3 tasks

Docker : Create official image #3356

zekth opened this issue Nov 16, 2019 · 45 comments
Labels
suggestion suggestions for new features (yet to be agreed)

Comments

@zekth
Copy link
Contributor

zekth commented Nov 16, 2019

@hayd started a while https://github.com/hayd/deno-docker

This can be ported to be integrated as the Official Deno docker image. So 3 steps are needed:

  • Integrate generation of docker image in the CI
  • Test the generated docker image
  • Publish image on the docker hub / github hub on release

Also do we plan to release on docker hub only or we push it to github registry also? It's just a matter of URL in the command in the CI but @ry and @piscisaureus have to create / set credentials in the project.

@hayd
Copy link
Contributor

hayd commented Nov 16, 2019

Ry has denoland on dockerhub.

I think we should solve glibc issue on Alpine before shipping, as I think Alpine is preferred target. #3243
I don't know how to do sccache on docker, the Amazon Linux build (from scratch) is pretty slow on GitHub actions ~45minutes. However this could be a lot quicker if we instead generated a compatible binary as part of regular CI (no need for building or even installing curl just ADD).

On https://github.com/hayd/deno-docker GitHub actions has my docker hub API key and pushes on tags to docker hub, very easy. See release.yml. Deno could do both :latest (every commit) and :VERSION.

@brandonkal brandonkal mentioned this issue Jan 9, 2020
43 tasks
@hayd
Copy link
Contributor

hayd commented Jan 12, 2020

Targeting x86_64-unknown-linux-musl might make this work... And also mean the Amazon Linux (Lambda) build becomes trivial (at the moment it's kinda a pain/slow to create a compatible binary).

denoland/rusty_v8#49

@aadamsx
Copy link

aadamsx commented Jan 26, 2020

Currently trying out https://github.com/hayd/deno-docker

@grant
Copy link
Contributor

grant commented Mar 30, 2020

Currently trying out https://github.com/hayd/deno-docker

This has worked for me too. Docker Hub link: https://hub.docker.com/r/hayd/deno

@andrewmclagan
Copy link

Keenly following the progress of this. Alpine Deno image is essential :-)

@gvatn
Copy link
Contributor

gvatn commented May 11, 2020

Note that several people have had performance problems when using rust on musl: https://andygrove.io/2020/05/why-musl-extremely-slow/

@hayd
Copy link
Contributor

hayd commented May 11, 2020

I am happy to help move over hayd/deno-docker to denoland, if that was desired.

Most are a very thin wrapper, extract the binary, create user and deno-dir, set up entrypoint:
https://github.com/hayd/deno-docker/blob/master/alpine.dockerfile
(The alpine ones uses glibc.)

@kesor created a musl built image recently! My suspicion is it makes sense to incorporate some of that into BUILD.gn/gn-args instead of maintaining a patch to apply:
#3711 (comment)

It'd be great to have some deno-specific benchmarks before dismissing musl here... however one concern is currently alpine offers llvm9 which means a larger deno binary (since no_inline_line_tables=false), so atm the musl alpine image is larger than alpine w/ glibc.

@andrewmclagan
Copy link

I mean 27MB is quite slim really.

@andrewmclagan
Copy link

I think once this is decided on and stable we could PR the official docker repo.

@zekth
Copy link
Contributor Author

zekth commented May 15, 2020

Also we can push to docker registry and also github registry don't you think?

@nihaals
Copy link

nihaals commented May 20, 2020

It's also standard to have the OS part of the tag instead of the repository name. Is it possible to get _/deno? It might be better to have an official image.

@cqh963852
Copy link

Deno image based on the buster is needed for development

@cqh963852
Copy link

Will we create a new reop just for dockerfile?

@Vanuan
Copy link

Vanuan commented May 29, 2020

Just wanted to try this out, found out that there's no official image yet. What's the workaround?
This one? https://hub.docker.com/r/hayd/alpine-deno/

@nihaals
Copy link

nihaals commented May 30, 2020

We can make a proposal to Docker to have an official image and then both deno and Docker will help work on the image to make it as efficient and stable as possible. Updates are also managed by Docker. It's probably better to do it from the start so people don't have to move from a deno image to an official image and there will almost definitely be confusion with the differences.

@lucemans
Copy link

lucemans commented Jun 1, 2020

Working on this repository atm: https://github.com/lucemans/docker-deno
Currently have smaller images, and different way of installing to avoid having to download extra software etc. Also should auto update to the latest deno version

@nihaals
Copy link

nihaals commented Jun 2, 2020

That's not how Docker images are meant to be designed... Everything should be installed beforehand not while creating the container

@hayd
Copy link
Contributor

hayd commented Jun 7, 2020

We can make a proposal to Docker to have an official image and then both deno and Docker will help work on the image to make it as efficient and stable as possible.

@nihaals what is the route for such a proposal?

@nihaals
Copy link

nihaals commented Jun 7, 2020

The first step is to create our final images. I've created a repo which can house the images while they are being developed and reviewed. What will happen is first we will need to create the images (I think @hayd will probably be able to kickstart this). Once we have a set of images that work, I can create a PR to the official images repo. This kickstarts the proposal.

During this time, people who know a lot more about maintaining secure and efficient Docker images will give feedback which will then be implemented (they may PR, I think it would most likely require us to implement the changes ourselves), and once the images are good enough, they will be accepted. Every update the images will be updated and a new PR will be made giving the new GitHub commit with the newer images. These images will again be verified (most likely with less feedback), and then updated on Docker Hub. It's this reason why anyone can host the repo during this phase, as every update will be manually verified by them.

It's usually standard to build from source in images and making sure to delete any kind of caches. I can also implement a testing system to make sure any images actually work as there may be a large number of PRs initially. I personally think we should focus on slim images (using Debian) as these tend to have the least issues (for example Alpine using musl).

Once the first proper images are done, automation just needs to be used to make sure they are updated with the latest version of deno (possibly also updating any checksums). We can take inspiration from the Node and Python images as they're very optimised. The main thing I'm unsure on is architecture support, the rest I've played around with. Part of the proposal is listing the supported architectures and if one requires a different image, listing that too. It may also be useful to move the repo to some kind of docker-deno org (I've currently reserved this name), but right now the images are going to require a lot of fine tuning.

Also, a modification for some of the 3rd party images will be needed to work here: we should be using official base images, as this will increase our likelihood of being accepted.

@nihaals
Copy link

nihaals commented Jun 7, 2020

We also need a member to confirm whether they are or aren't interested in maintaining the image (either a member needs to be related to the repo or we need to have "contacted" them (assuming this means we need permission)). We shouldn't even really consider moving forward until we get some response as this will block it from happening at some point and it might be that they would prefer to be the main maintainers.

@nathanblair
Copy link

I've also got a vested interest in seeing a musl-compatible binary for release on alpine. It could also be extrapolated to be run on void-linux as well. Watching this as well as #3711.

If there's anything I can do to help with troubleshooting the Dockerfile give me a holler.

@andrewmclagan
Copy link

Any movement on this?

@nihaals
Copy link

nihaals commented Jul 21, 2020

Just waiting on a response from a project member

@youngjuning
Copy link
Contributor

Just waiting on a response from a project member +1

@zekth
Copy link
Contributor Author

zekth commented Jul 31, 2020

@bartlomieju Do you think we can move the @hayd project in the official Deno repo in the future? Seems like it's mature enough.

@nihaals
Copy link

nihaals commented Jul 31, 2020

I’m not sure if that makes much sense if it doesn’t follow the official docker image guidelines
There’s really not much that can be done until a member gives their input on either hosting the repo themselves or allowing a community member to
Going by the lack of response in ~8 months, Docker doesn’t seem to be a current priority so you should probably stick to either installing it yourself or using the previously mentioned repo

@hayd
Copy link
Contributor

hayd commented Jul 31, 2020

I’m not sure if that makes much sense if it doesn’t follow the official docker image guidelines

Are there some specific things hayd/docker-deno images are doing/not doing? Either way we can address these as GitHub issues ...

Atm it seems like there's been no indication the correct docker team is watching/interested in managing a deno image (yet).

Moving to denoland seems positive way forward.

@nihaals
Copy link

nihaals commented Jul 31, 2020

I already explained in my write-up.

It's obviously up to the project members to decide, they may not want to even have an official image for whatever reason. The key point is that it's up to them. It's their project, they have the final say, even if it's open source

@hayd
Copy link
Contributor

hayd commented Jul 31, 2020

When you say "project member" do you mean of deno(land)? or a member of the docker official images team?

@nihaals
Copy link

nihaals commented Jul 31, 2020

I'm quoting Docker's docs, I believe they say member of upstream, which I guess in this case would be a member of the denoland org

@hayd
Copy link
Contributor

hayd commented Aug 1, 2020

I see (the original link you posted)

it is preferable to have upstream software authors maintaining their corresponding Official Images

This reads like it's a good start for dockerfiles to be moved to denoland if we want a deno Official Image in the future.

@nihaals
Copy link

nihaals commented Aug 1, 2020

I agree, the only reason I'm hesitant is that it could actually delay progress since it would be rewritten from scratch (not all the current main images are built from source and use official base images)
Your images are great for anyone who needs a docker image now and is probably very similar to what people would do in their own custom images (I want to be clear that I think they're definitely a great resource), but official docker images, understandably, have much tighter restrictions as they could end up having billions of pulls, so feels like a drastically different beast. Many people wouldn't consider checksum checks to be very important, but when you're creating images that effectively define what good, safe and secure images should look like, they may be important

As Docker images don't seem to be a current priority, this possible further delay could mean that we don't get official images for a further 6 months due to miscommunication or misunderstandings of the requirements of official images, compared to having a space where these images are already defined, so having them available to everyone could be as simple as getting an OK from a member or transferring the repo

@ry
Copy link
Member

ry commented Aug 1, 2020

I don’t think we (denoland) are going to do an official docker image right now. Mostly because I’m not sure we have the bandwidth to manage it. My attempt to manage the vscode extension, for example, has not been going well. Andy’s image is working fine.

Maybe in the future if it makes sense.

@nihaals
Copy link

nihaals commented Aug 1, 2020

Does this also mean hold back on a "Docker Official" image or are you giving permission for someone else to work on and submit it? Anyone is allowed to submit one for review as long as they have permission afaik

@hayd
Copy link
Contributor

hayd commented Aug 1, 2020

Permission is not a prerequisite... at least it's not mentioned in the docs.

I'm happy to take any feedback in order for hayd/deno-docker to meet any Official requirements, please create separate GH issues on that repo. That way we could more easily move it to denoland or Docker Official in the future.

(@ry Atm updates are bumping the version number and tagging releases, so docker would be much less arduous than vscode... But it would still be "another thing" to maintain!)

@hayd
Copy link
Contributor

hayd commented Aug 1, 2020

Just to note, if there is some suggestion this is important/useful for Official Images, @lucacasonato is a maintainer of hayd/deno-docker and also a member of denoland.

Anyway, perhaps let's move this discussion there for now.

@jiraguha
Copy link

jiraguha commented Aug 9, 2020

@ry It already makes sense now IMO. I think that having ASAP an official image will remove some frictions to production... As a Kubernetes user, I would be more confident to ship deno to production with an official image that has support for the community. And Kubernetes is only one of several use-cases where we would need an official docker image.

hayd/deno-docker might be at an early stage but why do not we move it to denoland and incrementally improve it from there? People from the community that have time to maintain it, would find it easy to do PR's @ denoland. If there is no official place for it, the community will attend to be scattered and will have hundreds of images of deno to select or improve?

I hope that many others share the same opinion about such an important topic :-)

@ghost
Copy link

ghost commented Nov 19, 2020

So what's holding the "Deno" team to publish official image on Docker Hub? or it's not really up to Deno team?

@stale
Copy link

stale bot commented Jan 18, 2021

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Jan 18, 2021
@Tzahile
Copy link

Tzahile commented Jan 18, 2021

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

I don't think it makes sense to close this issue, as it remains an issue.
Furthermore, google search leads here. Closing this issue without resolving it might be confusing.

@stale stale bot removed the stale label Jan 18, 2021
@LRNZ09
Copy link

LRNZ09 commented Jan 29, 2021

I totally quote @jiraguha, I'm new to Deno, and I'd like to try it within a Docker container first.

@stale
Copy link

stale bot commented Mar 30, 2021

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Mar 30, 2021
@manniL
Copy link

manniL commented Mar 30, 2021

Oh No! :(

@stale stale bot removed the stale label Mar 30, 2021
@lucacasonato lucacasonato added the suggestion suggestions for new features (yet to be agreed) label Mar 30, 2021
@Guileas
Copy link

Guileas commented May 23, 2021

Hy,
Any news on this topic?

@bartlomieju
Copy link
Member

Huge thanks to @hayd who's been maintaining Docker images for the past two years. Today these images were transferred to denoland org and are available at https://github.com/denoland/deno_docker and thus are now considered official. Dockerhub account will be setup shortly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
suggestion suggestions for new features (yet to be agreed)
Projects
None yet
Development

No branches or pull requests