Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker image for arm64 platform #655

Closed
IncidGeo opened this issue May 9, 2023 · 8 comments · Fixed by #891
Closed

Docker image for arm64 platform #655

IncidGeo opened this issue May 9, 2023 · 8 comments · Fixed by #891

Comments

@IncidGeo
Copy link

IncidGeo commented May 9, 2023

Hi there,
I get a "no matching manifest for linux/arm64/v8 in the manifest list entries" message using main docker image.
Is there a arm compatible Docker image for Martin ?

Thanks,

@nyurik
Copy link
Member

nyurik commented May 9, 2023

@IncidGeo hi, the #613 tried to add a docker image to the automated build process, but we are having major problems building Arm docker image using github actions -- such that I plan to revert that PR until we have a better solution. I would love to add Arm Martin docker images, but we could really use some help to automate it in a reliable way.

It seems the docker images for different platforms need to somehow be combined together into a single package (?) before uploading them to the ghcr?

See also #603 and #505

@IncidGeo
Copy link
Author

IncidGeo commented May 9, 2023

Thank you nyurik. So to date there is'nt even a non-perfect solution to dockerize martin in the development phase ?

@nyurik
Copy link
Member

nyurik commented May 9, 2023

@IncidGeo define "non-perfect" :) There is a simple solution - you can just build it yourself instead of relying on a public docker image. If you have an arm64 machine somewhere, simply build the arm64.docker file on it, and use it.

@IncidGeo
Copy link
Author

IncidGeo commented May 9, 2023

It's beyond my skills, but I'll look into it, thank you.

@nyurik
Copy link
Member

nyurik commented May 9, 2023

it's actually pretty simple:

  • login to your Arm machine
  • git clone --depth=1 https://github.com/maplibre/martin
  • cd martin
  • docker build -f arm64.Dockerfile -t mydockerimage .
  • docker run --rm -it mydockerimage (use mydockerimage as if you downloaded it from the docker hub / ghcr)

nyurik added a commit that referenced this issue May 9, 2023
This reverts commit c358ec5, as well as
any other related changes to the docker github action.

It is clearly not working, while also not allowing us to build proper
releases quicker.

Several ways to fix:
* Use 3rd party CI service to just build multi-platform docker images
* Use cross-compilation wiht github actions
* (???)

See also #655,  #603 and #505
@IncidGeo
Copy link
Author

IncidGeo commented May 11, 2023

Thank you very much nyurik.
I've finnally succed to create a arm compatible docker image on a my private docker repository and pull it from my docker-compose.yml. It works.
docker tag mydockerimage dockerusername/nameofthedockerimage:test
docker push dockerusername/nameofthedockerimage:test
In the docker-compose.yml :

version: '3'

services:

  djangoapp:
    build:
        context: .
        dockerfile: config/dockerfiles/Dockerfile_django_python
    volumes:
      - .:/opt/services/djangoapp/src
      - static_volume:/opt/services/djangoapp/static  
      - media_volume:/opt/services/djangoapp/media  
    networks:
      - nginx_network
      - database1_network
    depends_on:
      - database1

  nginx:
    restart : always
    build:
        context: .
        dockerfile: config/dockerfiles/Dockerfile_nginx
    ports:
      - 80:80
    volumes:
      - static_volume:/opt/services/djangoapp/static 
      - media_volume:/opt/services/djangoapp/media   
    depends_on:
      - djangoapp
    networks:
      - nginx_network

  database1:
    restart : always
    build:
        context: .
        dockerfile: config/dockerfiles/Dockerfile_postgres
    ports:
      - 5432:5432
    env_file:
      - config/db/database1_env
    networks:
      - database1_network
    volumes:
      - database1_volume:/var/lib/postgresql/data

  martin:
    image: dockerusername/nameofthedockerimage:test
    restart: unless-stopped
    ports:
      - 3000:3000
    environment:
      - DATABASE_URL=postgresql://postgresusername:password@database1/database1 
    depends_on:
      - database1

networks:
  nginx_network:
    driver: bridge
  database1_network:
    driver: bridge

volumes:
  database1_volume:
  static_volume:  
  media_volume: 

dockefile for postgres (as there is the same problem for arm machine and postgis) :

FROM postgres

RUN apt update
RUN apt install postgis -y
COPY ./config/db/db.sql /docker-entrypoint-initdb.d/

db.sql :
CREATE EXTENSION postgis;

Just need now to figure out how to set up Nginx, django, postgres, martin together in the yml or maybe in the nginx conf file because Martin can't connect to the database while django/Gunicorn can, but it's an other subject.

Edit1 : It is working properly with the github "Using with Docker Compose" example :

version: '3'

services:
  martin:
    image: dockerusername/nameofthedockerimage:test
    restart: unless-stopped
    ports:
      - "3000:3000"
    environment:
      - DATABASE_URL=postgresql://postgres:password@db/db
    depends_on:
      - db

  db:
    build:
        context: .
        dockerfile: Dockerfile_postgres
    restart: unless-stopped
    environment:
      - POSTGRES_DB=db
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=password
    volumes:
      - ./pg_data:/var/lib/postgresql/data

and

FROM postgres

RUN apt update
RUN apt install postgis -y
COPY ./config/db/db.sql /docker-entrypoint-initdb.d/

db.sql :
CREATE EXTENSION postgis;

Edit2: Ok i'm such a nut, bridge were missing in the docker-compose.yml, now everything is working fine (i've change the db name and service nam for readability)

version: '3'

services:

  djangoapp:
    build:
        context: .
        dockerfile: config/dockerfiles/Dockerfile_django_python
    volumes:
      - .:/opt/services/djangoapp/src
      - static_volume:/opt/services/djangoapp/static  
      - media_volume:/opt/services/djangoapp/media  
    networks:
      - nginx_network
      - db_network
    depends_on:
      - db
      - martin

  nginx:
    restart : always
    build:
        context: .
        dockerfile: config/dockerfiles/Dockerfile_nginx
    ports:
      - 80:80
    volumes:
      - static_volume:/opt/services/djangoapp/static 
      - media_volume:/opt/services/djangoapp/media   
    depends_on:
      - djangoapp
    networks:
      - nginx_network

  db:
    restart: unless-stopped
    build:
        context: .
        dockerfile: config/dockerfiles/Dockerfile_postgres
    ports:
      - 5432:5432
    env_file:
      - config/db/db_env
    networks:
      - db_network
      - martin_network
    volumes:
      - db_volume:/var/lib/postgresql/data

  martin:
    image: dockerusername/nameofthedockerimage:test
    restart: unless-stopped
    ports:
      - 3000:3000
    environment:
      - DATABASE_URL=postgresql://postgres:password@db/db 
    depends_on:
      - db
    networks:
      - db_network
      - martin_network
      - nginx_network


networks:
  nginx_network:
    driver: bridge
  db_network:
    driver: bridge
  martin_network:
    driver: bridge

volumes:
  db_volume:
  static_volume:  
  media_volume: 

nyurik added a commit to nyurik/martin that referenced this issue Sep 26, 2023
This is a far simpler CI path:
* Compiles Ubuntu targets outside of docker
* Copies targets into appropriate docker image
* No compilation inside docker - makes it far faster
* Runs all integration tests on all platforms using emulation against host-based postgis

Fixes maplibre#655
nyurik added a commit to nyurik/martin that referenced this issue Sep 26, 2023
This is a far simpler CI path:
* Compiles Ubuntu targets outside of docker
* Copies targets into appropriate docker image
* No compilation inside docker - makes it far faster
* Runs all integration tests on all platforms using emulation against host-based postgis

Fixes maplibre#655
@nyurik
Copy link
Member

nyurik commented Sep 26, 2023

Thanks for all the hard work! I was able to finally create a multi-target docker image, and it also gets created much faster because it simply reuses a binary compiled on the host (while still running through all the integration tests). Once #891 merges, and i do a proper release, we should be good to go

@IncidGeo
Copy link
Author

As always super thank you !

nyurik added a commit that referenced this issue Sep 26, 2023
This is a far simpler CI path:
* Compiles Ubuntu targets outside of docker
* Copies targets into appropriate docker image
* No compilation inside docker - makes it far faster
* Runs all integration tests on all platforms using emulation against
host-based postgis

Fixes #655
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants