Skip to content

Add Vector to the Airflow images #291

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
siegfriedweber merged 7 commits into from
Feb 3, 2023
Merged

Add Vector to the Airflow images #291

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
f4f26c6
Add Vector to the Airflow images
siegfriedweber Jan 20, 2023
b0d8e33
Add inotify-tools to Vector image
siegfriedweber Feb 2, 2023
c4b6da3
Merge branch 'main' into airflow-logging
siegfriedweber Feb 2, 2023
db217c9
Update changelog
siegfriedweber Feb 2, 2023
4d55c79
Set the next Vector image tag to 0.26.0-stackable1.1.0
siegfriedweber Feb 2, 2023
ca614c6
Use docker.stackable.tech/stackable instead of docker.stackable.tech/…
siegfriedweber Feb 3, 2023
b77f28c
Merge branch 'main' into airflow-logging
siegfriedweber Feb 3, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions airflow/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,15 @@
# Changelog

## [Unreleased]

### Changed

- Upgraded to the base image vector:0.26.0-stackable1.1.0. The new base image
provides Vector. The creation of the stackable user and group happens in the
stackable-base layer and is therefore removed from this image ([#291]).

[#291]: https://github.com/stackabletech/docker-images/pull/291

## [airflow-stackable0.5.0] - 2022-11-30

### Added
Expand Down
22 changes: 7 additions & 15 deletions airflow/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM registry.access.redhat.com/ubi8/ubi-minimal:8.6@sha256:c5ffdf5938d73283cec018f2adf59f0ed9f8c376d93e415a27b16c3c6aad6f45 AS airflow-build-image
Comment thread
siegfriedweber marked this conversation as resolved.
FROM docker.stackable.tech/stackable/vector:0.26.0-stackable1.1.0@sha256:a0555b112329409669e33ebfb4044b332ade5a8286cf6506e34f2608c66c05d4 AS airflow-build-image

ARG PRODUCT
ARG PYTHON
Expand Down Expand Up @@ -28,7 +28,7 @@ RUN microdnf update \
&& pip install --no-cache-dir apache-airflow[${AIRFLOW_EXTRAS}]==${PRODUCT} --constraint /tmp/constraints.txt

FROM prom/statsd-exporter:0.3.0@sha256:a9c27602d6f6b86527657922b6a87c12789f7f9b39a90f1513e8c665c941f26a as statsd-exporter
FROM registry.access.redhat.com/ubi8/ubi-minimal:8.6@sha256:e58664de16551db29fb0eaaeb3c4a44eaf95ad89a5b2399a1107041c4f2d6d34 as airflow-main-image
FROM docker.stackable.tech/stackable/vector:0.26.0-stackable23.4.0-rc1@sha256:a0555b112329409669e33ebfb4044b332ade5a8286cf6506e34f2608c66c05d4 AS airflow-main-image

ARG PRODUCT
ARG PYTHON
Expand Down Expand Up @@ -60,32 +60,24 @@ RUN microdnf install -y yum python${PYTHON} \
ENV HOME=/stackable
ENV AIRFLOW_USER_HOME_DIR=/stackable
ENV PATH=$PATH:/bin:$HOME/app/bin
ENV AIRFLOW_UID="50000"
ENV AIRFLOW_UID="1000"
Comment thread
siegfriedweber marked this conversation as resolved.
ENV AIRFLOW_HOME=$HOME/airflow

RUN mkdir -pv ${HOME} && \
mkdir -pv ${AIRFLOW_HOME} && \
RUN mkdir -pv ${AIRFLOW_HOME} && \
mkdir -pv ${AIRFLOW_HOME}/dags && \
mkdir -pv ${AIRFLOW_HOME}/logs

RUN chown --recursive stackable:stackable ${AIRFLOW_HOME}

# according to arch, copy binary to the name "tini"
RUN curl -o /usr/bin/tini https://repo.stackable.tech/repository/packages/tini/tini-$(arch)

COPY airflow/stackable/utils/entrypoint.sh /entrypoint
COPY --from=statsd-exporter /bin/statsd_exporter /stackable/statsd_exporter
COPY --from=statsd-exporter --chown=stackable:stackable /bin/statsd_exporter /stackable/statsd_exporter

RUN chmod a+x /entrypoint && \
chmod +x /usr/bin/tini

RUN mkdir -pv ${HOME} && \
mkdir -pv ${AIRFLOW_HOME} && \
mkdir -pv ${AIRFLOW_HOME}/dags && \
mkdir -pv ${AIRFLOW_HOME}/logs

RUN groupadd -r stackable --gid=1000 && \
useradd -r -g stackable --uid=${AIRFLOW_UID} -d /stackable stackable && \
chown -R stackable:stackable /stackable

COPY --from=airflow-build-image --chown=stackable:stackable /stackable/ ${HOME}/

USER stackable
Expand Down
8 changes: 8 additions & 0 deletions vector/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Changelog

## [vector-stackable1.1.0] - 2023-02-03

### Added

- Package inotify-tools added ([#291]).

[#291]: https://github.com/stackabletech/docker-images/pull/291

## [vector-stackable1.0.0] - 2022-12-12

### Added
Expand Down
8 changes: 8 additions & 0 deletions vector/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,12 @@ RUN microdnf update && \

FROM docker.stackable.tech/stackable/stackable-base:1.0.0-stackable1.0.0@sha256:18a6434eea00c911aaa3dec773e642933bbbfefa953153553998d16b54f349ad

# https://github.com/hadolint/hadolint/wiki/DL4006
SHELL ["/bin/bash", "-o", "pipefail", "-c"]

RUN rpm --install https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm && \
microdnf update && \
microdnf --assumeyes install inotify-tools && \
microdnf clean all

COPY --from=builder /build /