Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor docker deployment and user setup #863

Merged
merged 25 commits into from
May 15, 2024
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
71d4288
Fix #862, address #826 and maybe some other tickets: reimplemented th…
Jul 7, 2023
32a6fff
make unique DB container name and use it in communication from Photoview
Jul 19, 2023
3531a36
Merge branch 'photoview:master' into Enhance-docker-compose
kkovaletp Jul 19, 2023
de5a13b
Removed unnecessary healthcheck for photoview from docker-compose.exa…
Aug 16, 2023
70687c0
Set RWX permissions to the application's working folder for any user,…
Sep 13, 2023
a7c13ad
Enhanced the "Getting started" section in the readme; added the `help…
Nov 7, 2023
97294bc
Use `slim` base image for final photoview image
Jan 27, 2024
8039b80
Implement SQLite support according to the PR #851
Jan 28, 2024
dafa519
Merge commit 'b2d591bd1b7396ad2ec2b148cb2aa8865e478b74' into Enhance-…
Mar 30, 2024
49060c2
Merge branch 'photoview:master' into Enhance-docker-compose
kkovaletp Apr 13, 2024
cfd61bc
Removed deprecated `version` line from compose files; optimized docke…
Apr 13, 2024
bfc9c28
fix a typo in the username; add support of PostgreSQL; split and opti…
Apr 20, 2024
132d42e
Fixed some typos and styling in Readme, excluded dev-environment setu…
Apr 23, 2024
2897a47
Implemented many security improvements, suggested by @Omar007, switch…
Apr 23, 2024
e1c5a2a
forgot the compose file
Apr 23, 2024
ee0f333
move face models back to /app folder; comment out and document unnece…
Apr 24, 2024
ee9fc8d
Exclude Makefile in the root folder from git; documented multiple mou…
Apr 26, 2024
9895695
Merge branch 'photoview:master' into Enhance-docker-compose
kkovaletp Apr 26, 2024
ecb76f6
Fixed several bugs after complete testing cycle with all 3 DBs
Apr 26, 2024
1d0ca17
removed hardcoded port in Dockerfile
May 9, 2024
4189623
Pin the major version for the `photoview` image for stability
May 9, 2024
35aa3d9
Merge branch 'photoview:master' into Enhance-docker-compose
kkovaletp May 13, 2024
e33dad2
Revert back to the port 80 inside the container on product owner's re…
May 14, 2024
def5531
Provide a minimal compose file and update the readme accordingly
May 14, 2024
d4d719c
Handle incorrect media file and folder permissions; set correct permi…
May 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -1,12 +1,22 @@
.dockerignore
.git
.github
.gitignore
.husky
.prettierrc
.eslintcache
.vscode
.env
docker-compose example
docker-compose.yml
docker-compose-dev.yml
Dockerfile
Dockerfile-dev
example.env
Makefile
photos_path
screenshots
storage

ui/node_modules/
ui/build/
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ testing.env

# docker
docker-compose.yml
storage
database

# dependencies
node_modules/
Expand Down
83 changes: 38 additions & 45 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -20,75 +20,67 @@ ARG COMMIT_SHA
ENV COMMIT_SHA=${COMMIT_SHA:-}
ENV REACT_APP_BUILD_COMMIT_SHA=${COMMIT_SHA:-}

RUN mkdir -p /app
WORKDIR /app

# Download dependencies
COPY ui/package*.json /app/
RUN npm ci --omit=dev --ignore-scripts

# Build frontend
COPY ui /app
RUN npm run build -- --base=$UI_PUBLIC_URL
WORKDIR /app
RUN npm ci --omit=dev --ignore-scripts \
# Build frontend
&& npm run build -- --base=$UI_PUBLIC_URL
Comment on lines +26 to +28
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why merge these into one command. Wouldn't caching be more efficient by installing dependencies first. Then copy the source files afterwards. That way only npm run build needs to run when depdencies don't change?

COPY ui/package*.json /app/
RUN npm ci --omit=dev --ignore-scripts

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, we've discussed this topic in the Discord: https://discord.com/channels/794576839813234688/1223295288668717146/1229127778679390319

Here is my answer for transparency:
Regarding layers: building production image on CI won't benefit much (if at all) from build cache, as in our project merges to master happen not so often. Having a well-optimized image for users is much better, as it pulls faster and uses less space on the host. It is even more important for ARM-based systems, where usually system partition is limited in space and performance.
In the case of the PR testing CI job, I'm not sure how exactly the docker cache works in Github, but from my experience with other CI/CD services, the docker cache is related to the branch, so every new PR will start a new thread of cache and won't utilize the cache from other PRs or master, so here there are no benefits from having more layers too.


### Build API ###
FROM --platform=${BUILDPLATFORM:-linux/amd64} debian:bookworm AS api
ARG TARGETPLATFORM

COPY docker/install_build_dependencies.sh /tmp/
RUN chmod +x /tmp/install_build_dependencies.sh && /tmp/install_build_dependencies.sh

COPY docker/go_wrapper.sh /go/bin/go
RUN chmod +x /go/bin/go
COPY api /app
WORKDIR /app

ENV GOPATH="/go"
ENV PATH="${GOPATH}/bin:${PATH}"

ENV CGO_ENABLED 1

RUN go env

RUN mkdir -p /app
WORKDIR /app

# Download dependencies
COPY api/go.mod api/go.sum /app/
RUN go mod download

# Patch go-face
RUN sed -i 's/-march=native//g' ${GOPATH}/pkg/mod/github.com/!kagami/go-face*/face.go

# Build dependencies that use CGO
RUN go install \
github.com/mattn/go-sqlite3 \
github.com/Kagami/go-face

# Copy and build api source
COPY api /app
RUN go build -v -o photoview .
RUN chmod +x /tmp/install_build_dependencies.sh \
&& chmod +x /go/bin/go \
&& /tmp/install_build_dependencies.sh \
&& go env \
&& go mod download \
# Patch go-face
&& sed -i 's/-march=native//g' ${GOPATH}/pkg/mod/github.com/!kagami/go-face*/face.go \
# Build dependencies that use CGO
&& go install \
github.com/mattn/go-sqlite3 \
github.com/Kagami/go-face \
# Build api source
&& go build -v -o photoview .

### Copy api and ui to production environment ###
FROM debian:bookworm
FROM debian:bookworm-slim
ARG TARGETPLATFORM
WORKDIR /app

COPY api/data /app/data

RUN apt update \
# Create a user to run Photoview server
RUN groupadd -r photoview \
&& useradd -r -g photoview photoview \
&& mkdir -p /home/photoview \
&& chown -R photoview:photoview /app /home/photoview \
kkovaletp marked this conversation as resolved.
Show resolved Hide resolved
# Required dependencies
&& apt install -y curl gpg libdlib19.1 ffmpeg exiftool libheif1

# Install Darktable if building for a supported architecture
RUN if [ "${TARGETPLATFORM}" = "linux/amd64" ] || [ "${TARGETPLATFORM}" = "linux/arm64" ]; then \
apt install -y darktable; fi

# Remove build dependencies and cleanup
RUN apt purge -y gpg \
&& apt update \
&& apt install -y curl gpg libdlib19.1 ffmpeg exiftool libheif1 \
# Install Darktable if building for a supported architecture
&& if [ "${TARGETPLATFORM}" = "linux/amd64" ] || [ "${TARGETPLATFORM}" = "linux/arm64" ]; then \
apt install -y darktable; \
fi \
# Remove build dependencies and cleanup
&& apt purge -y gpg \
&& apt autoremove -y \
&& apt clean \
&& rm -rf /var/lib/apt/lists/*

COPY --from=ui /app/dist /ui
COPY --from=api /app/photoview /app/photoview
COPY --chown=photoveiw:photoveiw api/data /app/data
COPY --chown=photoveiw:photoveiw --from=ui /app/dist /ui
COPY --chown=photoveiw:photoveiw --from=api /app/photoview /app/photoview
kkovaletp marked this conversation as resolved.
Show resolved Hide resolved

ENV PHOTOVIEW_LISTEN_IP 127.0.0.1
ENV PHOTOVIEW_LISTEN_PORT 80
kkovaletp marked this conversation as resolved.
Show resolved Hide resolved
Expand All @@ -100,4 +92,5 @@ EXPOSE 80

HEALTHCHECK --interval=60s --timeout=10s CMD curl --fail 'http://localhost:80/api/graphql' -X POST -H 'Content-Type: application/json' --data-raw '{"operationName":"CheckInitialSetup","variables":{},"query":"query CheckInitialSetup { siteInfo { initialSetup }}"}'

USER photoview
ENTRYPOINT ["/app/photoview"]
73 changes: 49 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<img src="./screenshots/photoview-logo.svg" height="92px" alt="photoview logo" />

[![License](https://img.shields.io/github/license/viktorstrate/photoview)](./LICENSE.md)
[![License](https://img.shields.io/github/license/viktorstrate/photoview)](./LICENSE.txt)
[![GitHub contributors](https://img.shields.io/github/contributors/viktorstrate/photoview)](https://github.com/viktorstrate/photoview/graphs/contributors)
[![Docker Pulls](https://img.shields.io/docker/pulls/viktorstrate/photoview)](https://hub.docker.com/r/viktorstrate/photoview)
[![Docker builds](https://github.com/photoview/photoview/actions/workflows/build.yml/badge.svg?branch=master)](https://github.com/photoview/photoview/actions/workflows/build.yml)
Expand All @@ -10,9 +10,9 @@

Photoview is a simple and user-friendly photo gallery that's made for photographers and aims to provide an easy and fast way to navigate directories, with thousands of high-resolution photos.

You configure Photoview to look for photos and videos within a directory on your file system. The scanner automatically picks up your media and start to generate thumbnail images to make browsing super fast.
You configure Photoview to look for photos and videos within a directory on your file system. The scanner automatically picks up your media and starts to generate thumbnail images to make browsing super fast.

When your media has been scanned they show up on the website, organised in the same way as on the filesystem.
When your media has been scanned, they show up on the website, organised in the same way as on the filesystem.

> If you have questions regarding setup or development,
feel free to join the Discord server https://discord.gg/jQ392948u9
Expand All @@ -30,19 +30,20 @@ Password: **demo**
- [Main features](#main-features)
- [Supported Platforms](#supported-platforms)
- [Why yet another self-hosted photo gallery](#why-yet-another-self-hosted-photo-gallery)
- [Getting started - Setup with Docker](#getting-started---setup-with-docker)
- [Set up development environment](#setup-development-environment)
- [Getting started — Setup with Docker](#getting-started--setup-with-docker)
- [Advanced setup](#advanced-setup)
- [Set up development environment](#set-up-development-environment)
- [Sponsors](#sponsors)

## Main features

- **Closely tied to the file system**. The website presents the images found on the local filesystem of the server, directories are mapped to albums.
- **Closely tied to the file system**. The website presents the images found on the local filesystem of the server; directories are mapped to albums.
- **User management**. Each user is created along with a path on the local filesystem, photos within that path can be accessed by that user.
- **Sharing**. Albums, as well as individual media, can easily be shared with a public link, the link can optionally be password protected.
- **Made for photography**. Photoview is built with photographers in mind, and thus supports **RAW** file formats, and **EXIF** parsing.
- **Video support**. Many common video formats are supported. Videos will automatically be optimized for web.
- **Face recognition**. Faces will automatically be detected in photos, and photos of the same person will be grouped together.
- **Performant**. Thumbnails are automatically generated and photos first load when they are visible on the screen. In full screen, thumbnails are displayed until the high resolution image has been fully loaded.
- **Performant**. Thumbnails are automatically generated and photos first load when they are visible on the screen. In full screen, thumbnails are displayed until the high-resolution image has been fully loaded.
- **Secure**. All media resources are protected with a cookie-token, all passwords are properly hashed, and the API uses a strict [CORS policy](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS).

## Supported platforms
Expand All @@ -64,53 +65,77 @@ There exists a lot of open-source self-hosted photo galleries already. Here are

So why another one?
I love taking photos, and I store all of them on my local fileserver.
This is great because I can organize my photos directly on the filesystem so it's easy to move them or take backups. I want to be able to control where and how the photos are stored.
This is great because I can organize my photos directly on the filesystem, so it's easy to move them or take backups. I want to be able to control where and how the photos are stored.

The problem is however that RAW images are extremely tedious to navigate from a fileserver, even over the local network.
The problem is, however, that RAW images are extremely tedious to navigate from a fileserver, even over the local network.

My server holds a lot of old family pictures, that I would like my family to have access to as well.
My server holds a lot of old family pictures that I would like my family to have access to as well.
And some of the pictures I would like to easily be able to share with other people without the hassle of them having to make an account first.

Thus I need a solution that can do the following:
Thus, I need a solution that can do the following:

- A scan based approach that automatically organises my photos
- A scan-based approach that automatically organises my photos
- Support RAW and EXIF parsing
- Have support for multiple users and ways to share albums and photos also publicly
- Be simple and fast to use
- Be straightforward and fast to use

All of the photo galleries can do a lot of what I need, but no single one can do it all.
All the photo galleries can do a lot of what I need, but no single one can do it all.

## Getting started - Setup with Docker
## Getting started Setup with Docker

> This section describes how to get Photoview up and running on your server with Docker.
> Make sure you have Docker and docker-compose installed and running on your server
> Make sure you have Docker and docker-compose installed and running on your server.
> `make` should be installed as well if you'd like to use provided `Makefile`, which is optional (see step 4 for more details).
> `7zz` should be installed in case, you'd like to use it in scope of the backup scenario instead of the default .tar.xz format. Read the comment in the `Makefile`, located on top of the `backup` section for more details.

1. Make a new `docker-compose.yml` file on your computer, and copy the content of [docker-compose.example.yml](/docker-compose.example.yml) to the new file.
2. Edit `docker-compose.yml`, find the comments starting with `Change This:`, and change the values, to properly match your setup. If you are just testing locally, you don't have to change anything.
3. Start the server by running the following command
1. Download the content of the `docker-compose example` folder to the folder on your server, where you expect to host the Photoview internal data (database and cache files).
2. Rename downloaded files and remove the `example` from their names (so, you need to have `.env`, `docker-compose.yml`, and `Makefile` files).
3. Open these files in a text editor and read them. Modify where needed according to the documentation comments to properly match your setup. There are comments of 2 types: those, starting with `##`, are explanations and examples, which should not be uncommented; those, starting with `#`, are optional or alternative configuration parts, which might be uncommented in certain circumstances, described in corresponding explanations. It is better to go through the files in the next order: `.env`, `docker-compose.yml`, and `Makefile`.
4. In case, you don't have `make` installed in your system or don't want to use it for the Photoview management activities, you could use the same commands from the `Makefile` and run them in your shell directly, or create your own scripts. Make sure to apply or replace the variables from your `.env` first in this case. `Makefile` is provided just for your convenience and simplicity, but is optional.
5. Start the server by running the following command (or corresponding sequence of commands from the `Makefile`):

```bash
$ docker-compose up -d
make all
```

If the endpoint or the port hasn't been changed in the `docker-compose.yml` file, Photoview can now be accessed at http://localhost:8000

### Initial Setup

If everything is setup correctly, you should be presented with an initial setup wizard, when accessing the website the first time.
If everything is set up correctly, you should be presented with an initial setup wizard when accessing the website the first time.

![Initial setup](./screenshots/initial-setup.png)

Enter a new username and password.

For the photo path, enter the path in the docker container where your photos are located.
This can be set from the `docker-compose.yml` file under `api` -> `volumes`.
The default location is `/photos`
For the photo path, enter the path inside the docker container where your photos are located.
This can be set from the `docker-compose.yml` file under `photoview` -> `volumes`.
The default location is `/photos`.

A new admin user will be created, with access to the photos located at the path provided under the initial setup.

The photos will have to be scanned before they show up, you can start a scan manually, by navigating to `Settings` and clicking on `Scan All`

## Advanced setup

We suggest securing the Photoview instance before exposing it outside your local network: even while it provides read-only access to your media gallery and has basic user authentication functionality, it is not enough to protect your private media from malicious actors on the Internet.

Possible ways of securing a self-hosted service might be (but not limited to):

1. Configure a **Firewall** on your local network's gateway and allow only the intended type of incoming traffic to pass.
2. Use **VPN** to provide external access to local services.
3. Setting up a **Reverse proxy** in front of the service and forwarding all the traffic through it, exposing HTTPS port with strong certificate and cipher suites to the Internet. This could be one of the next products or something else that you prefer:
- [Traefic Proxy](https://doc.traefik.io/traefik/)
- [NGinx Proxy Manager](https://nginxproxymanager.com/guide/)
- [Cloudflare Gateway](https://www.cloudflare.com/zero-trust/products/gateway/)
4. Configure an external **Multi-Factor Authentication** service to manage authentication for your service (part of Cloudflare services, but you can choose anything else).
5. Configure **Web Application Firewall** to protect from common web exploits like SQL injection, cross-site scripting, and cross-site forgery requests (part of Cloudflare services, but you can choose anything else).
6. Use **Content Delivery Network** as an additional level of DDoS prevention: it can securely cache your media and let it be accessible from a wide list of servers on the Internet (part of Cloudflare services, but you can choose anything else).
7. Configure a **Rate Limit** of allowed number of requests from a user during specified time range to protect against DDoS attacks.
8. Set up an **Intrusion Detection/Prevention System** to monitor network traffic for suspicious activity and issue alerts when such activity is discovered.

Setting up and configuring of all these protections depends on and requires a lot of info about your local network and self-hosted services. Based on this info, the configuration flow and resulting services architecture might differ a lot between cases. That is why in the scope of this project, we can only provide you with this high-level list of possible ways of webservice protection. You'll need to investigate them, find the best combination and configuration for your case, and take responsibility to configure everything in the correct and consistent way. We cannot provide you support for such highly secured setups, as a lot of things might work differently because of security limitations.

## Set up development environment

### Local setup
Expand Down
Loading
Loading