Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added simple demo config #20

Merged
merged 6 commits into from Dec 24, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Expand Up @@ -8,6 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### 🚀 Added
- Add the uniqueness option for email, ip and random_num [#16](https://github.com/datanymizer/datanymizer/pull/16)
([@evgeniy-r](https://github.com/evgeniy-r))
- Add demo configuration and docker-compose example [#20](https://github.com/datanymizer/datanymizer/pull/20) ([@akirill0v](https://github.com/akirill0v))

### ⚙️ Changed
- Automate release process [#17](https://github.com/datanymizer/datanymizer/pull/17) ([@mgrachev](https://github.com/mgrachev))
Expand Down
2 changes: 1 addition & 1 deletion README.md
Expand Up @@ -107,7 +107,7 @@ And then start to make dump from your database instance:
``` shell
pg_datanymizer -f /tmp/dump.sql -c ./config.yml postgres://postgres:postgres@localhost/test_database
```

Uniqueness
It creates new dump file `/tmp/dump.sql` with native SQL dump for Postgresql database.
You can import fake data from this dump into new Postgresql database with command:

Expand Down
37 changes: 37 additions & 0 deletions demo/Dockerfile.pg_datanynizer
@@ -0,0 +1,37 @@
FROM rust:1.48 as cargo-chef
WORKDIR /usr/src

RUN apt-get update && \
apt-get dist-upgrade -y && \
apt-get install -y musl-tools && \
rustup target add x86_64-unknown-linux-musl
# We only pay the installation cost once,
# it will be cached from the second build onwards
RUN cargo install cargo-chef

FROM cargo-chef as planner
WORKDIR /usr/src
COPY . .
RUN cargo chef prepare --recipe-path recipe.json

FROM cargo-chef as cacher
WORKDIR /usr/src

RUN cargo install cargo-chef
COPY --from=planner /usr/src/recipe.json recipe.json
RUN cargo chef cook --target x86_64-unknown-linux-musl --release --recipe-path recipe.json

FROM cargo-chef as builder
WORKDIR /usr/src

COPY . .
# Copy over the cached dependencies
COPY --from=cacher /usr/src/target target
COPY --from=cacher $CARGO_HOME $CARGO_HOME
RUN cargo build --target x86_64-unknown-linux-musl --release

FROM postgres:13-alpine
WORKDIR /
COPY --from=builder /usr/src/target/x86_64-unknown-linux-musl/release/pg_datanymizer .
USER 1000
ENTRYPOINT ["./pg_datanymizer"]
39 changes: 39 additions & 0 deletions demo/Makefile
@@ -0,0 +1,39 @@
example_database_url="https://sp.postgresqltutorial.com/wp-content/uploads/2019/05/dvdrental.zip"
tmpfile="/tmp/dump.zip"
pause=5
.SILENT:

build:
docker-compose build

up:
echo -e "\n==> Start containers"
docker-compose up -d
echo -e "\n==> Pause ${pause}s after starting Postgresql"
sleep ${pause}

download: clean_file
echo -e "\n==> Download DVDRENTAL database"
curl $(example_database_url) -o "$(tmpfile)" && unzip $(tmpfile) -d /tmp

clean_file:
echo -e "\n==> Clean all temporary files"
rm -rf $(tmpfile) && rm -rf /tmp/dvdrental.tar

clean_db:
echo -e "\n==> Clean database if exists"
docker-compose exec pg_database psql -Upostgres -c "drop database if exists dvdrental;"

create_db: up clean_db
echo -e "\n==> Create empty database"
docker-compose exec pg_database psql -Upostgres -c "create database dvdrental;"

restore_db:
echo -e "\n==> Restore database from dump file"
docker-compose run --rm -v /tmp/dvdrental.tar:/tmp/dvdrental.tar -e PGPASSWORD=postgres pg_database pg_restore -h pg_database -Upostgres -d dvdrental /tmp/dvdrental.tar

bootstrap: download create_db restore_db clean_file

dump:
echo -e "\n==> Create dump"
docker-compose run --rm pg_datanymizer postgres://postgres:postgres@pg_database/dvdrental -c /dvdrental.yml > /tmp/fake_dump.sql
26 changes: 26 additions & 0 deletions demo/README.md
@@ -0,0 +1,26 @@
# [Data]nymizer demo

## Get started

1. Install `docker` and `docker-compose` to your development environment.
2. Go to `demo` directory:

``` shell
cd demo
```
and run:

``` shell
make bootstrap
```

It starts docker container with Postgresql, download demo database [PostgreSQL Sample Database](https://www.postgresqltutorial.com/postgresql-sample-database/), unpack it and restore to new `dvdrental` database.

3. For dumping this base, run:

``` shell
make dump
```
It makes new `/tmp/fake_dump.sql` file with fake rows in tables from `dvdrental.yml` configuration.

You can change configuration file and try again.
24 changes: 24 additions & 0 deletions demo/docker-compose.yml
@@ -0,0 +1,24 @@
version: '3.2'
services:

pg_datanymizer:
build:
context: ..
dockerfile: demo/Dockerfile.pg_datanynizer
image: pg_datanymizer
volumes:
- ./dvdrental.yml:/dvdrental.yml

pg_database:
image: postgres:13-alpine
environment:
POSTGRES_PASSWORD: "postgres"
depends_on:
- pg_datanymizer
ports:
- "5432:5432"
volumes:
- demo-volume:/demo/:rw

volumes:
demo-volume:
14 changes: 14 additions & 0 deletions demo/dvdrental.yml
@@ -0,0 +1,14 @@
filter:
only:
- public.actor
tables:
- name: actor
rules:
first_name:
first_name: ~
last_name:
last_name: ~
last_update:
datetime:
from: 1990-01-01T00:00:00+00:00
to: 2010-12-31T00:00:00+00:00