Docker build correct logs?

[Ouwen]

I am attempting to build the docker image and found the README.md to be outdated so I did a little guess work.

The docker image was built in the buildmimic/docker folder with the following command
docker build -t postgres/mimic .

Upon successful build of the image I ran

docker run --name mimic_db -p 5433:5432 
-e BUILD_MIMIC=1 
-e POSTGRES_PASSWORD=test 
-e MIMIC_PASSWORD=test 
-v /Users/oh11/Development/mimic/csv:/mimic_data 
-v /Users/oh11/Development/mimic/pg_data:/var/lib/PostgreSQL/data 
-d postgres/mimic

When I view the docker logs via docker logs -f <container_id> I get the following. Is this correct and should I wait for import? Or is something wrong?

The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
creating template1 database in /var/lib/postgresql/data/base/1 ... ok
initializing pg_authid ... ok
initializing dependencies ... ok
creating system views ... ok
loading system objects' descriptions ... ok
creating collations ... ok
creating conversions ... ok
creating dictionaries ... ok
setting privileges on built-in objects ... ok
creating information schema ... ok
loading PL/pgSQL server-side language ... ok
vacuuming database template1 ... ok
copying template1 to template0 ... ok
copying template1 to postgres ... ok

WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
syncing data to disk ... ok

Success. You can now start the database server using:

    pg_ctl -D /var/lib/postgresql/data -l logfile start

waiting for server to start....LOG:  database system was shut down at 2016-09-26 14:48:40 UTC
LOG:  MultiXact member wraparound protections are now enabled
LOG:  database system is ready to accept connections
LOG:  autovacuum launcher started
 done
server started
ALTER ROLE


/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/buildmimic

/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/setup.sh
CREATING MIMIC ... 
running create mimic user
LOG:  received fast shutdown request
LOG:  aborting any active transactions
LOG:  autovacuum launcher shutting down
waiting for server to shut down...LOG:  shutting down
.LOG:  database system is shut down
 done
server stopped
waiting for server to start....LOG:  database system was shut down at 2016-09-26 14:48:42 UTC
LOG:  MultiXact member wraparound protections are now enabled
LOG:  database system is ready to accept connections
LOG:  autovacuum launcher started
 done
server started
CREATE ROLE
CREATE DATABASE
You are now connected to database "mimic" as user "postgres".
CREATE SCHEMA
ALTER SCHEMA
Found all tables in /mimic_data - beginning import from .csv.gz files.
Found all tables in /mimic_data - beginning import from .csv.gz files.
Found all tables in /mimic_data - beginning import from .csv.gz files.
Found all tables in /mimic_data - beginning import from .csv.gz files.
Found all tables in /mimic_data - beginning import from .csv.gz files.
Found all tables in /mimic_data - beginning import from .csv.gz files.
Found all tables in /mimic_data - beginning import from .csv.gz files.
Found all tables in /mimic_data - beginning import from .csv.gz files.
Found all tables in /mimic_data - beginning import from .csv.gz files.
Found all tables in /mimic_data - beginning import from .csv.gz files.
Found all tables in /mimic_data - beginning import from .csv.gz files.
Found all tables in /mimic_data - beginning import from .csv.gz files.
Found all tables in /mimic_data - beginning import from .csv.gz files.
Found all tables in /mimic_data - beginning import from .csv.gz files.
Found all tables in /mimic_data - beginning import from .csv.gz files.
Found all tables in /mimic_data - beginning import from .csv.gz files.
Found all tables in /mimic_data - beginning import from .csv.gz files.
Found all tables in /mimic_data - beginning import from .csv.gz files.
Found all tables in /mimic_data - beginning import from .csv.gz files.
Found all tables in /mimic_data - beginning import from .csv.gz files.
Found all tables in /mimic_data - beginning import from .csv.gz files.
Found all tables in /mimic_data - beginning import from .csv.gz files.
Found all tables in /mimic_data - beginning import from .csv.gz files.
Found all tables in /mimic_data - beginning import from .csv.gz files.
Found all tables in /mimic_data - beginning import from .csv.gz files.
Found all tables in /mimic_data - beginning import from .csv.gz files.
/docker-entrypoint.sh: running postgres_create_tables.sql
CREATE SCHEMA
NOTICE:  schema "mimiciii" already exists, skipping
SET
DROP TABLE
NOTICE:  table "admissions" does not exist, skipping
CREATE TABLE
NOTICE:  table "callout" does not exist, skipping
DROP TABLE
CREATE TABLE
NOTICE:  table "caregivers" does not exist, skipping
DROP TABLE
CREATE TABLE
NOTICE:  table "chartevents" does not exist, skipping
DROP TABLE
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE FUNCTION
CREATE TRIGGER
NOTICE:  table "cptevents" does not exist, skipping
DROP TABLE
CREATE TABLE
NOTICE:  table "datetimeevents" does not exist, skipping
DROP TABLE
CREATE TABLE
DROP TABLE
NOTICE:  table "diagnoses_icd" does not exist, skipping
CREATE TABLE
NOTICE:  table "drgcodes" does not exist, skipping
DROP TABLE
CREATE TABLE
NOTICE:  table "d_cpt" does not exist, skipping
DROP TABLE
CREATE TABLE
NOTICE:  table "d_icd_diagnoses" does not exist, skipping
DROP TABLE
CREATE TABLE
NOTICE:  table "d_icd_procedures" does not exist, skipping
DROP TABLE
CREATE TABLE
NOTICE:  table "d_items" does not exist, skipping
DROP TABLE
CREATE TABLE
DROP TABLE
NOTICE:  table "d_labitems" does not exist, skipping
CREATE TABLE
NOTICE:  table "icustays" does not exist, skipping
DROP TABLE
CREATE TABLE
NOTICE:  table "inputevents_cv" does not exist, skipping
DROP TABLE
CREATE TABLE
NOTICE:  table "inputevents_mv" does not exist, skipping
DROP TABLE
CREATE TABLE
NOTICE:  table "labevents" does not exist, skipping
DROP TABLE
CREATE TABLE
NOTICE:  table "microbiologyevents" does not exist, skipping
DROP TABLE
CREATE TABLE
NOTICE:  table "noteevents" does not exist, skipping
DROP TABLE
CREATE TABLE
NOTICE:  table "outputevents" does not exist, skipping
DROP TABLE
CREATE TABLE
NOTICE:  table "patients" does not exist, skipping
DROP TABLE
CREATE TABLE
NOTICE:  table "prescriptions" does not exist, skipping
DROP TABLE
CREATE TABLE
DROP TABLE
NOTICE:  table "procedureevents_mv" does not exist, skipping
CREATE TABLE
NOTICE:  table "procedures_icd" does not exist, skipping
DROP TABLE
CREATE TABLE
NOTICE:  table "services" does not exist, skipping
DROP TABLE
CREATE TABLE
NOTICE:  table "transfers" does not exist, skipping
DROP TABLE
CREATE TABLE
/docker-entrypoint.sh: running postgres_load_data_gz.sql
SET
COPY 58976
COPY 34499
COPY 7567

[Ouwen]

When I run
docker exec -it <container_id> bash

I can connect to postgres and get the following output

root@648415631c8b:/# psql -U postgres         
psql (9.5.3)
Type "help" for help.

postgres=# \list
                                 List of databases
   Name    |  Owner   | Encoding |  Collate   |   Ctype    |   Access privileges   
-----------+----------+----------+------------+------------+-----------------------
 mimic     | mimic    | UTF8     | en_US.utf8 | en_US.utf8 | 
 postgres  | postgres | UTF8     | en_US.utf8 | en_US.utf8 | 
 template0 | postgres | UTF8     | en_US.utf8 | en_US.utf8 | =c/postgres          +
           |          |          |            |            | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.utf8 | en_US.utf8 | =c/postgres          +
           |          |          |            |            | postgres=CTc/postgres
(4 rows)

postgres=# \q
root@648415631c8b:/# psql -U postgres -d mimic
psql (9.5.3)
Type "help" for help.

mimic=# \dt
No relations found.

[tompollard]

Thanks @Ouwen. The Docker image and documentation is community maintained, so we would welcome a pull request with updates and improvements.

What in particular concerned you about the docker logs? No major issues jump out to me, though I have only taken a quick glance. The NOTICE: table "x" does not exist, skipping is nothing to worry about. We always drop a table before creating it, so you are just being informed that the table did not already exist when the drop command was run.

\dt would not be expected to return any relations in the example above because the schema is not specified. After running psql -U postgres -d mimic, try listing the schemas with \dn, then set the search path with set search_path to <schema>; before running \dt.

[Ouwen]

From looking at pg activity. It seems like the query is still running...

postgres=# select datname, pid, waiting, state, query from pg_stat_activity;                                                                                                                                                
 datname  | pid | waiting | state  |                               query                               
----------+-----+---------+--------+-------------------------------------------------------------------
 mimic    |  96 | f       | active | COPY  CHARTEVENTS FROM STDIN delimiter ',' csv header NULL ''
 postgres | 253 | f       | active | select datname, pid, waiting, state, query from pg_stat_activity;
(2 rows)

[Ouwen]

@tompollard Thanks Tom, I'll make a PR for the docker README while waiting for this import.

[tompollard]

Thanks, that would be great :)