MySQL -> Postgres Docker The assertion (NOT (NULL PGLOADER.MONITOR:*MONITORING-QUEUE*)) failed

[leematos]

Attempting to convert a MySQL DB in a docker container to a PostgresSQL DB in a docker container with the pgloader docker container:

docker run --name pgloader dimitri/pgloader:latest pgloader --verbose --debug mysql://gitlab@192.168.1.164:13306/gitlabhq_production postgresql://gitlab:password@192.168.1.164:15432/gitlabhq_production

WARNING:
Couldn't re-execute SBCL with proper personality flags (/proc isn't mounted? setuid?)
Trying to continue anyway.
mkdir -p #P"/tmp/pgloader/"
sb-impl::*default-external-format* :ANSI_X3.4-1968
tmpdir: #P"/tmp/pgloader/"
2017-06-17T20:10:04.024000Z NOTICE Starting pgloader, log system is ready.
2017-06-17T20:10:04.055000Z INFO Starting monitor
2017-06-17T20:10:04.064000Z INFO SOURCE: "mysql://gitlab@192.168.1.164:13306/gitlabhq_production"
2017-06-17T20:10:04.065000Z INFO TARGET: "postgresql://gitlab:password@192.168.1.164:15432/gitlabhq_production"
2017-06-17T20:10:04.066000Z DEBUG LOAD DATA FROM #<PGLOADER.MYSQL:MYSQL-CONNECTION mysql://gitlab@192.168.1.164:13306/gitlabhq_production {1009342E73}>

debugger invoked on a SIMPLE-ERROR in thread
#<THREAD "main thread" RUNNING {1009056423}>:
  The assertion (NOT (NULL PGLOADER.MONITOR:*MONITORING-QUEUE*)) failed with
  (NULL PGLOADER.MONITOR:*MONITORING-QUEUE*) = T.

Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [CONTINUE] Retry assertion.
  1: [ABORT   ] Exit application

(SB-KERNEL:ASSERT-ERROR (NOT (NULL PGLOADER.MONITOR:*MONITORING-QUEUE*)) (((NULL PGLOADER.MONITOR:*MONITORING-QUEUE*) T)) NIL NIL)

I've not been able to trigger another error by adjusting the MySQL connection or the Postgres side connection so it doesn't seem to be the problem. If I'm understanding correctly it thinks something is queued, when there shouldn't be?

[dimitri]

The monitoring queue is a separate thread responsible for monitoring the activity and taking care of the output of the logs, both to the console and on file. It is an internal facility, and it is always started when you execute a pgloader command. Which version of pgloader are you using?

The only idea I have right now is that maybe in your environment there's something preventing pgloader to create threads?

[hregis]

same error ! i use pgloader in Travis CI

debugger invoked on a SIMPLE-ERROR in thread
#<THREAD "main thread" RUNNING {1009056263}>:
  The assertion (NOT (NULL PGLOADER.MONITOR:*MONITORING-QUEUE*)) failed with
  (NULL PGLOADER.MONITOR:*MONITORING-QUEUE*) = T.

Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [CONTINUE] Retry assertion.
  1: [ABORT   ] Exit application

(SB-KERNEL:ASSERT-ERROR (NOT (NULL PGLOADER.MONITOR:*MONITORING-QUEUE*)) (((NULL PGLOADER.MONITOR:*MONITORING-QUEUE*) T)) NIL NIL)
0] ; 
; compilation unit aborted
;   caught 1 fatal ERROR condition

[dimitri]

Can you run pgloader --version please?

[leematos]

Yep!

docker run --rm --name pgloader dimitri/pgloader:latest pgloader --version
WARNING:
Couldn't re-execute SBCL with proper personality flags (/proc isn't mounted? setuid?)
Trying to continue anyway.
pgloader version "3.3.2~devel"
compiled with SBCL 1.2.4.debian

[hregis]

same version with docker : pgloader version "3.3.2 ~ devel"
but in Travis CI, if i use Ubuntu Trusty and the pgloader package (2.3.3~dev3-1.1), it's ok !
@leematos can you use Ubuntu Trusty ?

[dimitri]

The 2.x versions of pgloader are not maintained anymore, avoid them.

@leematos I am not sure how to debug your problem here unfortunately. Can you try to build a docker image locally? As you can see at https://travis-ci.org/dimitri/pgloader/jobs/244028040 we have no problem running pgloader within Travis for the tests...

[leematos]

Huh. I wonder if it something to do with me running this on a Mac. I can try and build my own image. I suspect I'll run into the same problem (which would be telling, it's something on my Mac then.)

[wuliwong]

Same error on OS-X 10.12.5

pgloader version "3.3.b5a593a"
compiled with SBCL 1.3.18
postgres (PostgreSQL) 9.6.3

When running
pgloader mysql://root@localhost/najarian2 pgsql:///najarian2

I am not using it in Docker.

[dimitri]

Just tried again on OSX with SBCL 1.3.18 and current pgloader sources and coudn't reproduce...

[dimitri]

Having a test case at hand where I saw the behavior, it seems to be related to abnormal termination of the pgloader process. Can you try again with the current master's branch code, which includes referenced patch to try and fix the situation?

[dimitri]

I think I nailed it, please consider opening this issue again if you still have problem.