Issue with pgloader migration from MySQL MariaDB to Supabase

[mashwishi]

I am encountering difficulties while attempting to migrate my MySQL MariaDB database to Supabase (PostgreSQL). My local database is set up using XAMPP, and Supabase is the target database hosted online. I've followed the migration instructions provided by Supabase in their documentation: Supabase MySQL Migration Guide.

I have encountered issues with the example provided in the documentation, specifically with lines 4 and 5:

load database
  from mysql://user:password@host/source_db
  into postgres://postgres:password@db.xxxx.supabase.co:6543/postgres
alter schema 'public' owner to 'postgres';
set wal_buffers = '64MB', max_wal_senders = 0, statement_timeout = 0, work_mem to '2GB';

In particular, I am using pgloader for the migration and have configured it as follows:

load database
  from mysql://user:password@host/source_db
  into postgres://postgres:password@db.xxxx.supabase.co:6543/postgres;

These lines seem to be causing an error. The error message I am receiving is:

.....0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0) :START 0 :COUNT 0) :LOCK #<SB-THREAD:MUTEX "Anonymous lock" (free)> :NOTIFY-PUSH NIL :NOTIFY-POP NIL) :DISABLE-TRIGGERS NIL :ON-ERROR-STOP T :COLUMNS NIL)
7: ((FLET "BODY-FN0" :IN LPARALLEL.KERNEL::MAKE-CHANNELED-TASK))
8: ((LAMBDA NIL :IN LPARALLEL.KERNEL::MAKE-CHANNELED-TASK))
9: (LPARALLEL.KERNEL::EXEC-TASK/WORKER #<unavailable argument> #S(LPARALLEL.KERNEL::WORKER :HANDSHAKE/FROM-WORKER #S(LPARALLEL.CONS-QUEUE:CONS-QUEUE :IMPL #S(LPARALLEL.RAW-QUEUE:RAW-QUEUE :HEAD NIL :TAIL NIL) :LOCK #<SB-THREAD:MUTEX #1="Anonymous lock" (free)> :CVAR NIL) :HANDSHAKE/TO-WORKER #S(LPARALLEL.CONS-QUEUE:CONS-QUEUE :IMPL #S(LPARALLEL.RAW-QUEUE:RAW-QUEUE :HEAD NIL :TAIL NIL) :LOCK #<SB-THREAD:MUTEX #1# (free)> :CVAR #<SB-THREAD:WAITQUEUE Anonymous condition variable {700EB74F73}>) :EXIT-NOTIFICATION #S(LPARALLEL.CONS-QUEUE:CONS-QUEUE :IMPL #S(LPARALLEL.RAW-QUEUE:RAW-QUEUE :HEAD NIL :TAIL NIL) :LOCK #<SB-THREAD:MUTEX #1# (free)> :CVAR NIL) :THREAD #<SB-THREAD:THREAD "lparallel" RUNNING {700EB00103}> :RUNNING-CATEGORY :DEFAULT :INDEX 1 :TASKS #S(LPARALLEL.SPIN-QUEUE:SPIN-QUEUE :HEAD #2=(LPARALLEL.SPIN-QUEUE::DUMMY) :TAIL #2#)))
10: (LPARALLEL.KERNEL::WORKER-LOOP #<LPARALLEL.KERNEL:KERNEL :NAME "lparallel" :WORKER-COUNT 4 :USE-CALLER NIL :ALIVE T :SPIN-COUNT 2000 {700EAE0023}> #S(LPARALLEL.KERNEL::WORKER :HANDSHAKE/FROM-WORKER #S(LPARALLEL.CONS-QUEUE:CONS-QUEUE :IMPL #S(LPARALLEL.RAW-QUEUE:RAW-QUEUE :HEAD NIL :TAIL NIL) :LOCK #<SB-THREAD:MUTEX #1="Anonymous lock" (free)> :CVAR NIL) :HANDSHAKE/TO-WORKER #S(LPARALLEL.CONS-QUEUE:CONS-QUEUE :IMPL #S(LPARALLEL.RAW-QUEUE:RAW-QUEUE :HEAD NIL :TAIL NIL) :LOCK #<SB-THREAD:MUTEX #1# (free)> :CVAR #<SB-THREAD:WAITQUEUE Anonymous condition variable {700EB74F73}>) :EXIT-NOTIFICATION #S(LPARALLEL.CONS-QUEUE:CONS-QUEUE :IMPL #S(LPARALLEL.RAW-QUEUE:RAW-QUEUE :HEAD NIL :TAIL NIL) :LOCK #<SB-THREAD:MUTEX #1# (free)> :CVAR NIL) :THREAD #<SB-THREAD:THREAD "lparallel" RUNNING {700EB00103}> :RUNNING-CATEGORY :DEFAULT :INDEX 1 :TASKS #S(LPARALLEL.SPIN-QUEUE:SPIN-QUEUE :HEAD #2=(LPARALLEL.SPIN-QUEUE::DUMMY) :TAIL #2#)))
11: (LPARALLEL.KERNEL::%CALL-WITH-TASK-HANDLER #<unavailable argument>)
12: ((LAMBDA NIL :IN LPARALLEL.KERNEL::CALL-WITH-WORKER-CONTEXT))
13: (LPARALLEL.KERNEL::CALL-WITH-WORKER-CONTEXT #<FUNCTION (LAMBDA NIL :IN LPARALLEL.KERNEL::ENTER-WORKER-LOOP) {700EAE032B}> #<FUNCTION FUNCALL> #<LPARALLEL.KERNEL:KERNEL :NAME "lparallel" :WORKER-COUNT 4 :USE-CALLER NIL :ALIVE T :SPIN-COUNT 2000 {700EAE0023}> #S(LPARALLEL.KERNEL::WORKER :HANDSHAKE/FROM-WORKER #S(LPARALLEL.CONS-QUEUE:CONS-QUEUE :IMPL #S(LPARALLEL.RAW-QUEUE:RAW-QUEUE :HEAD NIL :TAIL NIL) :LOCK #<SB-THREAD:MUTEX #1="Anonymous lock" (free)> :CVAR NIL) :HANDSHAKE/TO-WORKER #S(LPARALLEL.CONS-QUEUE:CONS-QUEUE :IMPL #S(LPARALLEL.RAW-QUEUE:RAW-QUEUE :HEAD NIL :TAIL NIL) :LOCK #<SB-THREAD:MUTEX #1# (free)> :CVAR #<SB-THREAD:WAITQUEUE Anonymous condition variable {700EB74F73}>) :EXIT-NOTIFICATION #S(LPARALLEL.CONS-QUEUE:CONS-QUEUE :IMPL #S(LPARALLEL.RAW-QUEUE:RAW-QUEUE :HEAD NIL :TAIL NIL) :LOCK #<SB-THREAD:MUTEX #1# (free)> :CVAR NIL) :THREAD #<SB-THREAD:THREAD "lparallel" RUNNING {700EB00103}> :RUNNING-CATEGORY :DEFAULT :INDEX 1 :TASKS #S(LPARALLEL.SPIN-QUEUE:SPIN-QUEUE :HEAD #2=(LPARALLEL.SPIN-QUEUE::DUMMY) :TAIL #2#)))
14: ((LAMBDA NIL :IN LPARALLEL.KERNEL::MAKE-WORKER-THREAD))
15: ((LAMBDA NIL :IN BORDEAUX-THREADS::BINDING-DEFAULT-SPECIALS))
16: ((FLET SB-UNIX::BODY :IN SB-THREAD::RUN))
17: ((FLET "WITHOUT-INTERRUPTS-BODY-151" :IN SB-THREAD::RUN))
18: ((FLET SB-UNIX::BODY :IN SB-THREAD::RUN))
19: ((FLET "WITHOUT-INTERRUPTS-BODY-144" :IN SB-THREAD::RUN))
20: (SB-THREAD::RUN)

What I am doing here?

Failed to connect to pgsql at "db.xxxx.supabase.co" (port 5432) as user "postgres": Database error: Socket error in "connect": EINTR (Interrupted system call)

Version

• pgloader version "3.6.9"
• compiled with SBCL 2.3.4

Additional Information:

• I have attempted to proceed with the migration using only lines 1 to 3. While the database structure and data are being migrated, I am uncertain if the migration is 100% complete without data loss or if there is an issue with the structure.
• The migration was initially successful, but subsequent attempts (2nd to 5th) resulted in the same error. Initially, I considered the possibility of a rate limit since I am on a free plan, but after waiting for 13 hours, the issue persists.

Request for Assistance:
I would appreciate any guidance or assistance in resolving this issue. If there are alternative configurations or steps that need to be taken, please provide the necessary information. Additionally, if there are ways to ensure a complete and accurate migration without encountering the socket error, I would be grateful for the guidance.

Thank you for your time and assistance in resolving this matter.

[ccfontes]

Exact same message here, but sqlite3 -> postgres.

This issue happens on the mac, while in linux goes beyond this point. Later crashes in Linux because I don't have enough RAM there, being the reason I switch to the mac.

Versions:

• Linux ("works"): pgloader version "3.6.7~devel", compiled with SBCL 2.1.11.debian
• Mac (crashes): pgloader version "3.6.9", compiled with SBCL 2.3.8

[mashwishi]

@ccfontes i found the issue, i got ip blocked. all you need to do is fix your supabase ip block list remove your ip