Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

supabase db diff fails with: psycopg2.OperationalError: could not translate host name "localhost" to address: Try again #1574

Closed
vixalien opened this issue Oct 12, 2023 · 23 comments

Comments

@vixalien
Copy link

Describe the bug
I'm trying to do supabase db diff but it fails with psycopg2.OperationalError: could not translate host name "localhost" to address: Try again. I even tried changing the DNS resolver from native to https, but it doesn't work

To Reproduce
Steps to reproduce the behavior:

  1. Try to do a supabase db diff

Expected behavior
Normal diff to get written

Screenshots

Logs:

$ supabase db diff --dns-resolver https
Connecting to local database...
Creating shadow database...
Setting up initial schema...
Applying migration 20220822165112_xyz.sql...
Applying migration 20220822165112_xyo.sql...
Diffing schemas: extensions,public
error diffing schema: error running container: exit 1:                                                                                                 
WARNING: Running pip as root will break packages and permissions. You should install packages reliably by using venv: https://pip.pypa.io/warnings/venv
WARNING: You are using pip version 21.1.1; however, version 23.2.1 is available.                                                                       
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.                                                    
Traceback (most recent call last):                                                                                                                     
  File "/usr/local/lib/python3.9/site-packages/schemainspect/misc.py", line 9, in connection_from_s_or_c                                               
    s_or_c.engine                                                                                                                                      
AttributeError: 'Session' object has no attribute 'engine'                                                                                             
                                                                                                                                                       
During handling of the above exception, another exception occurred:                                                                                    
                                                                                                                                                       
Traceback (most recent call last):                                                                                                                     
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 3212, in _wrap_pool_connect                                            
    return fn()                                                                                                                                        
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 301, in connect                                                          
    return _ConnectionFairy._checkout(self)                                                                                                            
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 761, in _checkout                                                        
    fairy = _ConnectionRecord.checkout(pool)                                                                                                           
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 419, in checkout                                                         
    rec = pool._do_get()                                                                                                                               
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/impl.py", line 145, in _do_get                                                          
    self._dec_overflow()                                                                                                                               
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__                                                   
    compat.raise_(                                                                                                                                     
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 211, in raise_                                                         
    raise exception                                                                                                                                    
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/impl.py", line 142, in _do_get                                                          
    return self._create_connection()                                                                                                                   
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 247, in _create_connection                                               
    return _ConnectionRecord(self)                                                                                                                     
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 362, in __init__                                                         
    self.__connect()                                                                                                                                   
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 605, in __connect                                                        
    pool.logger.debug("Error on connect(): %s", e)                                                                                                     
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__                                                   
    compat.raise_(                                                                                                                                     
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 211, in raise_                                                         
    raise exception                                                                                                                                    
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 599, in __connect                                                        
    connection = pool._invoke_creator(self)                                                                                                            
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/create.py", line 578, in connect                                                      
    return dialect.connect(*cargs, **cparams)                                                                                                          
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 584, in connect                                                     
    return self.dbapi.connect(*cargs, **cparams)                                                                                                       
  File "/usr/local/lib/python3.9/site-packages/psycopg2/__init__.py", line 127, in connect                                                             
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)                                                                             
psycopg2.OperationalError: could not translate host name "localhost" to address: Try again                                                             
                                                                                                                                                       
                                                                                                                                                       
The above exception was the direct cause of the following exception:                                                                                   
                                                                                                                                                       
Traceback (most recent call last):                                                                                                                     
  File "/usr/local/bin/migra", line 8, in <module>                                                                                                     
    sys.exit(do_command())                                                                                                                             
  File "/usr/local/lib/python3.9/site-packages/migra/command.py", line 121, in do_command                                                              
    status = run(args)                                                                                                                                 
  File "/usr/local/lib/python3.9/site-packages/migra/command.py", line 86, in run                                                                      
    m = Migration(                                                                                                                                     
  File "/usr/local/lib/python3.9/site-packages/migra/migra.py", line 31, in __init__                                                                   
    self.changes.i_from = get_inspector(                                                                                                               
  File "/usr/local/lib/python3.9/site-packages/schemainspect/get.py", line 14, in get_inspector                                                        
    c = connection_from_s_or_c(x)                                                                                                                      
  File "/usr/local/lib/python3.9/site-packages/schemainspect/misc.py", line 14, in connection_from_s_or_c                                              
    return s_or_c.connection()                                                                                                                         
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 1516, in connection                                                    
    return self._connection_for_bind(                                                                                                                  
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 1526, in _connection_for_bind                                          
    return self._transaction._connection_for_bind(                                                                                                     
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 747, in _connection_for_bind                                           
    conn = bind.connect()                                                                                                                              
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 3166, in connect                                                       
    return self._connection_cls(self, close_with_result=close_with_result)                                                                             
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 96, in __init__                                                        
    else engine.raw_connection()                                                                                                                       
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 3245, in raw_connection                                                
    return self._wrap_pool_connect(self.pool.connect, _connection)                                                                                     
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 3215, in _wrap_pool_connect                                            
    Connection._handle_dbapi_exception_noconnection(                                                                                                   
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 2068, in _handle_dbapi_exception_noconnection                          
    util.raise_(                                                                                                                                       
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 211, in raise_                                                         
    raise exception                                                                                                                                    
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 3212, in _wrap_pool_connect                                            
    return fn()                                                                                                                                        
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 301, in connect                                                          
    return _ConnectionFairy._checkout(self)                                                                                                            
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 761, in _checkout                                                        
    fairy = _ConnectionRecord.checkout(pool)                                                                                                           
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 419, in checkout                                                         
    rec = pool._do_get()                                                                                                                               
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/impl.py", line 145, in _do_get                                                          
    self._dec_overflow()                                                                                                                               
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__                                                   
    compat.raise_(                                                                                                                                     
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 211, in raise_                                                         
    raise exception                                                                                                                                    
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/impl.py", line 142, in _do_get                                                          
    return self._create_connection()                                                                                                                   
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 247, in _create_connection                                               
    return _ConnectionRecord(self)                                                                                                                     
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 362, in __init__                                                         
    self.__connect()                                                                                                                                   
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 605, in __connect                                                        
    pool.logger.debug("Error on connect(): %s", e)                                                                                                     
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__                                                   
    compat.raise_(                                                                                                                                     
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 211, in raise_                                                         
    raise exception                                                                                                                                    
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 599, in __connect                                                        
    connection = pool._invoke_creator(self)                                                                                                            
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/create.py", line 578, in connect                                                      
    return dialect.connect(*cargs, **cparams)                                                                                                          
  File "/usr/local/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 584, in connect                                                     
    return self.dbapi.connect(*cargs, **cparams)                                                                                                       
  File "/usr/local/lib/python3.9/site-packages/psycopg2/__init__.py", line 127, in connect                                                             
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)                                                                             
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) could not translate host name "localhost" to address: Try again                           
                                                                                                                                                       
(Background on this error at: http://sqlalche.me/e/14/e3q8)                                                                                            
                                                                                                                                                       
Try rerunning the command with --debug to troubleshoot the error.

Desktop (please complete the following information):

  • OS: arch rolling
  • Browser (if applicable: N/A
  • Version of CLI: v1.100.1
  • Version of supabase-js (if applicable): N/A
  • Version of Node.js (if applicable): v20.5.1

Additional context
Add any other context about the problem here.

@sweatybridge
Copy link
Contributor

This could be due to localhost resolving to ipv6 address on your machine. Could you give the beta version a try?

npx supabase@beta db diff

@ArnaudScheffler
Copy link

I think the real issue is that the meta container started during the diff use localhost as target db. In this case, localhost will resolve to the meta container itself and not the real host. Using host.docker.internal in supabase db diff --db-url postgresql://supabase_admin:PASSWORD@host.docker.internal:54328/postgres doesn't work because the first connexion to the db fail.

I think one solution would be to to replace localhost by host.docker.internal in env used by the meta container.

@sweatybridge
Copy link
Contributor

We use docker's host network mode to allow localhost in migra container to resolve to the host machine's ip. @ArnaudScheffler are you able to reproduce this issue using beta version?

@ArnaudScheffler
Copy link

Yes, I have the same issue with the beta version :

npx supabase@beta db diff --db-url postgresql://supabase_admin:PASSWORD@localhost:54328/postgres               1 ↵ arnaud@MBP-de-Arnaud-001
Connecting to remote database...
Creating shadow database...
Setting up initial schema...
Applying migration 20230717125841_init_pipeline.sql...
Applying migration 20230717125842_init_tera.sql...
Applying migration 20230717125843_init_colas.sql...
Applying migration 20230717125844_migration_to_deploy.sql...
Diffing schemas: extensions,private,public
error diffing schema: error running container: exit 1:                                                                                                 
WARNING: Running pip as root will break packages and permissions. You should install packages reliably by using venv: https://pip.pypa.io/warnings/venv
WARNING: You are using pip version 21.1.1; however, version 23.3.1 is available.                                                                       
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.                                                    
Traceback (most recent call last):                                                                                                                     
  File "/usr/local/lib/python3.9/site-packages/schemainspect/misc.py", line 9, in connection_from_s_or_c                                               
    s_or_c.engine                                                                                                                                      
AttributeError: 'Session' object has no attribute 'engine'

...

sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) could not connect to server: Connection refused                                           
    Is the server running on host "localhost" (::1) and accepting                                                                                      
    TCP/IP connections on port 54328?                                                                                                                  
could not connect to server: Connection refused                                                                                                        
    Is the server running on host "localhost" (127.0.0.1) and accepting                                                                                
    TCP/IP connections on port 54328?                                                                                                                  
                                                                                                                                                       
(Background on this error at: http://sqlalche.me/e/14/e3q8)                                                                                                

I still belive there is something wrong with localhost inside the meta container. Take a look at the following tests :

docker run -it --rm --network host public.ecr.aws/supabase/postgres:14.1.0.89 bash

root@docker-desktop:/# psql -h localhost -p 54328 -U postgres
psql: error: connection to server at "localhost" (::1), port 54328 failed: Connection refused
        Is the server running on that host and accepting TCP/IP connections?
connection to server at "localhost" (127.0.0.1), port 54328 failed: Connection refused
        Is the server running on that host and accepting TCP/IP connections?
root@docker-desktop:/# 
exit

Localhost doesn't resolve in container but it's ok outside :

psql -h localhost -p 54328 -U postgres
psql (14.8 (Homebrew), server 15.3)
WARNING: psql major version 14, server major version 15.
         Some psql features might not work.
Type "help" for help.

postgres=> 

@sweatybridge
Copy link
Contributor

Hmm my best guess is some differences between OS. As mentioned in their docs

The host networking driver only works on Linux hosts, and is not supported on Docker Desktop for Mac, Docker Desktop for Windows, or Docker EE for Windows Server.

I personally use an intel mac, and the command works

qiao@Qiaos-MBP cli % docker run -it --rm --network host public.ecr.aws/supabase/postgres:15.1.0.103 bash
root@docker-desktop:/# psql -h localhost -p 54322 -U postgres
Password for user postgres:
psql (15.3 (Ubuntu 15.3-1.pgdg20.04+1), server 15.1 (Ubuntu 15.1-1.pgdg20.04+1))
Type "help" for help.

postgres=> \q
root@docker-desktop:/# exit

Are you perhaps on windows?

@ArnaudScheffler
Copy link

Oh okay, it seems that host network can be tricky 😞.
I use a Mac with Apple silicon (macOS 13.4.1) and docker desktop (4.9.1).
I just updated docker desktop (4.24.2) but nothing changed.

@kamilogorek
Copy link
Member

kamilogorek commented Oct 23, 2023

@ArnaudScheffler give https://orbstack.dev/ a try, it unifies network issues between different OSs.

btw. Docker Desktop do not support ipv6 (https://docs.docker.com/desktop/troubleshoot/known-issues/) and your logs state that it indeed still tries to connect to ipv6 address.

@vixalien
Copy link
Author

vixalien commented Oct 23, 2023 via email

@ArnaudScheffler
Copy link

Thanks for the discovery @kamilogorek !

I've also tried 127.0.0.1, my Mac IPv4 address and it's DNS name :

root@docker-desktop:/# psql -h 192.168.1.89 -p 54328
psql: error: connection to server at "192.168.1.89", port 54328 failed: Connection refused
        Is the server running on that host and accepting TCP/IP connections?
root@docker-desktop:/# psql -h MBP-de-Arnaud-001 -p 54328
psql: error: connection to server at "MBP-de-Arnaud-001" (192.168.1.89), port 54328 failed: Connection refused
        Is the server running on that host and accepting TCP/IP connections?

I found a workaround for my specific use case : my target db is in fact in a k8s cluster. I initially used kubectl proxy to establish a connection but I found another way to expose the db so it doesn't use localhost anymore.

@kamilogorek
Copy link
Member

Closing the issue, as it seems like the original issue has been partially resolved or there is a working solution. Please do not hesitate to ping me if it is still relevant, and I will happily reopen it.

@vixalien
Copy link
Author

Hi. I'm sorry, but I didn't get the solution.

@soedirgo soedirgo reopened this Oct 24, 2023
@sweatybridge
Copy link
Contributor

@vixalien are you using docker desktop or just docker engine? https://docs.docker.com/desktop/install/archlinux/

I suspect host network requires docker desktop to work.

@vixalien
Copy link
Author

vixalien commented Oct 24, 2023 via email

@bricesuazo
Copy link

This could be due to localhost resolving to ipv6 address on your machine. Could you give the beta version a try?

npx supabase@beta db diff

This works for me. I'm using Windows.

@iamchubko
Copy link

iamchubko commented Dec 22, 2023

Updated cli today from 1.115.4 where everything worked to 1.123.4 and received this error, using Apple silicon.

Update: but now I see that 1.115.4 was released on November 27th and OP is using 1.100.1 with issue created in October so the cause might be different

@sweatybridge
Copy link
Contributor

This should have been resolved by #1852

@ivan-robert
Copy link

Hello I am getting exactly the same error word by word. It worked fine before but it just started doing this. Could it be due to the latest supabase updates ? (ipv6 migration for instance)

@sweatybridge
Copy link
Contributor

@ivan-robert could you create a new issue please? The original error concerns local diff only. It's completely unrelated to IPv6 migration.

@ivan-robert
Copy link

ivan-robert commented Feb 2, 2024

I can and will if necessary, but my usecase is exactly the same. I am only trying to run supabase diff --linked to pull the remote in a local instance

edit: I am currently writing the issue

@sweatybridge
Copy link
Contributor

I am only trying to run supabase diff --linked to pull the remote in a local instance

Yes a new issue would be good because this ticket is about diffing without --linked flag.

As a blind shot though, you can try npx supabase@beta db diff --linked.

@ivan-robert
Copy link

Tried the command but got the same error... Here is the issue : #1910

@PhilippS93
Copy link

PhilippS93 commented Feb 12, 2024

same error here on windows using
npx supabase@beta db pull --linked

@Abdulrehman1203
Copy link

Abdulrehman1203 commented Oct 18, 2024

Facing this issue when try to run project on docker:
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 2442, in _handle_dbapi_exception_noconnection
raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 146, in init
self._dbapi_connection = engine.raw_connection()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 3302, in raw_connection
return self.pool.connect()
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 449, in connect
return _ConnectionFairy._checkout(self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 1263, in _checkout
fairy = _ConnectionRecord.checkout(pool)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 712, in checkout
rec = pool._do_get()
^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/pool/impl.py", line 308, in _do_get
return self._create_connection()
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 390, in _create_connection
return _ConnectionRecord(self)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 674, in init
self.__connect()
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 900, in __connect
with util.safe_reraise():
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 146, in exit
raise exc_value.with_traceback(exc_tb)
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 896, in __connect
self.dbapi_connection = connection = pool._invoke_creator(self)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/create.py", line 643, in connect
return dialect.connect(*cargs, **cparams)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 621, in connect
return self.loaded_dbapi.connect(*cargs, **cparams)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/psycopg2/init.py", line 122, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) connection to server at "localhost" (::1), port 5432 failed: Connection refused
Is the server running on that host and accepting TCP/IP connections?
connection to server at "localhost" (127.0.0.1), port 5432 failed: Connection refused
Is the server running on that host and accepting TCP/IP connections?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

10 participants