django.setup() is called with `pupa update --scrape` #87

Closed
fgregg opened this Issue May 31, 2014 · 10 comments

Projects

None yet

4 participants

@fgregg
Contributor
fgregg commented May 31, 2014

@paultag and I discussed at tcamp14

> pupa update --scrape chicago
no pupa_settings on path, using defaults
Traceback (most recent call last):
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/__init__.py", line 129, in ensure_connection
    self.connect()
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/__init__.py", line 118, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 132, in get_new_connection
    return Database.connect(**conn_params)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/psycopg2/__init__.py", line 164, in connect
    conn = _connect(dsn, connection_factory=connection_factory, async=async)
psycopg2.OperationalError: could not connect to server: Connection refused
    Is the server running on host "localhost" (127.0.0.1) and accepting
    TCP/IP connections on port 5432?


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/fgregg/public/municipal-scrapers-us/.env/bin/pupa", line 9, in <module>
    load_entry_point('pupa==0.4.0-dev', 'console_scripts', 'pupa')()
  File "/home/fgregg/public/municipal-scrapers-us/.env/src/pupa/pupa/cli/__main__.py", line 32, in main
    django.setup()
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/__init__.py", line 21, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/apps/registry.py", line 106, in populate
    app_config.import_models(all_models)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/apps/config.py", line 190, in import_models
    self.models_module = import_module(models_module_name)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/importlib/__init__.py", line 109, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 2231, in _gcd_import
  File "<frozen importlib._bootstrap>", line 2214, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2203, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1129, in _exec
  File "<frozen importlib._bootstrap>", line 1448, in exec_module
  File "<frozen importlib._bootstrap>", line 321, in _call_with_frames_removed
  File "/home/fgregg/public/municipal-scrapers-us/python-opencivicdata/opencivicdata/models/__init__.py", line 2, in <module>
    from .jurisdiction import Jurisdiction, JurisdictionSession
  File "/home/fgregg/public/municipal-scrapers-us/python-opencivicdata/opencivicdata/models/jurisdiction.py", line 5, in <module>
    from .base import OCDBase, LinkBase, OCDIDField, RelatedBase
  File "/home/fgregg/public/municipal-scrapers-us/python-opencivicdata/opencivicdata/models/base.py", line 41, in <module>
    class OCDBase(models.Model):
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/models/base.py", line 135, in __new__
    new_class.add_to_class('_meta', Options(meta, **kwargs))
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/models/base.py", line 313, in add_to_class
    value.contribute_to_class(cls, name)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/models/options.py", line 166, in contribute_to_class
    self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/__init__.py", line 40, in __getattr__
    return getattr(connections[DEFAULT_DB_ALIAS], item)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/utils.py", line 231, in __getitem__
    conn = backend.DatabaseWrapper(db, alias)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/contrib/gis/db/backends/postgis/base.py", line 14, in __init__
    self.ops = PostGISOperations(self)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 166, in __init__
    if self.spatial_version < (1, 3, 4):
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/utils/functional.py", line 55, in __get__
    res = instance.__dict__[self.func.__name__] = self.func(instance)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 260, in spatial_version
    vtup = self.postgis_version_tuple()
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 434, in postgis_version_tuple
    version = self.postgis_lib_version()
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 414, in postgis_lib_version
    return self._get_postgis_func('postgis_lib_version')
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 404, in _get_postgis_func
    with self.connection.temporary_connection() as cursor:
  File "/usr/lib/python3.4/contextlib.py", line 59, in __enter__
    return next(self.gen)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/__init__.py", line 540, in temporary_connection
    cursor = self.cursor()
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/__init__.py", line 164, in cursor
    cursor = utils.CursorWrapper(self._cursor(), self)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/__init__.py", line 134, in _cursor
    self.ensure_connection()
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/__init__.py", line 129, in ensure_connection
    self.connect()
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/utils/six.py", line 549, in reraise
    raise value.with_traceback(tb)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/__init__.py", line 129, in ensure_connection
    self.connect()
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/__init__.py", line 118, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 132, in get_new_connection
    return Database.connect(**conn_params)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/psycopg2/__init__.py", line 164, in connect
    conn = _connect(dsn, connection_factory=connection_factory, async=async)
django.db.utils.OperationalError: could not connect to server: Connection refused
    Is the server running on host "localhost" (127.0.0.1) and accepting
    TCP/IP connections on port 5432?
@jamesturk
Member

thanks for reporting this
On May 31, 2014 10:37 AM, "Forest Gregg" notifications@github.com wrote:

@paultag https://github.com/paultag and I discussed at tcamp14

pupa update --scrape chicagono pupa_settings on path, using defaultsTraceback (most recent call last):
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/init.py", line 129, in ensure_connection
self.connect()
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/init.py", line 118, in connect
self.connection = self.get_new_connection(conn_params)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 132, in get_new_connection
return Database.connect(**conn_params)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/psycopg2/init.py", line 164, in connect
conn = _connect(dsn, connection_factory=connection_factory, async=async)psycopg2.OperationalError: could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?

The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/fgregg/public/municipal-scrapers-us/.env/bin/pupa", line 9, in
load_entry_point('pupa==0.4.0-dev', 'console_scripts', 'pupa')()
File "/home/fgregg/public/municipal-scrapers-us/.env/src/pupa/pupa/cli/main.py", line 32, in main
django.setup()
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/init.py", line 21, in setup
apps.populate(settings.INSTALLED_APPS)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/apps/registry.py", line 106, in populate
app_config.import_models(all_models)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/apps/config.py", line 190, in import_models
self.models_module = import_module(models_module_name)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/importlib/init.py", line 109, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 2231, in _gcd_import
File "", line 2214, in _find_and_load
File "", line 2203, in _find_and_load_unlocked
File "", line 1200, in _load_unlocked
File "", line 1129, in _exec
File "", line 1448, in exec_module
File "", line 321, in _call_with_frames_removed
File "/home/fgregg/public/municipal-scrapers-us/python-opencivicdata/opencivicdata/models/init.py", line 2, in
from .jurisdiction import Jurisdiction, JurisdictionSession
File "/home/fgregg/public/municipal-scrapers-us/python-opencivicdata/opencivicdata/models/jurisdiction.py", line 5, in
from .base import OCDBase, LinkBase, OCDIDField, RelatedBase
File "/home/fgregg/public/municipal-scrapers-us/python-opencivicdata/opencivicdata/models/base.py", line 41, in
class OCDBase(models.Model):
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/models/base.py", line 135, in new
new_class.add_to_class('_meta', Options(meta, **kwargs))
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/models/base.py", line 313, in add_to_class
value.contribute_to_class(cls, name)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/models/options.py", line 166, in contribute_to_class
self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/init.py", line 40, in getattr
return getattr(connections[DEFAULT_DB_ALIAS], item)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/utils.py", line 231, in getitem
conn = backend.DatabaseWrapper(db, alias)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/contrib/gis/db/backends/postgis/base.py", line 14, in init
self.ops = PostGISOperations(self)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 166, in init
if self.spatial_version < (1, 3, 4):
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/utils/functional.py", line 55, in get
res = instance.dict[self.func.name] = self.func(instance)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 260, in spatial_version
vtup = self.postgis_version_tuple()
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 434, in postgis_version_tuple
version = self.postgis_lib_version()
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 414, in postgis_lib_version
return self._get_postgis_func('postgis_lib_version')
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 404, in _get_postgis_func
with self.connection.temporary_connection() as cursor:
File "/usr/lib/python3.4/contextlib.py", line 59, in enter
return next(self.gen)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/init.py", line 540, in temporary_connection
cursor = self.cursor()
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/init.py", line 164, in cursor
cursor = utils.CursorWrapper(self._cursor(), self)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/init.py", line 134, in _cursor
self.ensure_connection()
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/init.py", line 129, in ensure_connection
self.connect()
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/utils.py", line 94, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/utils/six.py", line 549, in reraise
raise value.with_traceback(tb)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/init.py", line 129, in ensure_connection
self.connect()
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/init.py", line 118, in connect
self.connection = self.get_new_connection(conn_params)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 132, in get_new_connection
return Database.connect(**conn_params)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/psycopg2/init.py", line 164, in connect
conn = _connect(dsn, connection_factory=connection_factory, async=async)django.db.utils.OperationalError: could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?


Reply to this email directly or view it on GitHub
#87.

@jamesturk
Member

can't reproduce this w/ latest pupa - was this against master?

@paultag
Member
paultag commented Jun 5, 2014

Yeah, it was against master -- I think we isolated the invocation down to https://github.com/opencivicdata/pupa/blob/master/pupa/cli/__main__.py#L32

(I think that was the one)

@jamesturk
Member

from what I can tell, django.setup() doesn't require the DB to be there it simply populates the app list

so it is called, but i'm not getting the DB error

@fgregg
Contributor
fgregg commented Jun 11, 2014

I have postgres running, but haven't created a pupa user:

(.env)12:06 PM|fgregg-ThinkPad-X201|~/public/municipal-scrapers-us>pupa update --scrape chicago
no pupa_settings on path, using defaults
Traceback (most recent call last):
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/__init__.py", line 129, in ensure_connection
    self.connect()
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/__init__.py", line 118, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 134, in get_new_connection
    return Database.connect(**conn_params)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/psycopg2/__init__.py", line 164, in connect
    conn = _connect(dsn, connection_factory=connection_factory, async=async)
psycopg2.OperationalError: FATAL:  password authentication failed for user "pupa"
FATAL:  password authentication failed for user "pupa"


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/fgregg/public/municipal-scrapers-us/.env/bin/pupa", line 9, in <module>
    load_entry_point('pupa==0.4.0-dev', 'console_scripts', 'pupa')()
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/pupa-0.4.0_dev-py3.4.egg/pupa/cli/__main__.py", line 32, in main
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/__init__.py", line 21, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/apps/registry.py", line 106, in populate
    app_config.import_models(all_models)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/apps/config.py", line 190, in import_models
    self.models_module = import_module(models_module_name)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/importlib/__init__.py", line 109, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 2231, in _gcd_import
  File "<frozen importlib._bootstrap>", line 2214, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2203, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1129, in _exec
  File "<frozen importlib._bootstrap>", line 1448, in exec_module
  File "<frozen importlib._bootstrap>", line 321, in _call_with_frames_removed
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/opencivicdata-0.3.0-py3.4.egg/opencivicdata/models/__init__.py", line 2, in <module>
    from .jurisdiction import Jurisdiction, JurisdictionSession
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/opencivicdata-0.3.0-py3.4.egg/opencivicdata/models/jurisdiction.py", line 5, in <module>
    from .base import OCDBase, LinkBase, OCDIDField, RelatedBase
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/opencivicdata-0.3.0-py3.4.egg/opencivicdata/models/base.py", line 41, in <module>
    class OCDBase(models.Model):
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/models/base.py", line 125, in __new__
    new_class.add_to_class('_meta', Options(meta, **kwargs))
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/models/base.py", line 303, in add_to_class
    value.contribute_to_class(cls, name)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/models/options.py", line 166, in contribute_to_class
    self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/__init__.py", line 40, in __getattr__
    return getattr(connections[DEFAULT_DB_ALIAS], item)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/utils.py", line 231, in __getitem__
    conn = backend.DatabaseWrapper(db, alias)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/contrib/gis/db/backends/postgis/base.py", line 14, in __init__
    self.ops = PostGISOperations(self)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 166, in __init__
    if self.spatial_version < (1, 3, 4):
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/utils/functional.py", line 55, in __get__
    res = instance.__dict__[self.func.__name__] = self.func(instance)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 260, in spatial_version
    vtup = self.postgis_version_tuple()
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 434, in postgis_version_tuple
    version = self.postgis_lib_version()
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 414, in postgis_lib_version
    return self._get_postgis_func('postgis_lib_version')
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 404, in _get_postgis_func
    with self.connection.temporary_connection() as cursor:
  File "/usr/lib/python3.4/contextlib.py", line 59, in __enter__
    return next(self.gen)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/__init__.py", line 540, in temporary_connection
    cursor = self.cursor()
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/__init__.py", line 164, in cursor
    cursor = utils.CursorWrapper(self._cursor(), self)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/__init__.py", line 134, in _cursor
    self.ensure_connection()
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/__init__.py", line 129, in ensure_connection
    self.connect()
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/utils/six.py", line 549, in reraise
    raise value.with_traceback(tb)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/__init__.py", line 129, in ensure_connection
    self.connect()
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/__init__.py", line 118, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 134, in get_new_connection
    return Database.connect(**conn_params)
  File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/psycopg2/__init__.py", line 164, in connect
    conn = _connect(dsn, connection_factory=connection_factory, async=async)
django.db.utils.OperationalError: FATAL:  password authentication failed for user "pupa"
FATAL:  password authentication failed for user "pupa"
@jamesturk
Member

Thanks for this- I'll find time to try it with different settings to
reproduce.

On Wed, Jun 11, 2014 at 1:07 PM, Forest Gregg notifications@github.com
wrote:

I have postgres running, but haven't created a pupa user:

(.env)12:06 PM|fgregg-ThinkPad-X201|~/public/municipal-scrapers-us>pupa update --scrape chicago
no pupa_settings on path, using defaults
Traceback (most recent call last):
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/init.py", line 129, in ensure_connection
self.connect()
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/init.py", line 118, in connect
self.connection = self.get_new_connection(conn_params)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 134, in get_new_connection
return Database.connect(**conn_params)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/psycopg2/init.py", line 164, in connect
conn = _connect(dsn, connection_factory=connection_factory, async=async)
psycopg2.OperationalError: FATAL: password authentication failed for user "pupa"
FATAL: password authentication failed for user "pupa"

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/fgregg/public/municipal-scrapers-us/.env/bin/pupa", line 9, in
load_entry_point('pupa==0.4.0-dev', 'console_scripts', 'pupa')()
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/pupa-0.4.0_dev-py3.4.egg/pupa/cli/main.py", line 32, in main
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/init.py", line 21, in setup
apps.populate(settings.INSTALLED_APPS)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/apps/registry.py", line 106, in populate
app_config.import_models(all_models)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/apps/config.py", line 190, in import_models
self.models_module = import_module(models_module_name)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/importlib/init.py", line 109, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 2231, in _gcd_import
File "", line 2214, in _find_and_load
File "", line 2203, in _find_and_load_unlocked
File "", line 1200, in _load_unlocked
File "", line 1129, in _exec
File "", line 1448, in exec_module
File "", line 321, in _call_with_frames_removed
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/opencivicdata-0.3.0-py3.4.egg/opencivicdata/models/init.py", line 2, in
from .jurisdiction import Jurisdiction, JurisdictionSession
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/opencivicdata-0.3.0-py3.4.egg/opencivicdata/models/jurisdiction.py", line 5, in
from .base import OCDBase, LinkBase, OCDIDField, RelatedBase
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/opencivicdata-0.3.0-py3.4.egg/opencivicdata/models/base.py", line 41, in
class OCDBase(models.Model):
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/models/base.py", line 125, in new
new_class.add_to_class('_meta', Options(meta, **kwargs))
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/models/base.py", line 303, in add_to_class
value.contribute_to_class(cls, name)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/models/options.py", line 166, in contribute_to_class
self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/init.py", line 40, in getattr
return getattr(connections[DEFAULT_DB_ALIAS], item)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/utils.py", line 231, in getitem
conn = backend.DatabaseWrapper(db, alias)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/contrib/gis/db/backends/postgis/base.py", line 14, in init
self.ops = PostGISOperations(self)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 166, in init
if self.spatial_version < (1, 3, 4):
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/utils/functional.py", line 55, in get
res = instance.dict[self.func.name] = self.func(instance)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 260, in spatial_version
vtup = self.postgis_version_tuple()
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 434, in postgis_version_tuple
version = self.postgis_lib_version()
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 414, in postgis_lib_version
return self._get_postgis_func('postgis_lib_version')
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/contrib/gis/db/backends/postgis/operations.py", line 404, in _get_postgis_func
with self.connection.temporary_connection() as cursor:
File "/usr/lib/python3.4/contextlib.py", line 59, in enter
return next(self.gen)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/init.py", line 540, in temporary_connection
cursor = self.cursor()
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/init.py", line 164, in cursor
cursor = utils.CursorWrapper(self._cursor(), self)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/init.py", line 134, in _cursor
self.ensure_connection()
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/init.py", line 129, in ensure_connection
self.connect()
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/utils.py", line 94, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/utils/six.py", line 549, in reraise
raise value.with_traceback(tb)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/init.py", line 129, in ensure_connection
self.connect()
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/init.py", line 118, in connect
self.connection = self.get_new_connection(conn_params)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 134, in get_new_connection
return Database.connect(**conn_params)
File "/home/fgregg/public/municipal-scrapers-us/.env/lib/python3.4/site-packages/psycopg2/init.py", line 164, in connect
conn = _connect(dsn, connection_factory=connection_factory, async=async)
django.db.utils.OperationalError: FATAL: password authentication failed for user "pupa"
FATAL: password authentication failed for user "pupa"


Reply to this email directly or view it on GitHub
#87 (comment).

@jamesturk jamesturk added this to the pupa 0.4 milestone Jun 11, 2014
@Vanuan
Contributor
Vanuan commented Jun 20, 2014

So, is there a way to pass a different db user? Is setting DATABASE_URL supported?
Update Yes, it is!

@jamesturk jamesturk added this to the pupa 0.5 milestone Jul 10, 2014
@Vanuan
Contributor
Vanuan commented Jul 11, 2014

With some dirty hacks I've managed to run pupa update --scrape without database setup.

@jamesturk
Member

can you describe what you did, we're still trying to track this one down

@Vanuan
Contributor
Vanuan commented Jul 11, 2014

Comment out this line (there should be a conditional):

https://github.com/opencivicdata/pupa/blob/971e184ecbe65cc5c827cbc186cb30a7e4508d7e/pupa/cli/__main__.py#L32

Added this import where it's needed (not sure why):

import logging.config

Commented out this line (we don't need importers if we're scraping, should be imported inline):

https://github.com/opencivicdata/pupa/blob/971e184ecbe65cc5c827cbc186cb30a7e4508d7e/pupa/cli/commands/update.py#L10

@jamesturk jamesturk closed this Jul 14, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment