NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:redshift.psycopg2

[ziziermao]

I am trying to connect to redshift from my python code. my pip installed:

psycopg2==2.6.1
redshift-sqlalchemy==0.4.1
SQLAlchemy==1.0.9
and my virtual machine(ubuntu) has:

libpq-dev
python-psycopg2
But I am still getting

engine = create_engine('redshift+psycopg2://{}:{}@{}'.format(username, password, url))
File "/opt/project/env/local/lib/python2.7/site-packages/sqlalchemy/engine/init.py", line 386, in create_engine
return strategy.create(args, *kwargs)
File "/opt/project/env/local/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 51, in create
entrypoint = u._get_entrypoint()
File "/opt/project/env/local/lib/python2.7/site-packages/sqlalchemy/engine/url.py", line 131, in _get_entrypoint
cls = registry.load(name)
File "/opt/project/env/local/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 205, in load
(self.group, name))
NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:redshift.psycopg2
I tried SQLAlchemy version 0.8.0, but doesn't work either.
With the same config, I am able to run from my laptop (mac), but on linux, I guess some packages still missing? Any suggestion will be appreciated, thanks!

[graingert]

please install:

psycopg2==2.6.1
sqlalchemy-redshift==0.4.0
SQLAlchemy==1.0.9

[ziziermao]

I tried 0.4.0, then I got:
engine = create_engine('redshift+psycopg2://{}:{}@{}'.format(username, password, url))
File "/opt/project/env/local/lib/python2.7/site-packages/sqlalchemy/engine/init.py", line 332, in create_engine
return strategy.create(_args, *_kwargs)
File "/opt/project/env/local/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 50, in create
dialect_cls = u.get_dialect()
File "/opt/project/env/local/lib/python2.7/site-packages/sqlalchemy/engine/url.py", line 107, in get_dialect
cls = registry.load(name)
File "/opt/project/env/local/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 89, in load
return self.implsname
File "/opt/project/env/local/lib/python2.7/site-packages/distribute-0.6.24-py2.7.egg/pkg_resources.py", line 1988, in load
if require: self.require(env, installer)
File "/opt/project/env/local/lib/python2.7/site-packages/distribute-0.6.24-py2.7.egg/pkg_resources.py", line 2001, in require
working_set.resolve(self.dist.requires(self.extras),env,installer))
File "/opt/project/env/local/lib/python2.7/site-packages/distribute-0.6.24-py2.7.egg/pkg_resources.py", line 584, in resolve
raise DistributionNotFound(req)
DistributionNotFound: SQLAlchemy>=0.8.0

[graingert]

FYI you should create URL objects:

from sqlalchemy.engine import url as sa_url

db_connect_url = sa_url.URL(
            drivername='redshift+psycopg2',
            username=username,
            password=password,
            host=config['host'],
            port=config['port'],
            database=config.get('database'),
)
create_engine(db_connect_url)

[graingert]

can you post a Vagrantfile that repeats the problem? Or give me SSH on the box

[ziziermao]

Do you mean my box's vagrant config?

[graingert]

yeah

[ziziermao]

-- mode: ruby --

vi: set ft=ruby :

Vagrant.configure(2) do |config|
config.vm.box = "PROJECT"
config.vm.box_url = "https://mycomputer.com/vagrant/PROJECT.json"
config.vm.box_download_insecure = true
config.vm.hostname = "project"
config.vm.network "forwarded_port", guest: 123, host: 1123
config.vm.network "forwarded_port", guest: 2333, host: 2333

config.vm.box_check_update = false

config.vm.synced_folder "../project_web", "/opt/project/project_web"

config.vm.provider "virtualbox" do |vb|

# Display the VirtualBox GUI when booting the machine

vb.gui = true

# Customize the amount of memory on the VM:

 vb.memory = "2048"

end
end

[graingert]

Can you publish the repo including provisioning scripts

[ziziermao]

I am sorry, I cannot publish the repo, and I don't think they provide provisioning scripts yet :(

[ziziermao]

Could you please suggest that needed to be included when doing vagrant provisioning? I didn't find a provisioning script anywhere yet.

[ziziermao]

Do you think I need to add any postgresql related package on the vagrant?

[graingert]

I don't know, can you create and post a new Vagrant image that replicates the problem?

[ziziermao]

Thanks @graingert, now we switched to use postgresql directly, and it's working. Sorry I cannot provide the vagrant image :(

[graingert]

I don't mean your current Vagrant image, I mean a minimal version that reproduces the problem

[ziziermao]

I will try to find a mini version. So far I am not sure where they keep the vagrant images. Thanks so much for helping!

[graingert]

Sorry I don't mean that you should send me your actual vagrant images, I'd just like you to create a new Vagrant project so I can vagrant up and replicate your problem.

[paulsef]

I'm running into the same issue and I'm wondering if there's been any progress here. I'm on OSX running python version 2.7

psycopg2==2.6.1
SQLAlchemy==1.0.9
sqlalchemy-redshift==0.4.0

from sqlalchemy.engine import url as sa_url
from sqlalchemy import create_engine
db_connect_url = sa_url.URL(
            drivername='redshift+psycopg2',
            username='username',
            password='password',
            host='host',
            port=port,
            database='db',
)
create_engine(db_connect_url)

NoSuchModuleError                         Traceback (most recent call last)
<ipython-input-4-e58bd282584a> in <module>()
----> 1 create_engine(db_connect_url)

/Users/change/work/test/venv/lib/python2.7/site-packages/sqlalchemy/engine/__init__.pyc in create_engine(*args, **kwargs)
    384     strategy = kwargs.pop('strategy', default_strategy)
    385     strategy = strategies.strategies[strategy]
--> 386     return strategy.create(*args, **kwargs)
    387 
    388 

/Users/change/work/test/venv/lib/python2.7/site-packages/sqlalchemy/engine/strategies.pyc in create(self, name_or_url, **kwargs)
     49         u = url.make_url(name_or_url)
     50 
---> 51         entrypoint = u._get_entrypoint()
     52         dialect_cls = entrypoint.get_dialect_cls(u)
     53 

/Users/change/work/test/venv/lib/python2.7/site-packages/sqlalchemy/engine/url.pyc in _get_entrypoint(self)
    129         else:
    130             name = self.drivername.replace('+', '.')
--> 131         cls = registry.load(name)
    132         # check for legacy dialects that
    133         # would return a module with 'dialect' as the

/Users/change/work/test/venv/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.pyc in load(self, name)
    203         raise exc.NoSuchModuleError(
    204             "Can't load plugin: %s:%s" %
--> 205             (self.group, name))
    206 
    207     def register(self, name, modulepath, objname):

NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:redshift.psycopg2

[ziziermao]

I ended up using 'postgresql+psycopg2' directly.

On Fri, Apr 8, 2016 at 4:43 PM, Paul Minton notifications@github.com
wrote:

I'm running into the same issue and I'm wondering if there's been any
progress here. I'm on OSX running python version 2.7

psycopg2==2.6.1
SQLAlchemy==1.0.9
sqlalchemy-redshift==0.4.0

from sqlalchemy.engine import url as sa_urlfrom sqlalchemy import create_engine
db_connect_url = sa_url.URL(
drivername='redshift+psycopg2',
username='username',
password='password',
host='host',
port=port,
database='db',
)
create_engine(db_connect_url)

NoSuchModuleError Traceback (most recent call last)
in ()
----> 1 create_engine(db_connect_url)

/Users/change/work/test/venv/lib/python2.7/site-packages/sqlalchemy/engine/init.pyc in create_engine(_args, *_kwargs)
384 strategy = kwargs.pop('strategy', default_strategy)
385 strategy = strategies.strategies[strategy]
--> 386 return strategy.create(_args, *_kwargs)
387
388

/Users/change/work/test/venv/lib/python2.7/site-packages/sqlalchemy/engine/strategies.pyc in create(self, name_or_url, **kwargs)
49 u = url.make_url(name_or_url)
50
---> 51 entrypoint = u._get_entrypoint()
52 dialect_cls = entrypoint.get_dialect_cls(u)
53

/Users/change/work/test/venv/lib/python2.7/site-packages/sqlalchemy/engine/url.pyc in _get_entrypoint(self)
129 else:
130 name = self.drivername.replace('+', '.')
--> 131 cls = registry.load(name)
132 # check for legacy dialects that
133 # would return a module with 'dialect' as the

/Users/change/work/test/venv/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.pyc in load(self, name)
203 raise exc.NoSuchModuleError(
204 "Can't load plugin: %s:%s" %
--> 205 (self.group, name))
206
207 def register(self, name, modulepath, objname):

NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:redshift.psycopg2

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
#68 (comment)

[paulsef]

I know that's an option but I'm looking to use some of the
redshift-specific commands (copy, unload, etc).

On Fri, Apr 8, 2016 at 4:46 PM, ziziermao notifications@github.com wrote:

I ended up using 'postgresql+psycopg2' directly.

On Fri, Apr 8, 2016 at 4:43 PM, Paul Minton notifications@github.com
wrote:

I'm running into the same issue and I'm wondering if there's been any
progress here. I'm on OSX running python version 2.7

psycopg2==2.6.1
SQLAlchemy==1.0.9
sqlalchemy-redshift==0.4.0

from sqlalchemy.engine import url as sa_urlfrom sqlalchemy import
create_engine
db_connect_url = sa_url.URL(
drivername='redshift+psycopg2',
username='username',
password='password',
host='host',
port=port,
database='db',
)
create_engine(db_connect_url)

NoSuchModuleError Traceback (most recent call last)
in ()
----> 1 create_engine(db_connect_url)

/Users/change/work/test/venv/lib/python2.7/site-packages/sqlalchemy/engine/init.pyc
in create_engine(_args, *_kwargs)
384 strategy = kwargs.pop('strategy', default_strategy)
385 strategy = strategies.strategies[strategy]
--> 386 return strategy.create(_args, *_kwargs)
387
388

/Users/change/work/test/venv/lib/python2.7/site-packages/sqlalchemy/engine/strategies.pyc
in create(self, name_or_url, **kwargs)
49 u = url.make_url(name_or_url)
50
---> 51 entrypoint = u._get_entrypoint()
52 dialect_cls = entrypoint.get_dialect_cls(u)
53

/Users/change/work/test/venv/lib/python2.7/site-packages/sqlalchemy/engine/url.pyc
in _get_entrypoint(self)
129 else:
130 name = self.drivername.replace('+', '.')
--> 131 cls = registry.load(name)
132 # check for legacy dialects that
133 # would return a module with 'dialect' as the

/Users/change/work/test/venv/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.pyc
in load(self, name)
203 raise exc.NoSuchModuleError(
204 "Can't load plugin: %s:%s" %
--> 205 (self.group, name))
206
207 def register(self, name, modulepath, objname):

NoSuchModuleError: Can't load plugin:
sqlalchemy.dialects:redshift.psycopg2

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
<
#68 (comment)

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#68 (comment)

[mervekovan]

Had this problem locally with

Flask-SQLAlchemy (2.1)
psycopg2 (2.6.2)
SQLAlchemy (1.1.5)
sqlalchemy-redshift (0.5.0)

After some search, saw someone mention something about installing globally instead of virtualenv. And when I did install sqlalchemy-redshift globally, it worked just fine.

[graingert]

please do no pip install sqlalchemy-redshift globally

[graingert]

closing this until I get a docker or VM image that reproduces the issue.

[qqrs]

Also ran into this using anaconda 4.3.14, Python 2.7, Linux. Had to give up and use postgresql+psycopg2 directly as suggested above.

(Sorry, not able to provide docker/VM image. Posting to signal boost the postgresql+psycopg2 alternative.)

[jaaved]

using:postgresql+psycopg2 is throwing error:ModuleNotFoundError: No module named 'MySQLdb'
with version:
psycopg2==2.7.3.1
sqlalchemy==1.1.15
sqlalchemy-redshift== 0.7.0
And if I am using: db+postgresql then error:sqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:db.postgresql

[graingert]

@jaaved please don't co-opt other issues that are irrelevant to your problem. I can't resolve the issue without instructions on how to reproduce it.

[Ic3fr0g]

I too encountered this error. Occurs if you are using a conda env. If you install it on the default env it works.

[GuSuku]

I too encountered this issue when installed via conda install within a conda env. After hours of struggling, I ended up using postgresql+psycopg2

[graingert]

@GuSuku @Ic3fr0g etc, again I need a reproducible example to be able to fix this. eg a github repo connected to travis, a Docker image or a Vagrantfile/vm image