Python silent sqlite compile error #15300
Comments
|
Thanks, I'll have a look. But I could not reproduce on 10.8. Perhaps the log gives a clue. |
|
And with At least I know that I have seen this error, if Apple's sqlite is used and that does not have loadable-extensions enabled. Worth a try: |
|
I already redid And yes, confirmed that brewing with |
|
Hmm. What is your But still I don't have a clue, why it fails for you. |
|
From my build log (I know this is from the build of the tkinter module, but python spits out the details only for unsuccessful modules): From your log: (I added some newlines) Abstracting over the differences between tk and sqlite, I don't see a real problem here. At least, I think it's not a problem, what superenv removes here, but what it adds (and that is not shown in the log). To investigate further, I need the Here is the interesting part of my cc.log. It's the last block that contains the string "sqlite". I added newlines.: I have a Xcode-only setup, but my CLT-only system builds python with sqlite just fine, too. However, I should test the CLT system with homebrew in a non-standard dir. The only thing which makes me itch when I look at my cc.log are the paths to I think if we can debug this, we may be able to improve superenv. |
|
What I'd like to ask @mxcl at this point: Why do I find |
|
Here is the log, and I guess my only question is why is there a |
Because readline is keg-only and sqlite is not. stdenv has identical behavior. And yes the bug would be that The reason this doesn't effect stdenv is we put HOMEBREW's libpath first. superenv should be removing that line, but for some reason doesn't. So that's the bug, the library path fixing code has a bug. I repeat again: superenv is simple, you are trying to think of it as a complicated system, all the problems we have had with it are bugs in my code. |
|
For me: we also need to remove: |
|
@mxcl yep, that answers my questions. So I leave the python formula as is. Superenv is good. Don't get me wrong. I misunderstood the keg_only handling here. If I install homebrew somewhere different than |
|
How do I fix my path issue if I am using /usr/local/homebrew/bin as my directory? |
|
@Alives for you the SQLite module is broken too? It would help me diagnose I you could gist.github.com the output of your build |
|
https://gist.github.com/4020694 FYI: I had to rm python first and that may have left some libs behind.. not sure. |
|
Should I not just edit the python formula for now to make a one-off fix for the time being? |
|
@Alives thanks, on your system, too, python attempts to link against the sqlite from OS X. I don't know a one-liner that fixes this. (Except disabling the dynamic loadable sqlite extensions) |
|
Can you point out the lines that show that? Maybe I can put a bandaid on it for the time being. |
|
@Alives, I'd rather try to find where and why |
Unless you're doing GIS work and need to load up SpatiaLite. I'll bet the QGIS formula in homebrew-science could be affected by this since it contains a large amount of algorithms written in Python. A possible workaround could be to explicitly add Ultimately, the solution may be to flag SQLite as |
|
sqlite: Yep, it is a dupe, therefore it should be Besides that, I still have to debug the superenv + python behavior here. Why are the |
|
So...should we flag sqlite as keg-only? I forget what breaks when we do this. |
|
Yes. |
|
|
|
@staticfloat do you still have the problem that sqlite is not build for python? I think this is related to homebrew not being in /usr/local. Python seems to look there. As we can see, it linkts to my "production" homebrew in /usr/local. This is a bug. |
|
@staticfloat since your test systems seems a good candidate: Are you willing to help by temporary making |
|
I tried to do this locally and indeed Python links to the correct sqlite if the sqlite formula would be keg_only. Further, I tested Shall I open a PR for keg_only sqlite? |
|
this issue should be fixed now. |
|
I saw that your I double-checked, and |
|
Have you set an ENV var to build universal? Veerery strange. |
|
I opened a pull request to fix the "sqlite not found"-bug. Hope it gets pulled soon. Sorry for any inconvenience! |
|
You rock samueljohn! Thank you so much for your help towards this commit! :) Everything is running slick as a whistle again! :) |
|
I hit the failure to build with sqlite problem with an install of homebrew from about 7 days ago specifically because it's needed for iPython history support. 18 Jan 2013 Ran brew update and brew doctor, brew uninstall python, followed by brew install python and both import sqlite3 is good and iPython runs without complaining that there's no sqlite so can confirm fixed for OSX 10.8.2 on a clean install. Many Thanks and keep up the good work. |
|
I'm running into what I think is the same issue for python3 formula. Is there any plan to fix this as well? |
|
H On 21 Jan 2013, at 22:07, Pete Browne notifications@github.com wrote:
|
|
Hi, On 22 Jan 2013, at 00:07, Edward Hartley ed_hartley@mac.com wrote:
|
|
@petebrowne, thanks, I am going to fix that soon. @ReaddyEddy |
|
@ReaddyEddy @petebrowne Had the same issue with python3, resolved by doing a https://github.com/norioxkimura/homebrew/commit/89d3ee1f4d11c897b56a8c2516c0a61f7c5c31f9 Try installing from this maybe: https://gist.github.com/4651708 |
|
Sadly, I could not get uninstalling and reinstalling python did not work for me. I still get an error that sqlite is not found. (pycharm_repro)rburhum@peru.local ~/src/pycharm_repro/pycharm_repro $ python manage.py test
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/Users/rburhum/.virtualenvs/pycharm_repro/lib/python2.7/site-packages/django/core/management/__init__.py", line 443, in execute_from_command_line
utility.execute()
File "/Users/rburhum/.virtualenvs/pycharm_repro/lib/python2.7/site-packages/django/core/management/__init__.py", line 382, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Users/rburhum/.virtualenvs/pycharm_repro/lib/python2.7/site-packages/django/core/management/__init__.py", line 261, in fetch_command
klass = load_command_class(app_name, subcommand)
File "/Users/rburhum/.virtualenvs/pycharm_repro/lib/python2.7/site-packages/django/core/management/__init__.py", line 69, in load_command_class
module = import_module('%s.management.commands.%s' % (app_name, name))
File "/Users/rburhum/.virtualenvs/pycharm_repro/lib/python2.7/site-packages/django/utils/importlib.py", line 35, in import_module
__import__(name)
File "/Users/rburhum/.virtualenvs/pycharm_repro/lib/python2.7/site-packages/django_nose/__init__.py", line 4, in <module>
from django_nose.runner import *
File "/Users/rburhum/.virtualenvs/pycharm_repro/lib/python2.7/site-packages/django_nose/runner.py", line 18, in <module>
from django.core.management.commands.loaddata import Command
File "/Users/rburhum/.virtualenvs/pycharm_repro/lib/python2.7/site-packages/django/core/management/commands/loaddata.py", line 13, in <module>
from django.core import serializers
File "/Users/rburhum/.virtualenvs/pycharm_repro/lib/python2.7/site-packages/django/core/serializers/__init__.py", line 21, in <module>
from django.core.serializers.base import SerializerDoesNotExist
File "/Users/rburhum/.virtualenvs/pycharm_repro/lib/python2.7/site-packages/django/core/serializers/base.py", line 7, in <module>
from django.db import models
File "/Users/rburhum/.virtualenvs/pycharm_repro/lib/python2.7/site-packages/django/db/__init__.py", line 40, in <module>
backend = load_backend(connection.settings_dict['ENGINE'])
File "/Users/rburhum/.virtualenvs/pycharm_repro/lib/python2.7/site-packages/django/db/__init__.py", line 34, in __getattr__
return getattr(connections[DEFAULT_DB_ALIAS], item)
File "/Users/rburhum/.virtualenvs/pycharm_repro/lib/python2.7/site-packages/django/db/utils.py", line 92, in __getitem__
backend = load_backend(db['ENGINE'])
File "/Users/rburhum/.virtualenvs/pycharm_repro/lib/python2.7/site-packages/django/db/utils.py", line 24, in load_backend
return import_module('.base', backend_name)
File "/Users/rburhum/.virtualenvs/pycharm_repro/lib/python2.7/site-packages/django/utils/importlib.py", line 35, in import_module
__import__(name)
File "/Users/rburhum/.virtualenvs/pycharm_repro/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 31, in <module>
raise ImproperlyConfigured("Error loading either pysqlite2 or sqlite3 modules (tried in that order): %s" % exc)
django.core.exceptions.ImproperlyConfigured: Error loading either pysqlite2 or sqlite3 modules (tried in that order): dlopen(/Users/rburhum/.virtualenvs/pycharm_repro/lib/python2.7/lib-dynload/_sqlite3.so, 2): Library not loaded: /usr/local/lib/libsqlite3.0.8.6.dylib
Referenced from: /Users/rburhum/.virtualenvs/pycharm_repro/lib/python2.7/lib-dynload/_sqlite3.so
Reason: image not foundI also tried removing all homebrew packages and reinstalling them to be sure by installing sqlite first, then python, then the rest. Removing ".virtualenv" directory and recreating it. No luck. The only thing that fixes it is to do |
|
@rburhum I assume you did a It should have been fixed by explicitly adding the You are running from inside a virtualenv, can you test without the virtualenv if |
|
@samueljohn I did run You are right though. It works outside of virtualenv, and inside of virtualenv it is broken. Yesterday I removed ~/.virtualenvs yesterday and recreate it after logging in with a fresh upgraded terminal, the virtualenvs dir got recreated. Check out this output: Works outside virtualenv rburhum@peru.local ~ $ python -c "import sqlite3"Create virtualenv - notice error with sitecustomize. Try to import sqlite3 rburhum@peru.local ~ $ mkvirtualenv test
'import sitecustomize' failed; use -v for traceback
New python executable in test/bin/python
Installing setuptools............done.
Installing pip...............done.
virtualenvwrapper.user_scripts creating /Users/rburhum/.virtualenvs/test/bin/predeactivate
virtualenvwrapper.user_scripts creating /Users/rburhum/.virtualenvs/test/bin/postdeactivate
virtualenvwrapper.user_scripts creating /Users/rburhum/.virtualenvs/test/bin/preactivate
virtualenvwrapper.user_scripts creating /Users/rburhum/.virtualenvs/test/bin/postactivate
virtualenvwrapper.user_scripts creating /Users/rburhum/.virtualenvs/test/bin/get_env_details
(test)rburhum@peru.local ~ $ python -c "import sqlite3"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sqlite3/__init__.py", line 24, in <module>
from dbapi2 import *
File "/usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sqlite3/dbapi2.py", line 27, in <module>
from _sqlite3 import *
ImportError: dlopen(/Users/rburhum/.virtualenvs/test/lib/python2.7/lib-dynload/_sqlite3.so, 2): Library not loaded: /usr/local/lib/libsqlite3.0.8.6.dylib
Referenced from: /Users/rburhum/.virtualenvs/test/lib/python2.7/lib-dynload/_sqlite3.so
Reason: image not foundForce link sqlite and run import again inside virtualenv (test)rburhum@peru.local ~ $ brew link -f sqlite
Linking /usr/local/Cellar/sqlite/3.7.15.1... 9 symlinks created
(test)rburhum@peru.local ~ $ python -c "import sqlite3"
(test)rburhum@peru.local ~ $ Works fine that time. I guess it is due to sitecustomize then? This is the relevant portion of my bashrc file: # Setting up the VirtualEnv
export WORKON_HOME=$HOME/.virtualenvs
export VIRTUALENVWRAPPER_PYTHON=/usr/local/bin/python2.7
#export VIRTUALENVWRAPPER_VIRTUALENV_ARGS='--no-site-packages'
export PIP_VIRTUALENV_BASE=$WORKON_HOME
export PIP_RESPECT_VIRTUALENV=true
if [[ -r /usr/local/share/python/virtualenvwrapper.sh ]]; then
source /usr/local/share/python/virtualenvwrapper.sh
else
echo "WARNING: Can't find virtualenvwrapper.sh"
fi |
|
Thanks for the report. So workaround for now: I will put this on my todo list. |
|
Sure. Thanks to you @samueljohn |
|
I guess this is because during the creation of a virtualenv, the python stdlib (including the sqlite wrappers) are copied into the venv. Can somebody test if creating a new virtualenv with the sqlite keg_only (and _un_linked), works and |
rburhum@peru.local ~ $ python
Python 2.7.3 (default, Feb 6 2013, 18:10:34)
[GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.24)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> quit()
rburhum@peru.local ~ $ brew unlink sqlite
Unlinking /usr/local/Cellar/sqlite/3.7.15.2... 9 links removed
rburhum@peru.local ~ $ brew test python
Testing python
==> /usr/local/Cellar/python/2.7.3/bin/python -c import sqlite3
==> /usr/local/Cellar/python/2.7.3/bin/python -c import Tkinter; root = Tkinter.
rburhum@peru.local ~ $ mkvirtualenv test_sqlite
'import sitecustomize' failed; use -v for traceback
New python executable in test_sqlite/bin/python
Installing setuptools............done.
Installing pip...............done.
virtualenvwrapper.user_scripts creating /Users/rburhum/.virtualenvs/test_sqlite/bin/predeactivate
virtualenvwrapper.user_scripts creating /Users/rburhum/.virtualenvs/test_sqlite/bin/postdeactivate
virtualenvwrapper.user_scripts creating /Users/rburhum/.virtualenvs/test_sqlite/bin/preactivate
virtualenvwrapper.user_scripts creating /Users/rburhum/.virtualenvs/test_sqlite/bin/postactivate
virtualenvwrapper.user_scripts creating /Users/rburhum/.virtualenvs/test_sqlite/bin/get_env_details
(test_sqlite)rburhum@peru.local ~ $ brew test python
Testing python
==> /usr/local/Cellar/python/2.7.3/bin/python -c import sqlite3
==> /usr/local/Cellar/python/2.7.3/bin/python -c import Tkinter; root = Tkinter.
(test_sqlite)rburhum@peru.local ~ $ python -c "import sqlite3"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sqlite3/__init__.py", line 24, in <module>
from dbapi2 import *
File "/usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/sqlite3/dbapi2.py", line 27, in <module>
from _sqlite3 import *
ImportError: dlopen(/Users/rburhum/.virtualenvs/test_sqlite/lib/python2.7/lib-dynload/_sqlite3.so, 2): Library not loaded: /usr/local/lib/libsqlite3.0.8.6.dylib
Referenced from: /Users/rburhum/.virtualenvs/test_sqlite/lib/python2.7/lib-dynload/_sqlite3.so
Reason: image not found
(test_sqlite)rburhum@peru.local ~ $ brew link -f sqlite
Linking /usr/local/Cellar/sqlite/3.7.15.2... 9 symlinks created
(test_sqlite)rburhum@peru.local ~ $ python -c "import sqlite3"
(test_sqlite)rburhum@peru.local ~ $ |
|
Hmmmm ... I can't reproduce. Make sure you |
|
Still same behavior. Full log here: |
|
Okay does the virtualenv cache the python stdlib somehow? I deleted (due to other reasons my virtualenvs dir). Test wise you could set the ENV var where your virtualenvs are located Just please don't delete your venv :-) |
|
Did this also, no luck :( |
|
Any idea? What are your virtualenv ENV vars (from your shell's configuration script) like? Can you post them? |
|
Just wanted to report success after following clsung's comment, I first noticed sqlite problem when running ipython. Sorry if this is a noob question, is there any reason that sqlite shared library cannot be built/added after Python is already installed as part of "brew install sqlite3"? does Python link to static library if sqlite3 is not properly present? |
|
Not sure. |
|
I'm not sure if this is any help, but I encountered this problem a few months ago. IPython (which uses sqlite3 for history tracking) worked fine until I did a "brew update" and updated sqlite from 3.7.14 to 3.7.15.1. I reverted to the previous formula, returning to 3.7.14, and all was fine again. I presumed this indicated a problem with 3.7.15 and not with homebrew. Today I did a "brew update," and unlinked sqlite and then installed the current version. IPython gave a missing sqlite warning. I uninstalled python and re-installed it. IPython now works fine. So indeed the problem does appear to have been fixed. I'm running OS 10.7.5. |
|
Thanks for the round up. Absolutely correct. I made sqlite "keg_only" Sorry for the trouble. |
Symptom:
python -c import sqlitefails:Cause: Python build silently passes over a
_sqlitemodule build error. It's a large log that @samueljohn requested, the important bits are around line 918 inpython-install.log.sqlite-install.logis at the bottom.I checked to make sure that Python is not linking against system python:
Any ideas?
The text was updated successfully, but these errors were encountered: