New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ujson won't import on FreeBSD #180

Closed
ollyc opened this Issue Aug 5, 2015 · 12 comments

Comments

Projects
None yet
10 participants
@ollyc

ollyc commented Aug 5, 2015

I can't use ujson on FreeBSD 10.1.

$ virtualenv --python python3.4 env
Running virtualenv with interpreter /usr/local/bin/python3.4
Using base prefix '/usr/local'
New python executable in env/bin/python3.4
Also creating executable in env/bin/python
Installing setuptools, pip, wheel...done.

$ ./env/bin/pip install ujson
You are using pip version 7.0.3, however version 7.1.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting ujson
  Downloading ujson-1.33.zip (197kB)
    100% |████████████████████████████████| 200kB 1.4MB/s 
Building wheels for collected packages: ujson
  Running setup.py bdist_wheel for ujson
  Stored in directory: /home/oliver/.cache/pip/wheels/c0/b7/13/e66a0ddecdba15cf5178d05a3eda99d57591ae4f936f367793
Successfully built ujson
Installing collected packages: ujson
Successfully installed ujson-1.33

$ ./env/bin/python -c "import ujson"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: /tmp/env/lib/python3.4/site-packages/ujson.so: Undefined symbol "strreverse"

$ uname -a
FreeBSD myhost 10.1-RELEASE-p16 FreeBSD 10.1-RELEASE-p16 #0: Tue Jul 28 12:04:19 UTC 2015     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64
@jskorpan

This comment has been minimized.

Show comment
Hide comment
@jskorpan

jskorpan Aug 21, 2015

Member

That's weird because the strreverse function is defined in the source coude and shouln't resolve to an external dependency.

Could you try modifying the source and turn:
FASTCALL_ATTR INLINE_PREFIX void FASTCALL_MSVC strreverse(char* begin, char* end)

INTO

void strreverse(char* begin, char* end)

Member

jskorpan commented Aug 21, 2015

That's weird because the strreverse function is defined in the source coude and shouln't resolve to an external dependency.

Could you try modifying the source and turn:
FASTCALL_ATTR INLINE_PREFIX void FASTCALL_MSVC strreverse(char* begin, char* end)

INTO

void strreverse(char* begin, char* end)

@ollyc

This comment has been minimized.

Show comment
Hide comment
@ollyc

ollyc Aug 21, 2015

I just tried that and now I get a different error:

Python 3.4.3 (default, Aug  6 2015, 01:15:54) 
[GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final 208032)] on freebsd10
Type "help", "copyright", "credits" or "license" for more information.
>>> import ujson
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: /tmp/ultrajson/build/lib.freebsd-10.1-RELEASE-p16-amd64-3.4/ujson.so: Undefined symbol "Buffer_AppendShortHexUnchecked"

I just checked a couple of other python versions and this problem doesn't occur with other python versions (2.7, 3.3) on the same machine.

ollyc commented Aug 21, 2015

I just tried that and now I get a different error:

Python 3.4.3 (default, Aug  6 2015, 01:15:54) 
[GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final 208032)] on freebsd10
Type "help", "copyright", "credits" or "license" for more information.
>>> import ujson
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: /tmp/ultrajson/build/lib.freebsd-10.1-RELEASE-p16-amd64-3.4/ujson.so: Undefined symbol "Buffer_AppendShortHexUnchecked"

I just checked a couple of other python versions and this problem doesn't occur with other python versions (2.7, 3.3) on the same machine.

@zloidemon

This comment has been minimized.

Show comment
Hide comment
@zloidemon

zloidemon Sep 9, 2015

I have same issue on FreeBSD 10.2.

zloidemon commented Sep 9, 2015

I have same issue on FreeBSD 10.2.

@zloidemon

This comment has been minimized.

Show comment
Hide comment
@zloidemon

zloidemon Sep 9, 2015

It works with that patch:

diff --git a/lib/ultrajsonenc.c b/lib/ultrajsonenc.c
index 47016f6..2f1e867 100644
--- a/lib/ultrajsonenc.c
+++ b/lib/ultrajsonenc.c
@@ -148,7 +148,7 @@ void Buffer_Realloc (JSONObjectEncoder *enc, size_t cbNeeded)
   enc->end = enc->start + newSize;
 }

-FASTCALL_ATTR INLINE_PREFIX void FASTCALL_MSVC Buffer_AppendShortHexUnchecked (char *outputOffset, unsigned short value)
+void Buffer_AppendShortHexUnchecked (char *outputOffset, unsigned short value)
 {
   *(outputOffset++) = g_hexChars[(value & 0xf000) >> 12];
   *(outputOffset++) = g_hexChars[(value & 0x0f00) >> 8];
@@ -491,7 +491,7 @@ int Buffer_EscapeStringValidated (JSOBJ obj, JSONObjectEncoder *enc, const char
 #define Buffer_AppendCharUnchecked(__enc, __chr) \
                 *((__enc)->offset++) = __chr; \

-FASTCALL_ATTR INLINE_PREFIX void FASTCALL_MSVC strreverse(char* begin, char* end)
+void strreverse(char* begin, char* end)
 {
   char aux;
   while (end > begin)

zloidemon commented Sep 9, 2015

It works with that patch:

diff --git a/lib/ultrajsonenc.c b/lib/ultrajsonenc.c
index 47016f6..2f1e867 100644
--- a/lib/ultrajsonenc.c
+++ b/lib/ultrajsonenc.c
@@ -148,7 +148,7 @@ void Buffer_Realloc (JSONObjectEncoder *enc, size_t cbNeeded)
   enc->end = enc->start + newSize;
 }

-FASTCALL_ATTR INLINE_PREFIX void FASTCALL_MSVC Buffer_AppendShortHexUnchecked (char *outputOffset, unsigned short value)
+void Buffer_AppendShortHexUnchecked (char *outputOffset, unsigned short value)
 {
   *(outputOffset++) = g_hexChars[(value & 0xf000) >> 12];
   *(outputOffset++) = g_hexChars[(value & 0x0f00) >> 8];
@@ -491,7 +491,7 @@ int Buffer_EscapeStringValidated (JSOBJ obj, JSONObjectEncoder *enc, const char
 #define Buffer_AppendCharUnchecked(__enc, __chr) \
                 *((__enc)->offset++) = __chr; \

-FASTCALL_ATTR INLINE_PREFIX void FASTCALL_MSVC strreverse(char* begin, char* end)
+void strreverse(char* begin, char* end)
 {
   char aux;
   while (end > begin)
@Jahaja

This comment has been minimized.

Show comment
Hide comment
@Jahaja

Jahaja Jan 21, 2016

Member

Installed ultrajson on FreeBSD 10.2 using Virtualbox and could not reproduce this with 1.35. Reopen if this is still an issue.

Member

Jahaja commented Jan 21, 2016

Installed ultrajson on FreeBSD 10.2 using Virtualbox and could not reproduce this with 1.35. Reopen if this is still an issue.

@Jahaja Jahaja closed this Jan 21, 2016

xen0n added a commit to xen0n/ultrajson that referenced this issue May 30, 2016

Also mark the inline functions as always_inline
`inline` is not a requirement but only a suggestion, hence the compiler
has the freedom to not really compile the marked functions as inline.
Under some situations the affected functions could end up undefined in
the compiled library, causing its loading to fail!

For GCC at least, adding an `always_inline` attribute is enough to force
inlining. Not sure if this breaks other compilers but I cannot test, so
the definition is guarded by a check for GCC.

Really fixes esnme#180, which is also a problem on, at least, Gentoo with GCC
5.3.0.

WGH- added a commit to WGH-/ultrajson that referenced this issue Aug 27, 2016

added "static" to C functions, where possible
1. It reduces clutter in symbol table.
2. It fixes issues with C99 inline semantics for functions
   marked as inline (esnme#237, esnme#180, esnme#222), which manifests
   when compiled with GCC>=5.
@baverman

This comment has been minimized.

Show comment
Hide comment
@baverman

baverman Sep 7, 2016

Same error:

(boo) [abobrov@abobrov ~]$ pip install ultrajson
Collecting ultrajson
Installing collected packages: ultrajson
Successfully installed ultrajson-1.35
(boo) [abobrov@abobrov ~]$ python
Python 2.7.8 (default, Apr 13 2016, 14:03:40) 
[GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final 208032)] on freebsd10
Type "help", "copyright", "credits" or "license" for more information.
>>> import ultrajson
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: /home/abobrov/boo/lib/python2.7/site-packages/ultrajson.so: Undefined symbol "strreverse"
(boo) [abobrov@abobrov ~]$ uname -a
FreeBSD 10.2-STABLE-r286912 FreeBSD 10.2-STABLE-r286912 #0 r286912M: Wed Aug 19 13:15:14 UTC 2015 

baverman commented Sep 7, 2016

Same error:

(boo) [abobrov@abobrov ~]$ pip install ultrajson
Collecting ultrajson
Installing collected packages: ultrajson
Successfully installed ultrajson-1.35
(boo) [abobrov@abobrov ~]$ python
Python 2.7.8 (default, Apr 13 2016, 14:03:40) 
[GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final 208032)] on freebsd10
Type "help", "copyright", "credits" or "license" for more information.
>>> import ultrajson
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: /home/abobrov/boo/lib/python2.7/site-packages/ultrajson.so: Undefined symbol "strreverse"
(boo) [abobrov@abobrov ~]$ uname -a
FreeBSD 10.2-STABLE-r286912 FreeBSD 10.2-STABLE-r286912 #0 r286912M: Wed Aug 19 13:15:14 UTC 2015 
@christopher-abastar

This comment has been minimized.

Show comment
Hide comment
@christopher-abastar

christopher-abastar Sep 19, 2016

Same error:

ImportError: /home/xx/env/lib/python3.5/site-packages/ujson.so: Undefined symbol "strreverse"

uname -a
10.3-RELEASE-p7 FreeBSD 10.3-RELEASE-p7 #0: Thu Aug 11 18:38:15 UTC 2016 root@amd64-builder.daemonolo

christopher-abastar commented Sep 19, 2016

Same error:

ImportError: /home/xx/env/lib/python3.5/site-packages/ujson.so: Undefined symbol "strreverse"

uname -a
10.3-RELEASE-p7 FreeBSD 10.3-RELEASE-p7 #0: Thu Aug 11 18:38:15 UTC 2016 root@amd64-builder.daemonolo

@xmj

This comment has been minimized.

Show comment
Hide comment
@xmj

xmj Dec 12, 2016

Same issue with ujson 1.35, on FreeBSD 12.0-CURRENT with python 3.5.

xmj commented Dec 12, 2016

Same issue with ujson 1.35, on FreeBSD 12.0-CURRENT with python 3.5.

@mohd-akram

This comment has been minimized.

Show comment
Hide comment
@mohd-akram

mohd-akram Feb 5, 2017

Have the same issue on FreeBSD 11.0-RELEASE-p1 with Python 3.5.

mohd-akram commented Feb 5, 2017

Have the same issue on FreeBSD 11.0-RELEASE-p1 with Python 3.5.

@mohd-akram

This comment has been minimized.

Show comment
Hide comment
@mohd-akram

mohd-akram Feb 5, 2017

The version on master works, but not ujson 1.35. If you could release a new version, that would be great :)

mohd-akram commented Feb 5, 2017

The version on master works, but not ujson 1.35. If you could release a new version, that would be great :)

cordalace added a commit to cordalace/ultrajson that referenced this issue Nov 13, 2017

added "static" to C functions, where possible
1. It reduces clutter in symbol table.
2. It fixes issues with C99 inline semantics for functions
   marked as inline (esnme#237, esnme#180, esnme#222), which manifests
   when compiled with GCC>=5.

Patch info:
    Author: WGH <wgh@torlan.ru>
    Date:   Sat Aug 27 17:34:22 2016 +0300
@tshirtman

This comment has been minimized.

Show comment
Hide comment
@tshirtman

tshirtman May 22, 2018

Had the same issue on gentoo, installing from master did solve the issue.

tshirtman commented May 22, 2018

Had the same issue on gentoo, installing from master did solve the issue.

@jbandlow

This comment has been minimized.

Show comment
Hide comment
@jbandlow

jbandlow Jun 17, 2018

I also hit this (on termux) and to be explicit for others, installing directly from Github solved it:

pip install git+git://github.com/esnme/ultrajson.git

jbandlow commented Jun 17, 2018

I also hit this (on termux) and to be explicit for others, installing directly from Github solved it:

pip install git+git://github.com/esnme/ultrajson.git
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment