macOS support, and Python dependencies

[mvasilkov]

Hi all, these are two related issues:

1. After running pip3 install blingfire in macOS, which appears to work fine, it's impossible to use the library since the .dylib isn't there. And the error message is rather meh: AttributeError: 'NoneType' object has no attribute 'TextToWords'

2. It appears to require NumPy, but doesn't specify it as a dependency, so running from a venv without numpy in it also fails.

Both can be fixed by tuning the setup.py metadata, but wrt macOS, can we please actually support it? I can't imagine it being very different from the Linux build, if at all.

Thanks for open sourcing this!

[jialigit]

I also find this problem of not supporting macOs platform. Really hope for the open source of it.

[SergeiAlonichau]

Looking into it.

[ma2rten]

Same problem:

$ python3 -m virtualenv env
Using base prefix '/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7'
/Users/bosma/Library/Python/3.7/lib/python/site-packages/virtualenv.py:1041: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
New python executable in /Users/bosma/projects/env/bin/python3.7
Also creating executable in /Users/bosma/projects/env/bin/python
Installing setuptools, pip, wheel...done.
$ source env/bin/activate
$ pip install blingfire
Collecting blingfire
Downloading https://files.pythonhosted.org/packages/6b/aa/f999e3ca3e826cbdefe7dea7f90baf06db26448ef4cfd80f4800d84d660f/blingfire-0.0.9-py3-none-any.whl (248kB)
|████████████████████████████████| 256kB 1.5MB/s
Installing collected packages: blingfire
Successfully installed blingfire-0.0.9
$ python
Python 3.7.0 (default, Aug 17 2018, 21:14:48)
[Clang 9.1.0 (clang-902.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

from blingfire import *
Traceback (most recent call last):
File "", line 1, in
File "/Users/bosma/projects/env/lib/python3.7/site-packages/blingfire/init.py", line 5, in
import numpy as np
ModuleNotFoundError: No module named 'numpy'
text = 'This is the Bling-Fire tokenizer'
output = text_to_words(text)
KeyboardInterrupt

$ pip install numpy
Collecting numpy
Downloading https://files.pythonhosted.org/packages/43/6e/71a3af8680a159a141fab5b4d19988111a09c02ffbfdeb42175cca0fa341/numpy-1.16.3-cp37-cp37m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl (13.9MB)
|████████████████████████████████| 13.9MB 2.1MB/s
Installing collected packages: numpy
Successfully installed numpy-1.16.3
$ python
Python 3.7.0 (default, Aug 17 2018, 21:14:48)
[Clang 9.1.0 (clang-902.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

from blingfire import *
text = 'This is the Bling-Fire tokenizer'
output = text_to_words(text)
Traceback (most recent call last):
File "", line 1, in
File "/Users/bosma/projects/env/lib/python3.7/site-packages/blingfire/init.py", line 58, in text_to_words
o_len = blingfire.TextToWords(c_char_p(s_bytes), c_int(len(s_bytes)), byref(o_bytes), c_int(o_bytes_count))
AttributeError: 'NoneType' object has no attribute 'TextToWords'

[sandeepeecs]

Looking into it.

Any suggested fixes to make it work no macOS?

[ZPVIP]

there is no mac OS support:

BlingFire/scripts/blingfiretok.py

Lines 14 to 22 in 52a5d30

	# detect windows
	if platform.system() == "Windows":
	blingfire = cdll.LoadLibrary(os.path.join(path, "blingfiretokdll.dll"))
	# detect Mac OSX
	elif platform.system() == "Darwin":
	pass
	else:
	# detect linux
	blingfire = cdll.LoadLibrary(os.path.join(path, "libblingfiretokdll.so"))

[ankane]

fwiw, I was able to get it working on Mac with the following changes: master...ankane:mac. There's probably a cleaner way to do it.

[SergeiAlonichau]

Thank you, I will look at it today.

…

On Mon, Feb 24, 2020, 2:23 AM Andrew Kane ***@***.***> wrote: fwiw, I was able to get it working on Mac with the following changes: master...ankane:mac <master...ankane:mac>. There's probably a cleaner way to do it. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#9?email_source=notifications&email_token=AFYTXHPFJJ7XZ33HFRJQ2FTREON3TA5CNFSM4HG2S3RKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMXIY7A#issuecomment-590253180>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFYTXHOWMHUF3OBNA3JYWSTREON3TANCNFSM4HG2S3RA> .

[SergeiAlonichau]

Thank you, Andrew, for doing the change!

Starting from version 0.0.16 blingfire package supports Mac.

[ankane]

Awesome, thanks @SergeiAlonichau!