[FEATURE] Bokeh Python Wheel packages

[hacklschorsch]

Hi all,

for creating a package of our python application using the excellent Pynsist a .whl (Python Wheel) package would help us greatly.

Any reason there's no wheel available? Can I create one myself? Would you be interested in this?

Thanks,

Florian

[bryevdv]

Any reason there's no wheel available?

The purpose of wheels is to make it easier to distribute packages that have compiled or platform dependent characteristics. What is the purpose of having a wheel when there is no compiled code? Our release process is already very complicated by needing conda, CDN and NPM steps. Building and publishing wheels seems like extra work (for us) for little/no gain. We upload sdists to PyPI:

https://pypi.org/project/bokeh/#files

AFAIK you could use that to build a wheel if you need one.

cc @bokeh/dev

[hacklschorsch]

Thanks!

Wheels install faster, don't require the execution of code, and pynsist requires them for example. I tried building a wheel from bokeh, but I need a toolchain with node, npm and what not - could totally do it, but I would then also like upload my work or add to bokeh's CI to do it at release time or some such, to save others the hassle I am now going through.

My current workaround is to install bokeh in the venv which I am creating the installer from, and pynsist has a feature where it re-packages installed packages. But this forces me to setup up a venv with bokeh in it.

[bryevdv]

tried building a wheel from bokeh, but I need a toolchain with node, npm and what not -

These are not required in principle, as the sdist ships with all the BokehJS files already pre-built and included inside. But I don't know what modifications would be required to get bdist_wheel to use these instead of trying to start from scratch.

I personally mostly use conda for everything, so the best chance for any movement here is for someone with more interest and experience to push this forward. I would suggest starting by making it possible to build wheels from the sdist using the already-built BokehJS. Then, later we could discuss publishing wheels.

[bryevdv]

For reference, when building install or develop from an sdist, we force an otherwise undocumented --existing-js flag to use the existing packaged BokehJS files:

https://github.com/bokeh/bokeh/blob/branch-2.3/_setup_support.py#L177
https://github.com/bokeh/bokeh/blob/branch-2.3/_setup_support.py#L106

But I don't know how to make that also work automatically for bdist_wheel.

[bryevdv]

@hacklschorsch did you try building the wheel from the published sdist that I linked above? I specifically suggested building from the sdist on purpose. When I download the sdist from PyPI and run python setup.py bdist_wheel from the sdist it seems to work , and does not prompt for node or any BokehJS builds. Can you confirm the actual, exact steps that you took? (This is always advised, to prevent wasted communication)

[bryevdv]

@hacklschorsch will you be able to test wheel generation with the published sdist?

[hacklschorsch]

@hacklschorsch will you be able to test wheel generation with the published sdist?

@bryevdv Yes, should be easy enough. Sorry for letting you wait. Will do it on the weekend.

[bryevdv]

@hacklschorsch any update?

[hacklschorsch]

When I download the sdist from PyPI and run python setup.py bdist_wheel from the sdist it seems to work, and does not prompt for node or any BokehJS builds.

So does it here (Win10, 64 Bit, Python 3.7, cmder, using bokeh-2.2.3.tar.gz): python setup.py bdist_wheel generates a 8.9MB dist/bokeh-2.2.3-py3-none-any.whl as I would expect.

bokeh is BSD 3-Clause licensed, which means I can distribute that file in the installer of our (MIT-licensed) software. Not as cool as just telling the installer builder to fetch that wheel from online, but almost :)

Before I tried with the checked-out GIT repo; that didn't work. But it need not, for all I care.

Thank you!

[hacklschorsch]

Thanks again for your patience @bryevdv! I'd still love to have pre-built bokeh wheels online, for the above mentioned advantages. But you can close this issue if you don't think so.

Also, if you'd like me to check out how to integrate this in your CI pipline, I'll gladly do so.

Have a great weekend!

[bryevdv]

Also, if you'd like me to check out how to integrate this in your CI pipeline, I'll gladly do so.

Sure, we had to tear down our previous full release-automation to accommodate generating SRI hashes (our previous system was tag-initiiated but the hashes must be computed and checked in before the tag so that threw everything out the window). But I just got the "build" part of the automation back up and running. Our CI runs python -m release build <version> and the entry point to that is here:

https://github.com/bokeh/bokeh/blob/branch-2.3/release/__main__.py

If you follow that a bit you can see the function that builds the sdist, which is subsequently included in a "deployment tarball".:

bokeh/release/build.py

Lines 63 to 68 in 0f07317

	def build_sdist_packages(config: Config, system: System) -> ActionReturn:
	try:
	system.run("python setup.py sdist --install-js --formats=gztar")
	return PASSED("sdist package build succeeded")
	except RuntimeError as e:
	return FAILED("sdist package build did NOT succeed", details=e.args)

If you want to propose a step to build a wheel to add to the tarball that would go most of the way. It seems either the sdist can be untarred and used to generate a wheel (as above) or perhaps our setup.py can be modified to be able to generate a wheel directly (I guess that's "ideal" but don't know what that would take)

[hacklschorsch]

Just a quick status update - sorry for letting this sit around for three weeks. MentaLab who I help out with some coding decided to sponsor this. Nice! Will come around swiftly.

[bryevdv]

I will see what if anything can be done quickly/simply for 2.4 (no guarantees)

[kitmonisit]

May I add an upvote for a wheel as well. I'm using Poetry to manage my projects. This is what I get when I do a poetry show (truncated):

bokeh         (!) 2.3.1   Interactive plots and applications in the browser from Python
cursive-re        0.0.4   Readable regular expressions for Python 3.6 and up.
dill              0.3.3   serialize all of python

The exclamation point next to bokeh means that poetry was unable to build a wheel. Every time I add a new project dependency, say poetry add scikit-learn, it will repeat the entire installation process for bokeh.

[bryevdv]

@kitmonisit Do you mean install a wheel? Or build a wheel, from an sdist? Building bokeh from bare source requires an extensive non-Python toolchain to build the BohehJS library from typescript. It's never going to be a realistic goal for that to be as simple as it for "regular" python packages.

[kitmonisit]

@bryevdv Ah, sorry I wasn't clear. Poetry first tries to use a pre-built wheel for installation, and if it's not available, tries to build the wheel itself. What happened in my case is the latter (attempted to build the wheel locally). My upvote was for the availability of a pre-built wheel so that Poetry can just pull that in.

I can't say anything about sdist on behalf of Poetry, though, not very familiar with it.

[hacklschorsch]

Thank you very much @bryevdv !