Adding pypandoc-binary for linux

[AbdealiLoKo]

Hi, following up from #279

I was just installing pypandoc in a linux setup and thought I could use pypandoc-binary
but then got the error: ERROR: Could not find a version that satisfies the requirement pypandoc-binary (from versions: none)
And noticed that wheels for manylinux are not present

Was wondering if there was any reason not to add linux wheels ? Or is that something that can be easily done ?

[JessicaTegner]

@AbdealiJK thanks for opening this.

Since I don't know that much about linux, I had a hard time figuring out a way to make a manylinux binary that included pandoc, since pandoc comes in both dep and tar.gz options, and arm and amd.

If you have a suggestion, I would be more than happy to take another look at it :)

[AbdealiLoKo]

Hi @JessicaTegner
I see - I can help out with Linux if you like.

The steps for Linux involve:

1. Starting a docker container using the manylinuz docker image
2. Building the wheel inside the docker
3. Running 'auditwheel' to repair the wheel and make it manylinux compatible
4. Finally uploading this using twine with your pypi credentials

It's documented at: https://github.com/pypa/manylinux

If you would like to try it, I can help debug.
Or if you like I could attempt it but would need some guidance on how you want to set it up (as it requires your twine credentials)

[JessicaTegner]

@AbdealiJK If you look here https://github.com/JessicaTegner/pypandoc/actions/runs/3338893037

This is the latest run I have tried to implemented this with.
Basically from your steps, as you can see in the workflow we'll just have to:

• Start a docker container and build the wheel.
• Upload the artifacts to GitHub Actions.

Then the next couple jobs, will take care of publishing to gh releases and pypi.

[AbdealiLoKo]

@JessicaTegner That's awesome. Considering you already have the entire CI setup done up with github actions - I think we can just use the cibuildwheel tool's github action to generate this.

I have created a PR - #309

I found that there are 2 build systems in pypandoc:

• poetry is configured in pyproject.toml
• setup.py and setup_binary.py use setuptools
  And cibuildwheel will prioritize pyproject.toml's build-system as that is the standard in python now.
  So, I had to delete the pyproject.toml for it to work.

It looks like you're in the middle of moving to poetry - so, maybe that gets resolved once you fully move to poetry.

[AbdealiLoKo]

@JessicaTegner One question though before you do a release - you're aware that there is some issue with pandoc-citeproc when the wheels are getting built ?

  [INFO] Copying pandoc to /Users/runner/work/pypandoc/pypandoc/pypandoc/files ...
  Copying pandoc to /Users/runner/work/pypandoc/pypandoc/pypandoc/files ...
  [INFO] Making /Users/runner/work/pypandoc/pypandoc/pypandoc/files/pandoc executeable...
  Making /Users/runner/work/pypandoc/pypandoc/pypandoc/files/pandoc executeable...
  [INFO] Copying pandoc-citeproc to /Users/runner/work/pypandoc/pypandoc/pypandoc/files ...
  Copying pandoc-citeproc to /Users/runner/work/pypandoc/pypandoc/pypandoc/files ...
  Error:  Didn't copy pandoc-citeproc
  Traceback (most recent call last):
    File "/Users/runner/work/pypandoc/pypandoc/pypandoc/pandoc_download.py", line 156, in _handle_darwin
      shutil.copyfile(src, dst)
    File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/shutil.py", line 254, in copyfile
      with open(src, 'rb') as fsrc:
  FileNotFoundError: [Errno 2] No such file or directory: '/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpvd845m_m/tmp/usr/local/bin/pandoc-citeproc'
  Didn't copy pandoc-citeproc
  Traceback (most recent call last):
    File "/Users/runner/work/pypandoc/pypandoc/pypandoc/pandoc_download.py", line 156, in _handle_darwin
      shutil.copyfile(src, dst)
    File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/shutil.py", line 254, in copyfile
      with open(src, 'rb') as fsrc:
  FileNotFoundError: [Errno 2] No such file or directory: '/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpvd845m_m/tmp/usr/local/bin/pandoc-citeproc'
  [INFO] Done.

I didn't try looking into it cause it was failing in the test CI Download pandoc step

[JessicaTegner]

@AbdealiJK that's because since (I think pandoc 2.10) pandoc citeproc has been bundled into the main pandoc executable.
Maybe it would be a good idea to fix that with a pretty error message, or something like that.
Nothing is really going wrong, it's just because it can't find citeproc, which is expected.

[JessicaTegner]

Just to clarify. This is not a wheel issues...
This isn't really an error at all.

I looked into it, and it's because we use "logger.exception" to print our error message, which results in the stack trace to be printed automatically. A solution to this, would be to change all "logger.exception" in the various platform handlers to warning level instead... Or well, even just info, since it's expected behaviour unless you are using a very old pandoc.

[JessicaTegner]

With #311 out of the way, everything should be done.
@AbdealiJK if you have no further comments, I'll make a new release

[AbdealiLoKo]

Im good with the changes for wheels :+1 Thanks
I was looking through the source and was trying to do some minor cleanups
Mainly to drop old code from py2 (As we now have py3.6+)
It is not required in a release though ... So, we can go ahead with the current wheels publishing

I will create some PRs for the other cleanups so you can take a look ina while

[JessicaTegner]

Sounds good. Thanks for all your contributions

[JessicaTegner]

It seems I spoke too soon @AbdealiJK

take a look at this run: https://github.com/JessicaTegner/pypandoc/actions/runs/3346455236
And this run: https://github.com/JessicaTegner/pypandoc/actions/runs/3346157814

It seems it refuses to even start the release job at all. Can you take a look at this perhaps?

[AbdealiLoKo]

Checking

[AbdealiLoKo]

Created #313 which I THINK will solve the issue.
I am not very familiar with github actions - so it's based on the documentation I have rea about github actions.

I believe the conditions at ad520d1#diff-944291df2c9c06359d37cc8833d182d705c9e8c3108e7cfe132d61a06e9133ddR73 was not right
I explained the reasoning in the commit message

[AbdealiLoKo]

Closing as Linux 64bit wheels are present in pypandoc-binary 1.10
And pandoc does not have releases for Linux 32bit - so pypandoc-binary also does not have wheels for it