No module named '_bz2'

Trying to get https://github.com/Blizzard/heroprotocol running via Blizzard/heroprotocol#59

It includes a new version of mpyp (the library to read MPQ files), but it bails on No module named '_bz2'

Which is used in mpyq import bz2

I see in SciPy also bz2 is imported, any issues with that?

This is my code

package heroes

import org.graalvm.polyglot._

object Main extends App {

  val context = Context.newBuilder().allowIO(true).build
  val v = context.getPolyglotBindings()
  val _ = context.eval("python",
    """
      |from heroprotocol import protocol29406
      |from heroprotocol.mpyq import mpyq
      |
      |archive = mpyq.MPQArchive('/path/to/replay')
      |contents = archive.header['user_data_header']['content']
      |header = protocol.decode_replay_header(contents)
      |
      |# The header's baseBuild determines which protocol to use
      |baseBuild = header['m_version']['m_baseBuild']
      |try:
      |    protocol = __import__('protocol%s' % (baseBuild,))
      |except:
      |    print >> sys.stderr, 'Unsupported base build: %d' % baseBuild
      |    sys.exit(1)
      |
      |""".stripMargin)

}

Can put up a repo if you want?

[fangerer]

Hi Mark,

thank you for reporting this issue. There are good chances that we can support bz2 at least partially in the near future. As you mentioned, since it seems to be used in SciPy also, I'll look into this but I cannot promise anything.

[fangerer]

Hi Mark,

I'm happy to announce that we added basic support the bz2 module. It should now be possible to do simple compress/decompress operations. In order to be able to test this, you need to add a command line option: --llvm.libraries=libbz2.so. And, of course, your system is assumed to have libbz2 installed.

Unfortunately, not all of the original CPython tests are passing but this is also due to interaction with other modules we do not fully support yet.

I could also not confirm that your example is now working because the heroprotocol module seemingly needs Python 2. I assume you have your own fork of this module.

[Fristi]

That's cool! Should I download a new release of Graal or is the command line parameter enough?

A fork of heroprotocol which supposed to work with Python 3 can be found here https://github.com/petronny/heroprotocol

[fangerer]

The necessary changes are in our master branch. If you build Graalpython from GitHub, you should have the changes.
If you use GraalVM, then you need to wait for the next release. The next release is expected on Feb 1st (no warranty!).

[timfel]

The rc12 release now supports importing bz2

[Fristi]

With some tweaks I can confirm I am able to use https://github.com/petronny/heroprotocol 🎉

Some changes

• I had to replace ord(byte_array[0:1]) with struct.unpack('B', byte_array[0:1])
• The sources I'm working with are in a folder heroprotocol. The protocol python scripts all do import decoders from *. This doesn't work, I had to change it to: from heroprotocol.decoders import * . I think it's due the folder.
• for k,v in value.iteritems() didn't work and replaced with for k,v in value.items()

[timfel]

The ord issue is a bug on our end. I'm not sure about the import. The last issue is a Python2 -> Python3 change, Python3 does not have iteritems() on dict anymore