CoreHdfsFile init expects non-PyObjects

[simleo]

At the C level, the open_file method is based on a hack where the libhdfs C structs for the fs and file objects are passed directly to the init method of CoreHdfsFile (a Python object), taking advantage of the fact that the "O" format string in PyArg_Parse* doesn't perform any actual type checking, but rather stores the object as-is. This means that all hell can break loose if a PyObject is passed to the method and, later on, anything is attempted that tries to use the self->file or self->fs refs.

We don't see this much in practice because directly instantiating CoreHdfsFile is explicitly discouraged in the docs, but it's still possible (and not too hard) to break the interpreter due to this.

For instance, this is OK:

import pydoop.hdfs as hdfs

f = hdfs.open("foo")
print(f.readline())
f.close()

But this:

import pydoop.hdfs as hdfs

f = hdfs.open("foo")
core_file = f.f.raw
core_fs = f.fs.fs
core_file.__init__(core_fs, core_file, "hosts", "r")
print(f.readline())
f.close()

Crashes the JVM with a core dump.