Parsing error elftools.common.exceptions.ELFParseError: expected 4, found 0

[dhondta]

I get an error when trying to parse a simple hello-world ELF.

Steps to reproduce:

$ echo "f0VMRgEBAUhpIFdvcmxkCgIAAwABAAAAgIAECDQAAAAAuAQAAADNgOtYIAACACgABQAEAAEAAAAAAAAAAIAECACABAiiAAAAogAAAAUAAAAAEAAAAQAAAKQAAACkkAQIpJAECAkAAAAJAAAAugkAAAC5B5AECLsBAAAA66QAAADr6rsAAAAAuAEAAADNgA==" | base64 -d > hello-world

$ chmod +x hello-world

$ ./hello world
Hi World

$ xxd hello-world 
00000000: 7f45 4c46 0101 0148 6920 576f 726c 640a  .ELF...Hi World.
00000010: 0200 0300 0100 0000 8080 0408 3400 0000  ............4...
00000020: 00b8 0400 0000 cd80 eb58 2000 0200 2800  .........X ...(.
00000030: 0500 0400 0100 0000 0000 0000 0080 0408  ................
00000040: 0080 0408 a200 0000 a200 0000 0500 0000  ................
00000050: 0010 0000 0100 0000 a400 0000 a490 0408  ................
00000060: a490 0408 0900 0000 0900 0000 ba09 0000  ................
00000070: 00b9 0790 0408 bb01 0000 00eb a400 0000  ................
00000080: ebea bb00 0000 00b8 0100 0000 cd80       ..............

$ readelf -a hello-world 
ELF Header:
  Magic:   7f 45 4c 46 01 01 01 48 69 20 57 6f 72 6c 64 0a 
  Class:                             ELF32
  Data:                              2\'s complement, little endian
  Version:                           1 (current)
  OS/ABI:                            <unknown: 48>
  ABI Version:                       105
  Type:                              EXEC (Executable file)
  Machine:                           Intel 80386
  Version:                           0x1
  Entry point address:               0x8048080
  Start of program headers:          52 (bytes into file)
  Start of section headers:          309248 (bytes into file)
  Flags:                             0x80cd0000
  Size of this header:               22763 (bytes)
  Size of program headers:           32 (bytes)
  Number of program headers:         2
  Size of section headers:           40 (bytes)
  Number of section headers:         5
  Section header string table index: 4
readelf: Error: Reading 200 bytes extends past end of file for section headers
readelf: Error: Section headers are not available!

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  LOAD           0x000000 0x08048000 0x08048000 0x000a2 0x000a2 R E 0x1000
  LOAD           0x0000a4 0x080490a4 0x080490a4 0x00009 0x00009  W  0x9007b900

There is no dynamic section in this file.

$ python3
[...]
>>> from elftools.elf.elffile import ELFFile
>>> with open("hello-world", 'rb') as f:
...     elf = ELFFile(f)
... 
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/elftools/construct/core.py", line 351, in _parse
    return self.packer.unpack(_read_stream(stream, self.length))[0]
  File "/usr/local/lib/python3.8/dist-packages/elftools/construct/core.py", line 293, in _read_stream
    raise FieldError("expected %d, found %d" % (length, len(data)))
elftools.construct.core.FieldError: expected 4, found 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/elftools/common/utils.py", line 40, in struct_parse
    return struct.parse_stream(stream)
  File "/usr/local/lib/python3.8/dist-packages/elftools/construct/core.py", line 190, in parse_stream
    return self._parse(stream, Container())
  File "/usr/local/lib/python3.8/dist-packages/elftools/construct/core.py", line 647, in _parse
    subobj = sc._parse(stream, context)
  File "/usr/local/lib/python3.8/dist-packages/elftools/construct/core.py", line 353, in _parse
    raise FieldError(ex)
elftools.construct.core.FieldError: expected 4, found 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/dist-packages/elftools/elf/elffile.py", line 88, in __init__
    self._get_section_header_stringtable()
  File "/usr/local/lib/python3.8/dist-packages/elftools/elf/elffile.py", line 663, in _get_section_header_stringtable
    header=self._get_section_header(stringtable_section_num),
  File "/usr/local/lib/python3.8/dist-packages/elftools/elf/elffile.py", line 526, in _get_section_header
    return struct_parse(
  File "/usr/local/lib/python3.8/dist-packages/elftools/common/utils.py", line 42, in struct_parse
    raise ELFParseError(str(e))
elftools.common.exceptions.ELFParseError: expected 4, found 0

[sevaa]

It would be easier for the community to troubleshoot it if the OP did the fundamental legwork of hand parsing the sample. Specifically, what's that file like in hex, and how does that map to the ELF header structures. From looking at the Base64 representation, it's not immediately obvious that the ELF is well formed and pyelftools is to blame.

[dhondta]

Hi @sevaa , thank you for your reaction.
The reasons why I'm using this excellent library is that I'm not that experienced at parsing ELF header structures manually and that it is precisely aimed to parse it in an automated way.
While reporting this issue, I expect to get helped, not criticized. Could you tell me what is the difficulty of base64-decoding the provided sample and inspecting it as hex ? Is it really worth adding despite simplicity ?
I do not consider pyelftools is to blame, I only report an exception...

Ultimately, can somebody help fix this, please ?

[sevaa]

Readelf is telling you that the file is not well formed:

readelf: Error: Reading 200 bytes extends past end of file for section headers
readelf: Error: Section headers are not available!

The readelf output suggests that the ELF header claims 5 sections in file. That field is the 2 byte value at 0x30. There is indeed 5. That means there should be a section header table. Meanwhile, there is no such thing. Pyelftools tells you the same by throwing an exception (maybe with less details :)). To me, that suggests the problem is not with pyelftools, it's with the file. Where did you get this file data in the first place? It looks like the beginning of a larger, valid ELF file with the end cut off. The ELF format doesn't allow for that. Even XML with the end cut off stops being valid. :)

There is no criticism, only advice regarding the expectations. Please note this is the issue tracker specifically for pyelftools, not a general purpose forum devoted to ELF parsing (that would be StackOverflow :)). If you are creating an issue here, you are saying something is wrong with pyelftools. That's how issue trackers work. The burden of proof that pyelftools is indeed working incorrectly is on you.

If your issue is that the error message of the exception that pyelftools is throwing on this particular malformed file is not sufficiently descriptive, please say so. Pyelftools throwing an exception on a bad ELF file is not a bug.

[dhondta]

Where did you get this file data in the first place?

That was the right question to raise ;-)
I found this tiny ELF here.
While this executable runs normally with no error, and as you pointed out, it violates the ELF format. But as it runs and I get an exception with pyelftools, that's why I reported it here.

So, in pyelftools, shoudn't it be a simple warning instead of an exception ?

[sevaa]

The final sentence of that page's preamble says:

The final result is a completely corrupted x86 ELF Binary that still runs.

The file's author freely admits it's malformed. The fact that the ELF loader in Ubuntu Linux forgives this particular deviation from the format is an undocumented implementation detail. In a future version of Linux, they might remove this loophole.

So, in pyelftools, shoudn't it be a simple warning instead of an exception?

As a general rule, parser libraries are not expected to gracefully handle data that don't conform to the format. They throw exceptions instead.

On a side note, how exactly do you expect pyelftools to "do a warning"?

[eliben]

Thanks for helping investigate this, @sevaa

Seems like it's WAI.

[dhondta]

WAI ?...

[sevaa]

Works As Intended.

[dhondta]

OK, my bad.
@sevaa Thank you for your help and patience. As an answer to your question, this could be done for instance with logging but I understand this is a bad choice in this case. The current implementation choice seems totally logical. I wonder if other Linux distributions would run this tiny malformed ELF with no error. No time for testing unfortunately. That being said, thank you for your support.