Ghidra headless analysis failed

[xuing]

Thanks for the Project.
when I use GhIDA Decompiler, got a "Ghidra headless analysis failed" error.
many times with various files. same error.

GhIDA:: [INFO] Ghidra headless (timeout: 300s)
GhIDA:: [INFO] Waiting Ghidra headless analysis to finish...
GhIDA:: [!] 'module' object has no attribute 'killpg'
GhIDA:: [!] Ghidra headless analysis failed
GhIDA:: [!] Decompilation interrupted.

IDA Version： 7.2.181105 Windows
Python Version：Python 2.7.15
Ghidra Path ：C:\Program Files\ghidra_9.0.4
Java Version：jdk-11.0.4

and I also have this error

GhIDA:: [DEBUG] Reading GhIDA configuration
('GHIDA_CONF.load_save_cached_code', True)
('GHIDA_CONF.load_save_cached_comments', True)
GhIDA:: [DEBUG] code_cache_path: c:\users\edz\appdata\local\temp\decompiled_cache_3407694CA26AC630BBAD34D07BAFE340.json
GhIDA:: [DEBUG] loading decomp cache from json
GhIDA:: [!] error while loading code from c:\users\edz\appdata\local\temp\decompiled_cache_3407694CA26AC630BBAD34D07BAFE340.json
GhIDA:: [DEBUG] comments_cache_path: c:\users\edz\appdata\local\temp\comments_cache_3407694CA26AC630BBAD34D07BAFE340.json
GhIDA:: [DEBUG] loading comments cache from json
GhIDA:: [!] error while loading comments from c:\users\edz\appdata\local\temp\comments_cache_3407694CA26AC630BBAD34D07BAFE340.json
GhIDA:: [DEBUG] Registering handlers
GhIDA [DEBUG] ScreenEAHook initialized

running command line
"C:\Program Files\ghidra_9.0.4\support\analyzeHeadless.bat" . Temp -import C:\Users\EDZ\source\repos\ConsoleApplication1\test\3407694CA26AC630BBAD34D07BAFE340_JVLNU.xml -scriptPath "C:\Program Files\IDA 7.2\plugins\ghida_plugin\ghidra_plugin" -postScript FunctionDecompile.py 411357 c:\users\edz\appdata\local\temp\411357_5ddduh -noanalysis -deleteProject

[shogunlab]

I had the same issue on Windows, I believe that the problem has to do with line 266 in ghida_plugin\lib.py. The method killpg is used, when it should just be kill. This is hinted at by the error in the IDA console output GhIDA:: [!] 'module' object has no attribute 'killpg'.

After changing line 266 in ghida_plugin\lib.py from os.killpg(os.getpgid(p.pid), signal.SIGTERM) to os.kill(p.pid, -9), decompilation was successful for me. There may be a better way to deal with the error, but the change from killpg to kill appears to work for now. See below for additional info.

Before
ghida_plugin\lib.py, line 265-269

# Process timeout
if counter > COUNTER_MAX * 10:
    os.killpg(os.getpgid(p.pid), signal.SIGTERM)
    stop = True
    print("GhIDA:: [!] Decompilation error - timeout reached")
    continue

After
ghida_plugin\lib.py, line 265-269

# Process timeout
if counter > COUNTER_MAX * 10:
    os.kill(p.pid, -9)
    stop = True
    print("GhIDA:: [!] Decompilation error - timeout reached")
    continue

[leex2000]

I had the same issue on Windows, I believe that the problem has to do with line 266 in ghida_plugin\lib.py. The method killpg is used, when it should just be kill. This is hinted at by the error in the IDA console output GhIDA:: [!] 'module' object has no attribute 'killpg'.

After changing line 266 in ghida_plugin\lib.py from os.killpg(os.getpgid(p.pid), signal.SIGTERM) to os.kill(p.pid, -9), decompilation was successful for me. There may be a better way to deal with the error, but the change from killpg to kill appears to work for now. See below for additional info.

Before
ghida_plugin\lib.py, line 265-269

# Process timeout
if counter > COUNTER_MAX * 10:
    os.killpg(os.getpgid(p.pid), signal.SIGTERM)
    stop = True
    print("GhIDA:: [!] Decompilation error - timeout reached")
    continue

After
ghida_plugin\lib.py, line 265-269

# Process timeout
if counter > COUNTER_MAX * 10:
    os.kill(p.pid, -9)
    stop = True
    print("GhIDA:: [!] Decompilation error - timeout reached")
    continue

i get the problem too.
I have done like that，but the problem is still there.

GhIDA:: [INFO] Ghidra headless (timeout: 300s)
GhIDA:: [INFO] Waiting Ghidra headless analysis to finish...
GhIDA:: [INFO] Ghidra analysis completed!
GhIDA:: [!] No JSON object could be decoded
GhIDA:: [!] Ghidra headless analysis failed
GhIDA:: [!] Decompilation interrupted.

[xuing]

Yes, Just like @shogunlab said , killpg Is a issue. good job. it should be written like other statements to terminate process. compatible with multiple platforms as follows:

# Process timeout
if counter > COUNTER_MAX * 10:
    # Termiante the process!
    if os.name == 'posix':
        os.killpg(os.getpgid(p.pid), signal.SIGTERM)
    else:
        os.kill(p.pid, -9)
    stop = True
    print("GhIDA:: [!] Decompilation error - timeout reached")
    continue

(actually, termiante process should be a function.)

but it's not the core issue that why headless analysis failed . I think should figure out why it will time out .

[shogunlab]

Ah okay, yes it looks like the primary problem is the length of the decompilation from Ghidra, resulting in a timeout. I was able to successfully decompile very small functions, but got a timeout again after trying larger ones.

[starikman]

GhIDA:: [!] No JSON object could be decoded
I had the same problem when I had increased counter_max.
I dig deeply and realized something was going wrong with FunctionDecompile.py
I tested on Windows10 VM (python 2.7.16), so I created batch-file with same parameters to call headless analysis. I got success once I copied FunctionDecompile.py to same directory where xml-file was.
So I patched lib.py and error is gone. May be it will be back :)
https://gist.github.com/starikman/8f70be487196ed3391ded5b49d2399e0

[jimmy-sonny]

Thank you very much for pointing out this issue.

Yes, as @shogunlab correctly said, the length of the decompilation in Ghidra is the primary cause of the timeout.

It's possible to change the TIMEOUT value in lib.py. By default, it's set to 300 seconds, but it may be increased if needed.
Please, do not modify the value of COUNTER_MAX or SLEEP_LENGTH, since they are all related.

Then, as @xuing suggested, I've created a terminate_process function to handle both user and timeout interruption.

Regarding what @starikman is referring to, it's probably related to the issue of the relative path of FunctionDecompile.py. I've addressed this problem in 2c4f5db.

[fareedfauzi]

I've change the timeout value to 3000.
And "Ghida headless analysis failed" still happened. Anyone know?

Update:
I altered lib.py based fb7292e commit, the "Ghida decompilation started" takes very long time for a small function like picture below.

Update:
I update my Ghidra to latest version (9.1.2), now the Ghida decompilation works fine even for medium function! Thanks.