IDA-Headless .. Windows ?

[rtompsam]

I'm trying to run IDA-Headless on a windows environment and it fails to start . Any idea ?

C:\Program Files\IDA Pro 7.7>headless-ida ida64.exe "C:\Users\me\Desktop\IDAPYTHON\sample.exe"
Traceback (most recent call last):
File "c:\users\me\appdata\local\programs\python\python39\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "c:\users\me\appdata\local\programs\python\python39\lib\runpy.py", line 87, in run_code
exec(code, run_globals)
File "C:\Users\me\AppData\Local\Programs\Python\Python39\Scripts\headless-ida.exe_main.py", line 7, in
File "c:\users\me\appdata\local\programs\python\python39\lib\site-packages\headless_ida\cli.py", line 21, in headlessida_cli
headlessida = HeadlessIda(args.idat_path, args.binary_path)
File "c:\users\me\appdata\local\programs\python\python39\lib\site-packages\headless_ida\client.py", line 45, in init
raise Exception(
Exception: IDA failed to start: return code 1
=============== STDOUT ===============
=============== STDERR ===============

[DennyDai]

The error you're experiencing might be caused by the rpyc package not being installed in the Python environment that your IDA Pro is using. You can use idapyswitch (typically located in the same directory as your IDA Pro installation) to verify which Python version is being used by IDA Pro, and confirm that the rpyc package is installed for that version.

[rtompsam]

I just verified the version running on IDA and my Python Environment it's the same :

Tried running a script and also running the server no luck:

from headless_ida import HeadlessIdaRemote
headlessida = HeadlessIdaRemote("localhost", 1337, r"C:\Users\me\Desktop\test\test.exe")
Traceback (most recent call last):
File "", line 1, in
File "C:\Program Files\Python38\lib\site-packages\headless_ida\client.py", line 80, in init
self.conn.root.init(f.read())
File "C:\Program Files\Python38\lib\site-packages\rpyc\core\netref.py", line 240, in call
return syncreq(_self, consts.HANDLE_CALL, args, kwargs)
File "C:\Program Files\Python38\lib\site-packages\rpyc\core\netref.py", line 63, in syncreq
return conn.sync_request(handler, proxy, *args)
File "C:\Program Files\Python38\lib\site-packages\rpyc\core\protocol.py", line 718, in sync_request
return async_res.value
File "C:\Program Files\Python38\lib\site-packages\rpyc\core\async.py", line 108, in value
raise self._obj
Exception: IDA failed to start: return code 1

========= Remote Traceback (1) =========
Traceback (most recent call last):
File "c:\program files\python38\lib\site-packages\rpyc\core\protocol.py", line 359, in _dispatch_request
res = self._HANDLERS[handler](self, *args)
File "c:\program files\python38\lib\site-packages\rpyc\core\protocol.py", line 837, in _handle_call
return obj(*args, **dict(kwargs))
File "c:\program files\python38\lib\site-packages\headless_ida\server.py", line 26, in exposed_init
raise Exception(
Exception: IDA failed to start: return code 1

[DennyDai]

Could you please confirm whether you're able to successfully run import rpyc in the IDA Pro console?

If you're able to successfully execute import rpyc in the IDA Pro console, yet still encounter the same error with headless-ida, I recommend running the following command in terminal:
.\idat64.exe -S"c:\users\me\appdata\local\programs\python\python39\lib\site-packages\headless_ida\ida_script.py 54886" -P+ C:\Users\me\Desktop\IDAPYTHON\sample.exe
Then share the error message you receive from IDA Pro TUI after executing this command.

[rtompsam]

I ran .\idat64.exe -S"c:\users\me\appdata\local\programs\python\python39\lib\site-packages\headless_ida\ida_script.py 54886" -P+ C:\Users\me\Desktop\IDAPYTHON\sample.exe

Not getting any error but just hangs:

[DennyDai]

Were you able to successfully run import rpyc in the IDA Pro console?
Also by errors I mean including all the popup windows, were there any warning?

[rtompsam]

Yes I was able to import rpyc and no warnings

[DennyDai]

If your C:\Users\Username path includes spaces, there could potentially be issues. I've just pushed a fix for that (v0.4.2). However, if this wasn't the problem, it may be difficult to diagnose and troubleshoot further without being able to replicate the issue on my end.

[DennyDai]

BTW if it hangs, it typically indicates that IDA was able to load the headless-ida server script and the script is now awaiting a client connection. I've also noticed that you were running version 7.7 initially but switched to 8.3 in your latest response. Could you try running headless-ida ida64.exe "C:\Users\me\Desktop\IDAPYTHON\sample.exe" from the 8.3 directory and see if that works?

[rtompsam]

Let me try, BTW I downloaded the new version and there is an error :

Traceback (most recent call last):
File "c:\program files\python38\lib\runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "c:\program files\python38\lib\runpy.py", line 87, in run_code
exec(code, run_globals)
File "C:\Program Files\Python38\Scripts\headless-ida.exe_main.py", line 7, in
File "c:\program files\python38\lib\site-packages\headless_ida\cli.py", line 21, in headlessida_cli
headlessida = HeadlessIda(args.idat_path, args.binary_path)
File "c:\program files\python38\lib\site-packages\headless_ida\client.py", line 44, in init
f'{idat_path} -o"{tempidb.name}" -A -S"{escape_path(server_path)} {port}" -P+ {binary_path}', shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
File "c:\program files\python38\lib\site-packages\headless_ida\helpers.py", line 12, in escape_path
if _GetShortPathName(long_path, buffer, len(buffer)):
NameError: name 'long_path' is not defined

[DennyDai]

Oops my bad, just a typo should be just path
Fixed and pushed, probably gonna need to wait a few minutes until CI pushes version 0.4.3 to PyPi. v0.4.3 is now available on PyPi

[rtompsam]

Worked !! The issue is with helpers.py
Changed long_path to path:

if _GetShortPathName(path, buffer, len(buffer)):

import os
import ctypes

def escape_path(path):
if os.name == "nt":
_GetShortPathName = ctypes.windll.kernel32.GetShortPathNameW
_GetShortPathName.argtypes = [ctypes.c_wchar_p, ctypes.c_wchar_p, ctypes.c_uint]
_GetShortPathName.restype = ctypes.c_uint

    buffer = ctypes.create_unicode_buffer(len(path) + 1)
    if _GetShortPathName(path, buffer, len(buffer)): 
        return buffer.value
    else:
        raise Exception("Failed to get short path")
else:
    return f"\\\"{path}\\\""

[rtompsam]

Thanks !! Appreciate your help

[DennyDai]

Thank you for helping finding the issues :)