Currently the options for launching the process to be debugged are rather limited. You can either simply open a file which will then be launched immediately, or you can (depending on the process) launch it in the way you want and then attach the debugger afterwards. Command line arguments can be specified with initdbg but unless I'm mistaken there's no way to do it from the GUI.
I have been working on a "process launcher" to have more control over executable startup, which does not call CreateProcess but rather uses the NtCreateUserProcess system call directly. Using this I can create a process, suspend it, attach x64dbg or another debugger, and then resume (i.e. start) the process. While this is nice, it's really rather cumbersome and I'd like to be able to just do this from the debugger instead.
Something like this is what I have in mind:
The dialog parameters would normally be mostly empty or set to sensible defaults so that the behaviour stays the same as it is now unless explicitly changed.
While there are extremely many possible options to pass to the kernel when creating a process, there are at least two that I'd like to see in x64dbg and have written code for:
The reason I'm submitting this as an issue and not a pull request is because
It sounds like a good idea to me, however I don't think it will be used by 99% of the people and breaking the titanengine api is something I want to avoid because of GleeBug. The InitDebug functions only create a process and set some variables. It should be easy enough to extend this to support creating the process in a different way (environment variables could then also be changed).
You can change the command line post-launch in the debug menu by the way.
As for the GUI, you are right that it might be a problem. Currently a process is created with a command but if you start using pointers and such for context it is rather clumsy to do it that way and a new api would have to be created. Creating the GUI itself should be trivial. From what I saw its only a few edit boxes and nothing custom.
Not sure if I answered all your concerns/questions feel free to ping me.
Ah, of course I found the "change command line" menu option right after posting this issue. IMO it doesn't really belong in the debug menu though since it's a property of the process startup, not execution. (Not technically true since you can change the command line in the PEB during execution which x64dbg does, but most programs will receive a copy of this command line which the CRT makes so it can call main(argc, argv) with the proper arguments.) Also you still have to launch the program at least once with no parameters before you can set them, assuming you're using the 'Open' button from the GUI and not initdbg or launching x96dbg.exe with a command line set directly. Honestly my suggestion would be to just remove this button completely, the Debug menu is large enough as it is.
I'm OK with leaving the TitanEngine API intact, I was merely wondering if it would be a problem to break it. The InitDebug family of functions can simply forward to the new one. (Please help me think of a better name than InitDebugExEx or InitDebugEx2 though...)
Changing the environment variables is a good suggestion and easy to do. This would close #947, and if the environment variables are made to inherit from the parent process by default it could also close #948.
My would-be API currently looks like this:
_Out_ PHANDLE ProcessHandle,
_Out_ PHANDLE ThreadHandle,
_Out_opt_ PCLIENT_ID ClientId,
_Out_opt_ PPEB *RemotePebAddress,
_In_ PUNICODE_STRING ImagePath,
_In_ PUNICODE_STRING CommandLine,
_In_ PUNICODE_STRING WorkingDirectory,
_In_ ULONG ProcessFlags,
_In_ ULONG ThreadFlags,
_In_opt_ PVOID Environment,
_In_opt_ HANDLE ParentProcess,
_In_opt_ HANDLE Token,
_In_opt_ ULONG SessionId,
_In_opt_ PVOID DebugPort,
_In_opt_ ULONG NtGlobalFlag,
_In_ BOOLEAN NotifyCsr
As you can see it's very much NT-style, not Win32, so it's not possible to just replace calls to CreateProcess with this. However most of the arguments are either optional or trivially convertible, so it would be pretty easy to change InitDebug and co to use this. The biggest difference is the return value which is an NTSTATUS, and I'd really rather keep it that way because the various native calls involved all return NTSTATUS. It is trivial to convert an NTSTATUS to a Win32 error for use with SetLastError, but the other way around doesn't work so well and much of the error information is lost doing this.
I haven't looked into this much, but I believe the DebugPort parameter could be used to implement debugging of child processes to close #792.
My preferred approach would be to leave this API basically as is and put it in a separate module (say TitanEngine.Debugger.Launch.cpp) since the implementation is fairly big and the function itself doesn't really need to be aware of any debugging being done. The various InitDebug functions including the to-be-added new one can then just call this with ProcessFlags = DEBUG_PROCESS | DEBUG_ONLY_THIS_PROCESS like they already do with CreateProcess now.
DEBUG_PROCESS | DEBUG_ONLY_THIS_PROCESS