Add `DisassembleStream` method on Capstone class

[uxmal]

Would it be possible to add a method with the signature

IEnumerable<T1, T2, T3> DisassembleStream(byte[] bytes, int count, long startAddress)

to the Disassembler class that lazily disassembles the bytes in the byte stream?

The use case that I'm dealing with is tracing the disassembly of a basic block, and wanting to stop when I reach instructions like jmp or ret which terminate basic blocks. It would be very convenient if it were possible to say: "start disassembling these bytes at this position, and return a lazy stream of instructions." . I don't want to disassemble the entire .text segment if it turns out the first 10 bytes are code and the remaining 10000 are read-only data.

[9ee1]

The generic version of IEnumerable is defined as IEnumerable<T>. Could you provide a little more information on what T1, T2, and T3 are representing and perhaps we can think of a way?

[uxmal]

Sorry, I wrote in haste above. The enumerator should of course enumerate instructions! My proposed signature is:

IEnumerable<Instruction<TArchitectureInstruction, TArchitectureRegister, TArchitectureGroup, TArchitectureDetail>> DisassembleStream(byte[] bytes, long startAddress)

The idea is that Reko, the decompiler I'm writing, could get a deferred stream of instructions, which it can process until it determines that the boundary of a basic block has been reached (like a jmp or a ret instruction, for x86 processors). If it were to continue after such a terminating instruction, it might start disassembling data located after it. Reko does a lot of this when it scans a binary for instructions, and it seems wasteful to disassemble large address ranges if many of those addresses contain data.

If I were to code up my proposal, would you be willing to look it over?

[uxmal]

Here is a first stab:

        public IEnumerable<Instruction<TArchitectureInstruction, TArchitectureRegister, TArchitectureGroup, TArchitectureDetail>> DisassembleAll(byte[] code, long startingAddress)
        {
            if (code == null)
                throw new ArgumentNullException("code");
            byte[] miniBuf = new byte[0x0100];  //$REVIEW Is it likely that an architecture has instructions longer than this?
            int index = 0;
            long addr = startingAddress;
            while (index < code.Length)
            {
                //$REVIEW: inefficient copies slow us down, ideally there'd be a cursor-like
                // mechanism that would just update a pointer.
                Array.Copy(code, index, miniBuf, 0, Math.Min(code.Length - index, miniBuf.Length));
                var instr = Disassemble(code, 1, addr);
                if (instr == null)
                    yield break;
                yield return instr[0];
                addr = instr[0].Bytes.Length;
            }
        }

The efficiency of this code would be much improved if there were an overloaded version of the Disassemble method that took an extra int offset parameter. This parameter would be an offset into the code bytes from which the caller wishes to disassemble. That way we're not constrained to the beginning of the array, and don't have to perform all these copies to miniBuf. You'd need to propagate that extra parameter to NativeCapstone.Disassemble and change the call that passes pCode to instead pass pCode + offset.

I will try to code up a proposal on my fork of your project and have you review it, if you are willing.

[9ee1]

It seems all you're looking for is deferred dissemble of a buffer that terminates when a predicate is met. That can be easily done with very little complexity.

Let me code up something real quick for you on the develop branch and let me know if it does what you want.

[9ee1]

@uxmal BTW, your project is great! I see that you are considering using this project in it. Let me know how I can help make the integration effort easier!

[uxmal]

I'm not so sure if a design involving an explicit predicate is really necessary. The .NET idiom for a stream is to implement the IEnumerable / IEnumerator pattern, and have the client code decide whether or not to interrupt the enumeration.

I'm working on a code fragment on my fork of your project. I will post an update here when it's ready for review.

[uxmal]

When building the Capstone.NET project and running it, I'm getting the following exception in NativeInstruction.NativeX86Detail:

System.DllNotFoundException was unhandled
  HResult=-2146233052
  Message=Unable to load DLL 'Gee.External.Capstone.Proxy.dll': The specified module could not be found. (Exception from HRESULT: 0x8007007E)
  Source=Gee.External.Capstone
  TypeName=""
  StackTrace:
       at Gee.External.Capstone.CapstoneProxyImport.ArmDetail(IntPtr pDetail)
       at Gee.External.Capstone.NativeInstruction.get_NativeX86Detail() in d:\dev\jkl\reko\Capstone.NET\Gee.External.Capstone\NativeInstruction.cs:line 170
       at Gee.External.Capstone.X86.CapstoneX86Disassembler.CreateInstruction(NativeInstruction nativeInstruction) in d:\dev\jkl\reko\Capstone.NET\Gee.External.Capstone\X86\CapstoneX86Disassembler.cs:line 32
       at Gee.External.Capstone.CapstoneDisassembler`4.InstructionEnumerator.MoveNext() in d:\dev\jkl\reko\Capstone.NET\Gee.External.Capstone\CapstoneDisassembler.cs:line 417
       at CapstoneCMD.Program.ShowX86() in d:\dev\jkl\reko\Capstone.NET\CapstoneCMD\Program.cs:line 339
       at CapstoneCMD.Program.Main(String[] args) in d:\dev\jkl\reko\Capstone.NET\CapstoneCMD\Program.cs:line 47
       at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
       at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
       at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
       at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
       at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
       at System.Threading.ThreadHelper.ThreadStart()
  InnerException: 

It appears as if there is a missing project / DLL in the solution.@9ee1, can you help resolve this? I'm ready to commit to my fork if I only can get past this roadblock.

[9ee1]

@uxmal Yeah, agreed the idiom is to implement an enumerator. My suggestion for a predicate was because I was going to use a native Capstone function that does exactly this. But your approach works too and is probably more flexible than what I had in mind

Yeah, there is a missing DLL in the repository. I need to fix that. For now, you can download one of the releases and the DLL will be in the downloaded ZIP file.

[uxmal]

Adding the missing DLL as you state above fixes the problem I reported.

I've verified that enabling the code wrapped in #if !DISASSEMBLE_STREAM in CapstoneCMD\Program.cs works. All done!

[9ee1]

See my comment on the pull request.

[uxmal]

After the merge of pull request #8, this feature has been implemented. Closing, and thanks for accepting my contribution!