Feature request: ability to retrieve the `citem_t` corresponding to a given cursor position on a given line

[NSExceptional]

Given some code like this:

33|   v28 = a3;
34|   v27 = objc_retain(CFSTR("/Library/MobileSubstrate/DynamicLibraries/libFLEX.dylib"));
35|   v3 = objc_msgSend(&OBJC_CLASS___NSFileManager, "defaultManager");
36|   v26 = (void *)objc_retainAutoreleasedReturnValue(v3);

with the cursor right here (column 38 in the code above) for example:

            v
OBJC_CLASS__|_NSFileManager

funcEA = ...
line = 35
cursor = 38
item = fidl.lex_citem_at_pos(funcEA, line, cursor)

# Prints obj
item.cexpr.opname
# Prints address of _OBJC_CLASS_$_NSFileManager
item.cexpr.obj_ea

The explicit cursor position is important to me as I'm not working directly in IDA; I would like to be able to query arbitrary locations without using IDA's cursor API.

[NSExceptional]

I seem to have implemented this functionality using your code as a reference:

def citem_by_pos(psc, lineno, col):
    """
    Return the ctree item index corresponding to a zero-based (line, column) position.
    """
    
    if lineno >= psc.size():
        raise Exception("Line index out of bounds for function")
    
    line = psc[lineno].line

    # Position in the actual string, including color codes
    i = 0
    # Position in the displayed string, no color codes
    c = 0
    last_idx = None
    line_length = len(line)
    
    if col >= line_length:
        raise Exception("Column index out of bounds for line")

    # lex COLOR_ADDR tokens from the line of text
    while i < line_length:
        
        # Did we finally reach the given column?
        if c >= col:
            return last_idx

        # does this character mark the start of a new COLOR_* token?
        if line[i] == COLOR_ON:
            # yes, so move past the COLOR_ON byte
            i += 1

            # is this sequence for a COLOR_ADDR?
            if ord(line[i]) == COLOR_ADDR:
                # yes, so move past the COLOR_ADDR byte
                i += 1

                # A COLOR_ADDR token is followed by either 8, or 16 characters
                # (a hex encoded number) that represents an address/pointer.
                # in this context, it is actually the index number of a citem
                citem_index = int(line[i:i + COLOR_ADDR_SIZE], 16)
                i += COLOR_ADDR_SIZE

                # SANITY CHECK
                # NOTE: this value is arbitrary (although reasonable)
                # FIX: get this from cfunc.treeitems.size()
                if citem_index < 0x1000:
                    # save the extracted citem index
                    last_idx = citem_index
            
            # skip past the color code
            else:
                i += 1
        # does this character mark the end of a color code tag?
        elif line[i] == COLOR_OFF:
            # skip past COLOR_OFF as well as the color code
            i += 2
        
        # nothing we care about happened, keep lexing forward
        else:
            i += 1
            c += 1

    # Impossible!
    raise Exception("Fatal error in citem_by_pos()")

[carlosgprado]

The mapping between lines of code and citems is full of demons. Did you perform some tests to ensure that it works (reasonably) well?
The code looks nice, well commented and clean :)

[NSExceptional]

I didn't write any unit tests or anything but I wrote a webserver to send me the pseudocode and then output the type of the token under my cursor whenever I asked for it, and it seemed to work flawlessly! I could see casts, variables, assignment, reference, obj, etc.

I read up on the format of these strings; every COLOR_ON is followed by an address or another color tag, and if it is followed by a color tag then there will be a corresponding COLOR_OFF tag with the original color as well

My code won't detect malformed strings (i.e. a COLOR_OFF with no associated COLOR_ON) but it should work for valid strings I think!

[carlosgprado]

Sweet... since this is a pure addition without changes to the main code you can submit a PR if you think this could be relevant to others. I will set this issue to closed.

[NSExceptional]

Awesome! Will do :)