ReadOrWriteError exception raised when trying to use comment api in IDA 6.8

[arizvisa]

This issue manifested after fixing issue #59 with PR #60 (some prior conversation exists in those tickets, so start there if you need context for this).

Python>db.comment(0x006DA9CF, "123")
CRITICAL:root:internal.interface.priorityhook.callback(7186895, False) : Callback for "IDB_Hooks.cmt_changed" with priority (+0) raised an exception while executing <bound method type.old_changed of <class 'hooks.address'>>
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : Traceback ("IDB_Hooks.cmt_changed" was hooked at)
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : File "C:\Users\Hskyre\AppData\Roaming\Hex-Rays\IDA Pro\misc\hooks.py", line 1059, in ida_is_busy_sucking_cocks
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : make_ida_not_suck_cocks(idaapi.NW_INITIDA)
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : File "C:\Users\Hskyre\AppData\Roaming\Hex-Rays\IDA Pro\misc\hooks.py", line 993, in make_ida_not_suck_cocks
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : ui.hook.idb.add('cmt_changed', address.old_changed, 0)
WARNING:root:internal.interface.priorityhook.callback(7186895, False) :
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : Traceback (most recent call last):
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : File "C:\Users\Hskyre\AppData\Roaming\Hex-Rays\IDA Pro\base_interface.py", line 373, in closure
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : result = callable(*parameters)
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : File "C:\Users\Hskyre\AppData\Roaming\Hex-Rays\IDA Pro\misc\hooks.py", line 219, in old_changed
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : internal.comment.contents.set_address(ea, 0)
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : File "C:\Users\Hskyre\AppData\Roaming\Hex-Rays\IDA Pro\base_comment.py", line 971, in set_address
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : raise internal.exceptions.ReadOrWriteError(u"{:s}.set_address({:#x}, {:d}{:s}) : Unable to write name to address {:#x}.".format('.'.join(('internal', name, cls.name)), address, count, ', {:s}'.format(internal.utils.string.kwargs(target)) if target else '', address))
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : ReadOrWriteError: internal.comment.contents.set_address(0x6da9cf, 0) : Unable to write name to address 0x6da9cf.
WARNING:root:internal.interface.priorityhook.callback(7186895, False) :

[arizvisa]

Okay. @gool123456, were you able to confirm that database.tag(address, name, value) generates the same exception?

[arizvisa]

Also if you want to disable the comment cache because you need to get some other work done, you can just comment out the hook entirely in misc/hooks.py at line 993.

WARNING:root:internal.interface.priorityhook.callback(7186895, False) : File "C:\Users\Hskyre\AppData\Roaming\Hex-Rays\IDA Pro\misc\hooks.py", line 993, in make_ida_not_suck_cocks
WARNING:root:internal.interface.priorityhook.callback(7186895, False) : ui.hook.idb.add('cmt_changed', address.old_changed, 0)

[gool123456]

Python>database.tag(0x006DA9C8, '222', 'ok')
is Okey !

[arizvisa]

Okay. Lol. That's very confusing then. There must be something in that hook that's screwed up, let's double-check that the api isn't messed up.

The following code should reproduce the exception. Can you try it and let me know what it outputs? Ideally it should raise that ReadOrWriteError exception.

ea = 0x6da9cf
import internal
ok = internal.comment.contents.set_address(ea, 0)
print(ok)
ok = internal.comment.contents.set_address(ea, 1)
print(ok)
print(func.within(ea), databases.disasm(ea))

[gool123456]

@arizvisa haha, That's all right by GH-59 :

{'name': {u'': 1, u'222': 2}, 'address': {7186888: 1}}
{'name': {u'': 1, u'222': 2}, 'address': {7186888: 1, 7186895: 1}}
(True, u'6da9cf: mov byte ptr [eax], 0')

[gool123456]

@arizvisa hi ,
In addition, I would like to ask, how to create a structure with a specified size, I don’t understand the documentation ...

[gool123456]

How to set this add method type parameter ?
Python>t2=structure.new('t2')
Python>t2.add('f1', 'char')
Result :
KeyError: 'char'

[arizvisa]

@gool123456, just use t2.add('f1', chr), or you can use t2.add('f1', (int,1)). The idea is that you just use python's types (int, chr, str, tuple, list) to describe your type. So this way you can (int, $size) to control the size (byte, word, dword, qword, etc). If you just just int like t2.add('f1', int) it'll use the default size.

Then if you want to describe an array, you just wrap it in a list, like t2.add('f1', [(int, 1), 42]) means a 1-byte integer times 42.

After you figure out the size of your type, you can assign to t2.members.by('f1').typeinfo if you need it to correspond to exactly something. i.e. t2.members.by('f1').typeinfo = "char".

It's definitely a very strange way of thinking about types, but I couldn't come up with any other way to hide what IDAPython does with its typeflags like idaapi.FF_BYTE | idaapi.FF_DWRD | idaapi.FF_SIGN. With this way you don't have to memorize anything.

[gool123456]

@arizvisa that's nice, Interesting question!! In learning, ida is really a big knowledge base.

[gool123456]

@arizvisa hi, I'm so stupid, I have to come and ask you questions, I don't know what went wrong, I can't set the member's custom type , it awaly is 'int':

import idaapi
import idc


st1 =structure.by_name('TTDBTextInfoGroup')
struct_name = 'TTDBTextInfoGroup*'
f1 = st1.members.by(offset=0)

print f1.typeinfo.dstr()
ordinal = f1.typeinfo.get_ordinal()
f1.typeinfo.set_numbered_type(idaapi.cvar.idati,ordinal,BTF_STRUCT, struct_name)
print f1.typeinfo.dstr()

[arizvisa]

Can you print out the structure st1 to me? There might not be enough space in the field to promote it to an int.

This is actually related to issue #45. IDA doesn't return an error or anything, so to solve issue #45, we need to verify that the field was properly assigned.

[arizvisa]

Also, are you trying to change the type of the member at offset 0 to a structure?

If so, then you'll want to do:

st = struc.by('mystructure')
m = st.members.by(offset=0)
m.type = struc.by('structuremember')

[arizvisa]

You can also treat member.typeinfo as a string, meaning you can do:

m.typeinfo = 'char*'

[arizvisa]

@gool123456, I moved your question to a separate issue for tracking since it is different that this bug you reported. You can find it at #62.

[arizvisa]

Btw, I'm still looking at this issue and am trying to download a version of IDA 6.8 from archive.org in order to reproduce. The town that I'm currently at was hit by a hurricane and for some reason my d/l speed is around 5k/s-10k/s. Give me a little bit to get an older version of IDA so I can try and reproduce this.

[gool123456]

@arizvisa ok, Lucky to have your help! Don't worry, I can help you with something :)
Can the Google Device I gave you last time be downloaded?
Link : Ida6.8

[arizvisa]

Yes. That link seems to work. Thanks you.

I was also downloading it from this link too. https://archive.org/details/Yadi.skdRG9p6Vqu3NRckp

Currently my internet is very slow, though. So I'll update this issue when I finish downloading it and am able to reproduce.

[arizvisa]

Similar logs related to issue:

Exception type 1:

CRITICAL:root:internal.interface.priorityhook.callback(<idaapi.func_t; proxy of <Swig Object of type 'func_t *' at 0x04F33878> >, 269451364) : Callback for "IDB_Hooks.func_tail_removed" with priority (+0) raised an exception while executing <function func_tail_removed at 0x04EB22B0>
WARNING:root:internal.interface.priorityhook.callback(<idaapi.func_t; proxy of <Swig Object of type 'func_t *' at 0x04F33878> >, 269451364) : Traceback ("IDB_Hooks.func_tail_removed" was hooked at)
WARNING:root:internal.interface.priorityhook.callback(<idaapi.func_t; proxy of <Swig Object of type 'func_t *' at 0x04F33878> >, 269451364) :   File "C:\Users\root\AppData\Roaming\Hex-Rays\IDA Pro\misc\hooks.py", line 1059, in ida_is_busy_sucking_cocks
WARNING:root:internal.interface.priorityhook.callback(<idaapi.func_t; proxy of <Swig Object of type 'func_t *' at 0x04F33878> >, 269451364) :     make_ida_not_suck_cocks(idaapi.NW_INITIDA)
WARNING:root:internal.interface.priorityhook.callback(<idaapi.func_t; proxy of <Swig Object of type 'func_t *' at 0x04F33878> >, 269451364) :   File "C:\Users\root\AppData\Roaming\Hex-Rays\IDA Pro\misc\hooks.py", line 1019, in make_ida_not_suck_cocks
WARNING:root:internal.interface.priorityhook.callback(<idaapi.func_t; proxy of <Swig Object of type 'func_t *' at 0x04F33878> >, 269451364) :     ui.hook.idb.add('func_tail_removed', func_tail_removed, 0)
WARNING:root:internal.interface.priorityhook.callback(<idaapi.func_t; proxy of <Swig Object of type 'func_t *' at 0x04F33878> >, 269451364) : 
WARNING:root:internal.interface.priorityhook.callback(<idaapi.func_t; proxy of <Swig Object of type 'func_t *' at 0x04F33878> >, 269451364) : Traceback (most recent call last):
WARNING:root:internal.interface.priorityhook.callback(<idaapi.func_t; proxy of <Swig Object of type 'func_t *' at 0x04F33878> >, 269451364) :   File "C:\Users\root\AppData\Roaming\Hex-Rays\IDA Pro\base\_interface.py", line 373, in closure
WARNING:root:internal.interface.priorityhook.callback(<idaapi.func_t; proxy of <Swig Object of type 'func_t *' at 0x04F33878> >, 269451364) :     result = callable(*parameters)
WARNING:root:internal.interface.priorityhook.callback(<idaapi.func_t; proxy of <Swig Object of type 'func_t *' at 0x04F33878> >, 269451364) :   File "C:\Users\root\AppData\Roaming\Hex-Rays\IDA Pro\misc\hooks.py", line 789, in func_tail_removed
WARNING:root:internal.interface.priorityhook.callback(<idaapi.func_t; proxy of <Swig Object of type 'func_t *' at 0x04F33878> >, 269451364) :     for ea in database.address.iterate(min(missing), max(missing)):
WARNING:root:internal.interface.priorityhook.callback(<idaapi.func_t; proxy of <Swig Object of type 'func_t *' at 0x04F33878> >, 269451364) : ValueError: min() arg is an empty sequence

Exception type 2:

CRITICAL:root:internal.interface.priorityhook.callback(270667220, True) : Callback for "IDB_Hooks.cmt_changed" with priority (+0) raised an exception while executing <bound method type.old_changed of <class 'hooks.address'>>
WARNING:root:internal.interface.priorityhook.callback(270667220, True) : Traceback ("IDB_Hooks.cmt_changed" was hooked at)
WARNING:root:internal.interface.priorityhook.callback(270667220, True) :   File "C:\Users\root\AppData\Roaming\Hex-Rays\IDA Pro\misc\hooks.py", line 1059, in ida_is_busy_sucking_cocks
WARNING:root:internal.interface.priorityhook.callback(270667220, True) :     make_ida_not_suck_cocks(idaapi.NW_INITIDA)
WARNING:root:internal.interface.priorityhook.callback(270667220, True) :   File "C:\Users\root\AppData\Roaming\Hex-Rays\IDA Pro\misc\hooks.py", line 993, in make_ida_not_suck_cocks
WARNING:root:internal.interface.priorityhook.callback(270667220, True) :     ui.hook.idb.add('cmt_changed', address.old_changed, 0)
WARNING:root:internal.interface.priorityhook.callback(270667220, True) : 
WARNING:root:internal.interface.priorityhook.callback(270667220, True) : Traceback (most recent call last):
WARNING:root:internal.interface.priorityhook.callback(270667220, True) :   File "C:\Users\root\AppData\Roaming\Hex-Rays\IDA Pro\base\_interface.py", line 373, in closure
WARNING:root:internal.interface.priorityhook.callback(270667220, True) :     result = callable(*parameters)
WARNING:root:internal.interface.priorityhook.callback(270667220, True) :   File "C:\Users\root\AppData\Roaming\Hex-Rays\IDA Pro\misc\hooks.py", line 219, in old_changed
WARNING:root:internal.interface.priorityhook.callback(270667220, True) :     internal.comment.contents.set_address(ea, 0)
WARNING:root:internal.interface.priorityhook.callback(270667220, True) :   File "C:\Users\root\AppData\Roaming\Hex-Rays\IDA Pro\base\_comment.py", line 981, in set_address
WARNING:root:internal.interface.priorityhook.callback(270667220, True) :     raise internal.exceptions.ReadOrWriteError(u"{:s}.set_address({:#x}, {:d}{:s}) : Unable to write to cache for address {:#x}.".format('.'.join(('internal', __name__, cls.__name__)), address, count, ', {:s}'.format(internal.utils.string.kwargs(target)) if target else '', address))
WARNING:root:internal.interface.priorityhook.callback(270667220, True) : ReadOrWriteError: internal.comment.contents.set_address(0x10220dd4, 0) : Unable to write to cache for address 0x10220dd4.
WARNING:root:internal.interface.priorityhook.callback(270667220, True) : 

[arizvisa]

I was only able to reproduce these errors wiith a non-fresh instance of IDA. What this means is that you need to open a database, let it analyze to completion, close it, and then create a new database. This seems to happen right before the type-propagation phase (I think). To identify the "type-propagation" phase, it's right before the type library gets applied.

After planting some logging, it seems to be happening when erasing the cache + blog when no data is specified. This is happening in the finally case for a try-finally block in the implementation of contents._write.

[arizvisa]

Some of these issues are potentially related to issue #65.

[arizvisa]

I think PR #67 should fix the exception you mentioned, and 2 more that I found after being able to reproduce. There's a chance I didn't get them all as it depends on what hooks IDA executes (which depends on what it's analyzing). I was able to find and fix the following 3 exceptions:

• ReadOrWriteError -- comes from the idb.cmt_changed hook, and it complains about "Unable to write name to address XXX"
• ValueError -- comes from the idb.func_tail_removed hook, and it complains about min() being used with an empty sequence
• Some null-pointer exception -- comes from the idb.func_tail_removed hook, and it happens when trying to compare with the area_t.__ne__() method.

Check out PR #67 whenever you get a chance, and let me know if it fixes these 3 issues, and if it introduces any newer exceptions that we haven't seen yet.

[arizvisa]

(btw, sorry about the language barrier. I know I write English very "comprehensively" and so it might be difficult for you to understand.)

[gool123456]

@arizvisa Thanks~ So that's it, I understand. It does appear only when you open a non-latest instance. I use the newly opened instance and everything is normal. It doesn't affect the use. :)

[arizvisa]

@gool123456, okay...Can you try out PR #67 to see if it fixes those errors that are happening during auto-analysis? This is the last PR to confirm, a lot of thanks for your patience.

To try it out you can use git like the following which will check out the PR into its own branch.

$ git fetch
$ git fetch origin pull/67/head:fix-issue-61
$ git checkout fix-issue-61

Then you'll likely need to rebase the PR onto origin/master by doing:

$ git rebase origin/master

Don't forget that you'll need to "open a non-latest instance". So pick something small the first time, and then open up another file. It should display the auto-analysis and not list any of the exceptions that you mentioned. Although I'm not 100% sure I got every single one of them as I was only able to identify the 3 that I mentioned.

[gool123456]

@arizvisa You are awesome, The RP#61 is right. It's okay, I'm happy to test it for you. :)

[gool123456]

I should thank you for writing such a good tool . Like IDA-minsc is really easy to use, it helps a lot in my work.

[gool123456]

@arizvisa I tried to let one ida instance analyze multiple samples, which is normal.

[arizvisa]

Awesome. Thanks you for testing it out. I just merged the PR, so you can checkout master and do a git pull to use the current version with all the fixes you contributed.

[arizvisa]

Lol! Nice. ;-)
I'm glad you find it so useful in your work. Tell as many people as you can about it, and spread it around. ;-)

I'm totally trying to beat sark (http://github.com/tmr232/sark) in users. :-)
Actually, I'm really trying to beat IDAPython in users, because IDAPython is just so not friendly.

[arizvisa]

Feel free to close this issue whenever you feel like it as PR #67 has been merged.

Also, don't be afraid to create a new issue when asking questions about how to use ida-minsc.
ida-minsc tries to simplify a lot of IDAPython's capabilities, and since it's very different from coding Python it can be very confusing for the users whom have only written IDAPython.

[gool123456]

Ha ! Okay, I already have a few friends using . I believe you can. 👍 🥇

[gool123456]

@arizvisa The new version is much easier to use . Thanks!

[arizvisa]

Awesome. Happy to help, and thanks for your contribution. ;-)

Closing this issue as it's been resolved by PR #67