Table view takes a long time to sort

[kefir-]

I did a file search in autopsy that returned quite a lot of files. I tried to search the results by size, but after thinking for a number of minutes the results were not sorted. I clicked the Size column again, and then the sort indicator (little arrow) appeared on the heading, but the results are still not sorted.

Here's a screenshot:

[rcordovano]

I did a file attributes search that returned 12,265 files. I was unable to reproduce this problem.

[kefir-]

I ran some more tests to reproduce this here. This time I ran in a fresh instance of Autopsy and I reopened the already ingested case. I searched for files with size larger than X, and then checked if sorting worked. I started timing the sorting operation at one point, as it took some time (so did the search, but I didn't time the search operation):

• 10MB: ~1700 files, sorted ok

• 1MB: ~18000 files, sorted ok

• 100kb: ~55000 files, sorted ok after 20 seconds

• 10kb: ~145000 files, sorted ok after 75 seconds

• 1kb: ~270000 files, sorting failed after about 330 seconds

[kefir-]

I thought I'd create an index on the size column and retry after doing that, but simply running sqlite3 shell manually on the database and executing commands like this:

select count(*) from tsk_files where size > 1000;

they returned the same numbers as I got from the search results in the GUI, but they all seemed to return after less than a second. So if it's sqlite taking time, it must be in a more complex join operation, but I can't figure out a way to intercept the actual query so that I can try it manually and then attempt to optimize it. (As a side note, it'd be really nice if it were possible to enable sql query logging, and even nicer if I could enable logging of how long each query took.)

[rcordovano]

Interesting results, thanks. If I increase the number of results to be sorted in my tests, I expect I will be able to reproduce the problem.

However, the sorting in question is not done by SQL statements sent to the SQLite database. Instead, the data for the matching files is obtained from the database, then wrapped in NetBeans Node objects and displayed in a NetBeans OutlineView. After that, sorting the displayed data by column is done by NetBeans library code, not Autopsy code. I have found the NetBeans source in the past when I went looking for it, but the sorting in this case is essentially a black box to Autopsy.

[kefir-]

Strange that netbeans sorting is so slow, sorting these numbers of records ought to be pretty fast.

Just found this article on customising sorting in netbeans outlineview, sharing in case it could be useful for something.

http://netbeans.dzone.com/nbm-advanced-sorting-customization

Apparently the comparator can be overridden as well, if that's what is taking so long.

[bcarrier]

Thanks for the link. There are several places that we could use paging in the table results, but haven't because the sorting would be all messed up. That link may help that.