Brunnhilde was unable to export files from disk image

[bdietz]

I've been creating raw images of a USB flash drive with dd and dd forked and patched tools (ddrescue, dc3dd, dcfldd). I'm on a Mac, and the drive is Windows NTSF format. I've been testing .dd as output. When I run Brunnhilde over the image file with the -d option it's not able to export files and closes down. If I change the extension to .img and mount the files to my computer using Disk Utility, they mount, and I can run Brunnhilde over the volume just fine. I've also tested this with writing the image to .img. I am able to run Brunnhilde over .iso images of optical discs. I'm wondering if there's something peculiar about how dd-based tools create images. IIRC, I've successfully used Brunnhilde over raw files created using FTK Imager (.001). Thanks.

[bdietz]

.img file was mounted using DiskImageMounter, not Disk Utility.

[tw4l]

Hi Brian,

That's interesting. I have definitely been able to use Brunnhilde with (at least some) dd images in the past, but you may be right that there's an issue with at least some.

Brunnhilde relies on tsk_recover to export files from disk images. Can you test tsk_recover on its own against the dd image and report back?

Maybe a factor, since you mentioned ddreacue - is the image missing a sizable number of sectors?

[bdietz]

Thanks, Tim. tsk_recover can't determine the file system type. If I pass -f ntfs, it reports "unsupported file system type: ntfs". But I think I'm getting conflicting info in diskutil and Disk Utility about filesystem. I think I need to dig into this a bit more. Possibly something to do with partitions?

ddrescue reports that it rescued all the bits.

Next week, I'll dig in a bit more into tsk_recover options, play around with reimaging the disk, and also try imaging a different drive. Thanks for the quick response. It's reassuring to know that it ought to work.

Brian

[tw4l]

Hmm, interesting. I found this potentially relevant open issue. When you take a look at it next week, maybe try running mmls against the image and then passing the sector offset for the ntfs partition to tsk_recover as the post describes?

[bdietz]

Well...using the offset in tsk_recover worked...sorta. It looks like I recovered deleted files, and not the ones I see when I mount the .img file or even when I insert the usb stick.

I did figure out why I was getting what I thought was conflicting information about the filesystem. It's exfat. From https://wiki.sleuthkit.org/index.php?title=ExFAT_Implementation_Notes:

ExFAT shares the same partition code as NTFS, so the string displayed by mmls for type 0x07 is now "NTFS / exFAT (0x07)".

[bdietz]

Tim, disregard most that last comment. I just imaged another usb stick, and I'm having the same issues, including recovering the same unexpected files from the .img file. I'm not quite sure what's going on, but this has gone beyond an issue with Brunnhilde. Thanks for your responses. -Brian

[tw4l]

Hi Brian. I'm closing this for now, as the problem seems larger than Brunnhilde. But if you figure it out and think it's something that could be improved within Brunnhilde, please let me know.

[bdietz]

Hi Tim. Thanks, again for your responses. One last question (maybe), eve though this is closed. Can one use the offset option for tsk_recover in Brunnhilde? Or, would it be a possible feature. The use case is wanting to parse a partition of an image that tsk_recover is not able to parse otherwise.

[tw4l]

Hi Brian. That functionality isn't currently supported, but it sounds like a good addition to me. I'm on vacation starting tomorrow but will reopen this and plan to add the functionality when I return.

From your perspective, are there other tsk_recover flags/options besides offset that I should add to Brunnhilde?

[bdietz]

Super! Thanks. I think there's probably a use case for recovering allocated (-a) vs all files (-e). Enjoy your vacation.

[tw4l]

Hi Brian,

I added some additional tsk_recover functionality in branch dev-1.5.0. Want to try it out and let me know how it works for you? I've tested it on a few sample disks here but it would be great to confirm that it's behaving as expected with a wider sample.

There are a few other changes I'm planning to wrap into v1.5.0, so no great rush on this.

Thanks!

[tw4l]

(The README is updated as well and should be self-explanatory, but let me know if not and I can improve that as well)

[tw4l]

Hi Brian -- After some more testing on my own, I merged 1.5.0 into the Master branch and updated the version in PyPI. So go ahead and test on the main release :)

[bdietz]

Hi Tim, I really am trying to get to this to test. I tried it last week, without much luck, but when I tried tsk_recover on its own, I got some weird results. So, I'm trying to get back to this with a fresh image. I'm sorry for the delay.

[tw4l]

Hi Brian. No worries! I'll be curious to hear how it goes. In my testing I found it didn't work with images of optical media, which makes sense. Out of curiosity, was the problematic image formatted with exfat again?

…

On Mon, Jun 19, 2017 at 4:15 PM Brian Dietz ***@***.***> wrote: Hi Tim, I really am trying to get to this to test. I tried it last week, without much luck, but when I tried tsk_recover on its own, I got some weird results. So, I'm trying to get back to this with a fresh image. I'm sorry for the delay. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#20 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGchlH4QI0DGu808a5OIYS7hPp-E8dhrks5sFtbFgaJpZM4M-An3> .

[bdietz]

Tim, I'm really sorry to bombard you with emails today. I figured out the tsk problem. While -a gets allocated, I assumed no option would get it all. No option actually seems to only get the orphaned files. The -e option gets it all. Also incidentally, I didn't realize my attachments would post to the thread in github. I removed them.

…

On Tue, Aug 8, 2017 at 2:17 PM, Brian Dietz ***@***.***> wrote: Hey Tim, And to answer your question about the mystery disk, mmls shows partition two is "NTFS / exfat" (0 is the primary table and 1 is unallocated). When I run tsk recover using the f, o, and i options, I get only orphan files. If I add the allocated option, I get the files I'd expect. This isn't a brunnhilde issue, but since you asked, I thought I'd let you know. Brian On Tue, Aug 8, 2017 at 11:47 AM, Brian Dietz ***@***.***> wrote: > Hi Tim, > > I'm really sorry for the delay. I know it's bad form to request a feature > and then not to test it, especially since you were so prompt in addressing > the request. Naturally, respond when you can, and no rush. > > That said, I've tested the tsk options on a raw image made from a usb > flash drive using ddrescue. While I can extract files using tsk_recover, > I'm not able to do so with the tsk options in brunnhilde. I've tried to > document what I've done, shared in the attached file. My fear/hope is that > I'm making a simple mistake. My sense is that there's something off with > the options and reading the option value as the image file. (Which, again, > could be user error.) > > For what it's worth, I also have a raw image of just the fat32 partition, > and brunnhilde is able to extract files from that image without using any > of the tsk options in the command. > > I haven't tested this with other images yet, but I probably will. I can > share those results with you if it will be helpful. > > I heard great things about your SAA talk, and I'm sorry I wasn't there to > see it. > > Brian > > On Mon, Jun 19, 2017 at 6:16 PM, Tim Walsh ***@***.***> > wrote: > >> Hi Brian. No worries! I'll be curious to hear how it goes. In my testing >> I >> found it didn't work with images of optical media, which makes sense. Out >> of curiosity, was the problematic image formatted with exfat again? >> >> On Mon, Jun 19, 2017 at 4:15 PM Brian Dietz ***@***.***> >> wrote: >> >> > Hi Tim, I really am trying to get to this to test. I tried it last >> week, >> > without much luck, but when I tried tsk_recover on its own, I got some >> > weird results. So, I'm trying to get back to this with a fresh image. >> I'm >> > sorry for the delay. >> > >> > — >> > You are receiving this because you modified the open/close state. >> > >> > >> > Reply to this email directly, view it on GitHub >> > <#20 (comment) >> suecomment-309560773>, >> > or mute the thread >> > <https://github.com/notifications/unsubscribe-auth/AGchlH4QI >> 0DGu808a5OIYS7hPp-E8dhrks5sFtbFgaJpZM4M-An3> >> > . >> >> > >> >> — >> You are receiving this because you authored the thread. >> Reply to this email directly, view it on GitHub >> <#20 (comment)>, >> or mute the thread >> <https://github.com/notifications/unsubscribe-auth/AAWYKH7b6bUBBwGKieE4cJNsT-CQ-ohEks5sFvNRgaJpZM4M-An3> >> . >> > >

[tw4l]

Hi Brian. I think what was happening was a really simple off-by-one indexing error in how the tsk options were being added into the tsk_recover command. I've attempted a fix in this branch but don't have any suitable disk images to test.

Want to try downloading the code from that branch and trying with that local version of brunnhilde before I push it to a new bugfix release? Thanks!

[bdietz]

It works! Thank you. In the README, where it says, "Brunnhilde instructs tsk_recover to extract all files from disk images, including deleted files," would you mind confirming that that is the -e option?

…

On Thu, Aug 10, 2017 at 11:20 AM, Tim Walsh ***@***.***> wrote: Hi Brian. I think what was happening was a really simple off-by-one indexing error in how the tsk options were being added into the tsk_recover command. I've attempted a fix in this branch <https://github.com/timothyryanwalsh/brunnhilde/tree/dev-1.5.2> but don't have any suitable disk images to test. Want to try downloading the code from that branch and trying with that local version of brunnhilde before I push it to a new bugfix release? Thanks! — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#20 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAWYKGu_59vxGsFblnemrYCSExOepD0Cks5sWx_agaJpZM4M-An3> .

[tw4l]

Great! I wrapped this with another quick bug fix in release 1.5.3 and pushed it to PyPi so you should now be able to update with sudo pip uninstall brunnhilde && sudo pip install brunnhilde. Thanks for reporting the bug and I'm glad it's working now!