Limit number of partitions to be processed

[ealarcon-av]

Hello, i found your program on an answer on super user i think, its seems to be very promising, aside from its current limitations it seems very well done.

I'm trying to recover data from a ntfs partition, about 1TB of size, the hard disk "died" when deleting some files through linux, it appears that the MFT was corrupted because of failing sectors on the drive. So i made an image of the partition with ddrescue as you recomended and i'm using your program to try to recover the data.
Problems i've found:

• Big memory use, so much that swap got full and system collapsed.
  solution, increase swap to about 100GB, on an SSD and it uses about 40 GB of memory while running
• Lots of partitions found, i 've been running your program for over 120 hours, and the scan is still not finished, so i stopped it and put your solution to filter partitions with few files. i think now it doesn't go on line 823, but before that, on line 821.

I have read most issues here and the MFT seems to be very fragmented, so there are a lot of partitions available to scan. I've tried a windows program on the disk and the scan identified most of the files being on the firsts partitions founds on the disk.

¿How could i limit the processing to only the first 10 partitions of the disk?
I have no python-fu, so any help would be greatly appreciated

[Lazza]

Big memory use

This is true, I know. 😢

Most of the computation time (and also part of memory usage) should be consumed in trying to find the boundaries of partitions whose boot sector is lost.

This is the main distinguishing feature between RecuperaBit and other tools.

You might want to try forcefully disabling the find_boundary function inside ntfs.py by adding a return statement on its first line:

def find_boundary(self, part, mft_address, multipliers):
    return (None, None)

If the boot sector of the partition you care about is OK, this should cut the running time significantly.

[ealarcon-av]

I tried the recomendation on other issue to disable searching after 50 partitions, it seemed to work and i was able to recover the most important files i needed.
I used another ntfs external file to save the recovered files, but it created files with an extension .zone.identifier that i could'nt erase, i had to do a checkdisk on windows and then i was able to erase those files. The disk drive didn't have any problems before.

[Lazza]

Given that RecuperaBit also restores "dollar files" with their original name, it is advised not to restore into a NTFS partition for now.

[ealarcon-av]

For reference to anyone that needs it, lots of memory is required, RAM will never be enough, it will need swap, use a SSD drive as swap, it will be a lot faster.
MFT gets really fragmented, limit the processing to 20-50 partitions, most usefull files are found this way.
A 1TB hard drive used about 40-50GB SWAP and took about 90 hours to complete the scan (limited to 50 first partitions found), i recommend:

• RAM, all you can have, 16GB, 32 GB (never enough)
• SSD SWAP drive, 100GB
• Restore files to an ext4 formatted drive, metadata and "dollar files" are restored too and very difficult to delete from a ntfs partition
• This will not eat all the pc resources, so it still could be used, albeit a little slower
• A lot of patience, the process could take days (put a limit to the partitions to be processed), reconstruction of a damaged filesystem is not an easy task

A lot of thanks to @Lazza for this program

[thinrope]

Yes, I see it "can" require lots of resources and time. But for comparison, I just recovered some 4TB data from 6TB partition in 54 partitions and it never needed more than 9GB RAM, the processing took may be 24h, then restore another 24 (working with a pair of WD Gold drives).
Now comes the hard sorting out of actual data vs. fragments/0-bytes, merging all "partitions" together, and fixing metadata (timestamps) :-D

[Lazza]

lots of memory is required [...]
metadata and "dollar files" are restored too and very difficult to delete

Indeed these are two things that I plan to address. I will speak about this on March 3rd, 2021. The online webinar is organized by Basis Technology. In case you'd like to attend, here's a link with more information:

https://twitter.com/thelazza/status/1363559298450751490

[ealarcon-av]

Yes, I see it "can" require lots of resources and time. But for comparison, I just recovered some 4TB data from 6TB partition in 54 partitions and it never needed more than 9GB RAM, the processing took may be 24h, then restore another 24 (working with a pair of WD Gold drives).

Interesting, I supose it depends on how much files are in the partition and MFT fragmentation/damage.

Now my recomendation would be, if it takes to much time don't worry it is normal, if it uses too much memory be prepared to add more RAM/SWAP, still it is normal and expected