Creating new container with quick format cause drive to hang

[yangsfang]

Description:
When creating a new container with VeraCrypt, select quick format, progress bar is goes to 100%, but the interface appears it isn't completing. Abort button can be clicked. Clicking abort does not stop the hanging. VeraCrypt then appear busy and cannot be closed.
During this time, any Explorer interaction with the host volume (the partition where the container should reside) hangs. Windows Task Manager reports 93% external drive utilization.

Spec:
VeraCrypt 1.24-Update7 (64-bit)
Windows 10 Pro 1809 with September 2020 patch
System drive: WDC 256G SSD (not encrypted with VeraCrypt)
Portable drive: WD My Passport Ultra Metal 2TB
(465GB exFAT, 465GB NTFS, 931GB NTFS)

Steps to reproduce:

1. Open VeraCrypt (installed in Program files)
2. Create Volume
3. Create an encrypted file container
4. Standard VeraCrypt volume
5. Select File -> save to a file on external harddrive (Never save history)
6. Leave Encryption Option defaults (AES, SHA-512)
7. Enter Free space, 232GB (out of 465GB)
8. Enter password (<20 characters), use keyfile (4096bit, generated by VeraCrypt)
9. Large File Yes
10. Filesystem exFAT, Cluster default, Quick Format
11. Move mouse around until randomness is green, click Format. Dismiss warnings and continue.

Observed:
Progress bar quickly goes to 100%. Done: 100%, Speed empty, Left empty. but action does not finish. Abort button is available. Disk utilization is high, as if it is still formatting.

Expected:
Progress goes to 100% and done. Disk utilization is low.

[yangsfang]

I did some test to confirm creating 100MB and 4GB volumes both finished quickly. There are likely some threshold for which the quick format isn't finishing.

[idrassi]

This hang is due to Windows explicitly filling of file content with zeros so if the file container is very large, then this can take a lot of time.

"Quick Format" option only applies to VeraCrypt formating of the file container, it doesn't affect how Windows creates the file itself.
In your case, in order to accelerate file creation by Windows, you manually launch "VeraCrypt Format.exe" from a command prompt and specify the switch /FastCreateFile as documented at https://www.veracrypt.fr/en/Command%20Line%20Usage.html (VeraCrypt Format.exe /FactCreateFile): this will make Windows skip the part of filling file with zeros.

Can you please try with this switch and confirm that it works as expected?

Thanks.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[xnoreq]

@idrassi No, this doesn't work either.

Example invocation:

"c:\Program Files\VeraCrypt\VeraCrypt Format.exe" /create "test.tc" /size 10G /password "<enter your password here>" /hash sha-512 /encryption AES /filesystem none /quick /fastcreatefile

You don't check if SetFileValidData succeeds:

VeraCrypt/src/Common/Format.c

Line 409 in 3297693

SetFileValidData (dev, volumeSize.QuadPart);

And you don't open the file with SE_MANAGE_VOLUME_NAME which is required for SetFileValidData to succeed according to msdn.

Note that this works in Volume Expander because there this privilege is requested:

VeraCrypt/src/ExpandVolume/DlgExpandVolume.cpp

Line 797 in 3297693

if (!SetPrivilege (SE_MANAGE_VOLUME_NAME, TRUE))

---

This is also related to #388. The hangs from the GUI are worse because apparently Windows is horrible and such a blocking filesystem/kernel operation from a GUI application can make the entire desktop (including other applications) unresponsive.
It can also block shutdown...

--> Please add the "FastCreateFile" option to the GUI as well.

---

Lastly, there is #558.
One possibility to prevent the GUI from hanging and to make the operation cancelable would be the following:
After preallocating the file, I assume that writing metadata results in a write at the end of the file. This causes Windows to zero-fill the entire file, which could take minutes to hours or even days.

--> So after preallocating, why not seek e.g. in 100 MB chunks, writing a zero byte after each chunk, until the end of the file is reached.
Therefore, each zero-fill would be limited to much smaller chunks (instead of GBs or TBs at once) and the process could be canceled after each chunk. Additionally, progress information could be provided to the user.

[xnoreq]

@idrassi Any updates?

[James-Mat]

Just stumbled over this issue as well; used the same setup as you did and got confused over what VeraCrypt is doing there seemingly idling but using the disk with ~95%.
First thing I found was an older thread about mkfs.ntfs on unix causing VeraCrypt to seemingly hang, so I guessed that exFAT might cause the problem as well and used no filesystem for the initial creation instead; the issue persisted, and then I found this thread which explains the problem.

I think at least some sort of indicator or message what VeraCrypt is doing is necessary, even if it is no progress bar but a simple "Windows is filling the new volume with zeroes, this might take several hours depending on volume size".

[yangu-hury]

I have this problem too. Please try to quick format a container of 64GB and try to abort and see what happens. The system starts at 100% with a speed of 32GiB/s. My device maximum speed is just 3MB/s. If it is not a serious bug. I don't know that it is. It stays this way forever.

[xnoreq]

@idrassi I have fixed the issue here:

First, there was a bug in your SetPrivileges implementation which didn't check if it actually succeeded:
df8254d

Secondly, Format.exe didn't request the SE_MANAGE_VOLUME_NAME privilege required for SetFileValidData to succeed:
2d44395

To users: if this is merged then you have to start VeraCrypt Format.exe /fastcreatefile as administrator!

[andry81]

I've hitted the same issue trying to quick format 16TB new drive in Windows 7 x64 Pro SP1. The formatter does not respond and can not be terminated. Only reboot helps. The resulted image can be mounted, but disk is not readable. So, the quick format does not work for at least big volumes which has no sense.

[yangu-hury]

I tested it. It completes successfully if you wait for cancel or just wait for it. The problem is that It is not a quick format but a full format and the GUI gets no responsible. Impossible to cancel. If you cancel it. It will continue until the end, and will successfully end as if It wasn't cancelled. I tested with obviously small containers. Quick format has really no effect except broken the GUI ;).

[AnthonyB28]

experience this too with quick format. Its clearly writing to disk (100MB/s) for hours despite showing 100% done in veracrypt UI

[dt-flo]

Experiencing the same issue. I'm using /quick and /fastcreatefile. Any fixes planned?

[fzxx]

This hang is due to Windows explicitly filling of file content with zeros so if the file container is very large, then this can take a lot of time.

"Quick Format" option only applies to VeraCrypt formating of the file container, it doesn't affect how Windows creates the file itself. In your case, in order to accelerate file creation by Windows, you manually launch "VeraCrypt Format.exe" from a command prompt and specify the switch /FastCreateFile as documented at https://www.veracrypt.fr/en/Command%20Line%20Usage.html (VeraCrypt Format.exe /FactCreateFile): this will make Windows skip the part of filling file with zeros.

Can you please try with this switch and confirm that it works as expected?

Thanks.

Veracrypt has the same problem, Truecrypt, Cncrypt will not appear. @idrassi

[yangu-hury]

I will try it in command line to check if the same happens like in the GUI.

[AnthonyB28]

Just encountered this some more. The Quick Format assuredly results in a hang in the software with no way to abort. Comparably, if you terminate the process and rerun creation without Quick Format, it finishes in seconds. The Quick Format can take hours with a 100GB container. Win 10 and 11

[coralof]

I also just experienced this same issue. I was creating a 250GB (exFat) container on my external Samsung T7 SSD, and when trying to quick format, it finished to 100% in about 5 seconds, and then froze. I tried to click on Windows Explorer, and it ended up crashing the computer. After rebooting and starting again, I chose not to click Quick Format, and now it is proceeding perfectly fine.

OS: Windows 11 Pro, 22H2, build 22621.1702
Device: ThinkPad X13s
Processor: Snapdragon 8cx Gen 3
RAM: 16GB

[idrassi]

I have pushed a change to fix the quick format issue based on chunk idea proposed by @xnoreq : 579ce2f
It seems to do the job.
I will publish tomorrow a build that includes it so that others are able to test it.

[DStrelak]

I can confirm that:
Win 10 64b
VeraCrypt 1.26.5 (beta), 64b
16 TB container (exFAT, via USB)
Fast create option

Created the container in roughly 30 min

[BoffinBrain]

I've encountered this a few times over the last year. Good to see that it's being addressed.

So, if I'm reading this correctly, a quick format effectively takes the same amount of time as a full format, but due to the way Veracrypt shows the progress at 100% after getting to the backup header, but before it's actually been written to disk, we don't get any update until the I/O has caught up.

[idrassi]

@BoffinBrain: You've got the right idea. The "Fast Create" option is essentially our workaround to the Windows default of zero-filling a file.

From a pure performance angle, both Quick and Full formats might seem identical, but they diverge significantly in terms of security. During a Full Format, VeraCrypt fills the entire file with random data, ensuring attackers can't gauge the real usage of the file container. Meanwhile, in a Quick Format, Windows populates the file with zeros. This unfortunately allows attackers to deduce the actual amount of data written to the container, even if they can't decrypt the contents.

The "Fast Create" option, while speedy, has a security downside. Windows fills the file with lingering disk content (previously deleted data that's still in the disk sectors). This could inadvertently expose sensitive remnants to an attacker. To clarify, while they can see this content, they still can't access the encrypted data within the container.

[yangsfang]

Sorry for not responding (I've switched away from Veracrypt due to various reasons). But as the OP, I want to help you close out issues that have been fixed. So I ran the same test as I did in comment 1.

Spec:
VeraCrypt 1.26.7 (64-bit)
Windows 11 Pro 22H2 (22621.2070)
External drive: WD My Passport Ultra Metal 2TB (Formatted in NTFS)

1. Open VeraCrypt (installed in Program files)
2. Create Volume
3. Create an encrypted file container
4. Standard VeraCrypt volume
5. Select File -> save to a file on external harddrive (Never save history)
6. Leave Encryption Option defaults (AES, SHA-512)
7. Enter Free space, 130GB
8. Enter password (<20 characters), use keyfile (4096bit, generated by VeraCrypt)
9. Large File Yes
10. Filesystem exFAT, Cluster default, Quick Format
11. Move mouse around until randomness is green, click Format. Dismiss warnings and continue.

I can confirm that the progress bar is now showing progress of the file creation. It will successfully create the file when left alone.

Further, when the format is in progress, I can abort the format, and Veracrypt will not hang.

So I consider the original reported issue fixed. If anyone else have trouble with a similar issue, they can open a new issue with their exact repo steps.

---

@xnoreq Thank you for your contributions in #690 (comment). You identified various issues both in the underlying backend and the UI.

@idrassi I have fixed the issue here:

First, there was a bug in your SetPrivileges implementation which didn't check if it actually succeeded: df8254d

Secondly, Format.exe didn't request the SE_MANAGE_VOLUME_NAME privilege required for SetFileValidData to succeed: 2d44395

To users: if this is merged then you have to start VeraCrypt Format.exe /fastcreatefile as administrator!

@xnoreq I don't see your pull request to this repository. I see that you fixed it in your repo. Do you mind opening a PR to this repo?

"Quick Format" option only applies to VeraCrypt formating of the file container, it doesn't affect how Windows creates the file itself.

@idrassi I'm sure there are various technical reasons causing this. But this is really a user experience issue - nothing the user does should result in the UI hanging for hours not showing any progress and the computer hanging and refusing to shut down. It doesn't really matter if quick format is quick or not quick, as long as progress is being shown so the user knows what the computer is doing and they have the option to cancel (and that cancel actually works).

I can confirm that: Win 10 64b VeraCrypt 1.26.5 (beta), 64b 16 TB container (exFAT, via USB) Fast create option

@DStrelak This issue is about Quick Format and not Fast Create. As explained in #690 (comment), they are different.

---

Thanks all.

[LGFox]

I used "Fast Create" option while creating a 1TB encrypted file container on 6TB disk, but it corrupted file system on entire 6TB disk. Now the whole 6TB disk is displayed as RAW and I can't access my files on it. VeraCrypt supposed to create a file container. How did it corrupted the physical disk instead?