Very bad performance on NVME SSD

[stylersnico]

Hi,

Please don't take that as a complain, I only wish to help improve the software so I can use it.

I achieve very bad performance on Windows 10 LTSB 2016 with a Samsung EVO 960 250Gb and Veracrypt AES encryption:

Before :

After :

My hardware :

I7 7700k @ stock
16Gb Corsair DDR4
Samsung 960 Evo 250Gb with up to date Samsung NVME driver
Windows 10 LTSB 2016 up to date
Veracrypt 1.19 64 bits

Veracrypt Benchmark :

How can I manage to get close performance to non encrypted drive ? Is this even possible ?

Thanks and best regards,
Nicolas

[ghost]

It is problem not only for SSD.
I got the same problem with 2.5'' SSD and HDD with hardware RAID0.
Very low IOPS... 3-10 times less depending on hardware.
Common symptom: writing is less affected than reading.

Due to this issue CPU speed does not make a lot of sense to estimate how fast disk encryption will be.
So it would be very good if somebody can help to at least understand the reason.

UPDATE

I had to switch my storage system to DiskCryptor solution due to this issue.
It has approximately the same speed as unencrypted volume.
Hardware HDD RAID0 with on-board RAM cache gives me 10 times more read IOPS and +170 MB/s for linear read speed. Single SSD setup has 8 times more IOPS and + 90 MB/s for linear read.
So I can say it is 98% of "raw" speed.

The only drawback of DiskCryptor for me is no audits.
But I cannot accept so dramatic speed regression even if VeraCrypt is successor of TC and it was audited.
I read some opinions that TC was also slow on fast SSD disks. So I afraid it is some architecture problem initially created in TC.

[RcTomcat]

Is there any news on this matter?
It seems the iops still decrease drasticly.

[stylersnico]

@RcTomcat Unfortunatly I think it's not gonna change.
If you want strong encryption it's the price to pay :/

[InSys]

No news on this task? I have the same problem:

Without encryption (Windows 10, samsung ssd 960 evo m2 500gb):

With encryption (AES, mounted file container)

As you can see in the screenshots - the speed is less than 10-20 times. read more

Are there ways to solve this problem?

[304NotModified]

@InSys I'm curious how DiskCryptor performs on your system.

[tomasz1986]

I have the exact same issue, just on regular SATA SSDs.

DiskCryptor and BitLocker both perform very well (maybe ~5-10% decrease in the disk I/O), but performance with VeraCrypt is just abysmal. And yes, it is noticeable in normal workload, especially when operating on many small files at once. For this reason I have been using BitLocker for now (because DiskCryptor seems abandoned and does not support GPT).

I do not have top-notch hardware here but still... (AMD A10-7860K, 16GB RAM, 2 x SanDisk x110 256GB, Windows 10 LTSB 2016).

[InSys]

@304NotModified

Comparison of VeraCrypt and DiskCryptor on SSD:

As you can see, the performance is practically not degraded by DiskCryptor, but on VeraCrypt the performance drops in many times.

Click to view comparison of VeraCrypt and DiskCryptor on HDD

In order not to create a real HDD partition, I used a standard windows virtual disk real placed on single HDD: The degradation in performance on single HDD is not noticeable.

[nshtg]

Is there any effort to fix those ridiculous performance penaltys? Affect normal SSDs aswell not only NVMe ones.

[valnar1]

I noticed the same thing on a new laptop I planned to encrypt. I've loved TrueCrypt and VeraCrypt for a while now, but due to time constraints had to buy BestCrypt. I hope the devs get this figured out.

[ghost]

@tomasz1986

because DiskCryptor seems abandoned and does not support GPT

Why do you need GPT if you do not have partition more than 2TB?
Even large array are possible with RAID controllers (including integrated RAID).
Just create 2 disks: 1st - MBR (for booting, small up to 2TB), 2nd - GPT (any size).

[dartraiden]

Why do you need GPT

GPT is required for disabling CSM, And disabled CSM is required for SecureBoot.

[NameTheJew]

BUMP.
Is this still an issue?
Will using VeraCrypt FDE seriously drop write speeds down to 1MB/s?
Is there a Fix?

Also how is Trim and Provisioning effected when software FDE is implemented?

[CrownVic2]

I just encrypted my system drive with VeraCrypt (ver 1.23) on the almost identical HW configuration as listed in the first post. I see the same slow read speed in Samsung Magician and in CrystalDiskMark. Sequential read is down to 20% . Bitlocker compared to no-encryption is almost identical.

Now the good news. When I measured the real-time for programs to open, there wasn't much of a difference. Lightroom and Photoshop both take approx 2 seconds to be ready, just like with no encryption. Starting another windows 10 from Oracle VM VirtualBox takes 21.5 seconds with no encryption and 23 seconds with Veracrypt.

All programs are installed on my Samsung EVO 960 500GB and the startup time for each program is measured after a fresh system reboot.

[valnar1]

I know it costs money, but I'm very happy with BestCrypt. Also, the older DiskCryptor works great for a MBR SSD if you don't mind disabling UEFI and all the modern settings.

[ghost]

yeah but who wants to use something not open source to secure their shit

[valnar1]

Most people. We use Sophos Safeguard at work. Many use CheckPoint FDE. I build VPN's with Cisco, Fortinet, Palo Alto all day long. AES is a standard. Being open source is irrelevant.

[ghost]

Most people. We use Sophos Safeguard at work. Many use CheckPoint FDE. I build VPN's with Cisco, Fortinet, Palo Alto all day long. AES is a standard. Being open source is irrelevant.

Such tools like FDE, encrypting messengers and so on MUST be open source!

P.S. Let's stop flooding here.

[bxkx]

I've just ran into the same problem. Unfortunately I can't see a solid alternative... Bitlocker isn't available on all Windows versions and DiskCryptor is dead.

Btw, does your SSD also show a disk activity of 100% when only writing about 20 MB/s with a response time of a couple of thousands ms? I do have this problem when moving things from a HDD to encrypted SSD.

[valnar1]

bxkx, the people here would tell you not to use Bitlocker. Cuz it's not open source. sigh...

[ghost]

Plus bitlocker uses hardware-based encryption... which has it's own problems.

https://www.howtogeek.com/fyi/you-cant-trust-bitlocker-to-encrypt-your-ssd-on-windows-10/

[idrassi]

VeraCrypt tends to be slower for random read/write access because of its driver architecture and the way it handles IRP for I/O.

Because VeraCrypt supports file containers (which is not the case of DiskCryptor and Bitlocker), it can not handle IRPs in place and it must create a new IRP to the holding file for every read and which in turn causes a thread context switch. This explains the performance drop.

VeraCrypt uses the same IRP logic for both file containers and disks. Ideally, we should handle IRP in place for disks without context switch to have maximum performance and this requires a big architecture in order to keep both IRP logic working at the same time (a user can mount a file container and a disk simultaneously).

Some time ago I have implemented a prototype with a dual logic for file containers and disks but there was huge stability and reliability problems and I could find an easy solution for them.

Clearly the next version of VeraCrypt must address this and I hope there will be enough resources to work on this. Of course, external help is welcomed for such complex technical task especially that we want to keep file containers support.

[ghost]

@idrassi thank you for the explanation!

[ghost]

It explains a lot, but regarding this:

VeraCrypt tends to be slower for random read/write access

All my above posts are about sequential read and write.
So most likely it is also affected in some way by context switching.
I am using Far Manager to measure real file reading (to nul device or to another physical disk).

[JsBergbau]

Is there any update about the issue / an estimated release date for solving this issue? VeraCrypt is a great software and I would never trust Bitlocker in any way.

EDIT: So if there would be two different IRP paths the performance suffer would still exist for containers. So If you have for example a D: drive on a fast storage you should not create there a container and mount it, but instead encrypt the whole device and mount that for maximum performance. Right?

[daiveedx]

Bonjour @idrassi, based on your comment about IRP, what about having 2 separate "versions" of Veracrypt, one that supports both disk and file as you envision and one that implements IRP in place, supports only disk container and doesn't incur such performance loss.
Users could then have the choice.
I've been using Veracrypt with joy for so long but the more I invest in high-perf SSD the more I feel the pain of performance loss :(
Would you give guidelines to potential contributors on where exactly to look in the code for such separate implementation? Or maybe the dual logic prototype you mentioned is indeed about that?

[er1z]

@steamp0rt — it is possible to disable hardware encryption in group policy.

[fti7]

Any Updates here? This is a really high performance impact

[imennodenis]

Agree with @daiveedx comment. It would be nice to have two versions or even better to have an option while installing VeraCrypt (to support file containers or not).
@idrassi, what do you think?
Thanks!

[PhamHoangLong]

I want to donate for the project, but the only problem stopped me is this issue. We can think in 2 ways :
1 - Dev of this project does not want to fix this problem.
2 - This project is dead.
The very bad thing of Veracrypt's Dev is They don't fix it after 7 years.
I will donate when they fixed this problem.

[AdiK87]

Too bad. Wanted to use veracrypt. Would have happily donated some money. This makes the project unusable.

[Fireballcz]

Too bad. Wanted to use veracrypt. Would have happily donated some money. This makes the project unusable.

I must partially disagree. VeraCrypt is not unusable. There is no trustworthy alternative for Windows. By "trustworthy" I mean "open" + maximally realiable with bulletproof stability and FULLY compatible with Win10/11. The problem of low speed has no notable impact during office-like usage.
I always prefer security and privacy above speed.
So, if VeraCrypt is unusable, why not to use Bitlocker or other proprietary shit?

[TheRealKasumi]

So, if VeraCrypt is unusable, why not to use Bitlocker or other proprietary shit?

Because everyone knows there is no real alternative. But to deal with their frustration, they start crying instead of being thankful that awesome ppl have put a lot of effort, time and money into a open-source project that does a great job and even is free for everyone😋. Crying is just trendy these days I suppose😅.

[PhamHoangLong]

@TheRealKasumi
Don't be rude, kid. If there are issues with software, we (all users) have a rights to complain (report) about that. We (all users) have reported this issue about 7 years ago, and now, there are no plans for this. The devs should be have responsibility for their software, even it is free, If you think because this software is free, and you don't need to have any responsibility, Don't make it in the first place, live yourself and no one care. Your brain and your thought need to be fix by yourself, again, don't be rude and have responsible of your comments, kid.
BTW, I have donate for them a lot, and you should thank me, because of my money, it is free for you.
Good luck, kid.
Just in case, Sorry for my bad english grammar.

[TheRealKasumi]

@PhamHoangLong
Im not rude kid. I just see things a little different and have my own opinion on that. Instead of crying to the devs for 7 years, why not get together in a group and fix the issue when it’s so urgent? I guess everyone would appreciate a contribution in that direction. In the end you can’t force any other dev to do it for you. And like it or not… they do it when ever they feel like it. No matter how much you cry.

Also regarding responsibility… it’s not like something is really broken. In facts it is working perfectly reliable and stable. So in the Ende it is nothing more than a optimization problem.

And finally… donating money to open source projects to support them is a nice thing ofc. I am doing it myself regularly. However I feel like you are highly overestimating your impact there. Also it shouldn’t make you feel like you can then decide over other ppl and how they spend their time. So sorry kid, I don’t agree with some points there. No matter if you dislike that or not, that’s my opinion😋.

[AKmatiAK]

Just use Linux with LUKS :P

ok but seriously - we need to find ppl who will fix that and pay them.

[malwarepad]

I made a video for people who just want an outline of this thread with the thought that this issue needs more publicity...
https://www.youtube.com/watch?v=S69j4gbVb-I

[frubart]

Maybe it would be easier to just merge functions from DiskCryptor into Veracrypt and use funding to get an audit on these code parts. The German developer who took care of DiskCryptor also seems to be willing to develop it further if he gets some monetary compensation.

[thericle]

The changes that have been made since the release of version 1.25.9 indicate that this project is far from dead: https://sourceforge.net/projects/veracrypt/files/VeraCrypt%20Nightly%20Builds/

I still hope that this will be solved, since it's a serious issue and NVMe drives are becoming more and more popular, but I can understand that it can be very difficult, if it requires a change in architecture, especially for a security software where you have to make sure that any change doesn't introduce any new vulnerabilities.

[icv720]

Does this issue affect Linux device containers encryption too?
Because AFAIK (but I may be wrong) VeraCrypt in this case would make use of native encryption services (i.e. dm-crypt) for read/write data so I don't expect a massive performance loss.
Does anyone have an experience to share?
Thank you all.

[gashtal]

@idrassi This particular issue/feature request remains to be by far the most commented-on issue on this repository, and I doubt there is any fix/feature that is more sought after than this by VeraCrypt's community. You mentioned before that you had previously attempted a work around to improve VeraCrypt's performance but it didn't work out in the end due to reliability problems. Is there any more development happening in that direction in the background so that hopefully this performance problem could be addressed sometime in future, or is it a lost cause?

[Fireballcz]

Let me think a bit: if "suddenly", as by miracle, VeraCrypt appears with a solved problem, I'd rather wait for the evaluation of the "purity" of the source code...

[friedensfurz]

Hey @idrassi,
thank you for your continuous efforts on maintaining and improving VeraCrypt. I'm curious about the prototype with dual logic you mentioned a while ago. Do you mind sharing it, on a separate branch perhaps? It would be a great starting point for the community to contribute and help address those stability challenges.
Thanks in advance.

[PowerCoder]

I'm about to switch to PCIe 4.0 NVMe drives on two systems. Is this still an issue? One of those systems can not afford the slowdown. Which means I will finally have no other choice than to surrender to Bitlocker.

[ms2048]

Don't forget about DiskCryptor. I have switched to it a few years ago. It has less functionality than VeraCrypt and is Windows-only - but it is open source and is specifically designed for system volume encryption and encrypting whole partitions in general, and as such does not suffer from the performance issues. On SATA SSDs and on older NVMe drives it gives almost full speed (vs no encryption). Haven't used the newest ones yet.

However, over a year ago I have encrypted with DiskCryptor a virtual 100TB image formatted as NTFS, delivered by Synology ds1821+ via iSCSI with 8 physical 20TB IronWolf Pro disks in RAID6 via 20 GBit dedicated link.

I'm using it daily since - no issues so far! It doesn't reach 20 GBit for some reason, maxes out at about 4-6 GBit sustained read/write (probably because I haven't used NVMe cache yet) but it is still a lot faster than VeraCrypt.

[tradewatcher]

I'm no developer but I think one of the main performance penalties come from the fact the Veracrypt encrypt the whole drive (Its not NVME related, its probably NAND related). Keep in mind Veracrypt is from a time HDD was common.

When a drive is 100% full u have performance issues. Maybe a workaround could make ~25% of the drive unavailable and unused. So the SSD isn't "full" and the performance should heavily increase compared with 100% usage. 25% less amount of space is a noticeable penalty and maybe 10-20% "hidden" space is enough to dodge the major performance penalty. I think its worth to test this and this change doesn't affect the cryptographic way how Veracrypt works.

P.S.: I would also love to donate if Veracrypt become useable again in an acceptable way on NVME / SSD drives.

[Blacklands]

I'm no developer but I think one of the main performance penalties come from the fact the Veracrypt encrypt the whole drive (Its not NVME related, its probably NAND related). Keep in mind Veracrypt is from a time HDD was common.

It doesn't seem like this is the issue. But you can easily test it by using your SSD manufacturer's software to enable over-provisioning and have some percentage (20-30% if you want to be extreme) of the drive reserved for that. (Although I think high-end SSDs already have some over-provisioning built into the drive by default that you can't even turn off, as well.) Then Veracrypt won't even see that space on the drive. If you actually find a big performance difference, please report back!

[Fireballcz]

Tested with Samsung SSD with OP 20% and without OP many month ago. The speed results did not differ.
BTW: if there is no free space on SSD (10% min. recommended) the SSD performance is a bit lower. But just notable, not about 10% of unencrypted value. The poor SSD/NVMe performance is a core problem of Veracrypt.
As always I must say: thanks for Veracrypt. Security has bigger value than the performance itself. I can live with it. I have no problem with encoding vids, rendering graphics atc.

[ms2048]

I'm no developer but I think one of the main performance penalties come from the fact the Veracrypt encrypt the whole drive (Its not NVME related, its probably NAND related). Keep in mind Veracrypt is from a time HDD was common.

Nope, the exact reasons have been explained long time ago by the developer himself in the comment above: #136 (comment)

P.S.: I would also love to donate if Veracrypt become useable again in an acceptable way on NVME / SSD drives.

This would not help, unfortunately. I'd assume that the reverse would be helpful instead: paying for the development of the feature upfront. Provided it is not easy, it would also not be cheap though.

[wiesl]

@idrassi I unterstand that IRP handling under Windows (e.g. Windows 10) is not perfect. But why does performance drop down after some runtime/data written/data read (e.g. 50GB with NTFS) from e.g. ~10MByte/s to ~200kB/s.
And even 10MByte/s on a modern computer isn't much if something is copied.
Any ideas why this happens?
Might this be an IRP leak?
Unmounting/mounting/rebooting helps to get performance at least to 10MByte/s back.
Thnx.

[Fireballcz]

~10MByte/s to ~200kB/s?
This enormous drop is a system/drive caching problem, not VC problem. Cheap drives with "noname" chipsets, thats it. Im a long time user of TC and VC since its beginning. I NEVER met such performance drop, you are describing. The problem of VC is the maximum speed itself. If you have a 8500MBps NVMe drive and you are getting max about 80-150MBps performance.

[wiesl]

You are talking about MBit/s and not MByte/s?

If MBit/s, then it is about in the same range in MByte/s (sometimes it goes up to 20MByte/s).

To be precise: it happens with several WD Elements 5TB USB devices but also with several Samsung T7 Portable SSD - 2 TB USB devices.
Tried already to enable Windows cache in the drive settings without sucess.
Any ideas for this case?

[Fireballcz]

You are talking about MBit/s and not MByte/s?

MEGABYTES per second. Sorry Im NOT talking about external USB stuff (I have missed, that you are) but about internal NVMe drives. USB stuff is always much slover. Im not using another external drives than GB LAN devices.

[Yszty]

Encrypted samsung 990pro 2TB

---

Not encrypted

[luckylinux]

I am thinking about deploying Veracrypt Full Disk Encryption on an (very) Old System with a small SSD.

The performance seems to be quite bad indeed.

I know most of the Following Description is NOT for Windows, but I still believe there might be something similar going on.

I wonder if the Issues outlined here are similar to what happens in the GNU/Linux World (and maybe also FreeBSD) with ZFS: there is a HUGE write amplification going on (depending on file size and ZFS recordsize/volblocksize Dataset-ZVOL Settings) especially when running KVM Guests (Proxmox VE Host with ZFS, Guests typically using EXT4 as a Filesystem using a ZVOL Virtual Disk).

The Write Amplification can easily reach 20-50 x (meaning a fairly small 10 GB/day write inside a VM can result in 2000 - 5000 GB/day being written in the NAND) !!!

And it's also related to IOPS: heck I can't even use nano to open a 10 KB file without having to wait 30 Seconds for it to save.

One helper could be to turn off atime (for Access Time) using the noatime Option in GNU/Linux for the / Filesystem.

A quick Google Search brings up this to Disable Access Time on Windows.

In this way, each time you READ a file, you will NOT cause an additional WRITE. That will improve IOPS Performance and also reduce the SSD Wear/Tear.

The other possibility is that the clustersize (Unfamiliar with the Windows Term, sorry) is not configured Correctly. Just like ZFS ashift should be configured Correctly depending on the Disk Sector size (512B or 4kB = 4096B), maybe something is going on there.

So, when you combine the two, it might look as follows: you READ one (in reality: A LOT) File of say 6kB. Maybe your clustersize is set to 64kB. Just to update the Access Time (atime), you will need to WRITE a 6kB "physical" File but it will actually take 64kB, but of course the NAND has 4kB sectors, so you already see where this is going.

And maybe (on top of that) Veracrypt has ANOTHER value of sector/block/cluster size than NTFS ! Then you will have to write 64kB in another portion with the updated timestamp. And maybe TRIM is also not activated, so cleanup will be extremely slow.

Can anybody Benchmark if Write Amplification is what is causing the Issue here ? I mostly gave up on Windows, but I believe there are some Experienced People around here that could help 👍 !

EDIT 1
As a "Crude" benchmark you could copy a bunch of small Files from one Place to the Other and compare the SMART (using Crystal Disk Mark for instance) value of Total Bytes / LBAs Written. Then compare how much you transferred (e.g. 100000 Files x 0.01 MB each = 1'000 MB ~ 1 GB) to what SMART Reports. Compare total bytes WRITE before and after (maybe also the READ if you have atime turned on).

I suggest doing this with small Files as random I/O and IOPS falls much faster than sequential Writes. There is also more write Amplification and "fragmentation" / Overhead with small Files as well.

But of course, while doing this test, make sure that there is no significant "other stuff" going on (e.g. Antivirus Scanning, File Copy, several other Processes running, etc).

[gregtwallace]

Some time ago I have implemented a prototype with a dual logic for file containers and disks but there was huge stability and reliability problems and I could find an easy solution for them.

@idrassi Can you post the commit(s) for the prototype so others can review and potentially help?

[jbmorris289]

Figured I'd post my experience as well. Because SSD Model seems to be important, I am using a TeamGroup MP34 SSD as my boot drive. CPU has AES-NI capability.

[friedensfurz]

Also the VC Encryption parameters matter a lot. Here's a comparison for a Samsung 970 Evo 2TB SSD, (2048,128,64) seems to be a lot better for sequential writes:

[kriegste]

I personally would be happy with a non-container version of VeraCrypt that can only mount partitions but at full speed.