Volume mount performance issue

[erlotsman]

Issue description

When using the Azure Files integration with Azure Container Apps (as noted in the documentation), there are performance issues with the application and read/write operations inside the container.

Steps to reproduce

1. Deploy a WordPress image without volumes in Azure Container Apps.
2. Make sure everything is working properly.
3. Deploy a WordPress image with volume (/var/www/html).
4. The site starts to load in 10-15 seconds.

Expected behavior [What you expected to happen.]
I expect to get a fix for performance issues when using Azure Files integration.

Actual behavior [What actually happened.]
Now the site is loaded in 10-15 seconds.

[dariagrigoriu]

Please clarify what is the baseline here, how Azure Files are used specifically for the WordPress content, if Azure Files and the Azure Container Apps resources are in the same region, and what is the difference in performance with and without Azure Files.

[erlotsman]

Hello @dariagrigoriu, @howang-ms
Azure Files is used for volume.
Mounting takes place along the path /var/www/html.
The services are in the same region and in the same resource group.
All mounting actions were carried out according to the documentation: Use storage mounts in Azure Container Apps

The difference is that without volumes, the site loads quickly as expected.
With volume, the read/write speed deteriorates, and page loading can take up to 15 seconds.

[olivomarco]

From your setup it seems that you are running your entire WP install from Azure Files (your WP image will read lots of files for every request, as per how WP is built with all included files). I am wondering if you are incurring Azure Files limits with the traffic you are describing: https://docs.microsoft.com/en-us/azure/storage/files/storage-files-scale-targets
This could be diagnosed by looking at Azure Files metrics when traffic occurs.

One additional way you might want to try is to create a custom container WP image with everything (plugin, themes) packed in, and use Azure Storage only for uploads, maybe mounting the filesystem in the container (i.e. only in the wp-content/uploads folder).
One unofficial option (without mounted filesystems but with usage of a plugin for uploads) is described here: https://github.com/olivomarco/wp-container-apps/blob/main/wordpress-setup.md

HTH

[erlotsman]

Hello @olivomarco
We have standard limits as stated in the documentation

it is these restrictions that slow down the application

The 2nd approach is not suitable for us since we need to be able to install and change plugins from the admin panel.
Using this approach and the plugin will result in a data loss that will change outside the uploads folder.
Have no idea how to remove restrictions from Azure Files?

[olivomarco]

@erlotsman it's not a matter of storage limits: even if those could be upgraded, they cannot be upgraded to infinity, and there will always be a breaking point in your architecture. Containers scale as traffic comes in, while a fileshare cannot easily scale on any cloud.

I think you should focus on building a scalable architecture for WP. In the solution I proposed you can always install&upgrade your plugins and themes in your git, build the container image and run it (and that is scalable to virtual infinity: containers can scale horizontally without issues)
Yes, you lose the easiness of installing and upgrading WP with a click as a result (which is probably better in terms of security, BTW, since no change made by an attacker would be persistent across restarts), but scalability is ensured.
You could argue that even in my solution, where I use blob storage for uploads, I could encounter scalability limits: true, but you can still use CDN in that case to offload, since its content is user-facing.

[erlotsman]

That's all well and good, but not using volumes to solve a performance problem is not a satisfactory solution.
We expect to get a workable solution with volumes. Volumes are needed so that the data inside the container does not disappear. Your plugin approach works if you add plugins and update WordPress using an image instead of the admin panel.

[howang-ms]

@erlotsman , the performance issue seem to be the Azure File Storage itself, it is not related how Container Apps integrate with it.
Are you using the Standard tier Azure Storage? One of the options is using the Premium File Share, which is designed for low latency and high IOPS scenario.

More information can be found here:
https://docs.microsoft.com/en-us/azure/storage/files/storage-how-to-create-file-share?tabs=azure-portal
https://docs.microsoft.com/en-us/azure/storage/files/storage-files-scale-targets

[erlotsman]

Hello @howang-ms

I tried to use two types: premium and standard. In premium mode, uploading large files is faster, but reading a large number of small files is also slow.
app services have blob mount and it works fast, but unfortunately, this feature is not implemented for the container apps :(
for the container apps, the ability to use a file share is implemented, which is slow

[StianHaugland1]

I have the same issue. Ive tried both container apps and app service and mounting a volume just makes the wordpress instance unusable...

[frnode]

+1 Same issue.

[SophCarp]

Hi @frnode! Unfortunately, this is an issue from Azure File Share, and we can't do much from ACA.

[Josvds]

+1 same issue

[dhilgarth]

@SophCarp You could implement other volume mount possibilities such as Azure NetApp Files, Azure disk storage or Azure Blob Storage

[cforce]

Same here. Added a sleep into into init container . ugly hack
https://learn.microsoft.com/en-us/answers/questions/619134/very-slow-azure-file-volume-mount-on-azure-contain

[KianoMon]

@SophCarp Any update on this or do you know if there is a separate issue raised with the Azure Files team?

[DenislavPopov]

Same issue here, looks like the IOPS of Azure File Storage is not performant enough.

[mag-chang]

I have faced same issue using WordPress on Container Apps.

So, I tried to avoid this issue and I got good performance using syncing mounted Azure File Share to local storage such as /var/html/www.

I think we should copy or sync to local storage from mounted File Share for now.