Autotrim happening even when no autotrim selected

[Rudd-O]

My pool has an L2ARC device, which is getting autotrimmed during normal operation, not when trim is selected (sde is the L2ARC device here, see right axis purple line):

Strangely, the pool is not set to autotrim:

errors: No known data errors
[root@milena ~]# zpool get autotrim
NAME     PROPERTY  VALUE     SOURCE
backups  autotrim  off       default
[root@milena ~]# 

This is massively slowing down performance of the ARC, to the point that not using ARC is simply faster than using ARC.

I'm using latest master on Fedora 39 kernel

[root@milena ~]# uname -r
6.7.6-200.fc39.x86_64

This is likely related to the issue where ARC devices don't get trimmed at all: #14488

Related settings:

[root@milena parameters]# ls *trim*
l2arc_trim_ahead           zfs_trim_extent_bytes_min  zfs_trim_queue_limit  zfs_vdev_trim_max_active
zfs_trim_extent_bytes_max  zfs_trim_metaslab_skip     zfs_trim_txg_batch    zfs_vdev_trim_min_active
[root@milena parameters]# cat l2arc_trim_ahead 
100

Setting this to 0 stops trimming activity altogether. As a result, I/O congestion stalls drop to a minimum and interactive performance of the machine boosts enormously:

I expect the autotrim knob for the pool to turn off automatic trimming, even if the system-wide knob for L2ARC trimming is on. If I wanted the pool to autotrim, I would turn it on or leave it to the default.

[AllKind]

From the original pull request #9789 :

We also implement a "Trim Ahead" feature. It is a zfs module parameter,
expressed in % of the current write size. This trims ahead of the
current write size. A minimum of 64MB will be trimmed. The default is 0
which disables TRIM on L2ARC as it can put significant stress to
underlying storage devices. To enable TRIM on L2ARC we set
l2arc_trim_ahead > 0.

Consider setting l2arc_trim_ahead to a lower value.

[amotin]

Setting l2arc_trim_ahead=0 should disable L2ARC trimming. I could have sense for autotrim property to control it also, but L2ARC trim implementation is just completely different from other types of vdevs, so I am not exactly surprised that it is controlled separately.

I wonder whether #15843 improve the situation in your case. It may depend of what type of device you have that L2ARC on and whether it can execute TRIM simultaneously with data read.

[GregorKopka]

For devices exposed to random writes/frees I get why trim could be helpful to clear up the internal mapping tables of flash based memory... but is trimming L2ARC devices really beneficial/needed, given that they get written in a linear round-robin fashion?

(Bonus question: do the writes of the L2ARC update arrive at the drive(s) as one big coalesced write, so R/M/W in the flash would only happen on the beginning/end of the current write window anyway?)