unusual disk activity at startup

[xgiovio]

Hello, I have centos 7, latest kernel-ml and latest zfs 2.1.x. All works as it should. I have 3 pools, each of them with 600 snapshots.

What i noticed in the past months is that everytime I boot the os (single ssd), all pools are mounted but using "iostat -y 5" I can see activity of about 1 megabyte/s on each drive for about 5/10 minutes until all of them go to 0 mb/s. I discovered it because i have the system near me and after the start up i can feel the noise of all the discs together working.

I don't understand what is going on. Is it caching from disks to memory all the snapshots references? What cauld be the problem? Considering that i start and shutdown the system everyday , I would't put too much stress on the disks.

I have 50gb slogs, l2arcs 80 gb. 32gb ram ecc. Max arc size 25gb. No dedup.

I used iostat and zpool iostat to monitor the writing and reading. Iotop to monitor the processes.

The processes active during this readind/writing at statup are:
systemd udev
zed -f

I tried also to disable l2arc rebuild.
I set it in /etc/modprobe.d/zfs.conf and after reboot i can see that /sys/module/zfs/parameters/l2arc_rebuild_enabled is 0
However the reading from disks at startups using zpool iostat -v 1 remains. I need to understand what is the problem.

System information

Type	Version/Name
Distribution Name	centos
Distribution Version	7 latest
Kernel Version	5.15.13
Architecture	x64
OpenZFS Version	2.1.2

Describe the problem you're observing

unusual disk activity at startup

Describe how to reproduce the problem

reboot the system

Include any warning/errors/backtraces from the system logs

[AllKind]

Is it like this: #7132 ?
does restarting zed help?

[xgiovio]

Is it like this: #7132 ? does restarting zed help?

I tried the cmd: journalctl | grep zed

and i get these messages:

Feb 06 19:30:54 shareserver zed[2654]: eid=1593 class=config_sync pool='xpool2'
Feb 06 19:30:55 shareserver zed[2799]: eid=1594 class=config_sync pool='xpool2'
Feb 06 19:30:55 shareserver zed[2801]: eid=1595 class=config_sync pool='xpool2'
Feb 06 19:30:56 shareserver zed[3016]: eid=1596 class=config_sync pool='xpool2'
Feb 06 19:30:56 shareserver zed[3017]: eid=1597 class=config_sync pool='xpool2'
Feb 06 19:30:57 shareserver zed[3134]: eid=1598 class=config_sync pool='xpool2'
Feb 06 19:30:57 shareserver zed[3136]: eid=1599 class=config_sync pool='xpool2'
Feb 06 19:30:57 shareserver zed[3291]: eid=1600 class=config_sync pool='xpool2'
Feb 06 19:30:57 shareserver zed[3292]: eid=1601 class=config_sync pool='xpool2'
Feb 06 19:30:58 shareserver zed[3421]: eid=1602 class=config_sync pool='xpool2'
Feb 06 19:30:58 shareserver zed[3423]: eid=1603 class=config_sync pool='xpool2'
Feb 06 19:30:59 shareserver zed[3604]: eid=1604 class=config_sync pool='xpool2'
Feb 06 19:30:59 shareserver zed[3605]: eid=1605 class=config_sync pool='xpool2'
Feb 06 19:31:00 shareserver zed[3736]: eid=1606 class=config_sync pool='xpool2'
Feb 06 19:31:00 shareserver zed[3738]: eid=1607 class=config_sync pool='xpool2'
Feb 06 19:31:00 shareserver zed[3866]: eid=1608 class=config_sync pool='xpool2'
Feb 06 19:31:00 shareserver zed[3867]: eid=1609 class=config_sync pool='xpool2'
Feb 06 19:31:01 shareserver zed[4009]: eid=1610 class=config_sync pool='xpool2'
Feb 06 19:31:01 shareserver zed[4011]: eid=1611 class=config_sync pool='xpool2'
Feb 06 19:31:02 shareserver zed[4136]: eid=1612 class=config_sync pool='xpool2'
Feb 06 19:31:02 shareserver zed[4137]: eid=1613 class=config_sync pool='xpool2'
Feb 06 19:31:02 shareserver zed[4268]: eid=1614 class=config_sync pool='xpool2'
Feb 06 19:31:02 shareserver zed[4270]: eid=1615 class=config_sync pool='xpool2'
Feb 06 19:31:03 shareserver zed[4403]: eid=1616 class=config_sync pool='xpool2'
Feb 06 19:31:03 shareserver zed[4404]: eid=1617 class=config_sync pool='xpool2'
Feb 06 19:31:04 shareserver zed[4534]: eid=1618 class=config_sync pool='xpool2'
Feb 06 19:31:04 shareserver zed[4536]: eid=1619 class=config_sync pool='xpool2'
Feb 06 19:31:05 shareserver zed[4676]: eid=1620 class=config_sync pool='xpool2'
Feb 06 19:31:05 shareserver zed[4677]: eid=1621 class=config_sync pool='xpool2'
Feb 06 19:31:06 shareserver zed[4804]: eid=1622 class=config_sync pool='xpool2'
Feb 06 19:31:06 shareserver zed[4806]: eid=1623 class=config_sync pool='xpool2'
Feb 06 19:31:07 shareserver zed[4976]: eid=1624 class=config_sync pool='xpool2'
Feb 06 19:31:07 shareserver zed[4977]: eid=1625 class=config_sync pool='xpool2'

This for straight 10 minutes

[xgiovio]

this is definitely a bug
solved removing the autoexpand property and setting it to off

[xgiovio]

issue already present #7132 . closing this