Drawing coordinate grid around map window causes layout resizing problems

[MoltenRainbow]

What is the bug or the crash?

Issue relates to the layout.
When activating a grid around a map window, it appears to generate some sort of buffer or padding around the text labels.
This causes the "resize layout" button to not function properly anymore.
Instead of resizing the layout to meet the borders set along the frame, it generates extra space along the outside,

I have tried it with many different settings, changing values to see if anything makes any sort of difference.
The only two things that changed anything was to either 1)turn the coordinate labels off completely or 2)downsize the font.
Downsizing the font didn't solve the issue but the "padding" around the map frame did decrease. That makes me fairly certain that the problem is somehow related to the labels.

(the settings in the screenshots below are the ones that I have been using for as long as I can remember and I never ran into issues before)

Steps to reproduce the issue

1. Open a layout and set up a mapwindow with an active coordinate grid
2. Allow coordinates to be drawn
3. Press "resize layout" in the layout panel/tab
4. Image does not properly resize and keeps a buffer on all sides where the coordinates have been drawn.

when draw coordinates is turned off again, the resizing is done correctly and no buffer remains around the image.

Versions

QGIS-versie
3.28.11-Firenze
QGIS code revisie
b02458a
Qt versie
5.15.3
Python versie
3.9.5
GDAL/OGR versie
3.7.2
PROJ versie
9.3.0
EPSG-register databaseversie
v10.094 (2023-08-08)
GEOS versie
3.12.0-CAPI-1.18.0
SQLite versie
3.41.1
PDAL versie
2.5.5
PostgreSQL client versie
unknown
SpatiaLite versie
5.1.0
QWT versie
6.1.6
QScintilla2 versie
2.13.4
OS versie
Windows 10 Version 2009

Actieve Python plug-ins
archoltools
0.60
b4udignl2
2.3.0
FeatureGridCreator
3.0.0
pdokservicesplugin
4.1.5
quick_map_services
0.19.33
sminq
0.91
db_manager
0.1.20
grassprovider
2.12.99
MetaSearch
0.3.6
processing
2.12.99
sagaprovider
2.12.99

Supported QGIS version

• I'm running a supported QGIS version according to the roadmap.

New profile

• I tried with a new QGIS profile

Additional context

No response

[agiudiceandrea]

Also reported for QGIS 3.34: #54931.