`_pytest.main.Session.gethookproxy` is called many times with the same argument on large suites

[elbehery95]

When running --collect-only on a 3000 tests test suite (about 1000 test files each with 3 def test_*) gethookproxy gets called multiple times with the same fspath argument. Quickly inspecting line_profiler there seem to be a small room for improvement by caching the function call

E.g. the current implementation

(.venv) λ python -m line_profiler collect.py.lprof
Timer unit: 1e-06 s

Total time: 2.35111 s
File: <dir>\pytest\src\_pytest\main.py
Function: gethookproxy at line 543

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
   543                                               @profile
   544                                               def gethookproxy(self, fspath: "os.PathLike[str]"):
   545                                                   # Check if we have the common case of running
   546                                                   # hooks with all conftest.py files.
   547     11003      24086.9      2.2      1.0          pm = self.config.pluginmanager
   548     22006    1350630.7     61.4     57.4          my_conftestmodules = pm._getconftestmodules(
   549     11003     811497.5     73.8     34.5              Path(fspath),
   550     11003      70884.8      6.4      3.0              self.config.getoption("importmode"),
   551     11003      25038.9      2.3      1.1              rootpath=self.config.rootpath,
   552                                                   )
   553     11003      33742.8      3.1      1.4          remove_mods = pm._conftest_plugins.difference(my_conftestmodules)
   554     11003      12653.0      1.1      0.5          if remove_mods:
   555                                                       # One or more conftests are not in use at this fspath.
   556                                                       from .config.compat import PathAwareHookProxy
   557
   558                                                       proxy = PathAwareHookProxy(FSHookProxy(pm, remove_mods))
   559                                                   else:
   560                                                       # All plugins are active for this fspath.
   561     11003      13961.3      1.3      0.6              proxy = self.config.hook
   562     11003       8613.7      0.8      0.4          return proxy

Doing collection against the same test suite but with functools.lru_cache seems to be slightly faster

Timer unit: 1e-06 s

Total time: 0.56982 s
File: <dir>\pytest\src\_pytest\main.py
Function: gethookproxy at line 543

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
   543                                               @functools.lru_cache(maxsize=None)
   544                                               @profile
   545                                               def gethookproxy(self, fspath: "os.PathLike[str]"):
   546                                                   # Check if we have the common case of running
   547                                                   # hooks with all conftest.py files.
   548      1001       2603.2      2.6      0.5          pm = self.config.pluginmanager
   549      2002     482684.1    241.1     84.7          my_conftestmodules = pm._getconftestmodules(
   550      1001      69683.3     69.6     12.2              Path(fspath),
   551      1001       5946.0      5.9      1.0              self.config.getoption("importmode"),
   552      1001       2107.8      2.1      0.4              rootpath=self.config.rootpath,
   553                                                   )
   554      1001       3739.1      3.7      0.7          remove_mods = pm._conftest_plugins.difference(my_conftestmodules)
   555      1001       1030.0      1.0      0.2          if remove_mods:
   556                                                       # One or more conftests are not in use at this fspath.
   557                                                       from .config.compat import PathAwareHookProxy
   558
   559                                                       proxy = PathAwareHookProxy(FSHookProxy(pm, remove_mods))
   560                                                   else:
   561                                                       # All plugins are active for this fspath.
   562      1001       1310.9      1.3      0.2              proxy = self.config.hook
   563      1001        715.7      0.7      0.1          return proxy

Would appreciate your thoughts and feedback regarding this point if possible maybe I can submit this as a PR

[RonnyPfannschmidt]

interesting find, i vaguely recall we added some type of caching for this already, but its clearly not enough

i think adding the cache is a good help for now

[elbehery95]

Seems that pm._conftest_plugins returns different list depending on what is collected so far at the point of calling gethookproxy such that the same fspath returns different values depending on what is collected so far.

Simply adding lru cache will regress some of the tests like test_assertion.py::test_assertrepr_loaded_per_dir since the method depends on another internal state. I am trying to figure out some ideas and will share it here if possible

[RonnyPfannschmidt]

i believe to resolve, we'd have to add a helper for restringing new conftests to the remove lists