[8.0] fix(wms): correctly log the pilot job reference during the matching process

[aldbr]

Closes #7884

BEGINRELEASENOTES
*WorkloadManagement
FIX: correctly log the pilot job reference during the matching process
ENDRELEASENOTES

[aldbr]

https://github.com/DIRACGrid/DIRAC/actions/runs/12144642670/job/33864474014?pr=7924#step:10:935

--- Logging error ---
Traceback (most recent call last):
  File "/home/dirac/ServerInstallDIR/diracos/lib/python3.11/logging/__init__.py", line 1113, in emit
    stream.write(msg + self.terminator)
ValueError: I/O operation on closed file.
Call stack:
  File "/home/dirac/ServerInstallDIR/diracos/lib/python3.11/weakref.py", line 666, in _exitfunc
    f()
  File "/home/dirac/ServerInstallDIR/diracos/lib/python3.11/weakref.py", line 590, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
  File "/home/dirac/ServerInstallDIR/diracos/lib/python3.11/site-packages/DIRAC/Core/Utilities/DictCache.py", line 247, in _purgeAll
    deleteFunction(cache[cKey]["value"])
  File "/home/dirac/ServerInstallDIR/diracos/lib/python3.11/site-packages/DIRAC/WorkloadManagementSystem/DB/TaskQueueDB.py", line 1001, in __deleteTQIfEmpty
    result = self.deleteTaskQueueIfEmpty(tqId, tqOwnerDN, tqOwnerGroup)
  File "/home/dirac/ServerInstallDIR/diracos/lib/python3.11/site-packages/DIRAC/WorkloadManagementSystem/DB/TaskQueueDB.py", line 1027, in deleteTaskQueueIfEmpty
    retVal = self._query(sqlCmd, conn=connObj)
  File "/home/dirac/ServerInstallDIR/diracos/lib/python3.11/site-packages/DIRAC/Core/Utilities/MySQL.py", line 717, in _query
    self.log.debug(f"_query: {self._safeCmd(cmd)}")
  File "/home/dirac/ServerInstallDIR/diracos/lib/python3.11/site-packages/DIRAC/FrameworkSystem/private/standardLogging/Logging.py", line 352, in debug
    return self._createLogRecord(LogLevels.DEBUG, sMsg, sVarMsg)
  File "/home/dirac/ServerInstallDIR/diracos/lib/python3.11/site-packages/DIRAC/FrameworkSystem/private/standardLogging/Logging.py", line 417, in _createLogRecord
    self._logger.log(level, "%s", sMsg, exc_info=exc_info, extra=extra)
Message: '%s'
Arguments: ('_query: SELECT TQId FROM `tq_TaskQueues` WHERE Enabled >= 1 AND `tq_TaskQueues`.TQId = 26 AND `tq_TaskQueues`.TQId not in ( SELECT DISTINCT TQId from `tq_Jobs` )',)

[aldbr]

Update: ok funny, this is not a new bug: https://github.com/DIRACGrid/DIRAC/actions/runs/12140642296/job/33850877559

I am not sure to fully understand, but it looks like it is due to the weakref. When we are in __deleteTQIfEmpty, it's like the streams have already been cleaned up, and the logger, even if it's still alive, cannot emit the messages anymore.

May be we should not try to log/print messages in methods used by weakref? But then we take the risk of missing some errors

[fstagni]

This was not harming.

[aldbr]

Then, what's the point of having the following piece of code here?

https://github.com/DIRACGrid/DIRAC/pull/7924/files#diff-6e5904ade77b6cd702bb41532101e8027ba0ebc3a3430c98332efd246df5af5eR55-R56

[fstagni]

It was probably to handle cases of old pilots without a pilotRef.

[aldbr]

Thanks for the clarification, I reverted my change.

[fstagni]

All the logging code is in DIRAC.FrameworkSystem.private.standardLogging right? You can add this code in one of the existing module (here it feels a bit "lost").

[aldbr]

Indeed, but then it becomes weird to import a package coming from <something>.private.<something_else>.

May be the best solution would be to add it somewhere in DIRAC.FrameworkSystem.private.standardLogging and then call it through DIRAC.FrameworkSystem.Client.Logger.

[aldbr]

I applied the changes

[DIRACGridBot]

Sweep summary

Sweep ran in https://github.com/DIRACGrid/DIRAC/actions/runs/12317774899

Failed:

• integration
  cherry-pick d5aebd1 into integration failed
  check merge conflicts on a local copy of this repository

  git fetch upstream
  git checkout upstream/integration -b cherry-pick-2-d5aebd11d-integration
  git cherry-pick -x -m 1 d5aebd11d
  # Fix the conflicts
  git cherry-pick --continue
  git commit --amend -m 'sweep: #7924 fix(wms): correctly log the pilot job reference during the matching process' --author='aldbr <aldbr@outlook.com>'
  git push -u origin cherry-pick-2-d5aebd11d-integration
  
  # If you have the GitHub CLI installed the PR can be made with
  gh pr create \
       --label 'sweep:from rel-v8r0' \
       --base integration \
       --repo DIRACGrid/DIRAC \
       --title '[sweep:integration] fix(wms): correctly log the pilot job reference during the matching process' \
       --body 'Sweep #7924 `fix(wms): correctly log the pilot job reference during the matching process` to `integration`.
  
  Adding original author @aldbr as watcher.
  
  BEGINRELEASENOTES
  *WorkloadManagement
  FIX: correctly log the pilot job reference during the matching process
  ENDRELEASENOTES
  Closes #7952'