WMAgent: Implement adding of opportunistic resources with the new initialization model

[todor-ivanov]

Impact of the new feature
WMAgent

Is your feature request related to a problem? Please describe.

While developing the main parts of the new WMAgent initialization model, we did not consider adding everything that was previously there at once. One of the leftovers (to be implemented once we have everything properly tested) is the step for adding new opportunistic resources.

So far a list of sites was hard coded in the deploy-wmagent.sh script at:

WMCore/deploy/deploy-wmagent.sh

Lines 373 to 386 in de110cc

	echo "*** Setting up US opportunistic resources ***"
	if [[ "$HOSTNAME" == *fnal.gov ]]; then
	sed -i "s+forceSiteDown = \[\]+forceSiteDown = \[$FORCEDOWN\]+" $MANAGE_DIR/config.py
	for resourceName in {T3_US_NERSC,T3_US_OSG,T3_US_PSC,T3_US_SDSC,T3_US_TACC,T3_US_Anvil,T3_US_Lancium,T3_ES_PIC_BSC};
	do
	./manage execute-agent wmagent-resource-control --plugin=SimpleCondorPlugin --opportunistic \
	--pending-slots=$HPC_PEND_JOBS --running-slots=$HPC_RUNN_JOBS --add-one-site $resourceName
	done
	else
	sed -i "s+forceSiteDown = \[\]+forceSiteDown = \[$FORCEDOWN\]+" $MANAGE_DIR/config.py
	./manage execute-agent wmagent-resource-control --plugin=SimpleCondorPlugin --opportunistic \
	--pending-slots=$HPC_PEND_JOBS --running-slots=$HPC_RUNN_JOBS --add-one-site T3_ES_PIC_BSC
	fi
	echo "Done!" && echo

With the new docker container init procedures we can implement it as configurable and make the list to be refreshed at runtime. That way we can avoid hard coded values in the scripts and the need of additional PRs on every request to add a new opportunistic site.

Describe the solution you'd like

Fetch the list of opportunistic resources at runtime (possibly through WMAgent.secrets file or alternative) and check if new resource has showed up and if yes add it as usual. This would reduce the action of joining a new resource of the kind to just a WMAgent reboot.

Describe alternatives you've considered

None - It is a must do

Additional context
This issue is a followup on:

• Refactor/reorganize agent initialization and manage actions #11627

And is part of this meta issue:

• Deploy and run WMAgent with Docker container #11314

[todor-ivanov]

hi @anpicci

As we talked earlier today about the eventual solution of #11722 ... And we that should make this configurable... well for this to happen we'll need a more flexible way of parsing and loading all variables defined in the WMAgent.secrets file. I made the minor change needed and pushed it in my repository. I tested it and it behaves just as before + the additional functionality. So now you can load from WMAgent.secrets file any variable prefixed with "RESOURCE_" and use it later in your code.

Here is the commit: dmwm/CMSKubernetes@master...todor-ivanov:CMSKubernetes:WMAgent_Improve_loadSecrets Once you make your initial PR for fixing the issue I can create a PR to your branch so this change also gets into your final PR rather than having it as yet another one flying around.

[anpicci]

I re-opened this issue, since we need to complete the PR in dmwm/CMSKubernetes first

[amaltaro]

And now it can be closed for good, with the developments made in #11966 and the CMSKubernetes dependency linked in the WMCore pull request.
Thank you Andrea and Todor.