Skip to content

[8.0] Add arguments to DIRAC.initialize#6335

Merged
fstagni merged 7 commits into
DIRACGrid:integrationfrom
chrisburr:dirac-initialize-improve
Aug 19, 2022
Merged

[8.0] Add arguments to DIRAC.initialize#6335
fstagni merged 7 commits into
DIRACGrid:integrationfrom
chrisburr:dirac-initialize-improve

Conversation

@chrisburr

Copy link
Copy Markdown
Member

In order to make DIRAC.initialize more useful when writing scripts it needs to have arguments to expose some of the functionality which is currently only available via the command line. This pull request adds a proposal for what I think the arguments should be along with improving the user documentation by adding a tutorial on how to use the DIRAC client.

To summarise the some of the new options:

  • require_auth: By default DIRAC.initialize now raises an exception if it was unable to authenticate against the CS (hopefully with a hint about why). Setting require_auth=False disables this behavour.
  • log_level: Allows the log level to be changed. As part of this I've also changed LogLevel to be an enum (the implementation of this could be improved but that's not a user-facing change so it doesn't need to be tied to a release).
  • security_expression: Allows the caller to specify which properties are needed for the given users credentials. This be an expression to allow it to encode multiple paths by which the credentials can have access:
    SecurityProperty.NORMAL_USER | (SecurityProperty.JOB_ADMINISTRATOR & SecurityProperty.FC_MANAGEMENT)

I think it would be nice to replace the current constants with the SecurityProperty enum (and have most of this implemented in a local branch) but understand if we'd rather save that for later.

In the medium term I think LocalConfiguration/initialize/parseCommandLine should be refactored to make much of this functionality available to scripts. This would make it possible to have scripts fail earlier when credentials are missing rather then running until the authentication errors become problematic.

BEGINRELEASENOTES

*Core
NEW: DIRAC.initialize now provides keyword arguments to customise initialisation
NEW: SecurityProperty enum for expressing security properties which can be assigned to groups/hosts
NEW: LogLevel enum for choosing the logging level

ENDRELEASENOTES

Closes #6019

@chrisburr chrisburr marked this pull request as ready for review August 18, 2022 07:03
@andresailer

Copy link
Copy Markdown
Contributor

How do the SecurityProperty enums work with extensibility? At the moment I can just use any string as a property.

@chrisburr

Copy link
Copy Markdown
Member Author

At the moment they wouldn't. Is anyone actually extending them? The current list in vanilla DIRAC contains (unused) LHCb specific properties.

I can think of a few other ways of doing it that would allow extensions to add more properties if that's desirable though it would be good to know what the use cases are first.

@andresailer

Copy link
Copy Markdown
Contributor

Someone wanted to at some point: #4077 (comment)

Comment thread src/DIRAC/__init__.py
Comment thread docs/source/UserGuide/Tutorials/UsingDIRACFromPython/index.rst Outdated
else:
gLogger.warn("Running without remote configuration")
if returnErrors:
return S_OK(errorsList)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels like it deserves a comment why errors are returned in S_OK

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated the docstring to explain the situation.

@chrisburr chrisburr force-pushed the dirac-initialize-improve branch from 8a04231 to f07467f Compare August 18, 2022 13:01

@andresailer andresailer left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

couple more grammar or sphinx style things.

Comment thread docs/source/UserGuide/Tutorials/UsingDIRACFromPython/index.rst Outdated
Comment thread docs/source/UserGuide/Tutorials/UsingDIRACFromPython/index.rst Outdated
Comment thread docs/source/UserGuide/Tutorials/UsingDIRACFromPython/index.rst Outdated
Comment thread docs/source/UserGuide/Tutorials/UsingDIRACFromPython/index.rst Outdated
Comment thread docs/source/UserGuide/Tutorials/UsingDIRACFromPython/index.rst Outdated
Comment thread src/DIRAC/__init__.py Outdated
Comment thread src/DIRAC/__init__.py Outdated
@chrisburr chrisburr force-pushed the dirac-initialize-improve branch from e781ce1 to a33c28a Compare August 18, 2022 15:43

@fstagni fstagni left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not (for me) the easiest PR to understand. I don't think there's a specific hurry here, right?

.. _usingDIRACFromPython:

===============================
Using DIRAC Clients From Python

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is in this rst file is IMHO not a tutorial, but rather should be part of the DIRAC developer guide.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a lot of users that write python scripts against DIRAC for whom the developer documentation is excessive. I think there is a place for a tutorial that only documents clients without details about configuration/services/agents/testing.

Comment thread docs/source/UserGuide/Tutorials/UsingDIRACFromPython/index.rst Outdated
Comment thread src/DIRAC/Core/Security/Properties.py Outdated
@chrisburr chrisburr force-pushed the dirac-initialize-improve branch from a33c28a to 34aa1c9 Compare August 18, 2022 20:09
@chrisburr

Copy link
Copy Markdown
Member Author

This is not (for me) the easiest PR to understand. I don't think there's a specific hurry here, right?

It changes the behavour of DIRAC.initialize() which isn't ideal after 8.0.0 is released. We can discuss tomorrow.

@fstagni fstagni merged commit 3f873de into DIRACGrid:integration Aug 19, 2022
@DIRACGridBot DIRACGridBot added the sweep:ignore Prevent sweeping from being ran for this PR label Aug 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sweep:ignore Prevent sweeping from being ran for this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

parseCommandLine vs initialize

4 participants