Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Public dependency manager #297

Open
wants to merge 13 commits into
base: master
from

Conversation

Projects
None yet
3 participants
@moltob
Copy link
Contributor

commented May 12, 2019

In order to implement #288, this PR stores the dependency manager in a global registry class before task are loaded or executed.

This allows to get it from any code inside a task generator, e.g. from clean to read the previous execution result, see the contained sample.

The original discussion suggested to implement the registry as plain Python dict. I'd like to prevent string-based access and therefore implemented a very simply static class in new module doit.globals.

@coveralls

This comment has been minimized.

Copy link

commented May 12, 2019

Coverage Status

Coverage increased (+0.7%) to 99.736% when pulling 2ddf444 on moltob:feature/public-dependency-manager into 2b59b89 on pydoit:master.

@moltob

This comment has been minimized.

Copy link
Contributor Author

commented May 12, 2019

@schettino72 Where would I link a sample snippet into the docs, what would be a good spot?

@schettino72

This comment has been minimized.

Copy link
Member

commented May 12, 2019

I will rebase this before reviewing it...

@schettino72

This comment has been minimized.

Copy link
Member

commented May 12, 2019

Actually quite complicated to rebase this to master... I gave up.
I think it is easier no manually apply your changes. sorry for the trouble. I could do that myself later, not today.

Your Globals as Registry implementation is ok. I think the get_dep_manager turned out be useless. We can just ask user to import Globals and remove get_dep_manager. What do you think?

Regarding the documentation, your example i would like to put in a new HOW-TO section. dont worry about that now, we can do it later.

But Globals would need to be documented now, not sure where... here I would you use a simpler example (with just actions without clean).

@moltob

This comment has been minimized.

Copy link
Contributor Author

commented May 12, 2019

Don't worry, I'll fix the branch and will incorporate your comments.

@moltob

This comment has been minimized.

Copy link
Contributor Author

commented May 12, 2019

@schettino72 The diff is clean now, you can add your comments for the files, but I can easily just remove the access function as well. I'll do that tomorrow.

@moltob

This comment has been minimized.

Copy link
Contributor Author

commented May 13, 2019

I just rebased the code cleanly onto master locally, will push it tonight.

Regarding the sample I'd like to keep the clean. IMHO this is one of the main use cases for that feature. A slightly simplified sample will be part of the pushed branch.

@moltob moltob force-pushed the moltob:feature/public-dependency-manager branch from 232c228 to ce47f50 May 13, 2019

@moltob moltob changed the title (WIP) Public dependency manager Public dependency manager May 13, 2019

@moltob moltob force-pushed the moltob:feature/public-dependency-manager branch from b497297 to e22babe May 13, 2019

@moltob moltob force-pushed the moltob:feature/public-dependency-manager branch from e22babe to 0cca250 May 13, 2019

@moltob

This comment has been minimized.

Copy link
Contributor Author

commented May 13, 2019

@schettino72 Please pick up review, rebase onto master is complete. Please see my above comments on having clean in the sample.

class Globals:
"""Accessors to doit singletons.
:cvar dep_manager: The doit dependency manager, holding all persistent task data.

This comment has been minimized.

Copy link
@schettino72

schettino72 May 14, 2019

Member

Could you add the type here

@schettino72
Copy link
Member

left a comment

So it is still missing docs, right?
I guess this feature does not fit any of the existing pages... I would create a new page between task_args and uptodate.

Example is better now. There are so many use-cases for this feature...

I noticed sometimes users tend ignore a feature if the example is not trivial and do not reflect their use-case.
So I usually prefer minimal examples where the only feature being show is the one discussed. And leave to the user to figure out how put all doit features together to solve their problems :)

Show resolved Hide resolved tests/test_cmd_base.py
Show resolved Hide resolved tests/test_cmd_base.py Outdated
@moltob

This comment has been minimized.

Copy link
Contributor Author

commented May 19, 2019

@schettino72 Docs added.

The ``dep_manager`` attribute is initialized right before tasks are loaded, which means it allows to
be accessed during *all* task evaluation phases, in particular during:

* Task generation, i.e. from the body of any ``task_*`` function.

This comment has been minimized.

Copy link
@schettino72

schettino72 May 19, 2019

Member

i use the term 'task creation' instead of 'task generation'.

@@ -0,0 +1,31 @@
==========
Singletons

This comment has been minimized.

Copy link
@schettino72

schettino72 May 19, 2019

Member

the file is called 'globals' the the title is 'singletons'... we should use just one name...

you write the docs as if we would have many globals, not sure that would be the case. maybe we could expose the Reporter or some other internal stuff. But those are already documented elsewhere...

what about have this section of docs being specific to 'db backend' or something like that.

Another thing i just realized is that we are calling it 'dep_manager' as it is original function on doit itself.Wondering if we should use a more generic name. In practice users could use this to store some data. like a a task that stores and use a data itself in future runs. for this use-case the name dep_manager is completely innapropriate.

what do you think?

the dependency manager by returning it from the action. A `clean` action is then able to query
`dep_manager`for that result and perform the cleanup action:


This comment has been minimized.

Copy link
@schettino72

schettino72 May 19, 2019

Member

description looks good. but we also need to document the exposed API


The doit dependency manager holds the persistent state of `doit`. This includes relations of tasks
among each other or results, returned from their latest runs. It bascially consists of all the
information stored in `doit`'s database file.

This comment has been minimized.

Copy link
@schettino72

schettino72 May 19, 2019

Member

maybe add a note about doit different backends...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.