Hiding tasks from --listing

[JensRantil]

I have a larger fabric script that contains a bunch of different tasks. Initially I would issue

fab task1 task2 task3 ...

but then the number of tasks grew in proportion so much that I created a single task so that I could issue my command like so

fab bigtask

and it would issue all initial tasks using the execute(...) function. Now when I issue fab --list I get a fairly long list of tasks that I'm 99% of the times am not interested in executing. Request: Be able to hide certain tasks from --listing.

Now you may be wondering why I simply can't remove @task decorator and make basic function calls from my bigtask task. Two reasons:

• Some of the subtasks are using @runs_once decorator while others aren't. I expect that decorator to not work for non-@tasks. Am I wrong?
• I would love future users/colleagues to be able to use my script easily. Therefor, being able to hide certain tasks in the listing would making it way more clear for them which task to execute.

To concretize, my proposal is as follows: Never list tasks with names starting with an underscore, _, unless the filter proposed in #134 is in use. This proposal goes hand in hand with the Pythonesque way of making functions private.

What do you think of this?

[bitprophet]

I'm -1 on this, especially given that the underscore prefix is already in use for classic-style tasks, and even if it weren't, I think it's just too confusing/complicated an API for a feature that IMHO doesn't make a ton of sense. If it's a legit task, it should be in the list -- if it's not a legit task, it should just be a Python function and not a task!

There are several workarounds you could employ without changing Fabric core:

• Make the subtasks not actual tasks, if they're only intended to be subroutines and never invoked on their own. @runs_once should still work on this sort of function, IIRC, it doesn't require that it be a "real" task.
• If the subtasks need to remain accessible on their own, try placing them in a submodule and importing it -- that will namespace them, allowing you to e.g.:
  • display a nested view with fab --list -F nested, so at least you can easily visually see "top level" tasks vs subtasks.
  • import them inside the primary task functions so they don't show up at all in the main list -- they could still be used directly via eg fab -f submodule subtaskname. (Not 100% great Python style, but certainly useful in cases like this.)

If a good enough argument can be made that "a task but not a task" is a legit use case, I think the best real solution is to have an explicit @task kwarg, e.g. @task(hide=True).

[JensRantil]

Thanks for your answer! I'll respond to your comments inline.

Make the subtasks not actual tasks, if they're only intended to be subroutines and never invoked on their own.

Can I execute old-style tasks if I am using @task decorator?

@runs_once should still work on this sort of function, IIRC, it doesn't require that it be a "real" task.

That's good to know!

If the subtasks need to remain accessible on their own, try placing them in a submodule and importing it -- that will namespace them, allowing you to e.g. display a nested view with fab --list -F nested, so at least you can easily visually see "top level" tasks vs subtasks.

Yes, this is my best bet as of today. Not the prettiest, but would probably work enough.

import them inside the primary task functions so they don't show up at all in the main list -- they could still be used directly via eg fab -f submodule subtaskname. (Not 100% great Python style, but certainly useful in cases like this.)

It's the same with this as with the first counter argument; This won't make it possible for me to execute the tasks individually.

If a good enough argument can be made that "a task but not a task" is a legit use case, I think the best real solution is to have an explicit @task kwarg, e.g. @task(hide=True).

I am out of arguments. I do value simplicity, too. If you think this proposal will not take fabric further, I support you.

[bitprophet]

Can I execute old-style tasks if I am using @task decorator?

From the command line, no -- if the fabfile loaded contains @task-decorated functions, regular functions won't be accessible as tasks.

However, and this is partly why I brought it up later on, if a submodule contains only regular "helper" functions not decorated with @task, and you load that specific submodule on its own via fab -f submodule, Fabric should use old-style behavior and allow you to invoke those functions from the CLI.

This is slightly inelegant, but would work in the sense of having "second-class citizen" tasks that are normally invoked as subroutines within a "main" tasks, and do not show up normally, but can be loaded directly when you really need them.

I am out of arguments. I do value simplicity, too. If you think this proposal will not take fabric further, I support you.

Thanks, I appreciate it. I sympathize with the greater issue of "managing nontrivial collections of tasks is kind of crappy" because that is most definitely a problem, but I'm not convinced this particular approach is a great solution. I'll be re-examining this whole problem space soon with Invoke and hopefully something better will come out of it.

[apmorton]

I have a use case for hidden tasks in my project, as follows.

We are using fab to deploy some web applications, and each application has multiple 'environments' on the remote server, one for production, staging, testing etc. I am handling this like follows in my fab files

@task
def production():
    _setup_path('production')

@task
def staging():
    _setup_path('staging')

@task
def some_real_task():
    require('environment', provided_by=('staging', 'production'))
    # do the stuff with the things

the tasks 'staging' and 'production' are never actually run on their own, always before some other task like

fab staging deploy.full

if the staging and production functions are not tasks, it bails out giving me a list of available tasks

there are a few reasons we have the environments as tasks, and not arguments to each task, and i would like to keep the setup the way we have it.

a task decorator kwarg to hide the tasks seems like the best solution to me

also, i can see a few other use cases for hiding tasks

the way we have constructed our deployment tasks is with many small, simple tasks that are called with execute in the deploy.full task (which is also the default task for the deploy namespace). most of the times, the only command you need to use is deploy.full, but the ability to push out JUST a specific piece is required, so they are left as commands. but this leaves us with many, mostly unused, commands in the list.

if they could be hidden, it would increase the readability of the command list, and then perhaps a new flag --list-all could be implemented to list the hidden tasks.

[chrisgo]

Here is how I ended up doing mine visually ... doesn't help if you have 20+

Available commands:

    balancer          ......... <role> balancer role
    branch            ......... <branch> work on any specified branch
    database          ......... <role> database role
    dev               ......... <environment> set up dev environment
    developer         ......... <environment> set up developer environment
    master            ......... <branch> work on master branch
    production        ......... <environment> set up production environment
    project_setup     ..... (3) Setup project-specific installs by role
    release           Release code to an environment
    resque            ......... <role> resque role
    server_normalize  ..... (1) Initialize and normalize systems/providers
    server_setup      ..... (2) Setup basic server software by role
    staging           ......... <environment> set up staging environment
    www               ......... <role> www (web+app) role   

[imposeren]

you can specify all with list of tasks that should be displayed with fab --list

[monkey1016]

I think the issue with __all__ is that the command is no longer executable by itself on the command line. If I were to have the following tester.py:

from fabric.api import task
__all__ = ['tester1']

@task
def tester1():
    print "This one shows up."
@task
def tester2():
    print "This one doesn't."

If my init.py is as follows:

import tester

The only task that shows up when I run fab --list is tester1 and tester2 is no longer executable via fab

[quickshiftin]

I'm new to python, but here's what I have done. All the 'helper' code is in a class, then the fabfile has wrapper functions for anything I'd like to see via fab -l.
AppDeploy.py

from fabric.api import *

class AppDeploy:
    # Fabric environment
    env = None

    def __init__(self, env):
        self.env = env

    def private_helper(self):
        # Don't expose this function through fabfile.py

    def deploy_site(self):
        # Wrap this through fabfile.py

fabfile.py

from fabric.api import *
from AppDeploy import AppDeploy

deployer = AppDeploy(env)

@task
def deploy_site():
    deployer.deploy_site()

Good news is it keeps fabfile.py short and sweet. Downside is there's a little extra typing, and a tiny annoyance, AppDeploy will show in the listing with fab -l. I can live with that though, over seeing all the helper functions that might be in the fabfile.py without the backend class.

EDIT
Actually, I've been playing around this morning, and now using the @task decorator on all the functions in fabfile.py. Now AppDeploy is hidden from the fab -l listing and everything still works like a charm!

[sochoa]

From here (mail.python.org > python-ideas):

__all__ = []
def export(f):
    sys._getframe(1).f_globals.setdefault("__all__", []).append(f.__name__)
    return f

def do_stuff1():
    pass

@export
def do_stuff2():
    pass

Which results in this:

$ fab -f my-fabfile.py -l
Available commands:

    do_stuff2