Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

env var override support #1491

Merged
merged 11 commits into from Aug 9, 2023
Merged

env var override support #1491

merged 11 commits into from Aug 9, 2023

Conversation

casperdcl
Copy link
Sponsor Member

@casperdcl casperdcl commented Aug 8, 2023

  • environment variables to override defaults (TQDM_*)
    • e.g. in CI jobs, export TQDM_MININTERVAL=5 to avoid log spam
    • add tests
    • add docs (tqdm.utils.envwrap & e.g. export TQDM_MININTERVAL=5)
  • fix & update CLI completion
  • fix & update API docs
  • minor code tidy: replace os.path => pathlib.Path
  • fix documentation image hosting

@casperdcl casperdcl added p3-enhancement 🔥 Much new such feature question/docs ‽ Documentation clarification candidate to-merge ↰ Imminent c3-small 🕒 Complexity medium labels Aug 8, 2023
@casperdcl casperdcl self-assigned this Aug 8, 2023
@codecov
Copy link

codecov bot commented Aug 8, 2023

Codecov Report

Merging #1491 (d434a3c) into master (87414bc) will increase coverage by 0.13%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #1491      +/-   ##
==========================================
+ Coverage   88.29%   88.43%   +0.13%     
==========================================
  Files          26       26              
  Lines        1683     1703      +20     
  Branches      349      356       +7     
==========================================
+ Hits         1486     1506      +20     
  Misses        145      145              
  Partials       52       52              

@casperdcl casperdcl merged commit d69cc90 into master Aug 9, 2023
26 checks passed
Casper automation moved this from Next Release to Done Aug 9, 2023
@casperdcl casperdcl deleted the envwrap branch August 9, 2023 10:36
@casperdcl
Copy link
Sponsor Member Author

/tag v4.66.0 d69cc90

@biswaroop1547
Copy link

awesome 🚀

@sisrfeng
Copy link

For those who are curious about the implementation: (It seems to be a general and convinient method to "Override parameter defaults via `os.environ[prefix + param_name] " )

https://github.com/tqdm/tqdm/pull/1491/files#diff-a80e72b5f3679063ef2e742eba24c88112f0e189d34acc70cf2dc866e9af84b8R35

def envwrap(prefix, case_sensitive=False, literal_eval=False, is_method=False):
    """
    Override parameter defaults via `os.environ[prefix + param_name]`.
    Precedence (highest first):
    - call (`foo(a=3)`)
    - environ (`FOO_A=2`)
    - signature (`def foo(a=1)`)
    Parameters
    ----------
    prefix  : str
        Env var prefix, e.g. "FOO_"
    case_sensitive  : bool, optional
        If (default: False), treat env var "FOO_Some_ARG" as "FOO_some_arg".
    literal_eval  : bool, optional
        Whether to `ast.literal_eval` the detected env var overrides.
        Otherwise if (default: False), infer types from function signature.
    is_method  : bool, optional
        Whether to use `functools.partialmethod`. If (default: False) use `functools.partial`.
    Examples
    --------
    ```
    $ cat foo.py
    from tqdm.utils import envwrap
    @envwrap("FOO_")
    def test(a=1, b=2, c=3):
        print(f"received: a={a}, b={b}, c={c}")
    $ FOO_A=42 FOO_C=1337 python -c 'import foo; foo.test(c=99)'
    received: a=42, b=2, c=99
    ```
    """
    i = len(prefix)
    env_overrides = {k[i:] if case_sensitive else k[i:].lower(): v
                     for k, v in os.environ.items() if k.startswith(prefix)}
    part = partialmethod if is_method else partial

    def wrap(func):
        params = signature(func).parameters
        overrides = {k: v for k, v in env_overrides.items() if k in params}
        if literal_eval:
            return part(func, **{k: safe_eval(v) for k, v in overrides.items()})
        # use `func` signature to infer env override `type` (fallback to `str`)
        for k in overrides:
            param = params[k]
            if param.annotation is not param.empty:
                typ = param.annotation
                # TODO: parse type in {Union, Any, Optional, ...}
            else:
                typ = str if param.default is None else type(param.default)
            overrides[k] = typ(overrides[k])
        return part(func, **overrides)
    return wrap

@casperdcl
Copy link
Sponsor Member Author

casperdcl commented Aug 11, 2023

Well... tqdm.utils.envwrap API was updated in #1493 to fix cross-platform compatibility

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c3-small 🕒 Complexity medium p3-enhancement 🔥 Much new such feature question/docs ‽ Documentation clarification candidate to-merge ↰ Imminent
Projects
Casper
  
Done
3 participants