ctx Context can be used within shell tasks - to access context vars and secrets

[kumare3]

Signed-off-by: Ketan Umare ketan.umare@gmail.com

TL;DR

• Allows users to use ctx.* in shell tasks
• For secrets, it is necessary to use the attribute accessor api - ctx.secrets.group.key. The group and key are the actual group and key values to look for
• Also all inputs and outputs are now namesapced. So to access inputs it is necessary to use inputs.x

Example:

    t = ShellTask(
        name="test",
        script="""
            export EXEC_ID={ctx.execution_id}
            export SEC={ctx.secrets.group.key}
            cat {inputs.f}
            echo "Hello World {inputs.y} on  {inputs.j}"
            """,
        inputs=kwtypes(f=CSVFile, y=FlyteDirectory, j=datetime.datetime),
    )

all variables supported by flytekit.current_context are replaceable.

Type

• Bug Fix
• Feature
• Plugin

Are all requirements met?

• Code completed
• Smoke tested
• Unit tests added
• Code documentation added
• Any pending items have an associated Issue

Tracking Issue

flyteorg/flyte#2110

Follow-up issue

NA

[codecov]

Codecov Report

Merging #832 (619d9c4) into master (c20372e) will increase coverage by 0.07%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #832      +/-   ##
==========================================
+ Coverage   87.08%   87.15%   +0.07%     
==========================================
  Files         248      248              
  Lines       21919    21982      +63     
  Branches     2443     2450       +7     
==========================================
+ Hits        19088    19159      +71     
+ Misses       2406     2401       -5     
+ Partials      425      422       -3     

Impacted Files	Coverage Δ
flytekit/core/context_manager.py	91.12% <100.00%> (+1.91%)	⬆️
flytekit/extras/tasks/shell.py	85.71% <100.00%> (+0.34%)	⬆️
tests/flytekit/unit/core/test_context_manager.py	100.00% <100.00%> (ø)
tests/flytekit/unit/extras/tasks/test_shell.py	100.00% <100.00%> (ø)
flytekit/core/python_function_task.py	85.86% <0.00%> (-1.51%)	⬇️
tests/flytekit/unit/core/test_type_hints.py	96.54% <0.00%> (+<0.01%)	⬆️
flytekit/tools/translator.py	90.90% <0.00%> (+0.46%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c20372e...619d9c4. Read the comment docs.

[eapolinario]

can we make this an inner class to SecretsManager?

[kumare3]

can do