How to evaluate each step only once?

[denyslazarenko]

I have the following structure of my steps. The problem is that many steps are called more than once and it makes the process of training very slow. Is it possible somehow to simplify it?
more precisely, how to optimize this part? I would like to compute input_missing just once

[kamil-kaczmarek]

Hi @tankz0r,
Set Step.cache_output=True, so after fit_transform() or transform() steppy will cache outputs and just load them in consecutive calls to this step.

Let me know, if it worked for you :)

[kamil-kaczmarek]

BTW @tankz0r, what features you need, besides these already available?

[denyslazarenko]

you mean for Steppy? It is hard for me to judge now, it would be perfect to see the documentation to understand the full picture of it. By the way, what is the idea of steppy-tool?

[kamil-kaczmarek]

Hi @tankz0r, Documentation: https://steppy.readthedocs.io/en/latest/ Mind that it is still under construction, and we are improving it weekly :)

steppy-toolkit is complementary to the steppy. Our goal is to create very high quality implementations (with docs of course) of mostly Transformers that are ready to use in your steppy-based pipeline. For example you just do:

from toolkit.pytorch_recipes.models import MultiOutputUnet

my_step = Step(name='U-Net_segmentation',
               transformer=MultiOutputUnet(**parameters),
               input_steps=[preprocessing],
               persist_output=True)

and parametrize it in a way you want. All Tranformers will have the same Interface. The overall goal is to make sort of data scientist inventory of ready-to-use pieces of pipelines.