Skip to content

Local Storage Card with local K8s #2139

@ch-enfuse

Description

@ch-enfuse

Hi, I'm currently running a k3s cluster locally, and I've been wanting to test a workflow that has a @card decorator to see some html and see it from the UI that is being served from K8s. The worfllow is ran locally. I am testing this without an s3 instance (that was giving me some issues with locating the /cards content when the UI was asking for them to retrieve from the ui service)

here is my ~/.metaflowconfig/config.json
{
"METAFLOW_DEFAULT_METADATA": "service",
"METAFLOW_SERVICE_URL": "http://localhost:8080/"
}
I've created a /.metaflow directory with no contents and chowned to my user to work

  • the first run i make (Im testing this workflow ) has no .metaflow directory.
    This saves metadata about the run on /.metadata (a root folder owned by my user - not ideal but i want to mount this onto the ui service pod so that it can serve the contents later, i can do it from any folder but I haven't figured out how to have a consistent location for this yet
  • second run: there is a .metaflow/ directory all of the sudden, and my data is split. in /.metaflow directory where i expect things to be has only a NBFlow sub directory, and my local ./.metaflow directory has the mf.cards I'm missing on the other one.

Now every subsequent run after this, metaflow defaults everything to my local ./.metaflow directory, but what i need is
I want to figure out a way to have a consistent way of setting my local storage directory and make sure the cards content also land here.
is there some combination of env variables or config i can add to my config.json?
also, if I were to instead just go back to my s3 setup, what configurations should i look at? I haven't had luck finding sources on this.

I must be something pretty simple here but I'm very new to metaflow, any help would be appreciated!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions