Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better Credentials Support in Catalog #3634

Open
lordsoffallen opened this issue Feb 20, 2024 · 1 comment
Open

Better Credentials Support in Catalog #3634

lordsoffallen opened this issue Feb 20, 2024 · 1 comment
Labels
Issue: Feature Request New feature or improvement to existing feature

Comments

@lordsoffallen
Copy link
Contributor

Description

I would to refer this block in my catalog:

dataset#hf:
  type: hffinder.extras.hf.HFDataset
  filepath: data/01_raw/datasets/
  dataset_name: test/huggingface-datasets
  credentials: huggingface

This works when I have huggingface defined in the credentials.yml. I would like an option to make this optional and return None when that doesn't exist. Right now code fails if huggingface is not defined. I tried to play with oc.select but refers to existing keys and I think runs before the injection of credentials.

Context

Possible Implementation

Possible Alternatives

@lordsoffallen lordsoffallen added the Issue: Feature Request New feature or improvement to existing feature label Feb 20, 2024
@lordsoffallen
Copy link
Contributor Author

Current workaroung is what I put into settings.py file:

from unittest.mock import patch, Mock

patcher = patch(
    "kedro.io.data_catalog._get_credentials",
    Mock(side_effect=lambda name, credentials: credentials.get(name))
)
patcher.start()

I don't like mocking stuff but this function wasn't part of the catalog class so extending catalog class and overriding it wasn't possible or too much code was required.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Issue: Feature Request New feature or improvement to existing feature
Projects
Status: No status
Development

No branches or pull requests

1 participant