Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Step for fetching external resources #27

Closed
pydsigner opened this issue Aug 9, 2023 · 1 comment · Fixed by #42
Closed

Add Step for fetching external resources #27

pydsigner opened this issue Aug 9, 2023 · 1 comment · Fixed by #42
Labels
enhancement New feature or request good first issue Good for newcomers
Milestone

Comments

@pydsigner
Copy link
Owner

Add a step that uses a config file to pull in an external resource, either from the internet or from the local filesystem.

@pydsigner pydsigner added this to the 0.11.0 milestone Aug 9, 2023
@pydsigner pydsigner added the enhancement New feature or request label Aug 9, 2023
@pydsigner pydsigner modified the milestones: 0.11.0, 0.12.0 Aug 11, 2023
@pydsigner pydsigner added the good first issue Good for newcomers label Aug 11, 2023
@pydsigner
Copy link
Owner Author

We will have to extend the new explicit custody return values for Step, but only for input sources. This is important for handling resources that come from outside the filesystem, and does not really cost us anything; if we do not support checking whether non-FS resources have changed, they can simply be treated as always dirty. We will ideally want to have an API for custody management, which will allow end users to support their own resource types as needed. Output paths can remain strictly paths, since the flow of external data is strictly ingress in our approach to a file processing engine.

@pydsigner pydsigner linked a pull request Aug 12, 2023 that will close this issue
pydsigner added a commit that referenced this issue Aug 12, 2023
Add dependency entries for anchovy.include to pyproject.toml (#27)
pydsigner added a commit that referenced this issue Aug 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant