Run components from Viash Hub #816

hcannoodt · 2025-03-24T10:07:12Z

Describe your changes

Related issue(s)

Closes #xxxx

Type of Change

Breaking change (fix or feature that would cause existing functionality to not work as expected)
New functionality (non-breaking change which adds functionality)
Major change (non-breaking change which modifies existing functionality)
Minor change (non-breaking change which does not modify existing functionality)
Bug fix (non-breaking change which fixes an issue)
This change requires a documentation update

Checklist

Requirements:

I have read the CONTRIBUTING doc.
I have performed a self-review of my code by checking the "Changed Files" tab.
My code follows the code style of this project.

Tests:

I have added tests that prove my fix is effective or that my feature works.
New and existing unit tests pass locally with my changes.

Documentation:

Proposed changes are described in the CHANGELOG.md.
I have updated the documentation accordingly.

Test Environment

…t from Still allows any component command even if it doesn't make much sense

rcannood · 2025-03-31T09:02:55Z

This looks fantastic! Let me build it and try it out before I review, just in case.

rcannood

Hmmm...

When I try to run it, I get:

viash run vsh://toolbox/v0.1.0/yq -- --input codecov.yaml --output output.yaml

Fetching ANC from https://api-dev.viash-hub.dev/auto-net-config/auto-net-config
Result of fetching ANC from api-dev.viash-hub.dev: Some((AutoNetConfig(v1,2025-04-24T10:56:21+00:00,api-dev.viash-hub.dev/auto-net-config/auto-net-config,Viash-Hub,vsh,Hosts(packages.viash-hub.com,git.viash-hub.com,sshttp.viash-hub.com,images.viash-hub.com,packages.viash-hub.com,Gitlab,web-dev.viash-hub.dev,api-dev.viash-hub.dev,Https),Map(component -> Map(name -> Validation(500,^[a-zA-Z]\w*$), namespace -> Validation(500,^[a-zA-Z]\w*$)), package -> Map(name -> Validation(500,^[a-zA-Z]\w*$)))),{"api_version":"v1","hosts":{"back":"api-dev.viash-hub.dev","back_protocol":"https","front":"web-dev.viash-hub.dev","http":"packages.viash-hub.com","images":"images.viash-hub.com","sources":"packages.viash-hub.com","sources_type":"gitlab","ssh":"git.viash-hub.com","sshttp":"sshttp.viash-hub.com"},"name":"Viash-Hub","prefix":"vsh","provided_by":"api-dev.viash-hub.dev/auto-net-config/auto-net-config","valid_until":"2025-04-24T10:56:21+00:00","validation":{"component":{"name":{"length":500,"pattern":"^[a-zA-Z]\\w*$"},"namespace":{"length":500,"pattern":"^[a-zA-Z]\\w*$"}},"package":{"name":{"length":500,"pattern":"^[a-zA-Z]\\w*$"}}}}))
Fetching package bundle from: Https://api-dev.viash-hub.dev/package-bundle/toolbox/v0.1.0/yq?runner=executable
Exception in thread "ThreadProcess-spawn-Thread-4" java.io.FileNotFoundException: /home/rcannood/.viash/package-bundle-cache/toolbox-v0.1.0-yq-executable.tar.gz (No such file or directory)

There is probably a very logical explanation for this :)

Can we use ~~www.viash-hub.com~~ api.viash-hub.com instead of api-dev.viash-hub.dev?
Can we add a command, something like viash get ...? Or something like viash install ..., such that we can install a symlink the executable to .local/bin or something like that. Happy to move this to a separate github issue
Could we add some docs to explain how to use the functionality implemented by this command?

hcannoodt · 2025-04-23T12:13:14Z

Woopsie! Fetching packages when the cache folder doesn't exist yet should be fixed now.

hcannoodt · 2025-04-24T09:17:17Z

1 line in the documention for viash run was added and an example
Fixed some documentation inconsistencies too while I was touching it.

add a few lines on how to use viash run to a component using the package bundle strip a few stripMargin calls that are not needed

rcannood · 2025-04-24T14:30:20Z

I think the ANC is still set to AutoNetConfig.fetch("api-dev.viash-hub.dev").

Is it okay to delay this PR until it's changed to AutoNetConfig.fetch("api.viash-hub.dev")? And maybe allow an environment variable to change the setting? ^^

src/main/scala/io/viash/Main.scala

Copilot

Pull Request Overview

This PR adds functionality to run Viash components directly from Viash Hub by introducing a new package bundle syntax and auto-network configuration (ANC) system. The implementation allows users to execute components without local building using a vsh:// URI scheme.

Package bundle fetching from Viash Hub with caching and ETag support
Auto-network configuration (ANC) system for discovering Viash Hub endpoints
Documentation updates to consistently use "Viash" capitalization

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
src/main/scala/io/viash/helpers/autonetconfig/	New ANC system for discovering Viash Hub endpoints with caching
src/main/scala/io/viash/Main.scala	Package bundle fetching implementation with HTTP client and cache management
src/main/scala/io/viash/cli/CLIConf.scala	Updated CLI descriptions and added package bundle usage examples
src/main/scala/io/viash/config/dependencies/	Minor regex fix and documentation correction
CHANGELOG.md	Documentation of new package bundle functionality

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-10-01T07:49:04Z

src/main/scala/io/viash/Main.scala

+    Files.createDirectories(path)
+    val etag = Try(Source.fromFile(etag_path.toString).getLines().next()).toOption
+
+    val anc = AutoNetConfig.fetch("api-dev.viash-hub.dev")


Hard-coded development URL should be configurable or use production endpoint. Consider using a configuration parameter or environment variable to specify the ANC endpoint.

Suggested change

val anc = AutoNetConfig.fetch("api-dev.viash-hub.dev")

val ancEndpoint = sys.env.getOrElse("VIASH_ANC_ENDPOINT", "api-dev.viash-hub.dev")

val anc = AutoNetConfig.fetch(ancEndpoint)

I fixed the anc address, but while we should have some sort of setting to set anc addresses, I don't think this is the proper way to do it. We should have a system where other prefixes can be specified as well.

src/main/scala/io/viash/Main.scala

src/main/scala/io/viash/helpers/autonetconfig/AutoNetConfig.scala

src/main/scala/io/viash/Main.scala

src/main/scala/io/viash/cli/CLIConf.scala

src/main/scala/io/viash/config/dependencies/Dependency.scala

src/main/scala/io/viash/config/dependencies/Repository.scala

…iny bit

rcannood

LGTM!

hcannoodt added 5 commits March 19, 2025 17:16

Add option to fetch a package bundle from viash-hub to run a componen…

750e4cd

…t from Still allows any component command even if it doesn't make much sense

partial implementation to fetch auto-net-config

c608855

cleanup naming & add caching of anc

bf79aa3

Merge branch 'develop' into feature/package_bundle

a3d5f18

allow empty valid_until field

464d4ed

hcannoodt requested a review from rcannood March 28, 2025 13:50

rcannood reviewed Apr 23, 2025

View reviewed changes

Make sure the folders exist before trying to fetch a package bundle

aae7994

fix viash -> Viash in CLI documentation

19fab4f

add a few lines on how to use viash run to a component using the package bundle strip a few stripMargin calls that are not needed

hcannoodt force-pushed the feature/package_bundle branch from bd74c13 to 19fab4f Compare April 24, 2025 09:22

rcannood changed the title ~~Feature/package bundle~~ Run components from Viash Hub Apr 24, 2025

rcannood reviewed Jun 11, 2025

View reviewed changes

src/main/scala/io/viash/Main.scala Outdated Show resolved Hide resolved

hcannoodt added 3 commits August 11, 2025 15:02

Merge branch 'develop' into feature/package_bundle

f7e96d5

uniformize package sugar syntax, add option for latest in viash run

60e993c

Merge branch 'develop' into feature/package_bundle

d7d9a5b

rcannood requested a review from Copilot October 1, 2025 07:47

Copilot AI reviewed Oct 1, 2025

View reviewed changes

rcannood reviewed Oct 1, 2025

View reviewed changes

src/main/scala/io/viash/Main.scala Show resolved Hide resolved

rcannood reviewed Oct 1, 2025

View reviewed changes

src/main/scala/io/viash/cli/CLIConf.scala Outdated Show resolved Hide resolved

rcannood reviewed Oct 1, 2025

View reviewed changes

src/main/scala/io/viash/config/dependencies/Dependency.scala Show resolved Hide resolved

rcannood reviewed Oct 1, 2025

View reviewed changes

src/main/scala/io/viash/config/dependencies/Repository.scala Show resolved Hide resolved

hcannoodt added 3 commits October 1, 2025 10:09

fix vsh anc address

3e16578

fix format inconsistencies

c336e9f

convert enum to string explicitly. reduce spamminess of viash run a t…

ca030ed

…iny bit

hcannoodt requested a review from rcannood October 1, 2025 12:06

rcannood approved these changes Oct 1, 2025

View reviewed changes

hcannoodt merged commit e91a154 into develop Oct 1, 2025
20 of 31 checks passed

rcannood deleted the feature/package_bundle branch October 1, 2025 14:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Run components from Viash Hub #816

Run components from Viash Hub #816

Uh oh!

hcannoodt commented Mar 24, 2025 •

edited

Loading

Uh oh!

rcannood commented Mar 31, 2025 •

edited

Loading

Uh oh!

rcannood left a comment •

edited

Loading

Uh oh!

hcannoodt commented Apr 23, 2025

Uh oh!

hcannoodt commented Apr 24, 2025

Uh oh!

rcannood commented Apr 24, 2025

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Oct 1, 2025

Uh oh!

hcannoodt Oct 1, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rcannood left a comment

Uh oh!

Uh oh!

Uh oh!

	val anc = AutoNetConfig.fetch("api-dev.viash-hub.dev")
	val ancEndpoint = sys.env.getOrElse("VIASH_ANC_ENDPOINT", "api-dev.viash-hub.dev")
	val anc = AutoNetConfig.fetch(ancEndpoint)

Run components from Viash Hub #816

Run components from Viash Hub #816

Uh oh!

Conversation

hcannoodt commented Mar 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Describe your changes

Related issue(s)

Type of Change

Checklist

Test Environment

Uh oh!

rcannood commented Mar 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rcannood left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hcannoodt commented Apr 23, 2025

Uh oh!

hcannoodt commented Apr 24, 2025

Uh oh!

rcannood commented Apr 24, 2025

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Oct 1, 2025

Choose a reason for hiding this comment

Uh oh!

hcannoodt Oct 1, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rcannood left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

hcannoodt commented Mar 24, 2025 •

edited

Loading

rcannood commented Mar 31, 2025 •

edited

Loading

rcannood left a comment •

edited

Loading