batch-processing.qmd

---
title: Batch processing
order: 10
---

{{< include ../../_includes/_clone_template.qmd >}}

Within this project it's possible to do all of the same things mentioned in the ["Component" guide](../component/) such as build a target executable with `viash build` and test a component with `viash test`. However, doing this for all components in the repository can bet quite tedious.

Luckily, Viash provides a set of commands for building, testing or inspecting all Viash components in the current namespace (ns).

## Build all components

You can generate your first full development build using the [`viash ns build`](/reference/cli/ns_build.qmd) command.

```bash
viash ns build --setup cachedbuild
```
<details>
<summary>Output</summary>
```{bash echo=FALSE}
viash ns build --setup cachedbuild
```
</details>

Here are some useful optional arguments:

* `--parallel`: Run multiple builds in parallel.
* `--setup cachedbuild`: Build Docker images using the [`cachedbuild`](/reference/config/engines/docker/#setup_strategy) strategy.
* `--query demo`: Only select components that have 'demo' in the namespace or name.

## Test all components

You can run all of the component tests using the [`viash ns test`](/reference/cli/ns_test.qmd) command.

```bash
viash ns test
```
<details>
<summary>Output</summary>
```{bash echo=FALSE}
viash ns test
```
</details>


## Listing components

You can run list all components using the [`viash ns list`](/reference/cli/ns_list.qmd) command.

```bash
viash ns list
```
<details>
<summary>Output</summary>
```{bash echo=FALSE}
viash ns list
```
</details>

## Custom batch processing

The [`viash ns exec`](/reference/cli/ns_exec.qmd) command can be used to run a command on every component.

```bash
viash ns exec "echo Hello {}"
```
<details>
<summary>Output</summary>
```{bash echo=FALSE}
viash ns exec "echo Hello {}"
```
</details>

## Tips

### Parallel builds

Some commands shown above can be optimized by adding the [`--parallel`](/reference/cli/ns_build.qmd) option:

- `viash ns build --parallel` will build in parallel
- `viash ns test --parallel` will test in parallel

For example:


```bash
viash ns test --parallel
```
<details>
<summary>Output</summary>
```{bash echo=FALSE}
viash ns test --parallel
```
</details>

### Subset to components or namespaces

In a development context, one often needs to rebuild one or a few components rather than the full repository. For this situation, `viash ns` has the option to specify query arguments: `--query`, `query_name` and `query_namespace`. We refer to the [reference](/reference/cli/) documentation for details and illustrate the use using an example:

```bash
viash ns build --query "^.*columns$"
```
<details>
<summary>Output</summary>
```{bash echo=FALSE}
viash ns build --query "^.*columns$"
```
</details>

As shown here, the query arguments accept regular expressions.