Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename Target to Layout #393

Merged
merged 2 commits into from
Nov 30, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
16 changes: 8 additions & 8 deletions docs/getting_started/build_dashboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ sources:
tables:
penguin_table: https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2020/2020-07-28/penguins.csv

targets:
layouts:
- title: Penguins
source: penguin_source
views:
Expand Down Expand Up @@ -114,7 +114,7 @@ sources:
tables:
penguin_table: https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2020/2020-07-28/penguins.csv

targets:
layouts:
- title: Penguins
source: penguin_source
views:
Expand Down Expand Up @@ -146,7 +146,7 @@ sources:
tables:
penguin_table: https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2020/2020-07-28/penguins.csv

targets:
layouts:
- title: Penguins
source: penguin_source
views:
Expand Down Expand Up @@ -196,7 +196,7 @@ pipelines:
- type: columns
columns: ['species', 'island', 'sex', 'year', 'bill_length_mm', 'bill_depth_mm']

targets:
layouts:
- title: Penguins
pipeline: penguin_pipeline
views:
Expand Down Expand Up @@ -245,7 +245,7 @@ pipelines:
- type: columns
columns: ['species', 'island', 'sex', 'year', 'bill_length_mm', 'bill_depth_mm']

targets:
layouts:
- title: Penguins
pipeline: penguin_pipeline
views:
Expand All @@ -272,7 +272,7 @@ targets:

## 8. Customize the appearance and behavior

The default layout we get is less than ideal for this case since it cuts off one of our plots, leaves a lot of empty space, and does not resize responsively. We can get responsive plots by adding `sizing_mode` to the target and `responsive` to the views. By changing the `layout` and `height`, we can further customize how the dashboard looks.
The default layout we get is less than ideal for this case since it cuts off one of our plots, leaves a lot of empty space, and does not resize responsively. We can get responsive plots by adding `sizing_mode` to the layout and `responsive` to the views. By changing the `layout` and `height`, we can further customize how the dashboard looks.

::::{tab-set}

Expand Down Expand Up @@ -300,7 +300,7 @@ pipelines:
- type: columns
columns: ['species', 'island', 'sex', 'year', 'bill_length_mm', 'bill_depth_mm']

targets:
layouts:
- title: Penguins
pipeline: penguin_pipeline
layout: [[0], [1, 2]]
Expand Down Expand Up @@ -366,7 +366,7 @@ pipelines:
- type: columns
columns: ['species', 'island', 'sex', 'year', 'bill_length_mm', 'bill_depth_mm']

targets:
layouts:
- title: Penguins
pipeline: penguin_pipeline
layout: [[0], [1, 2]]
Expand Down
20 changes: 10 additions & 10 deletions docs/getting_started/core_concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ To specify how you want the data to be manipulated (filtered and transformed)
::::{grid} 1
:gutter: 3

:::{grid-item-card} `targets`
:::{grid-item-card} `layouts`
To create the views (e.g. table, plot) for your dashboard
:::

Expand All @@ -62,7 +62,7 @@ sources:
pipelines:
...:
...: ...
targets:
layouts:
- ...: ...
...: ...
```
Expand All @@ -76,7 +76,7 @@ The config section provides general settings which apply to the whole dashboard.
```{code-block} YAML
config:
title: The title of the overall application
layout: The layout to put the targets in ('grid', 'tabs', 'column')
layout: The layout to put the invidual (sub-)layouts in.
logo: A URL or local path to an image file
sync_with_url: Whether to sync app state with URL
template: The template to use for the monitoring application
Expand Down Expand Up @@ -182,12 +182,12 @@ pipelines:

See the [Transform Reference](../architecture/transform) for other transform types and for the relevant parameters.

## Targets (views)
## Layouts (views)

The `targets` section defines how the dashboard is going to look. The essential structure of a `targets` section is as follows:
The `layouts` section defines how the dashboard is going to look. The essential structure of a `layouts` section is as follows:

```{code-block} YAML
targets:
layouts:
- title: Dashboard title
pipeline: The pipeline driving these views
...: Dashboard parameters
Expand All @@ -196,16 +196,16 @@ targets:
...: View parameters
```

:::{dropdown} Expand this dropdown for a more complex example structure of `targets`
:::{dropdown} Expand this dropdown for a more complex example structure of `layouts`
:animate: fade-in-slide-down
```{code-block} yaml
targets:
layouts:
- title: The title of the monitoring endpoint
download:
format: When specified adds a section to the sidebar allowing users to download the filtered dataset
kwargs: Additional keyword arguments to pass to the pandas/dask to_<format> method
tables: Allows declaring a subset of tables to download
pipeline: The pipeline driving the views of this target. Each View can independently declare a pipeline or all use the shared pipeline defined at the target level
pipeline: The pipeline driving the views of this layout. Each View can independently declare a pipeline or all use the shared pipeline defined at the layout level
views: A list of metrics to monitor and display on the endpoint
- pipeline: The Pipeline driving the View
type: The type of View to use for rendering the table
Expand All @@ -225,7 +225,7 @@ Each view can be of a different type, but a good starting point is the `hvPlotVi
In your tutorial, the final dashboard included two `kinds` - scatter and histogram:

```{code-block} YAML
targets:
layouts:
- title: Penguins
pipeline: penguin_pipeline
layout: [[0], [1, 2]]
Expand Down
4 changes: 2 additions & 2 deletions docs/getting_started/lumen_architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@ Manipulating the data with `filters` and `transforms`.
::::{grid} 1
:gutter: 3

:::{grid-item-card} `targets`
:link: ../user_guide/dashboard.html#targets
:::{grid-item-card} `layouts`
:link: ../user_guide/dashboard.html#layouts

The presentation of the manipulated data with `views`.
:::
Expand Down
4 changes: 2 additions & 2 deletions docs/how_to/custom_components/local_components.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ pipelines:
- type: my_module.StableSor
by: island

targets:
layouts:
- title: Penguins
pipeline: penguin_pipeline
layout: [[0, 1], [2, 3]]
Expand Down Expand Up @@ -234,7 +234,7 @@ pipelines:
- type: my_library.my_module.StableSort
by: island

targets:
layouts:
- title: Penguins
pipeline: penguin_pipeline
layout: [[0, 1], [2, 3]]
Expand Down
42 changes: 40 additions & 2 deletions docs/how_to/data_intake/cache.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ This guide will show you how to locally cache data to speed up reloading from a

## Caching file

When working with large data in a non-optional file format, creating a local cache can be advantageous to save time when reloading the data. Caching a [source](../../reference/source) is done by using `cache_dir` followed by a directory. If the directory does not exist, it will be created. The data will be saved as [parquet](https://www.databricks.com/glossary/what-is-parquet) files in the cache directory.
When working with large data in a non-optional file format, creating a local cache can be advantageous to save time when reloading the data. Caching a [source](../../reference/source) is done by using `cache_dir` followed by a directory. If the directory does not exist, it will be created. The data will be saved as [parquet](https://www.databricks.com/glossary/what-is-parquet) files in the cache directory and the schema will be stored as a JSON file.

Below is an example of caching a 370 MB file that consists of almost 12 million rows.

```{warning}
Expand All @@ -32,7 +33,7 @@ sources:
kwargs:
engine: fastparquet

targets:
layouts:
- title: Table
source: large_source
views:
Expand Down Expand Up @@ -73,3 +74,40 @@ Depending on the source type data caching will cache the entire table or individ
```{note}
Lumen's [cache]() can be added to all [source types](../../reference/source).
```

## Precaching

Sources that support caching per query can be made to pre-cache specific Filter and SQLTransform combinations. To enable pre-caching you must initialize a `Pipeline` and then either programmatically request to populate the pre-cache OR supply the pre-cache configuration as part of the YAML specification.

A pre-cache definitions can take one of two forms

- A dictionary containing 'filters' and 'variables' dictionaries each containing lists of values to compute a cross-product for, e.g.

```python
{
'filters': {
<filter>': ['a', 'b', 'c', ...],
...
},
'variables':
<variable>: [0, 2, 4, ...],
...
}
}
```

- A list containing dictionaries of explicit values for each filter and variables.

```python
[
{
'filters': {<filter>: 'a'},
'variables': {<variable>: 0}
},
{
'filters': {<filter>: 'a'},
'variables': {<variable>: 1}
},
...
]
```
4 changes: 2 additions & 2 deletions docs/how_to/data_intake/files.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ sources:
tables:
local_table: local_table.csv

targets:
layouts:
- title: Table
source: local_source
views:
Expand Down Expand Up @@ -72,7 +72,7 @@ sources:
tables:
remote_table: https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2020/2020-07-28/penguins.csv

targets:
layouts:
- title: Table
source: remote_source
views:
Expand Down
6 changes: 3 additions & 3 deletions docs/how_to/data_output/download_data.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ sources:
tables:
penguins: https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2020/2020-07-28/penguins.csv

targets:
layouts:
- title: Table
source: penguins
download: csv
Expand All @@ -48,7 +48,7 @@ sources:
tables:
penguins: https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2020/2020-07-28/penguins.csv

targets:
layouts:
- title: Table
source: penguins
download: csv
Expand All @@ -73,7 +73,7 @@ sources:
tables:
penguins: https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2020/2020-07-28/penguins.csv

targets:
layouts:
- title: Table
source: penguins
download: csv
Expand Down
6 changes: 3 additions & 3 deletions docs/how_to/data_processing/branch_pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ pipelines:
- type: widget
field: island

targets:
layouts:
- title: Penguins
sizing_mode: stretch_width
views:
Expand Down Expand Up @@ -72,7 +72,7 @@ pipelines:
- type: columns
columns: ['species', 'island', 'bill_length_mm', 'bill_depth_mm']

targets:
layouts:
- title: Penguins
sizing_mode: stretch_width
views:
Expand Down Expand Up @@ -107,7 +107,7 @@ pipelines:
- type: columns
columns: ['species', 'island', 'bill_length_mm', 'bill_depth_mm']

targets:
layouts:
- title: Penguins
sizing_mode: stretch_width
views:
Expand Down
4 changes: 2 additions & 2 deletions docs/how_to/data_visualization/views.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ A view is the final output of a dashboard, but to be able to create a view, at l
The following example source is a table containing data about individual penguins with various measurements.
This source could have been [filtered](../../reference/filter/index.md) or [transformed](../../reference/transform/index.md) but is omitted to keep the example simple.

The views are located in the `targets` area and can take the form of various visual components.
The views are located in the `layouts` area and can take the form of various visual components.
Lumen includes many [view types](../../reference/view/index.md) and is built so that you can easily use components from the [Holoviz ecosystem](https://holoviz.org/), such as a [scatter plot from hvPlot](https://hvplot.holoviz.org/reference/pandas/scatter.html) or an [indicator from Panel](https://panel.holoviz.org/reference/index.html#indicators).

![](../../_static/excalidraw/lumen_dashboard.png)
Expand All @@ -35,7 +35,7 @@ sources:
tables:
penguin_table: https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2020/2020-07-28/penguins.csv

targets:
layouts:
- title: Table
source: penguin_source
views:
Expand Down
2 changes: 1 addition & 1 deletion docs/how_to/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ Lumen's How to Guides provide step by step recipes for solving essential problem
**[General](general/index)**
* [Validate a specification file](general/validate)
* [Build a dashboard in Python](general/pipeline_python)
* [Use variables and references](general/variables_and_references)
* [Override parameter defaults]() (under construction)
* [Use variables and references]() (under construction)
* [Set up authentication]() (under construction)

**[Data intake](data_intake/index)**
Expand Down
8 changes: 4 additions & 4 deletions docs/how_to/variables_and_references.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ pipelines:
- type: columns
columns: $variables.columns

targets:
layouts:
- title: Plot
pipeline: ticker_pipe
views:
Expand Down Expand Up @@ -93,7 +93,7 @@ sources:
type: live
urls: $csv.websites.url

targets:
layouts:
- title: Status of websites
source: live
views:
Expand Down Expand Up @@ -157,7 +157,7 @@ sources:
tables:
table: https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2020/2020-07-28/penguins.csv

targets:
layouts:
- title: Hello my dear friend {{ USER }}
source: source
views:
Expand Down Expand Up @@ -199,7 +199,7 @@ sources:
tables:
table: https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2020/2020-07-28/penguins.csv

targets:
layouts:
- title: {{ shell("echo hello from the shell") }}
source: source
views:
Expand Down