Skip to content

Commit

Permalink
Merge branch 'master' into hxgh-apiv1-sqllab-viz
Browse files Browse the repository at this point in the history
  • Loading branch information
hughhhh committed Apr 25, 2023
2 parents cca9d8a + e8121b1 commit 7411635
Show file tree
Hide file tree
Showing 189 changed files with 7,188 additions and 2,152 deletions.
2 changes: 2 additions & 0 deletions .github/workflows/chromatic-master.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,8 @@ jobs:
# Job steps
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0 # 👈 Required to retrieve git history
- name: Install dependencies
run: npm ci
working-directory: superset-frontend
Expand Down
116 changes: 57 additions & 59 deletions .github/workflows/docker-ephemeral-env.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Push ephmereral env image
name: Push ephemeral env image

on:
workflow_run:
Expand All @@ -17,12 +17,10 @@ jobs:
id: check
shell: bash
run: |
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
if [ -n "${{ (secrets.AWS_ACCESS_KEY_ID != '' &&
secrets.AWS_ACCESS_KEY_ID != '' &&
secrets.AWS_SECRET_ACCESS_KEY != '' &&
secrets.AWS_SECRET_ACCESS_KEY != '') || '' }}" ]; then
secrets.AWS_ACCESS_KEY_ID != '' &&
secrets.AWS_SECRET_ACCESS_KEY != '' &&
secrets.AWS_SECRET_ACCESS_KEY != '') || '' }}" ]; then
echo "has-secrets=1" >> "$GITHUB_OUTPUT"
fi
Expand All @@ -33,66 +31,66 @@ jobs:
runs-on: ubuntu-latest

steps:
- name: 'Download artifact'
uses: actions/github-script@v3.1.0
with:
script: |
const artifacts = await github.actions.listWorkflowRunArtifacts({
owner: context.repo.owner,
repo: context.repo.repo,
run_id: ${{ github.event.workflow_run.id }},
});
- name: "Download artifact"
uses: actions/github-script@v3.1.0
with:
script: |
const artifacts = await github.actions.listWorkflowRunArtifacts({
owner: context.repo.owner,
repo: context.repo.repo,
run_id: ${{ github.event.workflow_run.id }},
});
core.info('*** artifacts')
core.info(JSON.stringify(artifacts))
core.info('*** artifacts')
core.info(JSON.stringify(artifacts))
const matchArtifact = artifacts.data.artifacts.filter((artifact) => {
return artifact.name == "build"
})[0];
if(!matchArtifact) return core.setFailed("Build artifacts not found")
const matchArtifact = artifacts.data.artifacts.filter((artifact) => {
return artifact.name == "build"
})[0];
if(!matchArtifact) return core.setFailed("Build artifacts not found")
const download = await github.actions.downloadArtifact({
owner: context.repo.owner,
repo: context.repo.repo,
artifact_id: matchArtifact.id,
archive_format: 'zip',
});
var fs = require('fs');
fs.writeFileSync('${{github.workspace}}/build.zip', Buffer.from(download.data));
const download = await github.actions.downloadArtifact({
owner: context.repo.owner,
repo: context.repo.repo,
artifact_id: matchArtifact.id,
archive_format: 'zip',
});
var fs = require('fs');
fs.writeFileSync('${{github.workspace}}/build.zip', Buffer.from(download.data));
- run: unzip build.zip
- run: unzip build.zip

- name: Display downloaded files (debug)
run: ls -la
- name: Display downloaded files (debug)
run: ls -la

- name: Get SHA
id: get-sha
run: echo "::set-output name=sha::$(cat ./SHA)"
- name: Get SHA
id: get-sha
run: echo "::set-output name=sha::$(cat ./SHA)"

- name: Get PR
id: get-pr
run: echo "::set-output name=num::$(cat ./PR-NUM)"
- name: Get PR
id: get-pr
run: echo "::set-output name=num::$(cat ./PR-NUM)"

- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-west-2
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-west-2

- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@v1
- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@v1

- name: Load, tag and push image to ECR
id: push-image
env:
ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
ECR_REPOSITORY: superset-ci
SHA: ${{ steps.get-sha.outputs.sha }}
IMAGE_TAG: pr-${{ steps.get-pr.outputs.num }}
run: |
docker load < $SHA.tar.gz
docker tag $SHA $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
docker tag $SHA $ECR_REGISTRY/$ECR_REPOSITORY:$SHA
docker push -a $ECR_REGISTRY/$ECR_REPOSITORY
- name: Load, tag and push image to ECR
id: push-image
env:
ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
ECR_REPOSITORY: superset-ci
SHA: ${{ steps.get-sha.outputs.sha }}
IMAGE_TAG: pr-${{ steps.get-pr.outputs.num }}
run: |
docker load < $SHA.tar.gz
docker tag $SHA $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
docker tag $SHA $ECR_REGISTRY/$ECR_REPOSITORY:$SHA
docker push -a $ECR_REGISTRY/$ECR_REPOSITORY
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -647,7 +647,7 @@ FEATURE_FLAGS = {
}
```

If you want to use the same flag in the client code, also add it to the FeatureFlag TypeScript enum in [@superset-ui/core](https://github.com/apache-superset/superset-ui/blob/master/packages/superset-ui-core/src/utils/featureFlags.ts). For example,
If you want to use the same flag in the client code, also add it to the FeatureFlag TypeScript enum in [@superset-ui/core](https://github.com/apache/superset/blob/master/superset-frontend/packages/superset-ui-core/src/utils/featureFlags.ts). For example,

```typescript
export enum FeatureFlag {
Expand Down
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,12 @@ Superset provides:

## Screenshots & Gifs

**Video Overview**

https://user-images.githubusercontent.com/64562059/234390129-321d4f35-cb4b-45e8-89d9-20ae292f34fc.mp4

<br/>

**Large Gallery of Visualizations**

<kbd><img title="Gallery" src="superset-frontend/src/assets/images/screenshots/gallery.jpg"/></kbd><br/>
Expand Down
14 changes: 10 additions & 4 deletions RESOURCES/FEATURE_FLAGS.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,10 @@ These features are considered **unfinished** and should only be used on developm
- DASHBOARD_CACHE
- DASHBOARD_NATIVE_FILTERS_SET
- DISABLE_DATASET_SOURCE_EDIT
- DRILL_TO_DETAIL
- DRILL_BY
- ENABLE_ADVANCED_DATA_TYPES
- ENABLE_EXPLORE_JSON_CSRF_PROTECTION
- ENABLE_TEMPLATE_REMOVE_FILTERS
- HORIZONTAL_FILTER_BAR
- KV_STORE
- PRESTO_EXPAND_DATA
- REMOVE_SLICE_LEVEL_LABEL_COLORS
Expand All @@ -54,25 +53,31 @@ These features are **finished** but currently being tested. They are usable, but
- CONFIRM_DASHBOARD_DIFF
- DASHBOARD_EDIT_CHART_IN_NEW_TAB
- DASHBOARD_FILTERS_EXPERIMENTAL
- DASHBOARD_NATIVE_FILTERS
- DASHBOARD_VIRTUALIZATION
- DRILL_BY
- DRILL_TO_DETAIL
- DYNAMIC_PLUGINS: [(docs)](https://superset.apache.org/docs/installation/running-on-kubernetes)
- ENABLE_JAVASCRIPT_CONTROLS
- ESTIMATE_QUERY_COST
- GENERIC_CHART_AXES
- GLOBAL_ASYNC_QUERIES [(docs)](https://github.com/apache/superset/blob/master/CONTRIBUTING.md#async-chart-queries)
- HORIZONTAL_FILTER_BAR
- RLS_IN_SQLLAB
- SSH_TUNNELING [(docs)](https://superset.apache.org/docs/installation/setup-ssh-tunneling)
- USE_ANALAGOUS_COLORS
- VERSIONED_EXPORT

## Stable

These features flags are **safe for production** and have been tested.
These features flags are **safe for production**. They have been tested and will be supported for the foreseeable future.

[//]: # "PLEASE KEEP THE LIST SORTED ALPHABETICALLY"

- ALERTS_ATTACH_REPORTS
- ALLOW_ADHOC_SUBQUERY
- DASHBOARD_CROSS_FILTERS
- DASHBOARD_RBAC [(docs)](https://superset.apache.org/docs/creating-charts-dashboards/first-dashboard#manage-access-to-dashboards)
- DATAPANEL_CLOSED_BY_DEFAULT
- DISABLE_LEGACY_DATASOURCE_EDITOR
- DRUID_JOINS
- EMBEDDABLE_CHARTS
Expand All @@ -93,4 +98,5 @@ These features flags currently default to True and **will be removed in a future

[//]: # "PLEASE KEEP THE LIST SORTED ALPHABETICALLY"

- DASHBOARD_NATIVE_FILTERS
- GENERIC_CHART_AXES
1 change: 1 addition & 0 deletions UPDATING.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ assists people when migrating to a new version.
- [22798](https://github.com/apache/superset/pull/22798): To make the welcome page more relevant in production environments, the last tab on the welcome page has been changed from to feature all charts/dashboards the user has access to (previously only examples were shown). To keep current behavior unchanged, add the following to your `superset_config.py`: `WELCOME_PAGE_LAST_TAB = "examples"`
- [22328](https://github.com/apache/superset/pull/22328): For deployments that have enabled the "THUMBNAILS" feature flag, the function that calculates dashboard digests has been updated to consider additional properties to more accurately identify changes in the dashboard metadata. This change will invalidate all currently cached dashboard thumbnails.
- [21765](https://github.com/apache/superset/pull/21765): For deployments that have enabled the "ALERT_REPORTS" feature flag, Gamma users will no longer have read and write access to Alerts & Reports by default. To give Gamma users the ability to schedule reports from the Dashboard and Explore view like before, create an additional role with "can read on ReportSchedule" and "can write on ReportSchedule" permissions. To further give Gamma users access to the "Alerts & Reports" menu and CRUD view, add "menu access on Manage" and "menu access on Alerts & Report" permissions to the role.
- [22325](https://github.com/apache/superset/pull/22325): "RLS_FORM_QUERY_REL_FIELDS" is replaced by "RLS_BASE_RELATED_FIELD_FILTERS" feature flag. Its value format stays same.

### Potential Downtime

Expand Down
37 changes: 37 additions & 0 deletions docs/docs/databases/ocient.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
---
title: Ocient DB
hide_title: true
sidebar_position: 20
version: 1
---

## Ocient DB

The recommended connector library for Ocient is [sqlalchemy-ocient](https://pypi.org/project/sqlalchemy-ocient).

## Install the Ocient Driver

```
pip install sqlalchemy-ocient
```

## Connecting to Ocient

The format of the Ocient DSN is:

```shell
ocient://user:password@[host][:port][/database][?param1=value1&...]
```

The DSN for connecting to an `exampledb` database hosted at `examplehost:4050` with TLS enabled is:
```shell
ocient://admin:abc123@examplehost:4050/exampledb?tls=on
```

**NOTE**: You must enter the `user` and `password` credentials. `host` defaults to localhost,
port defaults to 4050, database defaults to `system` and `tls` defaults
to `unverified`.

## User Access Control

Make sure the user has privileges to access and use all required databases, schemas, tables, views, and warehouses, as the Ocient SQLAlchemy engine does not test for user or role rights by default.
4 changes: 2 additions & 2 deletions docs/docs/frequently-asked-questions.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ will do its own _GROUP BY_ and doing the work twice might slow down performance.
Whether you use a table or a view, the important factor is whether your database is fast enough to
serve it in an interactive fashion to provide a good user experience in Superset.

However, if you are using the SQL Lab, there is no such limitation, you can write sql query to join
multiple tables as long as your db account has access to the tables.
However, if you are using SQL Lab, there is no such limitation. You can write SQL queries to join
multiple tables as long as your database account has access to the tables.

### How BIG can my datasource be?

Expand Down
2 changes: 1 addition & 1 deletion docs/docs/miscellaneous/country-map-tools.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ The Country Maps visualization already ships with the maps for the following cou
## Adding a New Country

To add a new country to the list, you'd have to edit files in
[@superset-ui/legacy-plugin-chart-country-map](https://github.com/apache-superset/superset-ui/tree/master/plugins/legacy-plugin-chart-country-map).
[@superset-ui/legacy-plugin-chart-country-map](https://github.com/apache/superset/tree/master/superset-frontend/plugins/legacy-plugin-chart-country-map).

1. Generate a new GeoJSON file for your country following the guide in [this Jupyter notebook](https://github.com/apache/superset/blob/master/superset-frontend/plugins/legacy-plugin-chart-country-map/scripts/Country%20Map%20GeoJSON%20Generator.ipynb).
2. Edit the countries list in [legacy-plugin-chart-country-map/src/countries.ts](https://github.com/apache/superset/blob/master/superset-frontend/plugins/legacy-plugin-chart-country-map/src/countries.ts).
Expand Down
103 changes: 103 additions & 0 deletions docs/docs/miscellaneous/native-filter-migration.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
---
title: Migrating from Legacy to Native Filters
sidebar_position: 5
version: 1
---

##

The `superset native-filters` CLI command group—somewhat akin to an Alembic migration—
comprises of a number of sub-commands which allows administrators to upgrade/downgrade
existing dashboards which use the legacy filter-box charts—in combination with the
filter scopes/filter mapping—to use the native filter dashboard component.

Even though both legacy and native filters can coexist the overall user experience (UX)
is substandard as the already convoluted filter space becomes overly complex. After
enabling the `DASHBOARD_NATIVE_FILTERS` it is strongly advised to run the migration ASAP to
ensure users are not exposed to the hybrid state.

### Upgrading

The

```
superset native-filters upgrade
```

command—which provides the option to target either specific dashboard(s) or all
dashboards—migrates the legacy filters to native filters.

Specifically, the command performs the following:

- Replaces every filter-box chart within the dashboard with a markdown element which
provides a link to the deprecated chart. This preserves the layout whilst simultaneously
providing context to help owners review/verify said change.
- Migrates the filter scopes/filter mappings to the native filter configuration.

#### Quality Control

Dashboard owners should:

- Verify that the filter behavior is correct.
- Consolidate any conflicting/redundant filters—this previously may not have been
obvious given the embedded nature of the legacy filters and/or the non-optimal UX of the
legacy filter mapping (scopes and immunity).
- Rename the filters—which may not be uniquely named—to provide the necessary context
which previously was likely provided by both the location of the filter-box and the
corresponding filter-box title.

Dashboard owners may:

- Remove† the markdown elements from their dashboards and adjust the layout accordingly.

† Note removing the markdown elements—which contain metadata relating to the replaced
chart—prevents the dashboard from being fully restored and thus this operation should
only be performed if it is evident that a downgrade is not necessary.

### Downgrading

Similarly the

```
superset native-filters downgrade
```

command reverses said migration, i.e., restores the dashboard to the previous state.


### Cleanup

The ability to downgrade/reverse the migration requires temporary storage of the
dashboard metadata—relating to both positional composition and filter configuration.

Once the upgrade has been verified it is recommended to run the

```
superset native-filters cleanup
```

command—which provides the option to target either specific dashboard(s) or all
dashboards. Note this operation is irreversible.

Specifically, the command performs the following:

- Removes the temporary dashboard metadata.
- Deletes the filter-box charts associated with the dashboard†.

† Note the markdown elements will still remain however the link to the referenced filter-box
chart will no longer be valid.

Finally, the

```
superset native-filers cleanup --all
```

command will additionally delete all filter-box charts, irrespective of whether they
were ever associated with a dashboard.

#### Quality Control

Dashboard owners should:

- Remove the markdown elements from their dashboards and adjust the layout accordingly.
Loading

0 comments on commit 7411635

Please sign in to comment.