Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 4 additions & 6 deletions src/guide/admin/analysis-execution.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
# Analysis Execution

One of the primary functions of the Node UI is to allow admins to control the analyzes running on their node. This page
One of the primary functions of the Node UI is to allow admins to control the analyses running on their node. This page
describes the different features in the Node UI for managing and interacting with the analysis containers.

To see the statuses of the approved analyzes for your node, click the "Analyses" button in the menu bar at the top of
To see the statuses of the approved analyses for your node, click the "Analyses" button in the menu bar at the top of
the Node UI after logging in.

## Table Overview
The analysis table provides an overview of all the approved analyzes on your node as well as displays their current
The analysis table provides an overview of all the approved analyses on your node as well as displays their current
states in the "Run Status" column. The columns of the table have various options for sorting the entries
including being able to filter by certain categorical variables in addition to ordering by name or date.

::: info Detailed Dates
Hovering your mouse cursor over a date will show the exact date and time
:::

The search bar above the table allows the admin to quickly search for specific analyzes using keywords or identifiers.
The search bar above the table allows the admin to quickly search for specific analyses using keywords or identifiers.
Additionally, to get the latest entries without refreshing the whole page, simply click the white "refresh" button
above the table on the right side of the page.

Expand Down Expand Up @@ -49,8 +49,6 @@ During an analysis run, a node admin may want to halt an analysis container manu
yellow stop button will be enabled and can be used to prematurely stop the container. After stopping the run, the
green play button will change to indicate that the admin can rerun the analysis if they choose.

Analyzes that have run to completion will enter this "stopped" state by default.

### Deleting an Analysis Run

[![Analysis Delete Button](/images/node_ui_images/analysis_delete.png){width=40}](/images/node_ui_images/analysis_delete.png)
Expand Down
11 changes: 5 additions & 6 deletions src/guide/admin/analysis-review.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,18 @@
# Analysis Review
As a node administrator, you have a crucial role in analysis: If your node was part of an approved project, the
analyst who set it up can create analyzes with your node. Before a valid analysis is built, it requires approval.
As a node administrator, you have a crucial role in analysis execution: If your node was part of an approved project, the
analyst who set it up can create analyses with your node. Before a valid analysis is built, it requires approval.
Here, you may review the analysis code and see whether your provided node data will be used according to the study
protocol. If it is, approve the analyzes. If it is not, you may reject it, effectively blocking its execution on
your site. If you do so, we encourage you to inform the data analyst of the reason, so they can revise it. After approval
protocol. If it adheres to the protocol, then approval of the analysis is permitted, otherwise, you may reject it, effectively blocking its execution on your site. If you do so, we encourage you to inform the data analyst of the reason, so they can revise it. After approval
or rejection of each site, the analysis is built.

To review analysis, follow these steps:
1. Within the home area (top left), you need to select **Analyses** in the left navigation, and then the Option
**Incoming** to its right.
2. Here, you can view all approved, rejected, as well as analyzes awaiting either (Optional: Use the search bar at the
2. Here, you can view all approved, rejected, as well as analyses awaiting either (Optional: Use the search bar at the
top).
[![image](/images/ui_images/hub_analysis_review.png)](/images/ui_images/hub_analysis_review.png)
3. **Review** incoming analysis scripts by clicking the black download button on the right of the request.
4. **Approve** incoming analyzes by clicking the gray triple-bar button on the right of the anlaysis and then clicking the
4. **Approve** incoming analyses by clicking the gray triple-bar button on the right of the anlaysis and then clicking the
Comment thread
brucetony marked this conversation as resolved.
green approve button in the small popup.
5. **Reject** incoming analysis by clicking the gray triple-bar button on the right of the analysis and then clicking the
red reject button in the small popup.
Expand Down
121 changes: 100 additions & 21 deletions src/guide/admin/bucket-setup-for-data-store.md
Original file line number Diff line number Diff line change
@@ -1,44 +1,123 @@
# Bucket Setup for Data Store
# Bucket Setup for Data Store

Before an analysis can be executed in FLAME, the required data must be made accessible through a configured data store.
In this context, a bucket refers to a storage location—such as an S3 or MinIO bucket—that contains the dataset to be analyzed.
Creating a bucket is the first step in making data available to FLAME.
The bucket will later be referenced by a data store configuration, which connects the FLAME project to the actual data.
Before an analysis can be executed in FLAME, the required data must be made accessible through a configured data store.
In this context, a bucket refers to a storage location—such as an S3 or MinIO bucket—that contains the dataset to be
analyzed.
Creating a bucket is the first step in making data available to FLAME.
The bucket will later be referenced by a data store configuration, which connects the FLAME project to the actual data.

The examples shown below use the included MinIO object store, but these steps can be applied to other S3 stores.

::: info FLAME Node <0.0.10
As of flame-node v0.0.10, private buckets are supported and are set as the default. If you have flame-node <0.0.10
deployed, consider upgrading to the latest version, otherwise, only S3 buckets set to "Public" will work and any
instructions on this page mentioning "keys" are not applicable.
:::

## Creating a Bucket

To create a new bucket, go to the "Buckets" tab in the Administrator Area on the left side of the page and click "Create Bucket".
Give your bucket a name following the bucket naming rules, optionally select additional features, and click "Create Bucket" in the lower-right corner under the Features section.
To create a new bucket, go to the "Buckets" tab in the Administrator Area on the left side of the page and click "Create
Bucket".
Give your bucket a name following the bucket naming rules, optionally select additional features, and click "Create
Bucket" in the lower-right corner under the Features section.

[![Data Store Creator](/images/minio_images/bucket_create.png)](/images/minio_images/bucket_create.png)

To enable FLAME Nodes to access the data during development, the access policy of the newly created bucket must temporarily be set to "Public" in the summary tab. This setting is only required during the development phase and should be adjusted accordingly in production environments.
### Generating Access and Secret Keys

By default, S3 buckets are set to "Private", and accessing the data they contain requires the use of an **access key**
and **secret key**. These keys are also required when creating a data store in the Node UI for this bucket. In MinIO,
administrators and data stewards can generate a key pair by following these steps:

1. Navigate to the "Access Keys" tab in the Administrator Area on the left side of the page
2. Click the "Create access key" button in the top right corner of the page
3. Enter any desired metadata including a name and expiration date for the keys
* It is recommended you enable the "Restrict beyond user policy" toggle and further limit the access key
permissions. More details can be found in [Modifying Access Key Permissions](#modifying-access-key-permissions)
4. Click the "Create" button at the bottom

::: warning Save the Keys!
The secret key is **only** accessible during creation so it is important you record both keys now as you will need
them when creating a data store
:::

[![MinIO Access Key Generation](/images/minio_images/access_key_creation.png)](/images/minio_images/access_key_creation.png)

#### Modifying Access Key Permissions

Access keys have the same policies as the user who created them by default, however, it is advisable to limit the
permissions of the access keys used for data store creation. FLAME analyses only require the following permissions:

* Bucket
* `s3:GetBucketLocation`
* `s3:ListBucket`
* Data
* `s3:GetObject`

Thus, administrators and data stewards can use the following custom **Access Key Policy** (replacing `<bucket name>`
with the name of the appropriate bucket) for additional security:

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetBucketLocation",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::<bucket name>"
]
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::<bucket name>/*"
]
}
]
}
```

This policy can be applied either during access key creation or afterward by editing the access key settings.

### Uploading Data into a Bucket

To upload files or a directory into a bucket, navigate to "Object Browser" in the User Area and select the desired bucket.
Click the "Upload" button in the top-right corner and choose either "Upload File" or "Upload Folder". You can then select a file or folder from your local computer to upload.
To upload files or a directory into a bucket, navigate to "Object Browser" in the User Area and select the desired
bucket.
Click the "Upload" button in the top-right corner and choose either "Upload File" or "Upload Folder". You can then
select a file or folder from your local computer to upload.

[![Data Store Creator](/images/minio_images/bucket_upload.png)](/images/minio_images/bucket_upload.png)

If the upload starts successfully, a "Downloads/Uploads" tab will appear in the top-right corner, displaying the upload progress as a percentage.
If the upload starts successfully, a "Downloads/Uploads" tab will appear in the top-right corner, displaying the upload
progress as a percentage.

### Deleting a Bucket

A bucket can only be deleted after all its contents have been removed. To do this, go to the "Buckets" tab in the Administrator Area, select the bucket you want to delete, and then click the "Delete Bucket" button in the top-right corner.
A bucket can only be deleted after all its contents have been removed. To do this, go to the "Buckets" tab in the
Administrator Area, select the bucket you want to delete, and then click the "Delete Bucket" button in the top-right
corner.

## Integrate a Bucket in the Node
## Integrate a Bucket in the Node

To make data from a bucket available for analysis within FLAME, a corresponding data store must be created for the project that intends to use the data.
To make data from a bucket available for analysis within FLAME, a corresponding data store must be created for the
project that intends to use the data.

This process is explained in detail in the [Creating a Data Store](/guide/admin/data-store-management#creating-a-data-store) section. When configuring a data store for a MinIO or S3 bucket, the following parameters must be provided:
This process is explained in detail in
the [Creating a Data Store](/guide/admin/data-store-management#creating-a-data-store) section. When configuring a data
store for a MinIO or S3 bucket, the following parameters must be provided:

[![Data Store Creator](/images/minio_images/bucket_node.png)](/images/minio_images/bucket_node.png)

* **Server:** `node-datastore-minio`

* **Data Path:** `/` followed by the bucket name (e.g., `/hello-world`)

* **Data Store Type:** `S3`

* **Port:** `9000`
* **Hostname:** `node-datastore-minio`
* **Bucket Name:** `<bucket name>`
* **Access Key:** Generated access key for the listed, private S3 bucket
* **Secret Key:** Generated secret key for the listed, private S3 bucket
* **Port:** `9000`
Loading
Loading