Observe Entity Explorer

Observe Entity Explorer allows discovery of dependencies between Observe objects (datasets, worksheets, dashboards, monitors, stages, parameters).

NOTE: This tool uses the Observe GraphQL API, which is not [yet] a publicly documented API. Therefore this tool, or any application that you develop that is derived from it may stop working without notice in the future. If you have specific API needs that are not covered by our currently documented API found at https://developer.observeinc.com/, please contact us for more information.

Install Application

Observe Entity Explorer can run on Windows, Mac or Linux. Compiled are in Releases.

Install on OSX

1. Install .NET 8.0 SDK

   • Download Microsoft Downloads web site with all instructions, or:
     • Direct x64 installer
     • Direct arm64 (M1/M2) installer
   • OR
   • Homebrew brew install dotnet

2. Download Releases\ observe-entity-explorer.osx-x64.<version>.zip for Intel or observe-entity-explorer.osx-arm64.<version>.zip for M1/M2 machines, but do not extract the archive yet.

3. Remove the quarantine attribute that will otherwise stop the application from running this command in your terminal:

   xattr -d com.apple.quarantine observe-entity-explorer.osx*.zip

4. Extract the archive:

   unzip observe-entity-explorer.osx*.zip

5. Run the application from the extracted folder:

   dotnet ./observe-entity-explorer.dll

Install on Windows

1. Download Releases\ observe-entity-explorer.win-x64.<version>.zip

2. Extract the archive using File Explorer or:

   Expand-Archive observe-entity-explorer.win*.zip

3. Run the application from the extracted folder:

   .\observe-entity-explorer.exe

Install on Linux

1. Download Releases\ observe-entity-explorer.linux-x64.<version>.zip

2. Extract the archive:

   unzip observe-entity-explorer.linux*.zip

3. Run the application from the extracted folder:

   ./observe-entity-explorer

Run from Source

You can also run it from source.

First, install .NET 8.0 SDK from Microsoft Downloads web site.

Then clone, build and run:

# Clone repo to local folder
git clone https://github.com/observeinc/entity-explorer entity-explorer

# Change to local folder
cd entity-explorer

# For Win and Linux, build the project into single self-contained binary. Choose the right runtime
dotnet publish observe-entity-explorer.csproj --self-contained --runtime <linux-x64|win-x64> -p:PublishSingleFile=true -o bin/publish/entity-explorer

# For OSX, build the project into multiple files to be ran by dotnet command later. Choose the right runtime
dotnet publish observe-entity-explorer.csproj --self-contained --runtime <osx-x64|osx-arm64> -p:PublishSingleFile=false -o bin/publish/entity-explorer

# Run the application from where it was published to
bin/publish/entity-explorer/observe-entity-explorer & 

Usage

Open the Web Site

When application starts, it opens a web interface at http://localhost:50110. Open that web site in your web browser:

./observe-entity-explorer

info: Microsoft.Hosting.Lifetime[0]
      Now listening on: http://localhost:50110
info: Microsoft.Hosting.Lifetime[0]
      Application started. Press Ctrl+C to shut down.
info: Microsoft.Hosting.Lifetime[0]
      Hosting environment: Production
info: Microsoft.Hosting.Lifetime[0]
      Content root path: C:\observe\observe-entity-explorer-releases\2023.10.18.0\observe-entity-explorer.win.2023.10.18.0

Connect Options

Authenticate options:

• Account
  • Full URL of the environment https://#####.observeinc.com
  • Just customer ID of the environment #####. Production URL observeinc.com is assumed
  • Host name #####.observeinc.com without https://
• Username
  • Typically it is your email
• Password
  • Specify your password
• Token
  • Specify your API token if you have one issued to you
  • OR
  • Specify token taken from the GraphQL Bearer authentication in your browser devtools
• Delegate (must be > Reader)
  • Log into Observe UI to approve your sign-in
  • Return to this UI to complete sign-in

Connect with Delegate Login

To connect with delegation, you must have > Reader role. If you have it:

• Specify account and username
• Click "Connect with Observe UI" button
• Click on "Go to Observe to approve login request link"
• In new browser window, authenticate into Observe
• Confirm your sign-in request on "Accounts Settings" page
• Back in Observe Entity Explorer, click "Check for Completion and Connect" button to finish signing in

Connect with Multiple Accounts

Your connections are saved. You can switch between them any time.

You can open new tab with different connection ID.

Entity Summary

Summary screen shows basic statistics about types of entities in your Observe environment and links to the Dataset List Dashboard List for getting more details.

You can also see the entire tree of relationships of all objects in All Relationships button.

Dataset List

Select Dataset screen shows all datasets grouped by type (DataStream, Event, Resource, Log, Metric, etc).

Column	Description
Details	Open the Dataset Detail page
View	Open object page in Observe
Edit	Open object page in Observe in edit mode
Type	Type of object (Event, Resource, Interval)
Origin	What created this object (System, App, Terraform, User, DataStream)
ID	Object ID
Name	Object Name
Description	Description label, if present
Parts	Fields, primary and foreign keys, related keys
Uses	What this object uses as data inputs and data links
Used By	What uses this object (Data, Dashboard, Monitor)
Created	Who and when created this object
Updated	Who and when updates this object last
Transform 1h	credits_transform in last 1 hour
Transform 1d	credits_transform in last 1 day
Transform 1w	credits_transform in last 1 week
Query 1h	credits_adhoc_query in last 1 hour
Query 1d	credits_adhoc_query in last 1 day
Query 1w	credits_adhoc_query in last 1 week
Accel Conf.	Configured staleness target of the dataset
Accel Eff.	The target staleness of this dataset when taking downstream datasets
Accel Actl.	Staleness of the dataset (averaged over some moving window)
Accel Range	Actual accelerated range of the dataset

Dashboard List

Select Dashboard screen shows all dashboards.

Column	Description
Details	Links to the Dashboard Detail page
View	Open object page in Observe
Edit	Open object page in Observe in edit mode
Origin	What created this object (System, App, Terraform, User, DataStream)
ID	Object ID
Name	Object Name
Sections	Number of sections in dashboard
Widgets	Number of widgets in dashboard
Stages	Number of stages in dashboard
Parameters	Number of parameters in dashboard
Uses	What this object uses as data inputs and data links
Created	Who and when created this object
Updated	Who and when updates this object last

Monitor List

Select Monitor screen shows all monitors.

Column	Description
Details	Links to the Monitor Detail page
Notif	Open notification list page in Observe
Edit	Open object page in Observe in edit mode
Type	Type of the monitor (Count, Promote, Threshold, Facet, Log)
Origin	What created this object (System, App, Terraform, User, DataStream)
ID	Object ID
Name	Object Name
Comm.	Comment if it exists
Enabled	Is the monitor enabled
Actions	Number of actions in monitor
Stages	Number of stages in monitor
Uses	What this object uses as data inputs and data links
Created	Who and when created this object
Updated	Who and when updates this object last
Transform 1h	credits_monitor in last 1 hour
Transform 1d	credits_monitor in last 1 day
Transform 1w	credits_monitor in last 1 week
Accel Conf.	Configured staleness target of the dataset
Accel Eff.	The target staleness of this dataset when taking downstream datasets
Accel Actl.	Staleness of the dataset (averaged over some moving window)
Accel Range	Actual accelerated range of the dataset

Dataset Detail

Summary - Dataset

Summary information about the dataset.

Section	Description
Observe	Open object page in Observe
Name	Object name and ID
Description	Dataset description
Type	Type of dataset
Icon	Icon of the object
Origin	What created this object (System, App, Terraform, User, DataStream)
Created	Who and when created this object
Updated	Who and when updates this object last
Credits	credits_transform and credits_adhoc_query for 1 hour, 1 day, 1 week
Query Credits	credits_adhoc_query for each user for 1 hour, 1 day, 1 week
Important Fields	From/To/Label designated fields
Fields	Table of fields and their datatypes
Related Key	List of related keys
Foreign Key	List of foreign keys

Acceleration - Dataset

Dataset acceleration settings.

Ancestors - Dataset

List of all datasets feeding data into this dataset.

Descendants - Dataset

List of all datasets using data from this dataset.

Related Datasets Graph - Dataset

Visual diagram showing relationship between datasets and dashboards related to this dataset.

Stages - Dataset

Information about each stage in the dataset.

Section	Description
Details	Link to the Stage in the subsequent List
Type	Type of the stage (table, graph etc)
Name	Name of the stage
ID	ID of the stage
Uses	Objects providing data and links to this stage
Used By	Objects this stage is providing data and links to

This table is followed by the repeated sections for each stage, containing:

Section	Description
Name	Object name and ID
Inputs	Objects providing data and links to this stage
Used By	Objects this stage is providing data and links to
OPAL	The OPAL pipeline for this stage

You can click "Show/Hide input and output tables" to remove details and just focus on OPAL code.

Stages Graph - Dataset

Visual diagram showing relationship between inputs and stages of this dataset.

Related Dashboards

List of all dashboards using this dataset.

Related Monitors

List of all monitors using this dataset.

Dashboard Detail

Summary - Dashboard

Summary information about the dashboard.

Section	Description
Observe	Open object page in Observe
Name	Object name and ID
Source	What created this object (System, App, Terraform, User, DataStream)
Components	Number of various components in dashboard
Created	Who and when created this object
Updated	Who and when updates this object last

Ancestors/Inputs - Dashboard

List of all datasets feeding data into this dashboard.

Related Datasets - Dashboard

Visual diagram showing relationship between datasets related to this dashboard.

Parameters - Dashboard

Information about each parameter in the dashboard.

Section	Description
Name	Display name of parameter
ID	ID of parameter
Type	Type of parameter (resource-input, single-select, text, numeric, input)
Data Type	Data Type of parameter ()
Source Type	What kind of object is providing data to this parameter (Dataset, Stage or empty for text entry)
Source Object	Link to the object that is used as input
Source Column	The name of the column in the dataset or stage that is being used as value in the drop-down
Default Value	Default value, if specified
Allow Empty	Is empty value allowed

Stages - Dashboard

Information about each stage in the dashboard.

Section	Description
Details	Link to the stage in the subsequent list
Type	Type of the stage (table, graph etc)
Name	Name of the stage
ID	ID of the stage
Uses	Objects providing data and links to this stage
Used By	Objects this stage is providing data and links to
Params	Which parameters are being used by the stage

This table is followed by the repeated sections for each stage, containing:

Section	Description
Name	Object name and ID
Inputs	Objects providing data and links to this stage
Used By	Objects this stage is providing data and links to
OPAL	The OPAL pipeline for this stage

You can click "Show/Hide input and output tables" to remove details and just focus on OPAL code.

You can click "Show/Hide dashboard widget image preview" to remove images and just focus on OPAL code.

Stages Graph - Dashboard

Visual diagram showing relationship between inputs and stages of this dashboard.

Monitor Detail

Summary - Monitor

Summary information about the monitor.

Section	Description
Observe	Open object page in Observe
Name	Object name and ID
Comment	Comment if present
Type	Type of the monitor
Source	What created this object (System, App, Terraform, User, DataStream)
Components	Number of various components in dashboard
Enabled	Is this monitor enabled
Template	Is this monitor template
Created	Who and when created this object
Updated	Who and when updates this object last
Settings	Monitor settings
Notifications	Monitor notification settings
Info	Monitor detail information

Acceleration - Monitor

Monitor acceleration settings similar to those of Dataset

Ancestors - Monitor

List of all datasets providing data to this monitor.

Supporting - Monitor

List of all datasets supporting this monitor.

Related Datasets Graph - Monitor

Visual diagram showing relationship between datasets related to this monitor.

Stages - Monitor

Information about each stage in the monitor.

Logging

Logs are in /logs subfolder below where your application is installed with new files created every day:

logs\Observe.EntityExplorer.Console.2023-10-18.log
logs\Observe.EntityExplorer.Main.2023-10-18.log
logs\Observe.EntityExplorer.ObserveConnection.2023-10-18.log