Add documentation for JsonImporter by Phanindra899 · Pull Request #146 · apache/otava

Phanindra899 · 2026-03-30T23:58:42Z

This PR adds documentation for the JsonImporter.

The documentation explains:

Expected JSON file structure
Required and optional fields
Configuration example
Behavior and filtering logic
Limitations and constraints

Closes #128

Let me know if any changes are needed.

Phanindra899 · 2026-03-30T23:59:06Z

Hi @henrikingo,

I've added documentation for JsonImporter based on the current implementation.

Could you please review and let me know if any changes are needed?

Thanks!

Phanindra899 · 2026-03-31T00:10:55Z

Hi @henrikingo,

It looks like the GitHub Actions workflow didn’t run due to fork security restrictions.

Could you please approve the workflow so the checks can run?

Thanks!

henrikingo · 2026-04-02T20:49:09Z

Thank you

Can you please execute the commands in the new documentation and copy paste the output in this discussion.

If there are files involved thrn please cat or head their contents.

Phanindra899 · 2026-04-03T05:11:52Z

@henrikingo — I've verified the commands from the JSON documentation and confirmed they work as expected. Below is the minimal example I used for validation.

JSON File (`test_data/sample.json`)

[
  {
    "timestamp": 1711929600,
    "metrics": [
      { "name": "throughput", "value": 100 }
    ],
    "attributes": {
      "branch": "main"
    }
  }
]

Config File (`otava.yaml`)

tests:
  my_benchmark:
    type: json
    file: test_data/sample.json
    base_branch: main

Command Executed

otava analyze my_benchmark --config otava.yaml

Output

INFO: Computing change points for test my_benchmark...
time                       branch      throughput
-------------------------  --------  ------------
2024-04-01 00:00:00 +0000  main               100

Everything works as documented. Let me know if any adjustments are needed.

henrikingo · 2026-04-03T14:32:14Z

Yes, now with the added files it makes more sense :-)

Can you move the new files so that they are under otava/examples/json/*

I will add some other comments too, but now we have some meaningful direction to start from.

henrikingo · 2026-04-02T12:47:06Z

docs/JSON.md

+
+## Overview
+
+`JsonImporter` reads benchmark results from a local JSON file and feeds them into Otava for change-point analysis. It is the simplest data source to set up — no external database or service is required.


Json and Csv both have this property.

henrikingo · 2026-04-03T14:38:36Z

docs/JSON.md

+## Example Usage
+
+Run analysis on a test backed by a JSON file:
+otava analyze my_benchmark


Please change this to the command that you are using in your reply today:

otava analyze my_benchmark --config otava.yaml

otava/examples/json/data/sample.json

henrikingo · 2026-04-03T14:47:58Z

otava.yaml

@@ -0,0 +1,5 @@
+tests:


...and this under /examples/json/config/

Phanindra899 · 2026-04-03T18:16:19Z

Hi @henrikingo, I’ve addressed all the requested changes:
• Moved example files to otava/examples/json/data/ and config/
• Updated configuration paths in documentation
• Fixed the example usage command
• Added required license headers

Please let me know if anything else needs improvement. Thanks!

henrikingo · 2026-04-03T18:56:32Z

docs/JSON.md

+
+- **Type:** integer (Unix epoch seconds)
+- **Required:** yes
+- Identifies when the benchmark run occurred. Used for time-range filtering via `DataSelector`.


Sorry I missed this the first time. Actually, this should be the time that the commit was merged to the branch we are tracking. Typically this is recorded in git's history as a merge-commit.

In most cases this should NOT be the time that the benchmark was run. In particular, it often happens that you want to rerun a benchmark with the system under test ghecked out and built from this same githash. In such a case, the new results should be recorded with this same timestamp. For the same githash, the timestamp should be constant, regardless of when or how many times the benchmark was run.

henrikingo

Ok thanks! The structure is clearer now and I think this will work.

I have several more comments now on wording or style issues. This is normal when you contribute to a project the first time, even for experienced developers.

I tried to be thorough. Hopefully after this, there won't be many or any more changes.

docs/JSON.md

henrikingo · 2026-04-03T19:12:20Z

docs/JSON.md

+- Each object must have:
+  - `name` (string) — unique identifier for the metric within this run
+  - `value` (number) — the measured value
+- Metric names are collected dynamically across all entries in the file. Names must be consistent across runs for change-point analysis to be meaningful.


I suggest removing the first sentence. It sound like it is describing the intrenals of how our code works. But this is a user manual, they don't need to understand that. The second sentence can stay though.

henrikingo · 2026-04-03T19:17:57Z

docs/JSON.md

+### `attributes`
+
+- **Type:** object (string → string)
+- **Required:** yes if `branch` filtering is used


=> So really you are saying Required: no

docs/JSON.md

henrikingo · 2026-04-03T19:46:14Z

docs/JSON.md

+
+## Example Usage
+
+Run analysis on a test backed by a JSON file:


Suggest to change:

"Analyze test results stored in JSON format."

henrikingo · 2026-04-03T19:47:32Z

otava/examples/json/config/otava.yaml

+tests:
+  my_benchmark:
+    type: json
+    file: test_data/sample.json


This needs updating. The user should be able to try the provided example and it should work.

henrikingo · 2026-04-03T19:48:31Z

otava/examples/json/data/sample.json

+    "attributes": {
+      "branch": "main"
+    }
+  }


You'd better add more than one data point if you want to demo change point detection :-)

Two points also is not enough to trigger a change point...

Perhaps you should use the same data that is in the csv example (local_sample.csv). Just convert that to json.

…ibutes, fix paths, and update examples

Phanindra899 · 2026-04-04T04:10:48Z

HI @henrikingo ,Thanks for the detailed feedback! I’ve addressed all the points:
• clarified timestamp semantics
• removed internal implementation details
• updated wording (runs → results where applicable)
• simplified attributes section
• fixed config file paths
• improved sample JSON with multiple data points

Please let me know if anything else needs adjustment.

Phanindra899 · 2026-04-06T13:36:24Z

Hi @henrikingo, I’ve updated the JSON example to include multiple realistic data points based on the CSV example, so it now supports meaningful change-point detection.

Let me know if anything else should be improved.

Phanindra899 · 2026-04-06T13:43:16Z

Hi @henrikingo, I realized that some unrelated Sphinx documentation changes were included in this PR when merging branches.

If you’d prefer, I can separate those into a different PR and keep this one focused only on the JsonImporter documentation changes.

Please let me know how you’d like me to proceed

henrikingo · 2026-04-06T19:22:36Z

Sounds like you want to learn about the rebase command so you can undo that last merge com. Once you manage to do that, use git push -f to push a corrected series of commits to this same PR.

Phanindra899 added 6 commits March 30, 2026 17:38

Add structured documentation homepage replacing directory listing

38f2d14

Add documentation landing page replacing directory listing

d876ce6

Add initial Sphinx documentation setup with MyST support

67fc0a9

Ignore Sphinx build and virtual environment files

9241b07

Add documentation for JsonImporter

3017a85

Add Apache license header to JSON docs

14b45a8

Add example JSON data and config for documentation validation

6bb5371

henrikingo requested changes Apr 3, 2026

View reviewed changes

Fix license header for YAML example

682a4a4

henrikingo reviewed Apr 3, 2026

View reviewed changes

Improve JsonImporter docs: clarify timestamp semantics, simplify attr…

d1efe5b

…ibutes, fix paths, and update examples

Phanindra899 added 4 commits April 4, 2026 16:31

Fix Sphinx index to use docs/ as source

68479e9

Clean Sphinx config and add docs path

1f550a4

Use realistic multi-point JSON example for change-point detection

a26b564

Merge branch 'sphinx-docs-setup' into json-importer-docs

ed63782


		## Overview

		`JsonImporter` reads benchmark results from a local JSON file and feeds them into Otava for change-point analysis. It is the simplest data source to set up — no external database or service is required.


		## Example Usage

		Run analysis on a test backed by a JSON file:

Conversation

Phanindra899 commented Mar 30, 2026

Uh oh!

Phanindra899 commented Mar 30, 2026

Uh oh!

Phanindra899 commented Mar 31, 2026

Uh oh!

henrikingo commented Apr 2, 2026

Uh oh!

Phanindra899 commented Apr 3, 2026

JSON File (test_data/sample.json)

Config File (otava.yaml)

Command Executed

Output

Uh oh!

henrikingo commented Apr 3, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Phanindra899 commented Apr 3, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

henrikingo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Phanindra899 commented Apr 4, 2026

Uh oh!

Phanindra899 commented Apr 6, 2026

Uh oh!

Phanindra899 commented Apr 6, 2026

Uh oh!

henrikingo commented Apr 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

JSON File (`test_data/sample.json`)

Config File (`otava.yaml`)