docs(integration): Add github integration docs #124
Conversation
|
Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. WalkthroughAdds a new GitHub integration MDX doc describing installation/configuration of the Traceloop GitHub App, CI experiment scripts (Python/TypeScript), a GitHub Actions workflow, PR comment/dashboard result examples, and updates site navigation to include the new page. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes
Suggested reviewers
Poem
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches🧪 Generate unit tests (beta)
Comment |
![ellipsis-dev[bot]](https://avatars.githubusercontent.com/in/64358?s=60&v=4){kind=small}
There was a problem hiding this comment.
Caution
Changes requested ❌
Reviewed everything up to a010668 in 2 minutes and 22 seconds. Click for details.
- Reviewed
302lines of code in2files - Skipped
8files when reviewing. - Skipped posting
1draft comments. View those below. - Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.
1. integrations/github.mdx:33
- Draft comment:
Typographical error: "autorization" should be "authorization". - Reason this comment was not posted:
Marked as duplicate.
Workflow ID: wflow_fOvrQlhOAtSP6aeS
You can customize 
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
⛔ Files ignored due to path filters (8)
img/traceloop-integrations/github-app-auth-dark.pngis excluded by!**/*.pngimg/traceloop-integrations/github-app-auth-light.pngis excluded by!**/*.pngimg/traceloop-integrations/github-app-comment-dark.pngis excluded by!**/*.pngimg/traceloop-integrations/github-app-comment-light.pngis excluded by!**/*.pngimg/traceloop-integrations/github-app-exp-run-results-dark.pngis excluded by!**/*.pngimg/traceloop-integrations/github-app-exp-run-results-light.pngis excluded by!**/*.pngimg/traceloop-integrations/github-app-secrets-dark.pngis excluded by!**/*.pngimg/traceloop-integrations/github-app-secrets-light.pngis excluded by!**/*.png
📒 Files selected for processing (2)
integrations/github.mdx(1 hunks)mint.json(1 hunks)
🔇 Additional comments (4)
integrations/github.mdx (3)
207-207: Verify YAML code block identifier syntax.Line 207 uses an unusual identifier "yaml ci-cd configuration" for the code block. Verify this is the correct format for your documentation parser. Typically, code blocks use just the language identifier (e.g.,
yaml).
246-246: Verify link formatting renders correctly.Line 246 contains a link with an arrow symbol that may not render as intended:
[Generate one in Settings →](/settings/managing-api-keys). Consider either removing the arrow or verifying it displays correctly in the rendered documentation.
42-43: All referenced images exist in the repository.Verification confirms all 8 image files are present in
img/traceloop-integrations/:
- github-app-auth-light.png and -dark.png ✓
- github-app-secrets-light.png and -dark.png ✓
- github-app-comment-light.png and -dark.png ✓
- github-app-exp-run-results-light.png and -dark.png ✓
The documentation image references are valid and no action is needed.
mint.json (1)
203-206: LGTM: Navigation configuration is properly formatted.The integration documentation page has been correctly added to the site navigation. The reference path matches the filename, syntax is correct, and placement is consistent with existing integrations.
nirga
changed the title
nirga
left a comment
nirga
left a comment
There was a problem hiding this comment.
@nina-kollman Add a link to this from the experiments page with a
| </Info> | ||
| </Step> | ||
|
|
||
| <Step title="Configure Repository Access"> |
There was a problem hiding this comment.
missing the last step of confirmation you showed us
There was a problem hiding this comment.
It is here - number 3 :
|
@nina-kollman typos here we need to fix it before merging |
There was a problem hiding this comment.
Caution
Changes requested ❌
Reviewed 14290e9 in 2 minutes and 3 seconds. Click for details.
- Reviewed
31lines of code in1files - Skipped
0files when reviewing. - Skipped posting
2draft comments. View those below. - Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.
1. integrations/github.mdx:33
- Draft comment:
Good correction: 'autorization' has been fixed to 'authorization'. - Reason this comment was not posted:
Confidence changes required:0%<= threshold50%None
2. integrations/github.mdx:6
- Draft comment:
Typographical suggestion: The header 'Track experiments results in CI' might be rephrased. Consider using an apostrophe for clarity (e.g., 'Track experiments' results in CI') or rewording it to 'Track experiment results in CI'. - Reason this comment was not posted:
Marked as duplicate.
Workflow ID: wflow_Ri9V0HKilvNbE5ac
You can customize
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
integrations/github.mdx (1)
28-38: Fix outstanding typo in Step 2.Line 33 contains a spelling error that was previously flagged: "autorization" should be "authorization".
- After installing the app you will be redirected to a Traceloop autorization page. + After installing the app you will be redirected to a Traceloop authorization page.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
⛔ Files ignored due to path filters (2)
img/traceloop-integrations/github-app-comment-dark.pngis excluded by!**/*.pngimg/traceloop-integrations/github-app-comment-light.pngis excluded by!**/*.png
📒 Files selected for processing (1)
integrations/github.mdx(1 hunks)
| <Step title="Set up Your CI Workflow"> | ||
| Add a GitHub Actions workflow to automatically run Traceloop experiments on pull requests. | ||
| Below is an example workflow file you can customize for your project: | ||
| ```yaml ci-cd configuration | ||
| name: Run Traceloop Experiments | ||
|
|
||
| on: | ||
| pull_request: | ||
| branches: [main, master] | ||
|
|
||
| jobs: | ||
| run-experiments: | ||
| runs-on: ubuntu-latest | ||
|
|
||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Python | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: '3.11' | ||
|
|
||
| - name: Install dependencies | ||
| run: | | ||
| pip install traceloop-sdk openai | ||
|
|
||
| - name: Run experiments | ||
| env: | ||
| TRACELOOP_API_KEY: ${{ secrets.TRACELOOP_API_KEY }} | ||
| OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} | ||
| run: | | ||
| python experiments/run_ci_experiments.py | ||
| ``` |
There was a problem hiding this comment.
Remove extraneous text from YAML code fence.
Line 207 has extra text in the code fence that disrupts syntax highlighting. The code fence should only contain the language identifier.
- ```yaml ci-cd configuration
+ ```yaml
name: Run Traceloop Experiments🤖 Prompt for AI Agents
integrations/github.mdx around lines 204 to 237: the opening code fence contains
extraneous text "yaml ci-cd configuration" which breaks syntax highlighting;
remove that extra text so the fence is just the language identifier (```yaml)
and ensure the YAML block starts immediately after the fence with the existing
workflow content.
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
integrations/github.mdx(1 hunks)
🔇 Additional comments (2)
integrations/github.mdx (2)
33-33: Typo already fixed—good catch.The typo mentioned in past reviews has been corrected: "authorization" is now spelled correctly (was "autorization").
1-280: Comprehensive GitHub integration documentation—well structured.The documentation clearly covers the end-to-end workflow: GitHub App installation, repository configuration, experiment script creation (with Python and TypeScript examples), CI workflow setup, and results visualization. Code examples are runnable and well-commented. UI illustrations are provided with light/dark theme variants. The setup instructions are clear and actionable.
| <Step title="Set up Your CI Workflow"> | ||
| Add a GitHub Actions workflow to automatically run Traceloop experiments on pull requests. | ||
| Below is an example workflow file you can customize for your project: | ||
| ```yaml ci-cd configuration |
There was a problem hiding this comment.
Remove extraneous text from YAML code fence.
Line 207 contains extra text "ci-cd configuration" after the language identifier, which disrupts syntax highlighting. The code fence should only contain yaml.
- ```yaml ci-cd configuration
+ ```yaml
name: Run Traceloop Experiments🤖 Prompt for AI Agents
In integrations/github.mdx around line 207, the opening fenced code block
includes extraneous text ("ci-cd configuration") after the language identifier
which breaks syntax highlighting; remove the extra text so the fence is just
```yaml. Update the line to begin with exactly three backticks followed by
"yaml" and nothing else, leaving the rest of the block unchanged.
Important
Adds documentation for GitHub integration to run Traceloop experiments in CI/CD pipelines, including setup and usage instructions.
integrations/github.mdxwith a guide on setting up GitHub integration for Traceloop experiments.mint.jsonto include the new GitHub integration page under the "Integrations" group.This description was created by
for 14290e9. You can customize this summary. It will automatically update as commits are pushed.
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings.