feat(integrations-catalog): add public catalog (540 integrations)#2617
Merged
feat(integrations-catalog): add public catalog (540 integrations)#2617
Conversation
Adds a public, manually-synced catalog of all 540 CompAI integrations at integrations-catalog/. Each vendor gets integrations-catalog/integrations/<slug>.json with sanitized public metadata: vendor info, auth type, customer-facing setup instructions, credential field labels, check names + descriptions + severity, sync support flag. Implementation details intentionally excluded: check DSL (endpoint paths, request bodies, response parsing, aggregation), syncDefinition, internal DB IDs, credential field placeholders, logoUrl (to avoid exposing logo.dev publishable tokens). - scripts/sync.mjs: fetches /v1/internal/dynamic-integrations, dedups by slug, runs a secret-pattern scanner over every sanitized output and BLOCKS any file containing Stripe pk/sk, GitHub tokens, AWS AKIA, Anthropic keys, Jina keys, Slack tokens, JWTs, or Bearer tokens. Atomic-writes only when content hash differs. Stale cleanup only removes files whose slug is not in the authoritative list; preserves files whose fetch failed. - scripts/generate-readme.mjs: regenerates README.md with full alphabetical vendor table per category (SEO-friendly). Catalog is refreshed manually on demand rather than on a schedule — no cron, no GitHub Action. To refresh, run both scripts locally and push a PR. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Relocates catalog regeneration tooling from integrations-catalog/scripts to tools/integrations-catalog-sync/ for cleaner separation between consumer-facing catalog data and internal dev tooling. Also hardens the sync script: - Removes hardcoded fallback for COMPAI_INTERNAL_API_BASE — the env var is now strictly required. No internal endpoint leaks in source. - Adds tools/integrations-catalog-sync/README.md documenting env var contract, usage, safety guardrails, and the full list of stripped fields. No change to published catalog data — same 540 files, same sanitize rules. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
7 issues found across 544 files
Note: This PR contains a large number of files. During the trial, cubic reviews up to 50 files per PR. Paid plans get a higher limit.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="integrations-catalog/integrations/bamboohr.json">
<violation number="1" location="integrations-catalog/integrations/bamboohr.json:11">
P2: Setup instructions reference a subdomain input that is not present in `credentialFields`, creating inconsistent and misleading integration metadata.</violation>
</file>
<file name="integrations-catalog/integrations/auvik.json">
<violation number="1" location="integrations-catalog/integrations/auvik.json:7">
P2: Do not publish credential template placeholders in `baseUrl`; use a literal URL or `null` in the public catalog output.</violation>
</file>
<file name="integrations-catalog/integrations/affinity.json">
<violation number="1" location="integrations-catalog/integrations/affinity.json:11">
P2: The auth instructions encode the literal string `apikey:` instead of the user’s actual API key, which can lead to invalid credentials.</violation>
</file>
<file name="integrations-catalog/integrations/auth0.json">
<violation number="1" location="integrations-catalog/integrations/auth0.json:11">
P2: This setup stores an Auth0 Management API access token as a persistent credential, but those tokens expire (default 24h), so the integration will break after expiry unless users continually re-paste new tokens.</violation>
</file>
<file name="integrations-catalog/integrations/akamai.json">
<violation number="1" location="integrations-catalog/integrations/akamai.json:32">
P2: Use `type: "password"` for the Access Token field so sensitive token values are masked instead of displayed in plain text.</violation>
</file>
<file name="integrations-catalog/integrations/backblaze.json">
<violation number="1" location="integrations-catalog/integrations/backblaze.json:11">
P2: Backblaze auth setup instructions use `accountId:applicationKey`, but the general required format is `applicationKeyId:applicationKey`; this can break authentication for non-master app keys.</violation>
</file>
<file name="integrations-catalog/integrations/amplitude.json">
<violation number="1" location="integrations-catalog/integrations/amplitude.json:6">
P3: The `docsUrl` for Amplitude resolves to a 404 page, so users will be sent to a broken documentation link.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review, or fix all with cubic.
| "authConfig": { | ||
| "type": "custom", | ||
| "config": { | ||
| "setupInstructions": "1. Go to BambooHR > profile icon > API Keys > Add New Key\n2. Copy the API key\n3. Encode it: run echo -n YOUR_API_KEY:x | base64\n4. Enter the encoded result and your subdomain below", |
Contributor
There was a problem hiding this comment.
P2: Setup instructions reference a subdomain input that is not present in credentialFields, creating inconsistent and misleading integration metadata.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At integrations-catalog/integrations/bamboohr.json, line 11:
<comment>Setup instructions reference a subdomain input that is not present in `credentialFields`, creating inconsistent and misleading integration metadata.</comment>
<file context>
@@ -0,0 +1,45 @@
+ "authConfig": {
+ "type": "custom",
+ "config": {
+ "setupInstructions": "1. Go to BambooHR > profile icon > API Keys > Add New Key\n2. Copy the API key\n3. Encode it: run echo -n YOUR_API_KEY:x | base64\n4. Enter the encoded result and your subdomain below",
+ "credentialFields": [
+ {
</file context>
| "description": "Monitor Auvik network infrastructure for device inventory, alerts, and network management compliance", | ||
| "category": "Monitoring", | ||
| "docsUrl": "https://support.auvik.com/hc/en-us/articles/360031007111-Auvik-API-Integration-Guide", | ||
| "baseUrl": "https://auvikapi.{{credentials.region}}.my.auvik.com", |
Contributor
There was a problem hiding this comment.
P2: Do not publish credential template placeholders in baseUrl; use a literal URL or null in the public catalog output.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At integrations-catalog/integrations/auvik.json, line 7:
<comment>Do not publish credential template placeholders in `baseUrl`; use a literal URL or `null` in the public catalog output.</comment>
<file context>
@@ -0,0 +1,56 @@
+ "description": "Monitor Auvik network infrastructure for device inventory, alerts, and network management compliance",
+ "category": "Monitoring",
+ "docsUrl": "https://support.auvik.com/hc/en-us/articles/360031007111-Auvik-API-Integration-Guide",
+ "baseUrl": "https://auvikapi.{{credentials.region}}.my.auvik.com",
+ "authConfig": {
+ "type": "basic",
</file context>
Suggested change
| "baseUrl": "https://auvikapi.{{credentials.region}}.my.auvik.com", | |
| "baseUrl": null, |
| "authConfig": { | ||
| "type": "custom", | ||
| "config": { | ||
| "setupInstructions": "1. Log in to Affinity at https://affinity.co\n2. Go to Settings → API\n3. Copy your API Key\n4. Base64 encode 'api_key:' (with colon): echo -n 'apikey:' | base64\n5. Enter the encoded value below", |
Contributor
There was a problem hiding this comment.
P2: The auth instructions encode the literal string apikey: instead of the user’s actual API key, which can lead to invalid credentials.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At integrations-catalog/integrations/affinity.json, line 11:
<comment>The auth instructions encode the literal string `apikey:` instead of the user’s actual API key, which can lead to invalid credentials.</comment>
<file context>
@@ -0,0 +1,45 @@
+ "authConfig": {
+ "type": "custom",
+ "config": {
+ "setupInstructions": "1. Log in to Affinity at https://affinity.co\n2. Go to Settings → API\n3. Copy your API Key\n4. Base64 encode 'api_key:' (with colon): echo -n 'apikey:' | base64\n5. Enter the encoded value below",
+ "credentialFields": [
+ {
</file context>
| "authConfig": { | ||
| "type": "custom", | ||
| "config": { | ||
| "setupInstructions": "1. Go to Auth0 Dashboard > Applications > Create Application\n2. Select Machine to Machine\n3. Authorize the Auth0 Management API with read:users, read:logs, read:guardian_factors permissions\n4. Deploy and copy the access token\n5. Paste it below", |
Contributor
There was a problem hiding this comment.
P2: This setup stores an Auth0 Management API access token as a persistent credential, but those tokens expire (default 24h), so the integration will break after expiry unless users continually re-paste new tokens.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At integrations-catalog/integrations/auth0.json, line 11:
<comment>This setup stores an Auth0 Management API access token as a persistent credential, but those tokens expire (default 24h), so the integration will break after expiry unless users continually re-paste new tokens.</comment>
<file context>
@@ -0,0 +1,52 @@
+ "authConfig": {
+ "type": "custom",
+ "config": {
+ "setupInstructions": "1. Go to Auth0 Dashboard > Applications > Create Application\n2. Select Machine to Machine\n3. Authorize the Auth0 Management API with read:users, read:logs, read:guardian_factors permissions\n4. Deploy and copy the access token\n5. Paste it below",
+ "credentialFields": [
+ {
</file context>
| "helpText": "Client secret from your .edgerc file" | ||
| }, | ||
| { | ||
| "label": "Access Token", |
Contributor
There was a problem hiding this comment.
P2: Use type: "password" for the Access Token field so sensitive token values are masked instead of displayed in plain text.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At integrations-catalog/integrations/akamai.json, line 32:
<comment>Use `type: "password"` for the Access Token field so sensitive token values are masked instead of displayed in plain text.</comment>
<file context>
@@ -0,0 +1,70 @@
+ "helpText": "Client secret from your .edgerc file"
+ },
+ {
+ "label": "Access Token",
+ "type": "text",
+ "required": true,
</file context>
| "authConfig": { | ||
| "type": "custom", | ||
| "config": { | ||
| "setupInstructions": "1. Log in to Backblaze\n2. Go to Account → App Keys\n3. Note your Account ID and Application Key\n4. Encode them: echo -n 'accountId:applicationKey' | base64\n5. Paste the encoded value below", |
Contributor
There was a problem hiding this comment.
P2: Backblaze auth setup instructions use accountId:applicationKey, but the general required format is applicationKeyId:applicationKey; this can break authentication for non-master app keys.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At integrations-catalog/integrations/backblaze.json, line 11:
<comment>Backblaze auth setup instructions use `accountId:applicationKey`, but the general required format is `applicationKeyId:applicationKey`; this can break authentication for non-master app keys.</comment>
<file context>
@@ -0,0 +1,51 @@
+ "authConfig": {
+ "type": "custom",
+ "config": {
+ "setupInstructions": "1. Log in to Backblaze\n2. Go to Account → App Keys\n3. Note your Account ID and Application Key\n4. Encode them: echo -n 'accountId:applicationKey' | base64\n5. Paste the encoded value below",
+ "credentialFields": [
+ {
</file context>
| "name": "Amplitude", | ||
| "description": "Monitor Amplitude projects and user access for product analytics compliance", | ||
| "category": "Monitoring", | ||
| "docsUrl": "https://www.docs.developers.amplitude.com/analytics/apis/management-api/", |
Contributor
There was a problem hiding this comment.
P3: The docsUrl for Amplitude resolves to a 404 page, so users will be sent to a broken documentation link.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At integrations-catalog/integrations/amplitude.json, line 6:
<comment>The `docsUrl` for Amplitude resolves to a 404 page, so users will be sent to a broken documentation link.</comment>
<file context>
@@ -0,0 +1,35 @@
+ "name": "Amplitude",
+ "description": "Monitor Amplitude projects and user access for product analytics compliance",
+ "category": "Monitoring",
+ "docsUrl": "https://www.docs.developers.amplitude.com/analytics/apis/management-api/",
+ "baseUrl": "https://management.us.amplitude.com/",
+ "authConfig": {
</file context>
Suggested change
| "docsUrl": "https://www.docs.developers.amplitude.com/analytics/apis/management-api/", | |
| "docsUrl": "https://amplitude.com/docs/apis", |
Adds SYNC_MIN_INTERVAL_MS (default 100ms) to enforce a minimum gap between HTTP requests across all workers. Paces the sync to stay under the CompAI internal API throttle, which was triggering 429s at higher concurrency. Also logs backoff delays when retries fire for visibility. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
2 issues found across 2 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="tools/integrations-catalog-sync/sync.mjs">
<violation number="1" location="tools/integrations-catalog-sync/sync.mjs:59">
P2: Validate `SYNC_MIN_INTERVAL_MS` before use; invalid values currently become `NaN` and silently disable request pacing.</violation>
<violation number="2" location="tools/integrations-catalog-sync/sync.mjs:63">
P2: `pace()` is not concurrency-safe: concurrent workers can wake at the same time and issue back-to-back requests, bypassing the intended global throttle.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review, or fix all with cubic.
| return new Promise((resolve) => setTimeout(resolve, ms)); | ||
| } | ||
|
|
||
| const MIN_REQUEST_INTERVAL_MS = parseInt(process.env.SYNC_MIN_INTERVAL_MS || "100", 10); |
Contributor
There was a problem hiding this comment.
P2: Validate SYNC_MIN_INTERVAL_MS before use; invalid values currently become NaN and silently disable request pacing.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At tools/integrations-catalog-sync/sync.mjs, line 59:
<comment>Validate `SYNC_MIN_INTERVAL_MS` before use; invalid values currently become `NaN` and silently disable request pacing.</comment>
<file context>
@@ -56,16 +56,27 @@ function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
+const MIN_REQUEST_INTERVAL_MS = parseInt(process.env.SYNC_MIN_INTERVAL_MS || "100", 10);
+let nextAllowedAt = 0;
+
</file context>
Suggested change
| const MIN_REQUEST_INTERVAL_MS = parseInt(process.env.SYNC_MIN_INTERVAL_MS || "100", 10); | |
| const parsedMinInterval = Number.parseInt(process.env.SYNC_MIN_INTERVAL_MS || "100", 10); | |
| const MIN_REQUEST_INTERVAL_MS = Number.isFinite(parsedMinInterval) && parsedMinInterval >= 0 ? parsedMinInterval : 100; |
Comment on lines
+63
to
+65
| const now = Date.now(); | ||
| if (now < nextAllowedAt) await sleep(nextAllowedAt - now); | ||
| nextAllowedAt = Date.now() + MIN_REQUEST_INTERVAL_MS; |
Contributor
There was a problem hiding this comment.
P2: pace() is not concurrency-safe: concurrent workers can wake at the same time and issue back-to-back requests, bypassing the intended global throttle.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At tools/integrations-catalog-sync/sync.mjs, line 63:
<comment>`pace()` is not concurrency-safe: concurrent workers can wake at the same time and issue back-to-back requests, bypassing the intended global throttle.</comment>
<file context>
@@ -56,16 +56,27 @@ function sleep(ms) {
+let nextAllowedAt = 0;
+
+async function pace() {
+ const now = Date.now();
+ if (now < nextAllowedAt) await sleep(nextAllowedAt - now);
+ nextAllowedAt = Date.now() + MIN_REQUEST_INTERVAL_MS;
</file context>
Suggested change
| const now = Date.now(); | |
| if (now < nextAllowedAt) await sleep(nextAllowedAt - now); | |
| nextAllowedAt = Date.now() + MIN_REQUEST_INTERVAL_MS; | |
| while (true) { | |
| const now = Date.now(); | |
| if (now >= nextAllowedAt) { | |
| nextAllowedAt = now + MIN_REQUEST_INTERVAL_MS; | |
| return; | |
| } | |
| await sleep(nextAllowedAt - now); | |
| } |
Contributor
|
🎉 This PR is included in version 3.27.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds public catalog of all 540 CompAI integrations at
integrations-catalog/. Manually synced on demand — no cron, no GitHub Action.What's published
Each
integrations/<slug>.jsoncontains:Summary by cubic
Adds a public, sanitized catalog of 540 CompAI integrations under
integrations-catalog/to make integration metadata discoverable without exposing implementation details. Tooling moved totools/integrations-catalog-sync/and hardened; catalog data is unchanged and still refreshed manually.New Features
index.jsonand 540integrations/*.jsonwith vendor/category/URLs, auth type + setup steps + credential labels, checks (name/description/default severity), and capabilities incl. sync support.tools/integrations-catalog-sync/sync.mjs(fetch, sanitize, dedupe, content-hash idempotency, atomic writes; global request pacing viaSYNC_MIN_INTERVAL_MSwith 100ms default to avoid 429s; backoff logging; no schedule; requiresCOMPAI_INTERNAL_API_BASEandCOMPAI_INTERNAL_TOKEN) andtools/integrations-catalog-sync/generate-readme.mjs(category-organizedREADME.md).Migration
COMPAI_INTERNAL_API_BASE=<base> COMPAI_INTERNAL_TOKEN=<token> node tools/integrations-catalog-sync/sync.mjs && node tools/integrations-catalog-sync/generate-readme.mjs(optional: setSYNC_MIN_INTERVAL_MSto tune pacing)Written for commit 0dfb793. Summary will update on new commits.