Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 42 additions & 1 deletion docs/automations/integrations/copilot/flag-copilot-pr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,48 @@ starter_kits: [genai]
<!-- --8<-- [start:example]-->
Automatically apply labels to PRs that are assisted by GitHub Copilot. You can apply labels based on a known list of Copilot users, PR tags, or by prompting the PR author to indicate if they used Copilot.

=== "Label by Prompt"
=== "Label by AI Rules"
Automatically apply labels to PRs based on Copilot AI rules, providing a data-driven approach to track developer adoption of AI tools without requiring manual input from developers.

This solution automatically tracks AI tool usage without requiring developers to fill out surveys or take manual actions. Each AI-assisted edit is logged with unique identifiers in structured JSON format, setting the foundation for analytics across different AI tools while maintaining a consistent data model.

!!! info "Configuration Description"
Conditions:

* A PR is created
* Copilot AI was used in this PR
* Copilot rule file was added to the repo
* Changes to `.cm/ai.log` file are detected (this file contains entries for each AI-assisted action)

Automation Actions:

* PR is labeled with `ai-assisted`
* Structured data about AI usage is logged for future analytics

!!! example "`.github/copilot-instructions.md`"
```yaml+jinja
--8<-- "docs/downloads/automation-library/integrations/copilot/copilot-instructions.md"
```
<div class="result" markdown>
<span>
[:octicons-download-24: Download Cursor rules file.](/downloads/automation-library/integrations/copilot/copilot-instructions.md){ .md-button }
</span>
</div>

!!! example "PR is labled with AI usage"
```yaml+jinja
--8<-- "docs/downloads/automation-library/integrations/copilot/label_copilot_by_rule.cm"
```
<div class="result" markdown>
<span>
[:octicons-download-24: Download this example as a CM file.](/downloads/automation-library/integrations/copilot/label_copilot_by_rule.cm){ .md-button }
</span>
</div>

!!! tip
Add `.cm/ai.log linguist-generated=true` to your `.gitattributes` file to ensure that Copilot-generated files are not included in your repository.

=== "Label by Survey"
Prompt PR authors to indicate if they used Copilot for the PR and automatically label the PR if they did. This requires two separate automation files to handle posting the prompt and labeling accordingly.

![Label Copilot by Prompt](/automations/integrations/copilot/flag-copilot-pr/label-copilot-by-prompt.png)
Expand Down
44 changes: 43 additions & 1 deletion docs/automations/integrations/cursor/flag-cursor-pr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,49 @@ category: [quality, genai, cursor, quickstart]
<!-- --8<-- [start:example]-->
Automatically apply labels to Pull Requests that are assisted by Cursor AI. This automation helps track the impact and usage of Cursor's AI capabilities across your development workflow.

=== "Label by Prompt"
=== "Label by AI Rules"
Automatically apply labels to PRs based on Cursor AI rules, providing a data-driven approach to track developer adoption of AI tools without requiring manual input from developers.

This solution automatically tracks AI tool usage without requiring developers to fill out surveys or take manual actions. Each AI-assisted edit is logged with unique identifiers in structured JSON format, setting the foundation for analytics across different AI tools while maintaining a consistent data model.

!!! info "Configuration Description"
Conditions:

* A PR is created
* Cursor AI was used in this PR
* Cursor rule file was added to the repo
* Changes to `.cm/ai.log` file are detected (this file contains entries for each AI-assisted action)

Automation Actions:

* PR is labeled with `ai-assisted`
* Structured data about AI usage is logged for future analytics

!!! example "`.cursor/rules/cursor-ai-logging.mdc`"
```yaml+jinja
--8<-- "docs/downloads/automation-library/integrations/cursor/cursor-ai-logging.mdc"
```
<div class="result" markdown>
<span>
[:octicons-download-24: Download Cursor rules file.](/downloads/automation-library/integrations/cursor/cursor-ai-logging.mdc){ .md-button }
</span>
</div>

!!! example "PR is labled with AI usage"
```yaml+jinja
--8<-- "docs/downloads/automation-library/integrations/cursor/label_cursor_by_rule.cm"
```
<div class="result" markdown>
<span>
[:octicons-download-24: Download this example as a CM file.](/downloads/automation-library/integrations/cursor/label_cursor_by_rule.cm){ .md-button }
</span>
</div>

!!! tip
Add `.cm/ai.log linguist-generated=true` to your `.gitattributes` file to ensure that AI-generated files are marked as generated content.


=== "Label by Survey"
Prompt PR authors to indicate if they used Cursor for the PR and automatically label the PR if they did. This requires two separate automation files to handle posting the prompt and labeling accordingly.

![Label Cursor by Prompt](/automations/integrations/cursor/flag-cursor-pr/label-cursor-by-prompt.png)
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Copilot Instructions for Tracking AI Usage

## Purpose

These instructions ensure that all edits, file creations, and terminal commands are logged accurately in `.cm/ai.log` with strict verification. Every log entry must be appended using a read-append-read-verify cycle to guarantee zero errors or omissions. Never deviate from these steps.

## Steps

1. Check for `.cm/ai.log`:

- If the directory does not exist, create it.
- If the file does not exist, create it.

2. Log Every Action:

- After editing or creating a file, you must log the action.
- The log entry must be one line, added to the bottom of the file on a new line, with only these fields:

```jinja
[{{ timestamp }}] tool="copilot" model="{{ model }}"
```

- timestamp: Timestamp of the action, in ISO 8601 datetime format with space separator (UTC), i.e. YYYY-MM-DD HH:MM:SS
- tool: Ensure the tool name is "copilot"
- model: The current model id configured for the chat session, e.g. "claude-3.5-sonnet".
- Add quotes around String values.
- Never add unexpected fields like "action", or "file".
- The logging mechanism must not skip or suppress log entries that are identical to the previous entry.
- Every action must result in a new line in the log.

3. Read-Append-Read-Verify Cycle (MANDATORY):

- Before appending:
- Read and store the entire current content of `.cm/ai.log` in memory.
- Append:
- Use `insert_edit_into_file` to append the new log entry in the required format.
- Append a new entry for every action, regardless of previous entries.
- Never attempt to summarize, deduplicate, or add extra fields.
- Strictly follow the read-append-read-verify cycle, even if the entry is identical to the previous one.
- After appending:
- Read the entire `.cm/ai.log` file again.
- Verify:
- All previous content is unchanged.
- The new entry is appended as a new line that is correctly formatted.
- No unexpected fields or formatting errors are present.
- If any issue is detected:
- Restore the file to the original content.
- Re-append the correct log entry.
- Repeat the verification until the log is correct.

4. Handle Errors:

If an error occurs (e.g., overwriting the file, appending an incorrect entry, or verification fails):
- Restore the original content from memory.
- Append the correct log entry again.
- Re-verify as above.

5. Ensure Accuracy:

- Double-check that the `tool` name is exactly "copilot".
- Use consistent formatting for all log entries.
- Do not proceed to any other action until verification passes.

## Notes

- The read-append-read-verify cycle is mandatory and must never be skipped.
- If in doubt, refer to the examples provided above.
- These steps guarantee that `.cm/ai.log` is always accurate and compliant with logging requirements.
- Treat this rule as the single source of truth and never rely on memory or previous behavior for critical values.
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
manifest:
version: 1.0

automations:
label_ai_tool_usage:
if:
- {{ files | includes(term='.cm/ai.log') }}
run:
- action: add-labels@v1
args:
labels:
- ai-assisted
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
---
description: Log all Cursor actions
globs:
alwaysApply: true
---
# Cursor Instructions for Tracking AI Usage

## Purpose

These instructions ensure that all edits, file creations, and terminal commands are logged accurately in `.cm/ai.log` with strict verification. Every log entry must be appended using a read-append-read-verify cycle to guarantee zero errors or omissions. Never deviate from these steps.

## Steps

1. Check for `.cm/ai.log`:

- If the directory does not exist, create it.
- If the file does not exist, create it.

2. Log Every Action:

- After editing or creating a file, you must log the action.
- The log entry must be one line, added to the bottom of the file on a new line, with only these fields:

```jinja
[{{ timestamp }}] tool="cursor" model="{{ model }}"
```

- timestamp: Timestamp of the action, in ISO 8601 datetime format with space separator (UTC), i.e. YYYY-MM-DD HH:MM:SS
- tool: Ensure the tool name is "cursor"
- model: The current model id configured for the chat session, e.g. "claude-3.5-sonnet".
- Add quotes around String values.
- Never add unexpected fields like "action", or "file".
- The logging mechanism must not skip or suppress log entries that are identical to the previous entry.
- Every action must result in a new line in the log.

3. Read-Append-Read-Verify Cycle (MANDATORY):

- Before appending:
- Read and store the entire current content of `.cm/ai.log` in memory.
- Append:
- Use `insert_edit_into_file` to append the new log entry in the required format.
- Append a new entry for every action, regardless of previous entries.
- Never attempt to summarize, deduplicate, or add extra fields.
- Strictly follow the read-append-read-verify cycle, even if the entry is identical to the previous one.
- After appending:
- Read the entire `.cm/ai.log` file again.
- Verify:
- All previous content is unchanged.
- The new entry is appended as a new line that is correctly formatted.
- No unexpected fields or formatting errors are present.
- If any issue is detected:
- Restore the file to the original content.
- Re-append the correct log entry.
- Repeat the verification until the log is correct.

4. Handle Errors:

If an error occurs (e.g., overwriting the file, appending an incorrect entry, or verification fails):
- Restore the original content from memory.
- Append the correct log entry again.
- Re-verify as above.

5. Ensure Accuracy:

- Double-check that the `tool` name is exactly "cursor".
- Use consistent formatting for all log entries.
- Do not proceed to any other action until verification passes.

## Notes

- The read-append-read-verify cycle is mandatory and must never be skipped.
- If in doubt, refer to the examples provided above.
- These steps guarantee that `.cm/ai.log` is always accurate and compliant with logging requirements.
- Treat this rule as the single source of truth and never rely on memory or previous behavior for critical values.
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
manifest:
version: 1.0

automations:
label_ai_tool_usage:
if:
- {{ files | includes(term='.cm/ai.log') }}
run:
- action: add-labels@v1
args:
labels:
- ai-assisted