Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,8 @@ This [Canonical Plugin](https://make.wordpress.org/core/2022/09/11/canonical-plu
* **[Image Generation and Editing](docs/features/image-generation.md)** - Create and edit images from post content in the editor, also via the Media Library.
* **[Meta Description Generation](docs/experiments/meta-description.md)** - Generates meta description suggestions and integrates those with various SEO plugins.
* **Multi-Provider Support** - Works with popular AI providers like OpenAI, Google, and Anthropic.
* **[Refine Notes](docs/experiments/refine-notes.md)** - Automatically apply editorial notes to content.
* **[Review Notes](docs/experiments/review-notes.md)** - Reviews post content block-by-block and adds Notes with suggestions for Accessibility, Readability, Grammar, and SEO.
* **[Editorial Updates](docs/experiments/editorial-updates.md)** - Automatically apply editorial notes to content.
* **[Editorial Notes](docs/experiments/editorial-notes.md)** - Reviews post content block-by-block and adds Notes with suggestions for Accessibility, Readability, Grammar, and SEO.
* **[Title Generation](docs/experiments/title-generation.md)** - Generates title suggestions from content.

## Roadmap
Expand Down
159 changes: 10 additions & 149 deletions docs/experiments/review-notes.md → docs/experiments/editorial-notes.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# AI Review Notes
# AI Editorial Notes

## Summary

The AI Review Notes experiment adds a block-by-block AI editorial review to the WordPress post editor. Clicking "Generate Review Notes" in the post sidebar triggers the AI to examine each reviewable block and create WordPress Notes directly on the relevant blocks with concise, actionable suggestions across four categories: **Accessibility**, **Readability**, **Grammar**, and **SEO**.
The AI Editorial Notes experiment adds a block-by-block AI editorial review to the WordPress post editor. Clicking "Generate Editorial Notes" in the post sidebar triggers the AI to examine each reviewable block and create WordPress Notes directly on the relevant blocks with concise, actionable suggestions across four categories: **Accessibility**, **Readability**, **Grammar**, and **SEO**.

## Overview

### For End Users

When enabled, a "Generate Review Notes" button appears in the post status info panel (the sidebar area below the post status). Clicking it triggers a review pass:
When enabled, a "Generate Editorial Notes" button appears in the post status info panel (the sidebar area below the post status). Clicking it triggers a review pass:

1. The button label updates to show review progress (`Reviewing blocks… (2 of 8)`)
2. Each content block is sent individually to the AI for analysis
Expand All @@ -26,48 +26,8 @@ When enabled, a "Generate Review Notes" button appears in the post status info p

### For Developers

The experiment consists of:

1. **Experiment Class** (`WordPress\AI\Experiments\Review_Notes\Review_Notes`): Registers the ability, enqueues the block editor asset, and wires server-side hooks for Note author override and block metadata cleanup
2. **Ability Class** (`WordPress\AI\Abilities\Review_Notes\Review_Notes`): Receives a single block's content and returns structured JSON suggestions
3. **React Plugin** (`src/experiments/review-notes/`): Drives the UI and orchestrates block traversal, Note creation, and thread management via WordPress data stores

## Architecture & Implementation

### Key Hooks & Entry Points

`WordPress\AI\Experiments\Review_Notes\Review_Notes::register()` wires everything once the experiment is enabled:

- `wp_abilities_api_init` → registers the `ai/review-notes` ability
- `enqueue_block_editor_assets` → enqueues the React bundle whenever the block editor loads
- `rest_pre_insert_comment` (filter) → `maybe_set_ai_author()` — overrides the comment author to "WordPress AI" when `meta.ai_note` is `true`, so AI-generated Notes are not attributed to the authenticated user's account

### Assets & Data Flow

1. **PHP Side:**
- `enqueue_assets()` loads `experiments/review-notes` and localizes `window.aiReviewNotesData`:
- `enabled`: Whether the experiment is currently enabled

2. **React Side:**
- `index.tsx` registers the `ai-review-notes` plugin
- `ReviewNotesPlugin.tsx` renders the button inside `PluginPostStatusInfo`
- `useReviewNotes.ts` hook manages all state and orchestration:
- Flattens the block tree to get all descendants
- Filters to reviewable block types with sufficient content (≥ 20 chars), capped at 25 blocks
- Fetches Notes in two parallel requests:
- `GET /wp/v2/comments?type=note&status=hold&post=<id>&per_page=100` — pending Notes used as context to avoid repeating suggestions
- `GET /wp/v2/comments?type=note&status=approve&post=<id>&per_page=100` — resolved Note IDs; blocks with a resolved Note are skipped entirely
- Processes blocks in parallel batches of 4, calling the ability for each
- Creates new Note threads via `POST /wp/v2/comments` (with `meta: { ai_note: true }` to trigger the AI author override) and updates block `metadata.noteId`
- Subsequent runs append replies to existing Note threads

3. **Ability Execution:**
- Receives one block's content at a time (block type, plain text, post context, prior Notes, review types)
- Builds a structured prompt and sends it to the AI with the system instruction and a JSON schema for structured output
- Parses the JSON response, sanitizes each suggestion, and returns `{ suggestions: [...] }`
- Returns `{ suggestions: [] }` when the AI finds no issues
- Deduplicates against `existing_notes`: if a Note already contains a `[TYPE]` marker for a given review type, that type is skipped in the current run

### Block Types Reviewed

```typescript
Expand Down Expand Up @@ -167,7 +127,7 @@ Notes are `WP_Comment` objects with `comment_type = 'note'` and `status = 'hold'
### Endpoint

```text
POST /wp-json/wp-abilities/v1/abilities/ai/review-notes/run
POST /wp-json/wp-abilities/v1/abilities/ai/editorial-notes/run
```

### Authentication
Expand All @@ -179,7 +139,7 @@ See [TESTING_REST_API.md](../TESTING_REST_API.md) for authentication details (ap
#### Review a Paragraph Block

```bash
curl -X POST "https://yoursite.com/wp-json/wp-abilities/v1/abilities/ai/review-notes/run" \
curl -X POST "https://yoursite.com/wp-json/wp-abilities/v1/abilities/ai/editorial-notes/run" \
-u "username:application-password" \
-H "Content-Type: application/json" \
-d '{
Expand Down Expand Up @@ -209,7 +169,7 @@ curl -X POST "https://yoursite.com/wp-json/wp-abilities/v1/abilities/ai/review-n
#### Review an Image Block (Accessibility Only)

```bash
curl -X POST "https://yoursite.com/wp-json/wp-abilities/v1/abilities/ai/review-notes/run" \
curl -X POST "https://yoursite.com/wp-json/wp-abilities/v1/abilities/ai/editorial-notes/run" \
-u "username:application-password" \
-H "Content-Type: application/json" \
-d '{
Expand Down Expand Up @@ -242,7 +202,7 @@ import apiFetch from '@wordpress/api-fetch';

async function reviewBlock( blockType, blockContent, existingNotes = [] ) {
const result = await apiFetch( {
path: '/wp-abilities/v1/abilities/ai/review-notes/run',
path: '/wp-abilities/v1/abilities/ai/editorial-notes/run',
method: 'POST',
data: {
input: {
Expand All @@ -267,69 +227,27 @@ async function reviewBlock( blockType, blockContent, existingNotes = [] ) {
| `post_not_found` | The post ID passed does not exist |
| `insufficient_capabilities` | User lacks `edit_posts` (or `edit_post` for the specific post) |

## Extending the Experiment

### Customizing the System Instruction

Edit `includes/Abilities/Review_Notes/system-instruction.php` to adjust:

- Which review types apply to which block types
- How strictly prior suggestions are de-duplicated

### Filtering Preferred Models

```php
add_filter( 'wpai_preferred_text_models', function( $models ) {
return array(
array( 'openai', 'gpt-4o' ),
array( 'openai', 'gpt-4o-mini' ),
);
} );
```

### Disabling the Experiment Programmatically

```php
add_filter( 'wpai_feature_review-notes_enabled', '__return_false' );
```

### Adding Custom Review Types

The `review_types` input field accepts any string values. Pass additional type names from the JS side and update the system instruction to provide guidance for those types:

```javascript
// In your custom JS
await runAbility( 'ai/review-notes', {
block_type: 'core/paragraph',
block_content: '...',
review_types: [ 'accessibility', 'readability', 'grammar', 'seo', 'tone' ],
existing_notes: [],
} );
```

Then add guidance for the `tone` type to `system-instruction.php`.

## Testing

### Manual Testing Steps

1. **Enable the experiment:**
- Go to `Settings → AI`
- Enable the global toggle
- Enable **AI Review Notes**
- Enable **AI Editorial Notes**
- Ensure valid AI credentials are configured

2. **Run a review:**
- Create or open a post with a mix of block types (headings, paragraphs, an image without alt text, a list)
- Open the post sidebar (click the **Settings** button in the toolbar)
- Click **Generate Review Notes** in the post info panel
- Click **Generate Editorial Notes** in the post info panel
- Watch the progress counter advance (`Reviewing blocks… 2 of 8`)
- After completion, open the **Notes** panel (via the block toolbar or the comments icon)
- Verify Notes appear on relevant blocks, formatted as `[REVIEW_TYPE] Suggestion text.`
- Verify Notes show "WordPress AI" as the author rather than your account name

3. **Re-run accumulation:**
- Click **Generate Review Notes** a second time
- Click **Generate Editorial Notes** a second time
- Verify existing Note threads gain replies rather than new top-level Notes
- Verify prior suggestions are not repeated

Expand All @@ -348,53 +266,10 @@ Then add guidance for the `tone` type to `system-instruction.php`.
- All blocks already have Notes → second run skips repeats
- Disable experiment → button disappears from sidebar

### Automated Testing

**PHPUnit integration tests:**

```bash
npm run test:php
```

Test files:
- `tests/Integration/Includes/Abilities/Review_NotesTest.php` — Ability class tests
- `tests/Integration/Includes/Experiments/Review_Notes/Review_NotesTest.php` — Experiment class tests

Covers:
- Input/output schema structure
- `suggestions_schema()` OpenAI wrapper structure (name, strict, schema keys; inner type must be object)
- Empty content validation
- Mock-based suggestion return and structure
- Content sanitization
- Permission callbacks: no post ID path (editor, subscriber, logged-out), and post-specific path (valid post, missing post, insufficient edit_post, non-REST post type)
- `execute_callback` with missing post ID → WP_Error
- `get_existing_review_types_from_notes()`: type extraction, case normalisation, multiple types per Note, Notes without brackets
- Experiment hook registration (rest_pre_insert_comment)
- `ai_note` comment meta registered with `show_in_rest`
- `maybe_set_ai_author()`: overrides author when `ai_note` is true, passes through otherwise, handles WP_Error

**Playwright E2E tests:**

```bash
npm run wp-env:test start # Start the test environment
npm run test:e2e -- --grep "AI Review Notes"
```

Test file: `tests/e2e/review-notes.spec.ts`

Covers:
- Button visibility in editor sidebar
- Button busy/disabled state during review
- Suggestion count feedback after completion
- Empty result handling
- Button hidden when experiment is disabled
- No-op when post has no reviewable blocks

## Notes & Considerations

### Requirements

- WordPress 6.9+ (Notes feature required for block-level comment association)
- Valid AI credentials configured in `Settings → Connectors`
- User must have `edit_posts` capability (or `edit_post` for the specific post when a post ID is provided)
- The block editor must be active (classic editor is not supported)
Expand All @@ -404,7 +279,6 @@ Covers:
- Each block generates one API call; blocks are processed in parallel batches of 4
- The review is capped at 25 blocks per run to control cost
- Blocks with fewer than 20 characters of text are skipped
- AI temperature is set to 0.7

### Note Storage

Expand All @@ -420,16 +294,3 @@ Covers:
- Block metadata (`noteId`) is only persisted after the post is saved
- The 25-block cap means very long posts will have only the first 25 reviewable blocks analyzed per run
- Resolved blocks (approved Notes) are skipped in full; they will not receive new suggestions until the Note is un-resolved or deleted

## Related Files

- **Experiment:** `includes/Experiments/Review_Notes/Review_Notes.php`
- **Ability:** `includes/Abilities/Review_Notes/Review_Notes.php`
- **System Instruction:** `includes/Abilities/Review_Notes/system-instruction.php`
- **React Entry:** `src/experiments/review-notes/index.tsx`
- **React Plugin Component:** `src/experiments/review-notes/components/ReviewNotesPlugin.tsx`
- **React Hook:** `src/experiments/review-notes/hooks/useReviewNotes.ts`
- **PHPUnit Tests (Ability):** `tests/Integration/Includes/Abilities/Review_NotesTest.php`
- **PHPUnit Tests (Experiment):** `tests/Integration/Includes/Experiments/Review_Notes/Review_NotesTest.php`
- **E2E Tests:** `tests/e2e/review-notes.spec.ts`
- **Mock Fixtures:** `tests/e2e-request-mocking/responses/OpenAI/review-notes-suggestions.json`
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# Refine from Notes
# Editorial Updates

## Summary

The Refine from Notes experiment enables users to automatically apply pending editorial feedback/notes to their WordPress post content using AI. Clicking "Refine from Notes" in the post sidebar triggers the AI to contextually examine all blocks with pending Notes and modify the text according to the provided suggestions.
The Editorial Updates experiment enables users to automatically apply pending editorial feedback/notes to their WordPress post content using AI. Clicking "Editorial Updates" in the post sidebar triggers the AI to contextually examine all blocks with pending Notes and modify the text according to the provided suggestions.

## Overview

### For End Users

When enabled, a "Refine from Notes" button appears in the post status info panel (the sidebar area below the post status) assuming there is at least one Note pending on any block in the post. Clicking it triggers the refinement process:
When enabled, a "Editorial Updates" button appears in the post status info panel (the sidebar area below the post status) assuming there is at least one Note pending on any block in the post. Clicking it triggers the refinement process:

1. The button label updates to show progression across blocks (`Refining block (2 of 4)…`)
2. Each block that has a pending Note attached is sent to the AI alongside the Note's content.
Expand All @@ -24,46 +24,8 @@ When enabled, a "Refine from Notes" button appears in the post status info panel

### For Developers

The experiment consists of:

1. **Experiment Class** (`WordPress\AI\Experiments\Refine_Notes\Refine_Notes`): Registers the ability and enqueues the block editor asset.
2. **Ability Class** (`WordPress\AI\Abilities\Refine_Notes\Refine_Notes`): Receives a single block's content, surrounding context, and associated notes, parsing the resulting AI output back into plain string replacements.
3. **React Plugin** (`src/experiments/refine-notes/`): Drives the sidebar UI, discovers threaded Notes via WordPress data stores, processes block attributes iteratively, and manages Editor saving workflows.

## Architecture & Implementation

### Key Hooks & Entry Points

`WordPress\AI\Experiments\Refine_Notes\Refine_Notes::register()` wires everything once the experiment is enabled:

- `wp_abilities_api_init` → registers the `ai/refine-notes` ability
- `enqueue_block_editor_assets` → enqueues the React bundle whenever the block editor loads

### Assets & Data Flow

1. **PHP Side:**

- `enqueue_assets()` loads `experiments/refine-notes` and localizes `window.RefineNotesData`:
- `enabled`: Whether the experiment is currently enabled

2. **React Side:**

- `index.tsx` registers the `ai-refine-notes` plugin.
- `RefineNotesPlugin.tsx` conditionally renders the button inside `PluginPostStatusInfo`.
- `useRefineNotes.ts` hook manages all state and orchestration:
- Flattens the active block tree.
- Fetches active pending Notes via `GET /wp/v2/comments?type=note&status=hold&post=<id>&per_page=100`.
- Maps notes and child threaded-replies directly to their parent `blockClientId`.
- Skips any blocks that do not have active pending notes attached.
- Processes qualifying blocks in parallel batches of 4.
- Dispatches an `updateBlockAttributes` directly to the `core/block-editor` store with the returned refactored content.
- Triggers `wp.data.dispatch( 'core/editor' ).savePost()` to persist changes and create a revision.

3. **Ability Execution:**
- Receives target block type, current content, note texts, and optionally surrounding text context.
- Builds a standard prompt matching against the system instruction.
- Extracts plain string response from the AI and returns the direct replacement to the block content.

### Block Types Supported

Can safely run against any block. Output targets formatting of standard block markup (e.g. inner wrappers).
Expand Down Expand Up @@ -119,7 +81,7 @@ In both cases, users without the required capability receive an `insufficient_ca
### Endpoint

```text
POST /wp-json/wp-abilities/v1/abilities/ai/refine-notes/run
POST /wp-json/wp-abilities/v1/abilities/ai/editorial-updates/run
```

### Authentication
Expand All @@ -131,7 +93,7 @@ See [TESTING_REST_API.md](../TESTING_REST_API.md) for authentication details (ap
#### Refine a Paragraph Block

```bash
curl -X POST "https://yoursite.com/wp-json/wp-abilities/v1/abilities/ai/refine-notes/run" \
curl -X POST "https://yoursite.com/wp-json/wp-abilities/v1/abilities/ai/editorial-updates/run" \
-u "username:application-password" \
-H "Content-Type: application/json" \
-d '{
Expand All @@ -158,16 +120,3 @@ curl -X POST "https://yoursite.com/wp-json/wp-abilities/v1/abilities/ai/refine-n
| `notes_required` | No valid notes were supplied in the array |
| `post_not_found` | The post ID passed does not exist |
| `insufficient_capabilities` | User lacks `edit_posts` (or `edit_post` for the specific post) |

## Related Files

- **Experiment:** `includes/Experiments/Refine_Notes/Refine_Notes.php`
- **Ability:** `includes/Abilities/Refine_Notes/Refine_Notes.php`
- **System Instruction:** `includes/Abilities/Refine_Notes/system-instruction.php`
- **React Entry:** `src/experiments/refine-notes/index.tsx`
- **React Plugin Component:** `src/experiments/refine-notes/components/RefineNotesPlugin.tsx`
- **React Hook:** `src/experiments/refine-notes/hooks/useRefineNotes.ts`
- **PHPUnit Tests (Ability):** `tests/Integration/Includes/Abilities/Refine_NotesTest.php`
- **PHPUnit Tests (Experiment):** `tests/Integration/Includes/Experiments/Refine_Notes/Refine_NotesTest.php`
- **E2E Tests:** `tests/e2e/specs/experiments/refine-notes.spec.js`
- **Mock Fixtures:** `tests/e2e-request-mocking/responses/OpenAI/refine-notes-completions.json` and `tests/e2e-request-mocking/responses/OpenAI/refine-notes-responses.json`
Loading
Loading