Refine script generation prompts for better timing and mouse movement

[DeDuckProject]

Summary

Updated the script generation prompt instructions to provide more nuanced guidance on timing and mouse movement behavior, replacing a blanket pause recommendation with context-aware timing strategies.

Key Changes

• Removed the generic instruction to add 1500ms pauses on key visual states
• Added a new "Timing" section with refined pause guidance:
  • Use short 300ms pauses between most actions
  • Reserve longer 800ms pauses only after interactions with visible results (animations, state changes, panel openings)
  • Discourage stacking multiple pauses in sequence
• Added a new "Mouse movement" section with natural cursor movement guidance:
  • Recommend moving the mouse to a nearby point before clicking/hovering to avoid teleporting
  • Use page.mouse.move() with a single intermediate waypoint
  • Ensure coordinates are plausible for the viewport dimensions

Implementation Details

These changes improve the quality of generated scripts by:

1. Making pause timing more intentional and context-aware rather than uniform
2. Encouraging more natural user interaction patterns through intermediate mouse movements
3. Providing clearer constraints (single waypoint, plausible coordinates) to keep generated scripts realistic

https://claude.ai/code/session_01WpqwtdViucvCCdgxj6RCGG

[DeDuckProject]

/glimpse

[github-actions]

🧐 UI Demo Preview

Changes detected in: packages/core/src/generator/prompts.ts

What changed: The demo script generator has been updated with improved timing and mouse movement instructions. Scripts now use shorter, more precise pauses and simulate natural cursor movement between interactions for more realistic recordings.

📱 Can't see the preview? Open it directly

Demo script (auto-generated)

import type { Page } from '@playwright/test';

export async function demo(page: Page): Promise<void> {
  await page.goto('http://localhost:3000');
  await page.waitForLoadState('networkidle');
  await page.waitForTimeout(300);

  // Look for a script generator form or input area
  const diffInput = page.getByRole('textbox').first();
  const hasDiffInput = await diffInput.isVisible().catch(() => false);

  if (hasDiffInput) {
    await page.mouse.move(400, 300);
    await diffInput.click();
    await page.waitForTimeout(300);

    await diffInput.fill(`diff --git a/src/button.tsx b/src/button.tsx
- <button className="btn">Click me</button>
+ <button className="btn btn-primary">Click me</button>`);
    await page.waitForTimeout(300);
  }

  // Look for a generate button
  const generateBtn = page.getByRole('button', { name: /generate/i }).first();
  const hasGenerateBtn = await generateBtn.isVisible().catch(() => false);

  if (hasGenerateBtn) {
    await page.mouse.move(500, 400);
    await generateBtn.click();
    await page.waitForTimeout(800);
  }

  // Look for output/result area showing the generated script
  const outputArea = page.locator('pre, code, [data-testid*="output"], [data-testid*="result"], textarea').last();
  const hasOutput = await outputArea.isVisible().catch(() => false);

  if (hasOutput) {
    await page.mouse.move(640, 360);
    await outputArea.scrollIntoViewIfNeeded();
    await page.waitForTimeout(300);

    // Highlight the timing section in the output
    await page.mouse.move(640, 400);
    await page.waitForTimeout(300);
  }

  // Scroll through the page to show timing rules in the UI if displayed
  await page.mouse.move(640, 360);
  await page.evaluate(() => window.scrollTo({ top: 300, behavior: 'smooth' }));
  await page.waitForTimeout(800);

  await page.mouse.move(640, 500);
  await page.evaluate(() => window.scrollTo({ top: 600, behavior: 'smooth' }));
  await page.waitForTimeout(300);

  // Look for any tabs or sections related to timing/mouse movement
  const timingTab = page.getByRole('tab', { name: /timing/i }).first();
  const hasTimingTab = await timingTab.isVisible().catch(() => false);

  if (hasTimingTab) {
    await page.mouse.move(400, 200);
    await timingTab.click();
    await page.waitForTimeout(800);
  }

  // Look for any settings or configuration panel
  const settingsBtn = page.getByRole('button', { name: /settings|config|options/i }).first();
  const hasSettings = await settingsBtn.isVisible().catch(() => false);

  if (hasSettings) {
    await page.mouse.move(900, 100);
    await settingsBtn.click();
    await page.waitForTimeout(800);
  }

  // Scroll back to top to show the full page state
  await page.mouse.move(640, 200);
  await page.evaluate(() => window.scrollTo({ top: 0, behavior: 'smooth' }));
  await page.waitForTimeout(300);
}

---

Generated by git-glimpse