Skip to content

setup doc

Jump to bottom
hbt edited this page Jun 22, 2025 · 1 revision

Browser MCP Setup Guide for Reddit Research

Overview

Browser MCP enables Claude to automate browser tasks, including accessing Reddit URLs that are blocked by the WebFetch tool. This guide provides step-by-step instructions for setting up Browser MCP to enhance your Claude Code research.

Installation Steps

1. Install Browser MCP Chrome Extension

Option A: Chrome Web Store

# Navigate to Chrome Web Store
open "https://chromewebstore.google.com/detail/browser-mcp-automate-your/bjfgambnhccakkhmkepdoekmckoijdlc"

Extension Details:

  • Name: "Browser MCP - Automate your browser using VS Code, Cursor, Claude, and more"
  • Rating: 3.2/5 stars (63 reviews)
  • Publisher: Browser MCP team

2. Set Up MCP Server

Install MCP Server:

# Using npm (if Node.js available)
npm install -g @browsermcp/server

# Or using the official installer
curl -sSL https://browsermcp.io/install.sh | bash

Configure MCP Server:

# Start the MCP server
browser-mcp-server --port 8080

# Verify server is running
curl http://localhost:8080/health

3. Connect to Claude Desktop

Add MCP Configuration to Claude:

{
  "mcp_servers": {
    "browser": {
      "command": "browser-mcp-server",
      "args": ["--port", "8080"]
    }
  }
}

Reddit Research Automation

Target URLs for Data Collection

Blocked Reddit URLs to Access:

https://old.reddit.com/r/ClaudeAI/comments/1ky80kz/im_blown_away_by_claude_code_built_a_full/
https://old.reddit.com/r/ChatGPTCoding/comments/1kwiuzd/experienced_dev_looking_into_claude_code/
https://old.reddit.com/r/ClaudeAI/comments/1kwecnp/ultimate_claude_code_setup/
https://old.reddit.com/r/ClaudeCode/
https://old.reddit.com/r/ClaudeAI/comments/1lh9cb2/claude_code_changed_my_life/
https://old.reddit.com/r/ChatGPTCoding/comments/1ldmibg/is_claude_code_the_best_tool_in_the_market/
https://old.reddit.com/r/ClaudeAI/comments/1kl82t6/is_everyone_sleeping_on_claude_code/
https://old.reddit.com/r/ClaudeAI/comments/1j6v2dm/thoughts_on_claude_code_so_far/

Blocked Claude.ai URLs to Access:

https://claude.ai/chat/f792c06f-e555-4a39-a65b-92644c1ec74c
https://claude.ai/public/artifacts/5c1f645e-63e9-4876-b1b3-23a098558ee9

Access Issues Identified:

  • Reddit: old.reddit.com domain blocked by WebFetch
  • Claude.ai: Returns 403 errors for both chat and public artifact URLs
  • Root Cause: Domain-level access restrictions in WebFetch tool
  • Solution: Browser MCP can bypass these restrictions through actual browser automation

Automation Scripts

Reddit Data Extraction Script:

// Browser MCP automation for Reddit
async function extractRedditFeedback(url) {
  const browser = await getBrowserInstance();
  const page = await browser.newPage();
  
  await page.goto(url);
  
  // Extract post content
  const postData = await page.evaluate(() => {
    const title = document.querySelector('.title').textContent;
    const content = document.querySelector('.usertext-body').textContent;
    const upvotes = document.querySelector('.score').textContent;
    
    // Extract comments
    const comments = Array.from(document.querySelectorAll('.comment')).map(comment => ({
      author: comment.querySelector('.author').textContent,
      text: comment.querySelector('.usertext-body').textContent,
      score: comment.querySelector('.score').textContent
    }));
    
    return { title, content, upvotes, comments };
  });
  
  return postData;
}

Claude.ai Data Extraction Script:

// Browser MCP automation for Claude.ai
async function extractClaudeArtifact(url) {
  const browser = await getBrowserInstance();
  const page = await browser.newPage();
  
  // Handle potential authentication
  await page.goto(url);
  
  // Wait for content to load
  await page.waitForSelector('[data-testid="artifact"]', { timeout: 10000 });
  
  // Extract artifact content
  const artifactData = await page.evaluate(() => {
    const title = document.querySelector('h1').textContent;
    const content = document.querySelector('[data-testid="artifact"]').textContent;
    const metadata = {
      created: document.querySelector('[data-testid="timestamp"]')?.textContent,
      author: document.querySelector('[data-testid="author"]')?.textContent
    };
    
    return { title, content, metadata };
  });
  
  return artifactData;
}

async function extractClaudeChat(url) {
  const browser = await getBrowserInstance();
  const page = await browser.newPage();
  
  await page.goto(url);
  
  // Extract conversation content
  const chatData = await page.evaluate(() => {
    const messages = Array.from(document.querySelectorAll('[data-testid="message"]')).map(msg => ({
      role: msg.getAttribute('data-role'),
      content: msg.querySelector('.message-content').textContent,
      timestamp: msg.querySelector('.timestamp')?.textContent
    }));
    
    return { messages };
  });
  
  return chatData;
}

Data Collection Strategy

Phase 1: Manual Verification

  1. Test Browser MCP Setup

    • Verify Chrome extension is installed and active
    • Confirm MCP server connection to Claude
    • Test basic navigation to Reddit

  2. Validate Data Extraction

    • Access one blocked URL manually through Browser MCP
    • Verify data structure and quality
    • Refine extraction parameters

Phase 2: Automated Collection

  1. Batch Process Reddit URLs

    • Loop through all provided Reddit URLs
    • Extract structured data from each post
    • Save results in organized format

  2. Claude.ai Content Extraction

    • Access blocked Claude.ai chat conversation
    • Extract public artifact content for comparison
    • Parse research methodology and findings

  3. Enhanced Search

    • Search Reddit for additional "claude code" discussions
    • Monitor r/ClaudeCode for new posts
    • Track trending discussions

Phase 3: Analysis Integration

  1. Data Processing

    • Parse extracted Reddit data
    • Identify sentiment patterns
    • Categorize feedback themes

  2. Cross-Platform Comparison

    • Compare my research methodology with Claude.ai artifact
    • Identify gaps or additional insights from Claude.ai analysis
    • Merge findings from both approaches

  3. Research Enhancement

    • Update main research document with Reddit insights
    • Compare Reddit feedback with Hacker News data
    • Generate comprehensive user feedback matrix
    • Create unified research methodology comparison

Expected Data Output

Structured Reddit Data Format

{
  "url": "https://old.reddit.com/r/ClaudeAI/comments/...",
  "title": "Post title",
  "author": "reddit_username",
  "content": "Post content text",
  "upvotes": 150,
  "comment_count": 45,
  "timestamp": "2024-12-01T12:00:00Z",
  "comments": [
    {
      "author": "commenter_username",
      "text": "Comment text",
      "score": 25,
      "replies": []
    }
  ],
  "extracted_themes": {
    "positive_aspects": ["performance", "code_quality"],
    "negative_aspects": ["cost", "integration"],
    "mentioned_alternatives": ["cursor", "copilot"]
  }
}

Troubleshooting

Common Issues

  1. Extension Not Loading

    • Check Chrome extension permissions
    • Verify MCP server is running
    • Restart browser if needed

  2. Connection Errors

    • Confirm MCP server port (default 8080)
    • Check firewall settings
    • Validate Claude Desktop configuration

  3. Reddit Access Issues

    • Ensure you're logged into Reddit (if required)
    • Check Reddit rate limiting
    • Use old.reddit.com format for better compatibility

  4. Claude.ai Access Issues

    • Authentication Required: Claude.ai may require login session
    • Rate Limiting: Respect Claude.ai's usage policies
    • Dynamic Content: Artifacts may load asynchronously, use proper wait conditions
    • DOM Changes: Claude.ai interface may change, update selectors accordingly

Alternative Approaches

If Browser MCP setup proves challenging:

  1. Manual Data Collection: Access URLs manually and copy data
  2. Reddit API: Use official Reddit API (requires authentication)
  3. Third-party Tools: Consider other Reddit scraping tools

Next Steps

  1. Install and test Browser MCP setup
  2. Validate data extraction from one Reddit URL
  3. Test Claude.ai artifact and chat access
  4. Process all blocked Reddit URLs
  5. Extract Claude.ai research content for comparison
  6. Update main research document with findings
  7. Generate comprehensive Claude Code feedback analysis
  8. Create methodology comparison document

Setup Guide Created: 2025-06-22
Status: Ready for implementation

Clone this wiki locally