Skip to content

Keywords

Jump to bottom
briansemify edited this page Nov 4, 2025 · 9 revisions

Keyword Research & Management

Keyword Budget Estimation

Endpoint: POST /api/v1/keywords/budget/estimate

Purpose: Generate comprehensive budget estimates for organic keywords based on SEMrush data analysis. Budgets are calculated using incremental traffic opportunity - the cost of improving from a keyword's current search position to position #1.

Authentication: Required - Bearer Token (JWT)

Base URL: https://uat.services.semify.com/api/v1/keywords/budget/estimate or https://services.semify.com/api/v1/keywords/budget/estimate

Request

Required Fields

Field Type Description
url string Client business domain URL to analyze (e.g., "https://www.example.com")

Optional Fields

Field Type Default Description
categories array [] Business categories for filtering (array of strings)
seeds array [] Seed keywords for analysis (array of strings)
market string "us" Country/locale code (e.g., "us", "uk", "ca")
language string "en" Language code (e.g., "en", "es", "fr")
max_keywords integer 50 Maximum number of keywords to return (1-50)

Request Example

{
  "url": "https://www.example.com",
  "categories": ["digital marketing", "seo services"],
  "seeds": ["best seo", "digital marketing agency"],
  "market": "us",
  "language": "en",
  "max_keywords": 25
}

Response Structure

Success Response (200 OK)

{
  "data": {
    "url": "https://www.example.com",
    "market": "us",
    "language": "en",
    "total_keywords": 5,
    "keywords": [
      {
        "keyword": "duct cleaning near me",
        "monthly_searches": "60500",
        "cpc": "9.93",
        "position": "24",
        "previous_position": "24",
        "position_difference": "0",
        "url": "https://www.example.com/residential/duct-cleaning/",
        "traffic": "28.37",
        "traffic_cost": "47.17",
        "competition": "0.680",
        "number_of_results": "41500000",
        "monthly_trends": ["0.24", "0.20", "0.16", "0.11", "0.67", "0.67", "0.82", "0.55", "0.67", "0.82", "1.00", "1.00"],
        "current_trend": 1.0,
        "trend_summary": {
          "direction": "accelerating",
          "strength": "Very High",
          "category": "exploding",
          "description": "Explosive growth - high opportunity",
          "average_trend": 0.5758333333333333,
          "trend_strength": 0.89,
          "peak_month": 1.0,
          "lowest_month": 0.11
        },
        "budget_estimate": {
          "monthly_budget": "400.00",
          "quarterly_budget": "1200.00",
          "yearly_budget": "4800.00",
          "cost_per_acquisition": "0.03",
          "value_score": 5,
          "value_description": "Premium Value",
          "recommendation": "Premium opportunity - high budget with strong upward trend",
          "incremental_traffic_metrics": {
            "current_traffic": "338.80",
            "position_1_traffic_opportunity": "13818.20",
            "incremental_traffic_opportunity": "13479.40",
            "incremental_traffic_cost": "133850.44",
            "current_position_ctr": "0.56%",
            "position_1_ctr": "22.84%"
          }
        }
      }
    ],
    "estimated_budget": {
      "total_keywords": 5,
      "total_monthly_searches": "60830",
      "average_cpc": "9.29",
      "total_traffic_cost": "60.73",
      "estimated_monthly_budget": "21.26",
      "estimated_quarterly_budget": "63.77",
      "estimated_yearly_budget": "255.07",
      "average_competition": "0.484"
    },
    "debug_info": {}
  },
  "messages": [
    "Budget estimate generated successfully."
  ],
  "pagination": {
    "totalPages": 1,
    "maxRows": 0,
    "offset": 0,
    "page": 1,
    "totalRecords": 5
  },
  "error": false
}

Budget Calculation Methodology

Overview

The budget calculation determines the monthly budget needed to optimize a keyword from its current search position to position #1. The calculation is based on incremental traffic opportunity and does NOT use trend multipliers.

Key Principles

  1. Incremental Traffic Opportunity: The budget represents the cost of gaining additional traffic by moving from current position to position #1
  2. Position-Based CTR: Uses industry-standard click-through rates (CTR) based on search position
  3. No Trend Multipliers: Trend data is used for recommendations only, not budget calculations
  4. Constraint-Based: All budgets are bounded by minimum ($25) and maximum ($400) limits

Calculation Formula

Step-by-Step Calculation

Step 1: Apply Input Constraints

constrained_monthly_searches = max(25, monthly_searches)
constrained_cpc = max(2.75, min(60.00, cpc))

Constraints:

  • Minimum monthly searches: 25
  • Minimum CPC: $2.75
  • Maximum CPC: $60.00

Step 2: Get Click-Through Rate (CTR) Values

CTR values are retrieved from position-based lookup table:

current_position_ctr = CTR[current_position]
position_1_ctr = CTR[1] = 0.2284  // Always 22.84%

Key CTR Values:

Position CTR (%) Decimal Value
1 22.84% 0.2284
2 13.71% 0.1371
3 9.55% 0.0955
4 6.71% 0.0671
5 5.35% 0.0535
10 3.19% 0.0319
20 2.70% 0.0270
30 0.56% 0.0056
50 0.0224% 0.000224
100 0.0001% 0.0000001
999 (outside top 100) 0% 0.0

Step 3: Calculate Current Traffic

Current traffic = clicks currently received at the keyword's position:

current_traffic = constrained_monthly_searches × current_position_ctr

Example:

  • Monthly searches: 1,200
  • Position: 5 (CTR: 5.35%)
  • Current traffic = 1,200 × 0.0535 = 64.2 clicks/month

Step 4: Calculate Position #1 Traffic Opportunity

Traffic opportunity if ranking at position #1:

position_1_traffic_opportunity = constrained_monthly_searches × position_1_ctr

Example:

  • Monthly searches: 1,200
  • Position 1 CTR: 22.84%
  • Position 1 opportunity = 1,200 × 0.2284 = 274.08 clicks/month

Step 5: Calculate Incremental Traffic Opportunity

The additional clicks gained by moving from current position to position #1:

incremental_traffic_opportunity = position_1_traffic_opportunity - current_traffic

Example:

  • Position 1 opportunity: 274.08 clicks/month
  • Current traffic: 64.2 clicks/month
  • Incremental opportunity = 274.08 - 64.2 = 209.88 clicks/month

Note: If result is negative, it's set to 0 (shouldn't happen since position 1 has highest CTR).

Step 6: Calculate Incremental Traffic Cost (Base Budget)

This is the core budget calculation - the cost of the incremental traffic:

incremental_traffic_cost = incremental_traffic_opportunity × constrained_cpc

Example:

  • Incremental opportunity: 209.88 clicks/month
  • CPC: $3.45
  • Incremental traffic cost = 209.88 × $3.45 = $724.09/month

Step 7: Apply Budget Constraints

Apply minimum and maximum budget limits:

final_monthly_budget = max(25.00, min(400.00, incremental_traffic_cost))

Constraints:

  • Minimum budget: $25.00
  • Maximum budget: $400.00

Example:

  • Base budget: $724.09
  • After constraints: min($400, max($25, $724.09)) = $400.00/month

Step 8: Calculate Quarterly and Yearly Budgets

Simple multipliers:

quarterly_budget = monthly_budget × 3
yearly_budget = monthly_budget × 12

Example:

  • Monthly: $400.00
  • Quarterly: $400.00 × 3 = $1,200.00
  • Yearly: $400.00 × 12 = $4,800.00

Step 9: Calculate Cost Per Acquisition (CPA)

Cost per incremental click/acquisition:

if (incremental_traffic_opportunity > 0) {
    cost_per_acquisition = final_monthly_budget / incremental_traffic_opportunity
} else {
    cost_per_acquisition = final_monthly_budget / monthly_searches
}

Example:

  • Final monthly budget: $400.00
  • Incremental opportunity: 209.88 clicks
  • CPA = $400.00 / 209.88 = $1.91 per click

Complete Calculation Example

Input Data:

  • Monthly Searches: 1,200
  • CPC: $3.45
  • Current Position: 5

Calculation:

  1. Apply Constraints:

    • Constrained searches: 1,200 ✓
    • Constrained CPC: $3.45 ✓

  2. Get CTR Values:

    • Current position (5) CTR: 5.35% (0.0535)
    • Position 1 CTR: 22.84% (0.2284)

  3. Calculate Traffic:

    • Current traffic = 1,200 × 0.0535 = 64.2 clicks/month
    • Position 1 opportunity = 1,200 × 0.2284 = 274.08 clicks/month

  4. Calculate Incremental:

    • Incremental opportunity = 274.08 - 64.2 = 209.88 clicks/month

  5. Calculate Base Budget:

    • Base budget = 209.88 × $3.45 = $724.09/month

  6. Apply Constraints:

    • Final budget = min($400, max($25, $724.09)) = $400.00/month

  7. Calculate Extended Budgets:

    • Quarterly = $400.00 × 3 = $1,200.00
    • Yearly = $400.00 × 12 = $4,800.00

  8. Calculate CPA:

    • CPA = $400.00 / 209.88 = $1.91 per click

Final Result:

{
  "monthly_budget": "400.00",
  "quarterly_budget": "1,200.00",
  "yearly_budget": "4,800.00",
  "cost_per_acquisition": "1.91",
  "incremental_traffic_metrics": {
    "current_traffic": "64.20",
    "position_1_traffic_opportunity": "274.08",
    "incremental_traffic_opportunity": "209.88",
    "incremental_traffic_cost": "724.09",
    "current_position_ctr": "5.35%",
    "position_1_ctr": "22.84%"
  }
}

Response Fields

Top-Level Response

Field Type Description
data object Response data object
messages array Array of status messages
pagination object Pagination information
error boolean Error flag (false on success)

Data Object

Field Type Description
url string Requested URL
market string Market/locale code used
language string Language code used
total_keywords integer Total number of keywords returned
keywords array Array of keyword objects
estimated_budget object Summary budget estimates
debug_info object Debug information (empty if no errors)

Keyword Object

Field Type Description
keyword string The keyword phrase
monthly_searches string Monthly search volume
cpc string Cost per click
position string Current search position (1-100, or "999" for outside top 100)
previous_position string Previous search position
position_difference string Change in position
url string Landing page URL
traffic string Current traffic estimate
traffic_cost string Traffic cost estimate
competition string Competition level (0.0-1.0)
number_of_results string Total search results
monthly_trends array Array of 12 monthly trend values
current_trend number Most recent trend value
trend_summary object Trend analysis summary
budget_estimate object Budget calculation results

Budget Estimate Object

Field Type Description
monthly_budget string Monthly budget (formatted to 2 decimals)
quarterly_budget string Quarterly budget (monthly × 3)
yearly_budget string Yearly budget (monthly × 12)
cost_per_acquisition string Cost per incremental click
value_score integer Value score (1-5 scale)
value_description string Value description (Low Value, Budget-Friendly, Moderate Value, High Value, Premium Value)
recommendation string Budget recommendation text
incremental_traffic_metrics object Detailed incremental traffic calculations

Incremental Traffic Metrics Object

Field Type Description
current_traffic string Current clicks/month at current position
position_1_traffic_opportunity string Potential clicks/month at position #1
incremental_traffic_opportunity string Additional clicks available (position 1 - current)
incremental_traffic_cost string Base budget before constraints (incremental opportunity × CPC)
current_position_ctr string CTR at current position (as percentage)
position_1_ctr string CTR at position #1 (always 22.84%)

Trend Summary Object

Field Type Description
direction string Trend direction (accelerating, decelerating, stable)
strength string Trend strength (Very High, High, Moderate, Low, Very Low)
category string Trend category (exploding, rising, growing, stable, declining, falling, crashing)
description string Human-readable trend description
average_trend number Average of all monthly trend values
trend_strength number Range of variation in trend values
peak_month number Highest trend value
lowest_month number Lowest trend value

Estimated Budget Summary Object

Field Type Description
total_keywords integer Total number of keywords
total_monthly_searches string Sum of all monthly search volumes
average_cpc string Average CPC across all keywords
total_traffic_cost string Total traffic cost
estimated_monthly_budget string Estimated monthly budget (traffic_cost × 0.35)
estimated_quarterly_budget string Estimated quarterly budget (monthly × 3)
estimated_yearly_budget string Estimated yearly budget (monthly × 12)
average_competition string Average competition level

Value Scoring

Keywords are assigned a value score (1-5) based on budget thresholds:

Score Description Budget Range
1 Low Value ≤ $25.00
2 Budget-Friendly $25.01 - $50.00
3 Moderate Value $50.01 - $100.00
4 High Value $100.01 - $199.99
5 Premium Value ≥ $200.00

Budget Recommendations

Recommendations are generated based on budget tiers and trend analysis:

Budget Tiers

Tier Monthly Budget Range
High Budget ≥ $300.00
Medium-High Budget $200.00 - $299.99
Medium Budget $100.00 - $199.99
Low-Medium Budget $50.00 - $99.99
Low Budget < $50.00

Trend Thresholds

Category Trend Value Multiplier (Historical - No Longer Used)
Strong Upward ≥ 0.6 1.2 (historical)
Moderate Growth ≥ 0.4 1.1 (historical)
Stable -0.2 to 0.4 1.0
Slight Decline ≤ -0.2 0.9 (historical)
Strong Decline ≤ -0.4 0.8 (historical)

Note: Trend multipliers are no longer applied to budget calculations. Trend data is used only for recommendations.

Edge Cases

Keyword Already at Position #1

  • Incremental opportunity: 0
  • Budget: $25.00 (minimum)
  • Reason: No incremental traffic to gain

Keyword Outside Top 100 (Position 999)

  • Current CTR: 0%
  • Incremental opportunity: Full position 1 opportunity
  • Budget: Maximum opportunity cost (capped at $400)

Low Search Volume

  • Searches < 25: Constrained to 25
  • Budget: May still be calculated but will be minimal

High Budget Calculation

  • Budget > $400: Capped at $400
  • Incremental cost: Still calculated and shown in incremental_traffic_metrics

Zero Incremental Opportunity

  • CPA calculation: Falls back to monthly searches if incremental opportunity is 0

Error Responses

400 Bad Request

{
  "error": true,
  "messages": [
    "Please enter a client business domain URL."
  ],
  "status_code": 400
}

Common Causes:

  • Missing required url field
  • Invalid URL format
  • Invalid parameter types

401 Unauthorized

{
  "error": true,
  "messages": [
    "Authentication required"
  ],
  "status_code": 401
}

Common Causes:

  • Missing or invalid Bearer token
  • Expired token
  • Invalid authentication credentials

500 Internal Server Error

{
  "error": true,
  "messages": [
    "The keywords budget estimate api is not available."
  ],
  "status_code": 500
}

Common Causes:

  • API service unavailable
  • SEMrush API errors
  • Database connection issues

Best Practices

1. Run Budget Estimates First

Always run budget estimates before creating accounts or campaigns to understand investment requirements.

2. Analyze Incremental Traffic Metrics

Review incremental_traffic_metrics to understand:

  • Current traffic potential
  • Opportunity for improvement
  • Cost efficiency (CPA)

3. Consider Value Scores

Use value scores (1-5) to prioritize keywords:

  • Premium Value (5): High investment, high return potential
  • High Value (4): Good balance of cost and opportunity
  • Budget-Friendly (2-3): Lower cost entry points

4. Review Recommendations

Budget recommendations provide contextual guidance:

  • High budget + strong upward trend = Premium opportunity
  • High budget + declining trend = Risk warning
  • Low budget + high volume = Testing opportunity

5. Understand Budget Constraints

All budgets are constrained between $25-$400:

  • Budgets below $25 are increased to minimum
  • Budgets above $400 are capped at maximum
  • Check incremental_traffic_cost in metrics to see unconstrained value

6. Monitor Trend Data

While trends don't affect budget calculations, they provide valuable insights:

  • Exploding/Rising trends: Growing market opportunity
  • Declining trends: Market may be shrinking
  • Use trends for strategic planning, not budget adjustments

Example Use Cases

Use Case 1: New Client Onboarding

Goal: Understand budget requirements for a new client's keyword portfolio.

Process:

  1. Submit client's website URL
  2. Review estimated_budget summary for total investment
  3. Analyze individual keyword budgets
  4. Prioritize based on value scores
  5. Create budget plan based on recommendations

Use Case 2: Keyword Prioritization

Goal: Identify which keywords offer the best ROI.

Process:

  1. Run budget estimate for keyword set
  2. Sort by cost_per_acquisition (lower is better)
  3. Filter by value_score (4-5 for high priority)
  4. Review incremental_traffic_opportunity for potential impact
  5. Focus on keywords with high opportunity and reasonable CPA

Use Case 3: Budget Planning

Goal: Create quarterly/annual budget projections.

Process:

  1. Run budget estimates for all target keywords
  2. Sum monthly_budget values
  3. Multiply by 3 for quarterly, 12 for yearly
  4. Use estimated_budget summary for quick totals
  5. Adjust based on business priorities

Technical Notes

CTR Position Mapping

CTR values are based on industry-standard search result click-through rate studies. Position 1 receives the highest CTR (22.84%), with CTR decreasing significantly for lower positions.

Budget Calculation Philosophy

The incremental traffic cost approach ensures budgets reflect:

  • Real opportunity: Based on actual position improvement potential
  • Transparency: All calculations visible in incremental_traffic_metrics
  • Fairness: Position-based, not arbitrary multipliers

Trend Data Usage

Trend data is collected and analyzed but not used in budget calculations. This ensures:

  • Budgets are based on current opportunity, not historical trends
  • More predictable and transparent pricing
  • Focus on actual traffic potential rather than trend speculation

Performance Considerations

  • API response time: Typically 2-10 seconds per request
  • Rate limiting: Recommended 1-2 second delay between requests
  • Token expiration: Tokens expire after 120 minutes

Support & Resources

For additional support or questions:

  • Review error messages in response for specific issues
  • Verify authentication token is valid and not expired
  • Ensure input data meets required format and constraints

Version History

Current Version

  • Removed: Trend multipliers from budget calculations
  • Added: Incremental traffic metrics in response
  • Enhanced: Detailed CTR and traffic calculations
  • Improved: Transparent budget calculation methodology

Previous Version

  • Used trend multipliers for budget adjustments
  • Less detailed traffic metrics
  • Trend-based budget modifications

Formula Summary

Step 1: Constrain Inputs
  constrained_searches = max(25, monthly_searches)
  constrained_cpc = max(2.75, min(60.00, cpc))

Step 2: Get CTRs
  current_ctr = CTR[current_position]
  position_1_ctr = CTR[1] = 0.2284

Step 3: Calculate Traffic
  current_traffic = constrained_searches × current_ctr
  position_1_opportunity = constrained_searches × position_1_ctr

Step 4: Calculate Incremental
  incremental_opportunity = position_1_opportunity - current_traffic

Step 5: Calculate Base Budget
  base_budget = incremental_opportunity × constrained_cpc

Step 6: Apply Constraints
  monthly_budget = max(25, min(400, base_budget))

Step 7: Extend Budgets
  quarterly_budget = monthly_budget × 3
  yearly_budget = monthly_budget × 12

Step 8: Calculate CPA
  cpa = monthly_budget / incremental_opportunity

Last Updated: November 4 2025
API Version: v1
Endpoint: /api/v1/keywords/budget/estimate

Sidebar

Clone this wiki locally