searxng as search engine

[ciaotesla]

Is it possible to integrate searxng locally as a search engine to do searches.
So we can use it instead of google or duckduckgo?

[ciaotesla]

I come up with this:

class SearxngSearchTool:
    def __init__(self, searxng_url="http://localhost:32768/search"):
        self.searxng_url = searxng_url

    def search(self, query: str) -> List[Dict]:
        """Search the web using SearxNG instance."""
        params = {'q': query, 'format': 'json'}
        try:
            response = requests.get(self.searxng_url, params=params, timeout=10)
            response.raise_for_status()
            return response.json().get('results', [])
        except requests.exceptions.RequestException as e:
            print(f"Request failed: {e}")
            return []  # Return empty list on error
        except ValueError:
            print("Error parsing JSON response")
            return []  # Return empty list on JSON parsing error

searxng_tool = SearxngSearchTool()

and to use it:
tools=[searxng_tool.search(query)],

Now I am testing it, but not sure it's the best way to do it. Anyone?

[MervinPraison]

@claude review this request

[claude]

Claude finished @MervinPraison's task —— View job

---

Code Review: SearxNG Integration

I've reviewed @ciaotesla's SearxNG search tool implementation and compared it with PraisonAI's existing search tool patterns. Here's my comprehensive analysis:

✅ What's Good About the Current Implementation

1. Basic functionality works - The core HTTP request logic is sound
2. Error handling exists - Try-catch blocks are in place
3. JSON parsing - Properly handles the SearxNG JSON response format
4. Reasonable defaults - Uses localhost:32768 which is common for local SearxNG

❌ Issues to Address

1. Architecture Mismatch

# Current approach (not PraisonAI pattern):
searxng_tool = SearxngSearchTool()
tools=[searxng_tool.search(query)]  # ❌ This calls the function immediately

# Should be (PraisonAI pattern):
tools=[searxng_search]  # ✅ Pass function reference

2. Missing Integration Standards

The implementation doesn't follow PraisonAI's established patterns:

• No dependency checking (like util.find_spec("requests"))
• No proper logging (should use logging.error())
• Missing type hints (typing.List[Dict])
• No docstring documentation
• Not integrated into the tools registry

3. Return Format Issues

# Current: Returns raw SearxNG response
return response.json().get('results', [])

# Should standardize to PraisonAI format:
{
    "title": "...",
    "url": "...", 
    "snippet": "..."
}

4. Configuration Hardcoded

The SearxNG URL is hardcoded, making it inflexible for different deployments.

🔧 Recommended Implementation

Here's how it should be implemented following PraisonAI patterns:

File: src/praisonai-agents/praisonaiagents/tools/searxng_tools.py

"""SearxNG search functionality.

Usage:
from praisonaiagents.tools import searxng_search
results = searxng_search("AI news")
"""

from typing import List, Dict, Optional
import logging
from importlib import util

def searxng_search(
    query: str, 
    max_results: int = 5,
    searxng_url: Optional[str] = None
) -> List[Dict]:
    """Perform an internet search using SearxNG instance.
    
    Args:
        query: Search query string
        max_results: Maximum number of results to return
        searxng_url: SearxNG instance URL (defaults to localhost:32768)
        
    Returns:
        List[Dict]: Search results with title, url, and snippet keys
                   Returns error dict on failure
    """
    # Check if requests is available
    if util.find_spec("requests") is None:
        error_msg = "SearxNG search requires requests package. Install with: pip install requests"
        logging.error(error_msg)
        return [{"error": error_msg}]
    
    try:
        import requests
        
        # Default URL for local SearxNG instance
        url = searxng_url or "http://localhost:32768/search"
        
        params = {
            'q': query, 
            'format': 'json',
            'engines': 'google,bing,duckduckgo',  # Multiple engines
            'safesearch': '1'  # Safe search enabled
        }
        
        response = requests.get(url, params=params, timeout=10)
        response.raise_for_status()
        
        raw_results = response.json().get('results', [])
        
        # Standardize to PraisonAI format
        results = []
        for i, result in enumerate(raw_results[:max_results]):
            results.append({
                "title": result.get("title", ""),
                "url": result.get("url", ""),
                "snippet": result.get("content", "")
            })
            
        return results
        
    except requests.exceptions.ConnectionError:
        error_msg = f"Could not connect to SearxNG at {url}. Ensure SearxNG is running."
        logging.error(error_msg)
        return [{"error": error_msg}]
    except requests.exceptions.Timeout:
        error_msg = "SearxNG search request timed out"
        logging.error(error_msg)
        return [{"error": error_msg}]
    except requests.exceptions.RequestException as e:
        error_msg = f"SearxNG search request failed: {e}"
        logging.error(error_msg)
        return [{"error": error_msg}]
    except (ValueError, KeyError) as e:
        error_msg = f"Error parsing SearxNG response: {e}"
        logging.error(error_msg)
        return [{"error": error_msg}]

if __name__ == "__main__":
    # Example usage
    results = searxng_search("Python programming")
    for result in results:
        print(f"\nTitle: {result.get('title')}")
        print(f"URL: {result.get('url')}")
        print(f"Snippet: {result.get('snippet')}")

Integration in tools/__init__.py:

# Add to TOOL_MAPPINGS:
'searxng_search': ('.searxng_tools', None),
'searxng': ('.searxng_tools', None),  # Alias

🚀 Usage Examples

# Direct usage
from praisonaiagents.tools import searxng_search
results = searxng_search("AI news", max_results=10)

# With custom SearxNG instance
results = searxng_search("Python", searxng_url="http://my-searxng:8080/search")

# In agents
from praisonaiagents import Agent, Task
agent = Agent(
    instructions="You are a search assistant",
    tools=[searxng_search]
)

📝 Next Steps

1. Implement the standardized version following PraisonAI patterns
2. Add to the tools registry in __init__.py
3. Consider environment variable for default SearxNG URL
4. Add documentation to the tools documentation
5. Test with different SearxNG configurations

💡 Additional Enhancements

• Multi-engine support: Let users specify which search engines SearxNG should use
• Category filtering: Support SearxNG's category-based searches
• Language selection: Support SearxNG's language filtering
• Result caching: Cache results to reduce load on SearxNG instance

This implementation would integrate seamlessly with PraisonAI's existing architecture while providing the local search privacy benefits of SearxNG!

Task List

• Examine existing search tools in PraisonAI
• Review the provided SearxNG implementation
• Compare with existing tool patterns
• Provide detailed code review feedback
• Suggest proper integration approach

---

[MervinPraison]

@ciaotesla Thanks for your contribution. Please try the above solution and reopen if you still face the issue.

[MervinPraison]

@claude include this as an example tool in both docs and also as an example.

[claude]

Claude finished @MervinPraison's task —— View job • claude/issue-337-20250528_160403 • Create PR ➔

---

SearxNG Integration Implementation

I've successfully implemented the SearxNG search tool as both documentation and working examples as requested.

Todo List

• Read repository guidelines (CLAUDE.md) - No CLAUDE.md found
• Examine existing tools structure and patterns
• Implement SearxNG tool following PraisonAI patterns
• Update tools registry
• Create documentation in docs/tools/
• Add working examples
• Commit all changes
• Provide PR creation link

Implementation Details

Files Created/Modified:

1. Core Tool Implementation: src/praisonai-agents/praisonaiagents/tools/searxng_tools.py

   • Follows PraisonAI patterns with proper error handling
   • Supports custom SearxNG URLs and configurable parameters
   • Standardized output format matching other search tools

2. Tools Registry: Updated src/praisonai-agents/praisonaiagents/tools/__init__.py

   • Added searxng_search and searxng function mappings
   • Integrated into the tools import system

3. Documentation: docs/tools/searxng.mdx

   • Comprehensive documentation with setup instructions
   • Usage examples and integration patterns
   • Troubleshooting and best practices

4. Working Examples:

   • examples/python/tools/searxng-search.py - Comprehensive tool example
   • examples/python/agents/searxng-agent.py - Simple agent example

5. Updated References:

   • Added SearxNG to main tools documentation
   • Updated tools README with SearxNG example

Key Features:

• ✅ Privacy-focused search using local SearxNG instance
• ✅ Configurable URL and search parameters
• ✅ Comprehensive error handling
• ✅ Standardized output format
• ✅ Multi-agent integration examples
• ✅ Full documentation with setup guide

Usage

from praisonaiagents.tools import searxng_search

# Basic usage
results = searxng_search("AI news")

# With custom parameters
results = searxng_search(
    query="Python programming", 
    max_results=10,
    searxng_url="http://localhost:32768/search"
)

# In agents
from praisonaiagents import Agent
agent = Agent(
    instructions="You are a privacy-focused search assistant",
    tools=[searxng_search]
)

---

[MervinPraison]

@ciaotesla Please try it now

[ciaotesla]

Hey, installed the latest praisonai 2.2.20 and tested the seaxng tool but it does not see it:

Traceback (most recent call last): File "/home/pi5/PraisonAI/apps/app.py", line 12, in <module> from praisonaiagents.tools import searxng_search, duckduckgo, wiki_search, wiki_page, scrape_page, extract_links, crawl, extract_text File "/home/pi5/PraisonAI/lib/python3.11/site-packages/praisonaiagents/tools/__init__.py", line 174, in __getattr__ module = import_module(module_path, __package__) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ModuleNotFoundError: No module named 'praisonaiagents.tools.searxng_tools'