MevaSearch — ACTIVE CRAWLER
The official search engine crawler for MevaSearch, building a safe and filter-free map of the internet.
Our bot uses the following User-Agent string:
WebAtlasBot/2.0 (+https://mevasearch.com/bot)
Always verify the bot via reverse DNS to ensure the request originates from MevaSearch infrastructure.
WebAtlasBot is designed to maintain a minimal load on web servers:
- Respects
robots.txtdirectives (Disallow,Allow). - Supports
Crawl-delayto prevent server strain. - Verifies
noindexandcanonicaltags. - Uses adaptive crawling — automatically slows down based on your server's response times.
All crawled content is processed and stored in the WebAtlas Index — our proprietary global database designed for high-performance retrieval and semantic analysis.
By being part of the index, your site becomes discoverable through our search infrastructure, reaching researchers, developers, and educators looking for high-quality technical content.
User-agent: WebAtlasBot
Disallow: /private/
Disallow: /admin/
Disallow: /tmp/
User-agent: WebAtlasBot
Disallow: /
User-agent: WebAtlasBot
Allow: /
User-agent: WebAtlasBot
Crawl-delay: 10
If you cannot modify robots.txt, add the following tag to the <head> section of any page you want excluded:
<meta name="robots" content="noindex">WebAtlasBot collects data to power the MevaSearch engine. We focus on high-quality, educational, and technical content. Our goal is not aggressive indexing, but rather creating a knowledge base resistant to algorithmic manipulation — a filter-free, semantic map of the internet's most valuable information.
Questions, feedback, or specific crawling requirements?
- Email: bot[at]mevasearch.com
- Website: mevasearch.com
- Developer: WebOrbiton Team
© 2026 MevaSearch. All rights reserved.