Skip to content

runawaydevil/scrapiptv

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
.gitignore
.gitignore
 
 
CONTRIBUTING.md
CONTRIBUTING.md
 
 
LICENSE
LICENSE
 
 
README.md
README.md
 
 
config.example.py
config.example.py
 
 
masteriptv.py
masteriptv.py
 
 

Repository files navigation

MasterIPTV - Universal IPTV List Extractor

Python License Version Status

Universal IPTV playlist extractor with multi-method support and intelligent caching

Features • Installation • Usage • Configuration • Documentation

Author: runawaydevil | Contact: runawaydevil@pm.me | Version: 0.01

Overview

MasterIPTV is a universal IPTV playlist extractor that supports multiple extraction methods including Xtream Codes API and direct URL downloads. Built with efficiency in mind, it features persistent caching, parallel processing, and robust error handling for reliable playlist management.

Features

Core Functionality

  • Multi-Method Support: Xtream Codes API and direct URL extraction
  • Persistent Caching: JSON-based cache system for rapid updates
  • Parallel Processing: Multi-threaded extraction for optimal performance
  • Interrupt Protection: Safe Ctrl+C handling with automatic cache preservation
  • Auto-Retry Logic: Automatic retry on temporary failures
  • Connection Testing: Pre-extraction connectivity validation
  • Multiple Providers: Support for unlimited provider configurations

Technical Highlights

  • Zero-dependency core (requests only)
  • Optional Cloudflare bypass support
  • Thread-safe operations
  • Comprehensive logging
  • Cross-platform compatibility (Windows, Linux, macOS)

Supported Methods

Method 1: Xtream Codes API (Recommended)

The Xtream Codes method extracts playlists by category, enabling:

  • Granular cache management per category
  • Parallel category processing
  • Incremental updates (only changed categories are re-downloaded)
  • Significantly faster subsequent executions

Performance:

  • Initial extraction: 30-60 minutes (large playlists)
  • Cached updates: 30 seconds

Method 2: Direct URL

Traditional M3U download via direct URL:

  • Simple HTTP(S) download
  • Cloudflare bypass support (with cloudscraper)
  • Suitable for providers without API access

Installation

Requirements

  • Python 3.7 or higher
  • pip package manager

Dependencies

Required:

pip install requests

Optional (for Cloudflare bypass):

pip install cloudscraper

Setup

git clone https://github.com/runawaydevil/masteriptv.git
cd masteriptv

Usage

Basic Configuration

Edit masteriptv.py and configure the PROVIDERS dictionary:

PROVIDERS = {
    'provider_name': {
        'name': 'Provider Display Name',
        'method': 'xtream',  # or 'url'
        
        # Xtream Codes configuration
        'base_url': 'http://server.example.com',
        'username': 'your_username',
        'password': 'your_password',
        
        # Direct URL configuration
        'url': 'http://server.example.com/playlist.m3u',
        
        'enabled': True
    }
}

Execution

Standard execution (uses cache):

python masteriptv.py

Force full refresh:

python masteriptv.py --force

Output

Generated M3U files are saved to the output/ directory:

output/
  └── provider_name_20260125_120000.m3u

Configuration

Global Settings

OUTPUT_DIR = 'output'      # Output directory for M3U files
CACHE_DIR = 'cache'        # Cache storage directory
MAX_WORKERS = 8            # Parallel threads (Xtream method)
TIMEOUT = 45               # Request timeout in seconds

Multiple Providers Example

PROVIDERS = {
    'primary': {
        'name': 'Primary Provider',
        'method': 'xtream',
        'base_url': 'http://primary.example.com',
        'username': 'user1',
        'password': 'pass1',
        'enabled': True
    },
    'backup': {
        'name': 'Backup Provider',
        'method': 'url',
        'url': 'http://backup.example.com/playlist.m3u',
        'enabled': True
    },
    'disabled': {
        'name': 'Disabled Provider',
        'method': 'xtream',
        'base_url': 'http://old.example.com',
        'username': 'user3',
        'password': 'pass3',
        'enabled': False
    }
}

Cache System

Architecture

The Xtream method implements a persistent JSON cache:

Structure:

cache/
  └── cache_<hash>.json

Behavior:

  • Automatic save every 10 requests
  • Automatic save on Ctrl+C interrupt
  • Automatic save on error conditions
  • Per-provider cache isolation via hash

Cache Management

View cache:

ls cache/

Clear cache:

# Windows
rmdir /s /q cache

# Linux/macOS
rm -rf cache

Force refresh:

python masteriptv.py --force

Interrupt Handling

The application implements safe interrupt handling:

Ctrl+C behavior:

  1. Immediate processing halt
  2. Automatic cache persistence
  3. Progress statistics display
  4. Clean exit

Example output:

WARNING: INTERRUPT DETECTED (Ctrl+C)
======================================================================
Saving cache before exit...
Cache saved successfully: cache/cache_a1b2c3d4.json
Categories cached: 150

Next execution will resume from checkpoint.
======================================================================
Exiting...
======================================================================

Example Output

======================================================================
MasterIPTV - Universal IPTV List Extractor
Version 0.01 by runawaydevil
GitHub: https://github.com/runawaydevil
======================================================================
Press Ctrl+C at any time to safely stop execution
======================================================================

======================================================================
PROVIDER: Primary Provider
======================================================================
Method: Xtream Codes API (with cache)
Testing connection to Xtream server...
Connection successful!
  Status: Active
  Expires: 2026-12-31

PHASE 1: Fetching categories...
  150 live TV categories
  400 VOD categories
  50 series categories

PHASE 2: Fetching streams in parallel...

Fetching live streams from 150 categories...
  120/150 categories already cached
  BR - MOVIES: 450 items
  Progress: 150/150 categories processed
  Total: 15000 live streams found

PHASE 3: Building M3U file...
  Processing 15000 live channels...
  Processing 50000 VOD items...
  Processing 5000 series...

M3U created: output/primary_20260125_120000.m3u
   Total entries: 70000

Execution time: 1850.5s (30.8 min)

======================================================================
FINAL SUMMARY
======================================================================

Primary Provider: SUCCESS
   Method: xtream
   File: output/primary_20260125_120000.m3u
   Entries: 70,000

======================================================================
COMPLETED
======================================================================

Documentation

Determining Provider Method

Xtream Codes detection: Access: http://your-server.com/player_api.php?username=xxx&password=xxx

If the response is JSON containing user information, the provider supports Xtream Codes API.

Method selection:

  • Use xtream for API-enabled providers (faster, cached)
  • Use url for direct M3U URLs (simpler, no cache)

Performance Considerations

Xtream Method:

  • First execution: 30-60 minutes (large playlists)
  • Subsequent executions: 30 seconds (with cache)
  • Network requests: 400-500 (first run), 5-10 (cached)
  • Bandwidth: 50-100 MB (first run), 1-5 MB (cached)

URL Method:

  • Execution time: 2-5 minutes
  • Network requests: 1
  • Bandwidth: Varies by playlist size

Troubleshooting

Connection errors:

  1. Verify credentials
  2. Test server accessibility
  3. Check provider status
  4. Retry after delay

Timeout errors: Increase timeout in configuration:

TIMEOUT = 90  # Increase to 90 seconds

Connection pool warnings: Reduce worker threads:

MAX_WORKERS = 5  # Reduce from 8 to 5

Cache corruption:

# Clear cache
rm -rf cache/

# Or force refresh
python masteriptv.py --force

Import errors:

# Install required dependencies
pip install requests

# Optional: Cloudflare bypass
pip install cloudscraper

Technical Specifications

System Requirements

  • Python 3.7+
  • 100MB free disk space (cache)
  • Network connectivity

Dependencies

  • requests (required)
  • cloudscraper (optional)

Supported Platforms

  • Windows 7/8/10/11
  • Linux (all distributions)
  • macOS 10.12+

License

MIT License - see LICENSE for details.

Copyright (c) 2026 runawaydevil

Author

runawaydevil

Contributing

Contributions are welcome. Please follow these guidelines:

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/enhancement)
  3. Commit changes (git commit -am 'Add enhancement')
  4. Push to branch (git push origin feature/enhancement)
  5. Open a Pull Request

See CONTRIBUTING.md for detailed guidelines.

Acknowledgments

  • Python Software Foundation
  • requests library maintainers
  • cloudscraper contributors
  • Open source community

Legal Notice

This software is provided for educational purposes only. Use of IPTV playlists must comply with local laws and provider terms of service. The author assumes no responsibility for misuse of this software.

Version 0.01 | Report Bug | Request Feature

Copyright (c) 2026 runawaydevil | Licensed under MIT

About

take the control of your iptv

Resources

Readme

License

MIT license

Contributing

Contributing
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages