NSE Pre-Open Market Data Fetcher

This project fetches pre-open market data from NSE India API and converts it to CSV format for analysis.

Files

• fetch_nse_preopen.py - Original Python script using requests (may face anti-bot protection)
• fetch_nse_curl.py - Python script using curl commands (may face anti-bot protection)
• manual_fetch.py - Manual approach for processing downloaded JSON data
• fetch_data.bat - Windows batch script to fetch all data at once
• requirements.txt - Python dependencies

Quick Start (Recommended)

Option 1: Use the batch script (Windows)

# Run the batch script to fetch all data and convert to CSV
fetch_data.bat

Option 2: Manual approach

# 1. Get the curl commands
python manual_fetch.py

# 2. Run the curl commands manually (copy-paste from output)
# 3. Process the downloaded JSON files
python manual_fetch.py --process

Option 3: Try the Python scripts (may not work due to NSE protection)

# Install dependencies
pip install requests pandas

# Try the requests-based script
python fetch_nse_preopen.py

# Or try the curl-based script
python fetch_nse_curl.py

Market Segments

The scripts fetch data for these NSE market segments:

• ALL - All securities
• NIFTY - NIFTY index stocks
• BANKNIFTY - Bank NIFTY index stocks
• SME - Small and Medium Enterprises
• FO - Futures & Options
• OTHERS - Other securities

Output

Data is saved in data/YYMMDD/ directory:

• CSV files: all.csv, nifty.csv, banknifty.csv, sme.csv, fo.csv, others.csv
• Raw JSON files: all.json, nifty.json, etc. (for reference)

Data Structure

Each CSV contains flattened market data with columns like:

• metadata_* - Stock metadata (symbol, company name, etc.)
• market_* - Market data (prices, volumes, changes, etc.)
• fetch_timestamp - When the data was fetched

Notes

• NSE has anti-bot protection, so automated scripts may not always work
• The manual approach using curl commands is more reliable
• Data is typically available during pre-market hours (9:00-9:15 AM IST)
• Files are appended to if they already exist (for multiple fetches per day)

Troubleshooting

If you get "Resource not found" or 401 errors:

1. Use the manual approach with curl commands
2. Try accessing the NSE website in a browser first
3. Check if you're running during pre-market hours
4. The API may have changed - check NSE website for updates