News Sentiment Stock Analysis

A data pipeline for analyzing the correlation between financial news sentiment and stock price movements using FinBERT sentiment analysis.

Overview

This project collects financial news, analyses sentiment using a pre-trained FinBERT model, and correlates sentiment trends with stock returns to identify potential trading signals.

Features

• 🔍 News Collection: Automated scraping from NewsAPI
• 🧠 Sentiment Analysis: FinBERT-powered financial sentiment scoring
• 📈 Price Integration: Historical stock data via yfinance
• 📊 Correlation Analysis: Statistical sentiment-price relationships

Project Structure

social-sentiment-stock/
├── src/
│   ├── news_scraper.py              # News data collection
│   ├── news_sentiment.py            # Sentiment analysis pipeline
│   └── sentiment_price_correlation.py # Correlation analysis
├── data/
│   ├── raw/                         # Raw scraped news data
│   ├── sentiment/                   # Processed sentiment data
│   └── merged/                      # Combined datasets
└── requirements.txt                 # Dependencies

Quick Start

1. Install dependencies

   pip install -r requirements.txt

2. Set up NewsAPI key

   export NEWSAPI_KEY="your_api_key_here"

3. Run the pipeline

   # Collect news data
   python src/news_scraper.py TSLA NVDA
   
   # Analyze sentiment
   python src/news_sentiment.py
   
   # Generate correlations
   python src/sentiment_price_correlation.py

Output

• Raw Data: data/raw/{SYMBOL}_news.csv - Original news articles
• Sentiment: data/sentiment/{SYMBOL}_daily_sentiment.csv - Daily sentiment scores
• Analysis: Combined sentiment and price correlation data

Model

Uses ProsusAI/FinBERT - a BERT model fine-tuned for financial sentiment analysis with three classes (negative, neutral, positive). Outputs sentiment scores from -1 (negative) to +1 (positive).