Skip to content
BranchesTags

Repository files navigation

✈️ Flight Price Prediction - Data 400 Capstone Project

Airplane in sky

📋 Table of Contents

🎯 Introduction

Hi! Welcome to our Data 400 capstone project! 👋

This project aims to address the question: Can we predict flight prices?
Using machine learning and automated data collection from the Amadeus Flight API, We've built a project that gathers real-time flight data for major cross-country flight patterns and develops predictive models to forecast ticket prices.

Why does this matter?
Airlines use complex algorithms to dynamically price tickets based on demand, seasonality, route popularity, and countless other factors. By analyzing these patterns, we can help travelers make more informed decisions about when to book their flights! 💰✈️

📊 Project Overview

Key Features

  • Robust Data Workflow: GitHub Actions run our workflow 2x daily to collect up-to-date flight data
  • Real-Time API Integration: Uses Amadeus Flight Offers Search API
  • Large-Scale Dataset: ~700,000 observations collected
  • Machine Learning Models: Multiple ML approaches to predict flight prices
  • Quality Data Visualizations: Many visualizations showing the capabilities of our data

Technologies Used

  • Python 3.8+ for data processing, visualizing, and modeling
  • Amadeus API for data collection
  • GitHub Actions for automated workflows
  • Pandas & NumPy for data manipulation
  • Scikit-learn for machine learning
  • Matplotlib/Seaborn for visualization
  • Dash & Plotly for interactive dashboard
  • ExchangeRate-API for live currency conversion

📁 Repository Structure

Data400_Final_Project/
│
├── .github/workflows/           # GitHub Actions automation
│   └── collect_flights.yml      # Scheduled workflow for daily data collection
│
├── code/                         # Analysis and modeling scripts
│   ├── EDA.ipynb                 # EDA & data visualization
│   ├── data_concatenation.ipynb  # Data concatenation (for saving example dataset)
│   ├── feature_engineering.ipynb # Feature engineering and modeling preparation
│   ├── modeling.ipynb            # Machine learning models
│   └── aircraft_dashboard.ipynb  # Interactive aircraft usage dashboard
│
├── data/                         # Flight data storage
│   ├── final_flight_data.csv     # Example dataset (first ~500,000 observations collected)
│
├── collect_flights.py            # Main data collection script
├── requirements.txt              # Python package dependencies
├── .gitignore                    # Git ignore rules
└── README.md                     # You are here

📄 File Descriptions

collect_flights.py

This script:

  • Connects to the Amadeus API using secure credentials
  • Queries flight offers for major airport routes
  • Handles pagination and rate limiting
  • Stores data in structured CSV format
  • Uploads data directly to Dropbox
  • Includes error handling and logging

.github/workflows/

Contains the GitHub Actions configuration that automatically runs collect_flights.py on a schedule. This ensures we're constantly gathering fresh data without manual intervention. The workflow runs twice daily at 12:00 AM and 12:00 PM UTC.

code/

Houses all analysis and modeling scripts:

  • Exploratory Data Analysis: Understanding price distributions, seasonal patterns, route popularity, and much more!
  • Feature Engineering: Creating meaningful predictors from raw data
  • Modeling: Building and evaluating machine learning models

data/

Stores a sample of the data for our project. Note: Large data files are gitignored to avoid file size issues.

requirements.txt

Lists all Python packages needed to run the project. Install with:

pip install -r requirements.txt

🔄 Data Collection

The automated data collection pipeline follows these steps:

  1. GitHub Actions Trigger 📅

    • Workflow runs twice daily
    • Can also be triggered manually

  2. API Authentication 🔐

    • Authenticates Amadeus API credentials

  3. Flight Search 🔍

    • Queries major US airport pairs (e.g., JFK↔LAX)
    • Searches for flights across multiple departure dates
    • Collects various cabin classes and booking options

  4. Data Storage 💾

    • New data is saved to Dropbox as a .csv with a date and time label

  5. Error Handling ⚠️

    • Logs API errors and rate limit issues
    • Implements retry logic for failed requests
    • Sends notifications for critical failures

API Rate Limits

  • The Amadeus API has very strict usage limits
  • Multiple API keys are used in each data collection run

📊 Dataset Description

Amadeus Variables

Each flight observation includes the following variables:

Variable Name Type Description Source/Notes
flight_id string Unique identifier for the specific flight Engineered
origin string Origin airport code (e.g., 'JFK', 'LAX') API
destination string Destination airport code API
departure_date datetime Scheduled departure date and time API
arrival_date datetime Scheduled arrival date and time API
price float Target variable - Flight price in Euros API
currency string Currency of the price (EUR, transformed to USD) API
airline string Operating airline carrier code API
airline_name string Full airline name mapped from carrier code Engineered - Dashboard
aircraft_type string Aircraft model codes (e.g., '32Q', '73H') API - Dashboard
stops integer Number of stops (0 = nonstop, etc.) API
duration string Total flight duration (e.g., 'PT5H30M') API
cabin_class string Booking class (ECONOMY, PREMIUM_ECONOMY, BUSINESS, FIRST) API
seats_available integer Number of seats remaining at this price API
booking_class string Fare booking code API
days_until_departure integer Days between search date and departure Engineered
search_date datetime When the price was recorded API
route string Route identifier (e.g., 'JFK → LAX') Engineered - Dashboard*

*Dashboard-exclusive variables are used in the interactive aircraft usage dashboard for filtering and visualization.

Engineered Features

Features used for Modeling:

  • Historical Prices

    • lagged price features
    • rolling window statistics

  • Route Features

    • route competition features
    • route durations
    • route volatility

  • Temporal Features

    • red-eye indicators
    • time of day indicators
    • flight durations
    • departure day features

  • Airline Features

    • airline volatility
    • premium vs lowcost carrier

  • Bookable Seats Features

    • distance from 4 bookable seats (see EDA)
    • scarcity indicators

🛠️ Installation & Setup

Prerequisites

  • Python 3.8 or higher
  • pip package manager
  • Amadeus API credentials (Get them here)

Installation Steps

  1. Clone the repository

    git clone https://github.com/wbartlett1/Data400_Final_Project.git
cd Data400_Final_Project

  2. Install dependencies

    pip install -r requirements.txt

  3. Set up API credentials

    Create a .env file in the root directory:

    AMADEUS_API_KEY=your_api_key_here
AMADEUS_API_SECRET=your_api_secret_here

  4. Verify setup

    python collect_flights.py --test

🚀 Usage

Collecting Flight Data

Run manual data collection:

python collect_flights.py

Running Analysis

Open notebooks in Jupyter:

jupyter notebook code/EDA.ipynb
jupyter notebook code/feature_engineering.ipynb  
jupyter notebook code/modeling.ipynb
jupyter notebook code/aircraft_dashboard.ipynb

📈 Exploratory Analysis

Price by Airline

image We can see which airlines are higher-end and which are low-cost carriers. We can use this to improve our model's predictive power through engineering related features.

Route-Level Visualization

image We can see which routes are more expensive than others.

Booking Window Visualization

image We can see the pricing trends by how many days before departure the flight's price is observed. There is a clear drop-off at 7 days out, which can help us with modeling and feature engineering.

Temporal Visualization

image Price by day of week and time of day.

Bookable Seats Visualization

image Price by how many seats are left bookable. We can see the drop-off at 4 bookable seats, which can help us with modeling and feature engineering.

🎛️ Interactive Aircraft Usage Dashboard

We've built an interactive dashboard to analyze aircraft usage patterns across our flight data.

Dashboard Features

12 Interactive Filters:

  • Basic Filters: Airline, Route, Origin, Destination
  • Aircraft Filters: Aircraft Family, Aircraft Brand
  • Flight Characteristics: Number of Stops (checkboxes), Cabin Class (checkboxes), Red-eye flights
  • Advanced Filters:
    • Currency toggle (EUR/USD with live exchange rates)
    • Days Until Departure slider (0-14 days)
    • Departure Hour slider (24-hour)
    • Price Range slider (with currency conversion)
    • Flight Duration slider (in hours)
    • Bookable Seats slider

Visualization:

  • Interactive donut chart showing aircraft type distribution
  • Aircraft sorted by manufacturer (Airbus → Boeing → Bombardier → Embraer)
  • Labels shown only for aircraft types representing ≥5% of usage
  • Hover tooltips display aircraft family, brand, count, and percentage

Live Features:

  • Real-time EUR to USD exchange rate via ExchangeRate-API
  • Dynamic currency conversion for price filters
  • Responsive filter panel with organized sections

Running the Dashboard

jupyter notebook code/aircraft_dashboard.ipynb

Run all cells, then access the dashboard at http://127.0.0.1:8050/ (or http://localhost:8050/ works too!)

📈 Machine Learning Model Performance

Machine learning models tackle the question: Can we predict a tomorrow's price for a cross-country flight?

Modeling Approach:

  • 80/20 temporal split
  • Hyperparameter tuning via GridSearchCV & RandomizedSearchCV
  • 3-fold cross-validation
  • Evaluation Metric: Mean Absolute Error (MAE)

Models Tested:

  • Linear Regression
  • Decision Tree
  • Random Forest
  • XGBoost

Model Performance:

image With our best model, Random Forest, we can predict the price of a cross-country flight tomorrow within $15 on average! This model has an R² of 0.991, explaining over 99% of the variation in tomorrow's price.

Feature Importance:

image Price history and trends explain ~91.7% of the model's predictive power. Other important features are related to airline and route volatility, competition indicators, and more.

Results Discussion

The superiority of the tree models indicate a non-linear relationship between features and the target. We have highly predictive models, which can help us understand pricing strategies and tendencies. Using this model, we could help people save money on flights! This project is for academic purposes only.

❓ Questions ❓

Have any questions? Contact us at our email below or open an issue!

📚 References & Resources

🙏 Acknowledgments

  • Professor Bilen
  • Dickinson College
  • Amadeus for providing API access

Made by Will Bartlett & Kevin Tran
Dickinson College
Data Analytics Department

🔗 Quick Links

Last Updated: December 2025

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages