Skip to content

Berkanktk/datasets

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

7 Commits
data
data
 
 
raw
raw
 
 
scripts
scripts
 
 
.gitignore
.gitignore
 
 
README.md
README.md
 
 

Repository files navigation

Datasets Project

A comprehensive collection of datasets and tools for aviation and threat intelligence. Includes aircraft specifications, airport data, countries, MITRE ATT&CK techniques, and interactive dashboards for data exploration.

Datasets

Aircraft Data

  • Source: ICAO Doc 8643 Aircraft Type Designators
  • Records: 10,113 aircraft
  • Fields: Manufacturer, model, ICAO code, class, wake category, engine specs, dimensions, performance, capacity
  • Files:
    • data/aircraft/all_aircrafts.json - Complete aircraft dataset
    • data/aircraft/commercial_aircrafts.json - Curated commercial aircraft (165 models)

MITRE ATT&CK Framework

  • Records: 697 active techniques (222 parent + 475 sub-techniques)
  • Fields: ID, name, tactics, platforms, description, mitigations
  • Files:
    • data/mitre_attack/techniques.json - Active technique dataset with mitigations
    • data/mitre_attack/techniques_deprecated.json - Deprecated/revoked techniques (filtered out)

Geographic Data

  • Airports: Airport codes, locations, and metadata
  • Countries: Country codes, names, and geographic data
  • Cities: World cities by population threshold (GeoNames)
  • Islands: World islands with coordinates and alternate names
  • Use case: Flight distance calculations, route planning

Risk & Travel Data

  • Country Risk: Safety rankings aggregated from GPI, GTI, TravelSafe, and US News
  • Passports: Visa requirements for all origin-destination country pairs

Visual & Map Data

  • Flags: Country flag images in SVG and PNG (multiple resolutions)
  • Vector Maps: jsVectorMap-compatible world map path data

Features

Interactive Dashboard

File: data/aircraft/dashboard.html

A web-based tool for searching, filtering, and exploring aircraft data.

Features:

  • Advanced Search - Search by manufacturer, model, ICAO code
  • Smart Filtering - Class, wake category, engine type, capacity range, range distance
  • Quick Filters - Commercial, Large, Jets, High Capacity, Long Range
  • Sorting - Click any column header to sort (asc/desc)
  • Pagination - 25, 50, 100, or 250 rows per page
  • Export - Download filtered results as CSV
  • Modal Details - Click any aircraft to see full specifications

How to Use:

python3 -m http.server 8000
# Visit: http://localhost:8000/data/aircraft/dashboard.html

Flag Explorer

File: data/flags/index.html

A browser-based explorer for browsing and searching country flags.

How to Use:

python3 -m http.server 8000
# Visit: http://localhost:8000/data/flags/index.html

Scripts

Data Processing

  • scripts/aircraft/fetch_aircraft_api.py - Generate commercial aircraft reference data
  • scripts/aircraft/cross_reference_aircraft.py - Match and merge aircraft from multiple sources
  • scripts/aircraft/clean_aircraft.py - Filter aircraft data (commercial only)
  • scripts/mitre-attack/extract_techniques.py - Extract MITRE ATT&CK techniques from enterprise-attack.json
  • scripts/mitre-attack/techniques_stats.py - Generate statistics on techniques dataset

Running Scripts

# Extract MITRE ATT&CK techniques
python3 scripts/mitre-attack/extract_techniques.py

# Cross-reference aircraft data
python3 scripts/aircraft/cross_reference_aircraft.py

Data Schema

Aircraft Record

{
  "manufacturer_name": "Airbus",
  "model": "A320",
  "icao": "A320",
  "class": "M",
  "wake": "M",
  "engine_type": "J",
  "engine_number": "2",
  "engine_manufacturer": "CFM International",
  "engine_model": "CFM56-5A",
  "engine_thrust": "24750 lbf",
  "length": "37.6 m",
  "wingspan": "35.8 m",
  "tail_height": "11.76 m",
  "range": "6100 km",
  "ceiling": "43000 ft",
  "max_speed": "876 km/h",
  "passenger_capacity": "194",
  "is_variant": "0"
}

MITRE Technique Record

{
  "id": "T1055",
  "name": "Process Injection",
  "tactics": ["defense-evasion", "privilege-escalation"],
  "platforms": ["Windows", "macOS", "Linux"],
  "is_subtechnique": false,
  "description": "...",
  "mitigations": [
    {
      "name": "Exploit Protection",
      "description": "..."
    }
  ]
}

Planned APIs

1. Flight Distance Calculator

Calculate flight properties between any two airports.

Features:

  • Distance - Great circle distance
  • Flight Time - Estimated duration based on aircraft type
  • Intermediate Points - Geodesic path for map polylines
  • Initial/Final Bearing - Navigation headings
  • Emission Estimates - CO2 based on fuel burn
  • Price Estimation - Rough fare estimates
  • Aircraft Matching - Suggest aircraft for route

Endpoint Example:

GET /api/flight-distance?from=JFK&to=LHR&aircraft=A320

Response:

{
  "from": { "code": "JFK", "name": "New York", "lat": 40.64, "lon": -73.78 },
  "to": { "code": "LHR", "name": "London", "lat": 51.47, "lon": -0.46 },
  "distance_km": 5570,
  "distance_nm": 3008,
  "aircraft": "Airbus A320",
  "estimated_flight_time_hours": 7.2,
  "initial_bearing": 51.2,
  "final_bearing": 96.5,
  "emissions_co2_kg": 18500,
  "estimated_price_usd": 450,
  "polyline_points": []
}

2. Aircraft Specifications API

Query detailed aircraft information.

Endpoints:

  • GET /api/aircraft - List all aircraft (paginated)
  • GET /api/aircraft/:id - Get specific aircraft specs
  • GET /api/aircraft/search?q=Boeing - Search aircraft
  • GET /api/manufacturers - List all manufacturers

3. Threat Intelligence API

Query MITRE ATT&CK techniques and mappings.

Endpoints:

  • GET /api/techniques - List all techniques
  • GET /api/techniques/:id - Get technique details
  • GET /api/tactics - List tactics
  • GET /api/mitigations - Get mitigations for techniques

Statistics

Aircraft Dataset

  • Total Records: 10,113
  • Manufacturers: 232+
  • With Capacity Data: ~70%
  • Major Classes: Small, Medium, Large, Heavy
  • Engine Types: Jet, Piston, Turboprop

MITRE ATT&CK

  • Total Techniques: 697 (active, non-deprecated)
  • Parent Techniques: 222
  • Sub-techniques: 475
  • Tactics: 14 (Reconnaissance, Resource Development, Initial Access, etc.)
  • Top Tactic: Stealth (21.2%)
  • Platforms: Windows (69%), macOS (51%), Linux (49%)
  • Mitigation Coverage: 84.1% of techniques have mitigations

Tech Stack

  • Frontend: Vanilla HTML, CSS, JavaScript (no frameworks)
  • Backend: Python 3
  • Data Format: JSON, CSV
  • Server: Python http.server for development
  • Libraries:
    • requests - HTTP requests for scraping
    • beautifulsoup4 - HTML parsing
    • geopy - Geographic calculations (for future APIs)

Project Structure

datasets/
├── README.md
├── data/
│   ├── aircraft/
│   │   ├── all_aircrafts.json        # Complete aircraft database (ICAO Doc 8643)
│   │   ├── all_aircrafts.csv         # Same in CSV format
│   │   ├── commercial_aircrafts.json # Curated commercial aircraft
│   │   └── dashboard.html            # Interactive aircraft explorer
│   ├── airports/
│   │   └── airports.json             # Airport codes and metadata
│   ├── cities/
│   │   ├── cities500.zip             # Cities with population >= 500
│   │   └── all_cities.zip            # All cities
│   ├── countries/
│   │   ├── all.json                  # All countries
│   │   ├── all-independent.json      # Independent countries only
│   │   ├── countries.json            # Country name list
│   │   └── country/                  # Individual country files
│   ├── country_risk/
│   │   └── safest-countries2026.json # Safety/risk rankings
│   ├── flags/
│   │   ├── index.html                # Flag Explorer demo
│   │   ├── svg/                      # SVG flags
│   │   └── w160/ .. w2560/           # PNG flags at various resolutions
│   ├── islands/
│   │   └── islands.json              # World islands
│   ├── mitre_attack/
│   │   ├── techniques.json           # Active ATT&CK techniques
│   │   └── techniques_deprecated.json
│   ├── passports/
│   │   ├── passport-index.json       # Visa requirements (nested)
│   │   └── passport-index-2026-03-17.json.csv
│   └── vector_maps/
│       ├── world.js                  # Equirectangular projection
│       ├── world-merc.js             # Mercator projection
│       └── world_merc_en.js
├── scripts/
│   ├── aircraft/
│   └── mitre-attack/
└── raw/
    └── enterprise-attack.json        # MITRE ATT&CK STIX source

Getting Started

1. Setup

source .venv/bin/activate

2. View Dashboards

python3 -m http.server 8000
# Aircraft: http://localhost:8000/data/aircraft/dashboard.html
# Flags:    http://localhost:8000/data/flags/index.html

3. Process Data

python3 scripts/mitre-attack/extract_techniques.py
python3 scripts/aircraft/cross_reference_aircraft.py

Data Sources

  • Aircraft: ICAO Doc 8643 Aircraft Type Designators, Wikipedia
  • MITRE ATT&CK: Official ATT&CK Framework (v19.1)
  • Airports: OurAirports
  • Cities / Islands: GeoNames (CC BY 4.0)
  • Countries: REST Countries API (v3.1)
  • Flags: flagcdn.com
  • Passports: Passport Index
  • Country Risk: IEP Global Peace Index, IEP Global Terrorism Index, TravelSafe, US News

License & Attribution

  • MITRE ATT&CK: CC-BY-4.0
  • GeoNames: CC BY 4.0
  • Aircraft Data: Per source license
  • This Project: MIT

Contributing

To add features or datasets:

  1. Add data to data/ or raw/ directories
  2. Create a processing script in scripts/
  3. Update this README
  4. Test with the dashboard

Last Updated: 2026-07-26
Status: Active Development
Maintained By: Berkan Kutuk

About

A comprehensive collection of various processed datasets.

Topics

aviationcountrymitre-attack

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages