feat: add comprehensive test suite and CI workflow#71
Open
AdityaAsopa wants to merge 7 commits intoisro:masterfrom
Open
feat: add comprehensive test suite and CI workflow#71AdityaAsopa wants to merge 7 commits intoisro:masterfrom
AdityaAsopa wants to merge 7 commits intoisro:masterfrom
Conversation
The spacecraft_missions data had deeply inconsistent schemas — mass appeared as 'weight', 'lift-off_mass', 'spacecraft_mass', 'mass_at_lift-off' and 5 other variants; dates ranged from 'April 19, 1975' to '22 October 2008' to '26-05-1999' across 15+ formats; KALPANA-1 had mission_life stored in the 'mission' field as '7 Years'; and TES appeared as a duplicate entry. spacecrafts.json had only id+name for 113 records. launchers.json had only id for 81 records. customer_satellites.json mixed 'GERMANY' with 'Germany' and 'UK' with 'UNITED KINGDOM'. This commit introduces scripts/normalize_data.py — an idempotent pipeline that parses all date formats to ISO 8601, extracts numeric mass_kg and power_watts from free-text fields (handling edge cases like '15 Sq.m Solar Array generating 1360W'), classifies orbits (LEO/SSO/GEO/Lunar/Failed), infers mission status from launch date + mission life, and normalizes country names. The scraper was re-run against isro.gov.in and the fresh data is merged with existing records — no data is lost, only enriched. All 5 data files now have consistent, documented schemas. spacecrafts are enriched with launch date, vehicle, orbit type, and status from missions. Launchers are classified into 8 vehicle families. All API endpoints remain backward-compatible — same URLs, same structure, just cleaner data. API handlers: removed unused 'fs' imports, fixed misleading variable names (customer_satellites.js loaded data into a var called 'launchers'), added Content-Type: application/json headers, and sanitized error responses. Root endpoint now returns a JSON directory of all available endpoints.
Returns aggregate statistics computed at runtime from the five data collections: record counts, mission status distribution, orbit type breakdown, mission type breakdown, spacecraft status, customer satellite country distribution and total mass, and launcher vehicle family counts. All values derived directly from the JSON data files — nothing hardcoded. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each collection endpoint now accepts filter parameters: - /api/spacecrafts?status=active&orbit_type=GEO&mission_type=Communication&launch_vehicle=PSLV - /api/launchers?vehicle_family=PSLV - /api/customer_satellites?country=Germany&launcher=PSLV-C37 - /api/centres?state=Karnataka - /api/spacecraft_missions?status=active&orbit_type=SSO&mission_type=Remote+Sensing&launch_site=SDSC+SHAR Filters are case-insensitive and composable — multiple params narrow results. Unknown params are ignored. Empty array returned (not 404) when filters match no records. No changes to existing response shape when called without params.
Adds /api/:resource/:id endpoints for all five collections: - GET /api/spacecrafts/:id - GET /api/launchers/:id - GET /api/customer_satellites/:id - GET /api/centres/:id - GET /api/spacecraft_missions/:id Returns the matching record directly (not wrapped), 404 if not found, 400 if ID is not a valid integer. No new data added — reads directly from existing normalized JSON files. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
/api/spacecrafts/:id now includes a _links object: - _links.self: canonical URL for the spacecraft - _links.mission: URL to the matching spacecraft_missions record (when available) /api/spacecraft_missions/:id now includes a _links object: - _links.self: canonical URL for the mission - _links.spacecraft: URL to the matching spacecrafts record (when available) Cross-links are resolved by case-insensitive name matching between the two datasets. All 64 mission records have a matching spacecraft entry. No link is added when no match is found — no guessing. Collection endpoints and other individual endpoints are unchanged.
Adds schema definitions and automated validation for all five data files: Schemas (schemas/*.schema.json): - Enforce required fields (id, name, status where applicable) - Constrain orbit_type and status to known enum values - Validate ISO 8601 date format on all date fields - Require numeric types for mass_kg, power_watts, altitude_km - Reject unknown fields (additionalProperties: false) Validation script (scripts/validate_schemas.js): - Zero-dependency beyond ajv (devDependency only, not runtime) - Runs locally: npm run validate - Exits 0 on success, 1 on any schema violation CI workflow (.github/workflows/validate_data.yml): - Triggers on any PR or push to master that touches data/ or schemas/ - Runs on ubuntu-latest with Node 20 - Fails the PR if any data file violates its schema All five data files currently pass validation (confirmed pre-commit).
Adds Jest-based tests across three layers:
API handler tests (tests/api/):
- spacecrafts, launchers, customer_satellites, centres, spacecraft_missions
- Correct HTTP status and Content-Type on all endpoints
- Correct record counts returned with no filters
- Filter behaviour: case-insensitive, composable, unknown params ignored,
empty array (not 404) when no matches
- stats: all top-level sections present; every distribution sums to its
collection total; total_mass_kg is positive
- spacecrafts/:id and spacecraft_missions/:id: valid ID returns record,
non-existent ID returns 404, non-integer returns 400, _links present
and correctly formatted
Data integrity tests (tests/data/integrity.test.js):
- All IDs are unique positive integers
- No trailing whitespace in name fields
- All non-null dates match ISO 8601 format
- All non-null numeric fields are positive numbers
- status and orbit_type constrained to known enums
- country names are title case (regression guard against all-caps)
- centres use lowercase field names (regression against Place/State)
- No values in wrong fields (mass field cannot be a string name)
CI (`.github/workflows/test.yml`):
- Runs on every PR and push to master
- Installs deps, runs jest, then runs schema validation
- Both must pass for CI to go green
Test runner: Jest 29 (devDependency only, zero runtime impact).
All tests pass against current normalized data.
AdityaAsopa
added a commit
to AdityaAsopa/isro_api
that referenced
this pull request
Mar 12, 2026
- CHANGELOG.md: full project history (v1.0.0 → v1.1.0) documenting all 7 PRs (isro#65–isro#71) in Keep a Changelog format; Unreleased section for today's work - index.html: complete rewrite — space-themed mission control dashboard; live Chart.js visualisations (orbit distribution, mission status, vehicle families, top countries by satellite count); animated counters fed from /api/stats; responsive star-field background; endpoint quick-reference cards - style.css: full rewrite with CSS custom properties; dark space palette (#080818 bg, #06b8ee accent); responsive grid at 900 px and 600 px breakpoints - api/timeline.js: GET /api/timeline — aggregates launch dates from spacecraft_missions, spacecrafts, and customer_satellites into a unified chronological event stream; supports ?date=MM-DD, ?month=YYYY-MM, ?year=YYYY, ?range=YYYY,YYYY query params - isro_api_plan.md: big-picture vision document (10 major platform moves) - social_posts.md: LinkedIn posts and X thread for all 7 PRs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a Jest-based test suite covering all API handlers, the stats endpoint, individual resource endpoints, and data integrity — plus a CI workflow that runs on every PR.
Test coverage
API handler tests (
tests/api/)Collection endpoints (
spacecrafts,launchers,customer_satellites,centres,spacecraft_missions):Content-Type: application/jsonStats endpoint (
tests/api/stats.test.js):totals,spacecraft_missions,spacecrafts, etc.)total_mass_kgis a positive numberIndividual resource endpoints (
tests/api/spacecrafts-id.test.js,spacecraft_missions-id.test.js):_links.selfalways present;_links.mission/_links.spacecraftpresent when a match existsData integrity tests (
tests/data/integrity.test.js)These act as a regression guard — they fail if a scraper update or manual edit silently corrupts the normalized schema:
YYYY-MM-DD)mass_kg,power_watts, etc.) are positive numbersstatusandorbit_typeconstrained to defined enumscentres.jsonuses lowercase field names (place,state— notPlace,State)CI workflow (
.github/workflows/test.yml)Runs on every PR and every push to
master:npm test— Jest suite must passnpm run validate— JSON Schema validation must passBoth gates must be green for CI to pass.
Design decisions
mockReq/mockReshelper. Fast, no ports, no teardown.jestis adevDependencyonly113,64,75, etc.) — this is intentional. If a scraper adds or removes records, the tests will flag it for human review rather than silently passing.Notes
This PR cherry-picks all feature commits from PRs #65–#70 so the tests are self-contained. As those PRs merge individually into
master, this branch can be rebased cleanly — the test files themselves are net-new additions with no conflicts.Test plan
npm install && npm test— all tests passnpm run validate— all schemas pass