Skip to content
Crowd-sourced internet speed tests using M-Lab data and user tests on a website, with charts, maps, and raw data downloads.
Branch: master
Clone or download
ryanrolds Merge pull request #85 from noahbrenner/logos
Add logos for TAO and US Ignite to results page
Latest commit 88aba83 May 23, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.ebextensions Maximum Effort! Aug 14, 2018
app Merge pull request #85 from noahbrenner/logos May 23, 2019
bin Push Jun 5, 2018
config Merge branch 'master' into remove_turbo_links May 11, 2019
data Added fake state level data May 19, 2019
doc/app Maximum Effort! Aug 14, 2018
lib/tasks Switch to using geoid May 19, 2019
public Replaced Louisville to Americans; Replaced SpeedUpYourCity.com, Speed… Apr 15, 2019
test Maximum Effort! Aug 14, 2018
vendor/assets/javascripts Maximum Effort! Aug 14, 2018
.dockerignore Wrapping up Apr 28, 2019
.editorconfig Add .editorconfig Apr 27, 2019
.env_sample Maximum Effort! Aug 14, 2018
.gitignore Minor adjustments May 19, 2019
.ruby-version Implement GeoRuby for parsing WKT shapes May 13, 2019
CODE_OF_CONDUCT.md Initial files Apr 18, 2019
CONTRIBUTING.md Initial files Apr 18, 2019
Dockerfile gem updates for security reasons Apr 27, 2019
Gemfile Implement GeoRuby for parsing WKT shapes May 13, 2019
Gemfile.lock Minor adjustments May 19, 2019
GeoIPASNum.dat Maximum Effort! Aug 14, 2018
InstallSpeedUpAmerica.sh Install Script May 4, 2019
LICENSE Create LICENSE Sep 5, 2018
Mlab-SpeedUp-Integration-bd0a76086c36.p12 Maximum Effort! Aug 14, 2018
Procfile Maximum Effort! Aug 14, 2018
README.md Updated data May 19, 2019
Rakefile Push Jun 5, 2018
buildspec.yml Adding build specification. Apr 30, 2019
config.ru Push Jun 5, 2018
docker-compose.yaml Working on test data May 16, 2019
local.env.template More work Mar 30, 2019

README.md

Speed Up America

The project vision is an open source nation-wide map that pulls individual internet speed test data from M-Lab and breaking down the results on maps and charts by points, census blocks, ISP, date range, and speed. Census block data and FCC 477 data will used to supplement both the analysis and maps.

The current implentation of SpeedUp has been deployed to these cities.

Digital Inclusion

The project can be used as part of a digital inclusion strategy to learn where inequities are in your community. SpeedUp can help citizens, businesses, policymakers and others better understand where Americans can access high-quality Internet service, and where there are needs, allowing cities to track and improve performance through key policies, ISP agreements, and partnerships.

Replace FCC 477 Data

All current digital inclusion maps rely on FCC 477 data which is ISP self-reported, notriously incomplete, misleading, gameable by ISPs, and not detailed enough. Let's get better, more accurate, crowd-source speed data directly from citizens to make better decisions and drive policy.

Project History

In April 2016, Louisville Metro Government’s OPI2 Innovation Team,  PowerUp Labs and other partners launched a web-based application aimed to increase transparency about Internet service quality in Louisville at a hackathon. Louisville worked partners to open source "SpeedUp" so that any local government or organization can launch this application for their community.

The application, SpeedUpYourCity.com, the local deployment of "SpeedUp", collects and publicly shares user-generated information about local broadband service speeds, rates and service quality in Louisville. It also incorporates the Measurement Lab Test, which is integrated with Google.com, and greatly increases the number of tests that the application collects.

Articles and Write-Ups

About the Speed Test and the Data

The data is displayed on an interactive map and available for free download, with the goal of increasing transparency about Internet service quality in America and to continue the conversation around fiber in your community. Citizens can visit the site from any device to take the free Internet service test, and is supplemented by Google's M-Lab tests. The data provided by the test and short survey is stored in a publicly available database, combined with other results, and published to the online map in a form that does not identify contributors, and allows direct raw data download.

This test does not collect information about personal Internet traffic such as emails, web searches, or other personally identifiable information. 

About the Original Developers

The SpeedUpLouisville.com project started at a local civic hackathon led by the Civic Data Alliance and hosted by Code Louisville and Code for America. Eric Littleton, Jon Matar and the PowerUp Labs software development team later volunteered to continue the work started during the hackathon. LVL1, a local makerspace, also provided funding for the paid web tools required to complete the project.

Deployment & Operation

The SpeedUpAmerica project utilizes the following technologies for operation:

  • Ruby on Rails
  • MySQL
  • MapBox (API Key Required)
  • Map Technica (API Key Required)
  • MLab (Special configuration instructions)

Setup

These instructions work on Linux, Windows and MacOS and only need to be performed once, unless you reset your database or config files.

Install Docker and Docker Compose.

Depending on your OS, you may have to make sure to use copy instead of cp.

$ git clone https://github.com/Hack4Eugene/SpeedUpAmerica.git
$ git clone  https://github.com/Hack4Eugene/speedupamerica-migrator.git
$ cd SpeedUpAmerica
$ cp local.env.template local.env
$ docker-compose up -d mysql
$ docker-compose up --build migrator
$ docker-compose run migrator rake db:seed
$ docker-compose run frontend rake secret

Use the ouput from rake secret as the value for SECRET_KEY_BASE in your local.env. Go to Mapbox and create an account. Set MAPBOX_API_KEY to the public token or a new token.

If you want a basic dataset to work with run:

$ docker-compose exec -T mysql mysql -u suyc -psuyc suyc < data/zip_codes.sql
$ docker-compose exec -T mysql mysql -u suyc -psuyc suyc < data/census_tracts.sql
$ docker-compose exec -T mysql mysql -u suyc -psuyc suyc < data/submissions.sql

These instructions assume Windows users are not using the WSL, which has documented problems with Docker's bind mounts. Installing and configuring Docker for Windows to work with the WSL is outside the scope of this document.

Updating boundaries

When boundaries are updated each developer must reload their boundary tables:

$ docker-compose exec -T mysql mysql -u suyc -psuyc suyc <<< "TRUNCATE census_boundaries;"
$ docker-compose exec -T mysql mysql -u suyc -psuyc suyc <<< "TRUNCATE zip_boundaries;"
$ docker-compose exec -T mysql mysql -u suyc -psuyc suyc < data/zip_codes.sql
$ docker-compose exec -T mysql mysql -u suyc -psuyc suyc < data/census_tracts.sql

Running

$ docker-compose up -d

The site can be accessed at http://localhost:3000/. The Ruby app is configured to not cache and it doesn't require restarting the Docker container to load changes, unless it's a config change. Just make your changes and reload the page. First page load make take a little bit. See docker-compose logs frontend for stdout/stderr.

Stopping

$ docker-compose stop

Data tasks

There are just the tasks that have been run to populate and prepate the data for operation. The other tasks need investigated and documented.

Importing M-Lab submissions:

$ docker-compose run frontend rake import_mlab_submissions

Populating submissions with Census Tract

$ docker-compose run frontend rake update_pending_census_codes

Creating new submissions.sql

$ docker-compose exec mysql mysqldump --no-create-info -u suyc -psuyc suyc submissions > data/submissions.sql

Importing Census and Zip Code boundaries

Assumes you have these files in data/:

$ docker-compose exec -T mysql mysql -u suyc -psuyc suyc <<< "TRUNCATE census_boundaries;"
$ docker-compose exec -T mysql mysql -u suyc -psuyc suyc <<< "TRUNCATE zip_boundaries;"
$ docker-compose run frontend rake populate_census_tracts
$ docker-compose run frontend rake populate_zip_boundaries

Once imported you can update the SQL files by:

$ docker-compose exec mysql mysqldump --no-create-info -u suyc -psuyc suyc census_boundaries > data/census_tracts.sql
$ docker-compose exec mysql mysqldump --no-create-info -u suyc -psuyc suyc zip_boundaries > data/zip_codes.sql

Creating test data

After loading boundaries and submissions you can distribute the submissions across all Zip Codes and Census Tracts by running:

$ docker-compose run frontend rake create_test_data

Governance and contribution

See CONTRIBUTING.md.

Committers:

  • Diego Kourchenko

Technical Committee:

  • Matt Sayre
  • Chris Ritzo
  • Chris Sjoblom
  • Antonio Ortega
  • Mark Davis
  • Ryan Olds
You can’t perform that action at this time.