KPCC on Rails
Ruby CSS HTML JavaScript CoffeeScript
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
app
bin
config
db Enforce uniqueness of remote articles at the database level. Aug 16, 2016
lib
public
spec
vendor/assets
.gitignore Add task to automatically import image assets from the style guide. Jul 25, 2016
Gemfile
Gemfile.lock
Guardfile
README.md
Rakefile Fix Resque namespace setup Dec 10, 2014
circle.yml
config.ru
package.json

README.md

SCPRv4

Build Status

SCPRv4 is the code that runs SCPR.org. It is approximately the fourth generation of the software for the site.

Requirements

SCPRv4 is a Rails app, and runs in production on Ruby 2.1.

It connects to the following services:

  • MySQL is used as the primary datastore for CMS data.
  • Elasticsearch powers article search and display.
  • Redis is used by Resque, the queuing system used for async jobs.
  • Memcached is used in production for caching, but isn't required for development.
  • Newsroom is a small Node.js app that is used to communicate editing status inside the CMS, including the completion of import jobs inside Outpost.
  • NFS is used to make uploaded audio files available on all SCPRv4 servers.
  • Assethost is used to manage all photo and video assets for content.

Quick Tour

SCPRv4 basically integrates a CMS and frontend. CMS conventions are provided through Outpost, but all data models are here inside the SCPRv4 project.

And speaking of data models... there are a lot of them, and some of them might seem like they overlap.

Content Models

Before SCPRv4, there was Mercer. In Mercer, there were different models for blog entries, show segments, events and news stories, and each had very defined rules about where they could exist.

ContentBase was our effort to first make all of those content types available on the home page, and then secondly to make them far more interchangable.

Since then, ContentBase loosely evolved into the Article model. It would be fair to say that in SCPRv4, "content" is any model that can be cast as an Article.

Currently, that list includes Abstract, BlogEntry, ContentShell, Event, ExternalEpisode, ExternalSegment, NewsStory, PijQuery, ShowEpisode, and ShowSegment.

Each of these models includes a to_article function, which is used to shape the content into the standardized form that Article needs. Article objects are stored in Elasticsearch, and are used directly in various index contexts around the site.

CMS Login

User accounts for the CMS are in the AdminUser model.

Caching

SCPRv4 uses two kinds of caching:

  • As-Needed Partial Caching: These are normal Rails partial caches, generated and stored on-demand when content views are generated.
  • Periodic Caches: A handful of expensive caches are computed out-of-band and updated periodically rather than on-demand. The most important is the cache for homepage content, which requires loading up all homepage content objects, sorting them by section, etc.

Rake Tasks

There are some backend tasks that are worth knowing:

  • scprv4:index_all: Index all articles and models to Elasticsearch. During normal operation these are generated as content is updated, but this task can be necessary when importing data into a new deployment.
  • scprv4:sync_remote_articles: Trigger a SyncRemoteArticles job.
  • scprv4:sync_external_programs: Trigger a SyncExternalPrograms job.
  • scprv4:fire_content_alarms: Call ContentAlarm.fire_pending to publish all pending content with a current alarm.
  • scprv4:sync_audio: Trigger AudioSync::Pending and AudioSync::Program jobs.
  • scprv4:schedule:build: Trigger a BuildRecurringSchedule job.
  • scprv4:cache: Trigger jobs to build all of the periodic caches.

Scheduler

SCPRv4 uses rufus-scheduler to manage periodic job execution. You can find that schedule defined in lib/tasks/scheduler.rake.

In production, scheduler runs on a worker instance.

Asset Sync

AssetSync (run via this rake task) listens to a Redis server for AssetHost to send PUB/SUB notifications that an asset has been updated. When an asset is updated externally, we need to delete our cache of it so that the newest version is picked up.

In production, assetsync runs on a worker instance.

Getting Started

  • Install Ruby 2.1.x
  • Install Node.js 5.x
  • Install MySQL (5.6.x should be fine) / Elasticsearch (1.5.x) / Redis
  • Install Docker & Docker-Machine (for development only)
  • Acquire a secrets.yml file via the wiki
  • bundle install and npm install
  • Get a Deploybot token for a full DB backup and run rake db:pull (DEPLOYBOT TOKEN HERE)
  • rake scprv4:index_all to index articles and models into Elasticsearch
  • rake scprv4:cache:all to cache homepage, tweets, etc