SCPRv4 is the code that runs SCPR.org. It is approximately the fourth generation of the software for the site.
SCPRv4 is a Rails app, and runs in production on Ruby 2.1.
It connects to the following services:
- MySQL is used as the primary datastore for CMS data.
- Elasticsearch powers article search and display.
- Redis is used by Resque, the queuing system used for async jobs.
- Memcached is used in production for caching, but isn't required for development.
- Newsroom is a small Node.js app that is used to communicate editing status inside the CMS, including the completion of import jobs inside Outpost.
- NFS is used to make uploaded audio files available on all SCPRv4 servers.
- Assethost is used to manage all photo and video assets for content.
SCPRv4 basically integrates a CMS and frontend. CMS conventions are provided through Outpost, but all data models are here inside the SCPRv4 project.
And speaking of data models... there are a lot of them, and some of them might seem like they overlap.
Before SCPRv4, there was Mercer. In Mercer, there were different models for blog entries, show segments, events and news stories, and each had very defined rules about where they could exist.
ContentBase was our effort to first make all of those content types available on the home page, and then secondly to make them far more interchangable.
Since then, ContentBase loosely evolved into the
Article model. It would
be fair to say that in SCPRv4, "content" is any model that can be cast as an
Currently, that list includes
Each of these models includes a
to_article function, which is used to shape
the content into the standardized form that
are stored in Elasticsearch, and are used directly in various index contexts
around the site.
User accounts for the CMS are in the
SCPRv4 uses two kinds of caching:
- As-Needed Partial Caching: These are normal Rails partial caches, generated and stored on-demand when content views are generated.
- Periodic Caches: A handful of expensive caches are computed out-of-band and updated periodically rather than on-demand. The most important is the cache for homepage content, which requires loading up all homepage content objects, sorting them by section, etc.
There are some backend tasks that are worth knowing:
scprv4:index_all: Index all articles and models to Elasticsearch. During normal operation these are generated as content is updated, but this task can be necessary when importing data into a new deployment.
scprv4:sync_remote_articles: Trigger a
scprv4:sync_external_programs: Trigger a
ContentAlarm.fire_pendingto publish all pending content with a current alarm.
scprv4:schedule:build: Trigger a
scprv4:cache: Trigger jobs to build all of the periodic caches.
In production, scheduler runs on a worker instance.
AssetSync (run via this rake task) listens to a Redis server for AssetHost to send PUB/SUB notifications that an asset has been updated. When an asset is updated externally, we need to delete our cache of it so that the newest version is picked up.
In production, assetsync runs on a worker instance.
- Install Ruby 2.1.x
- Install Node.js 5.x
- Install MySQL (5.6.x should be fine) / Elasticsearch (1.5.x) / Redis
- Install Docker & Docker-Machine (for development only)
- Acquire a
secrets.ymlfile via the wiki
- Get a Deploybot token for a full DB backup and run
rake db:pull (DEPLOYBOT TOKEN HERE)
rake scprv4:index_allto index articles and models into Elasticsearch
rake scprv4:cache:allto cache homepage, tweets, etc