Adding a new module

Details the process of adding new module to limelight, from config to front end javascript

NB: Using a realtime collector as an example

alphagov/deployment

(backdrop)

1. Create buckets in backdrop write

   • generate token for bucket using backdrop/generate-token.sh or something
   • add name of module to dict, alphabetically ordered, as key, val = token
   • add DIFFERENT token for staging and production to preview

2. Change bucket access in backdrop.read

   • add entry for bucket into backdrop read config
   • just add entry in production here, because it will work everywhere, weirdly
   • Setup for raw queries if necessary in /backdrop/read/config/development.py
   • restart backdrop if you make a change

backdrop-ga-realtime-collector

3. Add targets to realtime collector
   • backdrop-ga-realtime-collector/to_upload/targets.ENVIRONMENT.json
   • add json object including name{url,token}
   • again, different tokens for preview vs. staging & production

Jenkins

Redeploying at this point will give you quicker feedback, feel free to do this at the end

8. Redeploy in deploy.preview.alphagov.co.uk
   1. go to Preview deploy applications
   2. build with parameters
   3. write.backdrop
   4. read.backdrop
   5. backdrop-ga-realtime-collector

backdrop-ga-realtime-collector

4. Add query file to realtime collector
   • create new query file: {collector,target,query{ids,metrics,filters}}
   • config/cronjobs: add query file & credentials file

github.gds/puppet

5. Setup proxy buckets on preview and staging/production
   • puppet/extdata/development.csv: add in new rule e.g. 'my_neat_realtime_feature',true
   • then
     • puppet/govuk/manifests/apps/publicapi.pp
     • add new pointer variable to the rule you've setup in development.csv
       • it looks like $enable_my_neat_realtime_feature_buckets = str2bool(extlookup('my_neat_realtime_feature','no'))
       • that looks up the feature flag rule, the 'no' means str2bool will default to false if it can't find the rule, or the rule is set to false. This is nice for turning things off if stuff is blowing up
       • add new object to backdrop buckets: {path:'some_path/my_neat_realtime_feature', name: bucket_to_proxy_to, 'enabled':$enable_my_neat_realtime_feature_buckets}
   • run bundle exec rake lint
   • run bundle exec rake test
   • ! make sure you're not committing in any personal credentials, come on dude.

5a. Ask someone (Alex or Rob) with access to gds/deployment to add the same feature flag to the ext data files.

limelight

6. Set up query in limelight

   • backdrop_stub_controller.rb: write in new StubConfig with query
     • StubConfig.new({'service' => 'PARENT-SERVICE', 'api_name' => 'my-neat-realtime-feature'}, 'dummy_data.json')
   • to test
     • setup enviornment variable to be development and use the stub data to test by running 'RAILS_ENV=development BACKDROP_URL=/backdrop_stub bowl limelight'
     • ping e.g. limelight.dev.gov.uk/backdrop_stub/performance/PARENT-SERVICE/api/my-neat-realtime-feature --> should return some stub json as set in the StubConfig
     • when you get something back, write in a .feature test in e.g. PARENT-SERVICE/my-neat-realtime-feature.feature in cucumber, may need to write extra steps for specific feature

7. Set up module in limelight

   • add collections and views to controller that will deal with your feature
     • add collection to someService/feature.js
     • use a view, maybe something like extensions/graph.js
     • wrap this up in a controller which calls in the collection and view and initialises it e.g. var myFeatureCollection = new FeatureCollection([],{element, collection, options}); var view = new View({ el: $("#myModuleId"), collection: myFeatureColletion, service: someService });
   • make sure there is a corresponding .erb template that has the element you're operating on in it with a placeholder element like <div id="myModuleId"></div>