Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
There are 4 Applications
1.1 Main / SearchApp:
This is the main app where entities are listed and edited.
modules: CRUD, Data, elements, Navigation and SolrSearch
**dependency declaration: **loader.js
This is an input form for users in the data entry group. Currently it just allows it’s users to create entities.
dependency declaration: data-entry-loader.js
**modules: **data-entry, elements, Data
This creates graphs using d3/nvd3. The three entities pull their data from solr, and user data comes from the django backend.
dependency declaration: reporting-loader.js
**modules: **reporting, elements, SolrSearch
The monitor app is used to configure and monitor data imports
dependency declaration: monitoring-loader.js
**modules: **monitor, elements
2. Design decisions
To aid with this the Bacon library was used to provide a more fully featured event system. It provides Functional programming tools for events.
RequireJS is used to provide a module system.
Models and Collections are found in folders called data
templates, surprisingly enough are found in templates!
views in views
tests are in tests on a per module basis
- This is with a view that modules could be reused across projects and therefore have their test folders brought with them.
Container views store their subviews in a variable called childViews, destroy is invoked on each of these subviews to clean up after they are removed from the display
3. Dev Environment
This outlines the requirements for getting a dev environment set up for the front end. There are two main build tools used, guard from ruby for compiling sass/compass and grunt for managing the js build requirements.
It would probably be nice to move the sass/compass stuff to grunt too.
You will need to have ruby installed on your machine, with the bundle gem. Try using rvm https://rvm.io/ to manage this.
bundler (1.3.5, 1.0.21)
then from /static run
to install the sass/compass/guard dependencies
and once installed
bundle exec guard
to watch for changes to the sass files, it builds them with sourcemaps which is good for dev on chrome
Details here on how to use sourcemaps in chrome, https://developers.google.com/chrome-developer-tools/docs/css-preprocessors
Combined with livereload it’s great, but you’ll have to set that up yourself.
Output should be optimized for production.
This can be done in config.rb, instructions are contained within that file.
The stylesheets are broken up into smallish components.
The app also uses an icon font generated on http://icomoon.io/app/#/select
You can add new ones by uploading the corroborator.dev.svg file from /static/fonts/corroborator
3.2 Js Dependencies
Node deps are installed using npm install. you must also have the grunt cli installed globally and karma, so
npm install -g grunt-cli karma
Frontend dependencies are managed using bower.
jquery.ui and leaflet don’t come with builds.
If you find yourself in a situation where you need to re-install them, go to the root folder of the project, run npm install and then grunt build, these projects usually have the dist folder in the .gitignore
There are a few libraries in there that shouldn’t be, but looking at the loader files will tell you what the real deps are.
backbone.paginator, q, socket.io, spinjs
Handlebars is used to generate the main html templates. These are all precompiled. To compile the templates run grunt handlebars. Template file declarations have been moved into the grunt folder, and are referenced at the top of Gruntfile.js.
karma is the test runner and jasmine is the assertion library, run karma start from static/js to have a watcher watch your folders for changes to files.
test files should be in the test folder of modules and end in Spec.js
html fixtures are created via the jasmine jquery library which also provides jquery like assertions
In the grunt file there is also a karma:ci task that can be run to have the tests run once
Karma config is in karma.conf.js and the test entry point is in test/test-main.js. This should pretty much mirror your loader.js file with regard to dependencies.
Generate the documentation/annotated source by running grunt docco. The docs will appear in docs/annotated-source
4. Backbone Applications
4.1 Search/Main Application
The app is divided into 5 modules Navigation, SolrSearch, CRUD, Data and elements.
Data provides the models and collections that are shared with the other modules. It also prepares the data before sending it to the django backend.
There’s nothing too special here, the main entity collections listen for new content on the searchBus defined in lib/streams.js
There are two models for each entity, one for normal save and display and one for persisting via the multisave
collections.js provide the collection instances
actor.js, bulletin.js and incident.js provide the model and collection definitions.
collection-mixins provides code that is shared between the collections using the underscore extend function.
These are populated by solr.
The initial intention had been to use the data from solr for the main display of entities, but this proved impractical, because the data from solr could easily be stale, even if it were not when downloaded initially, it could become so if another user edited the entity.
Saved searches are stored in saved-search-collection.js
Locations in LocationCollection
these are populated via the Bootstrap variable
SolrSearch is responsible for triggering solr searches and displaying the results.
Init for the application occurs in lib/SolrSearch/main.js
18.104.22.168 Result Display:
Results.init starts a listener in results.js that watches for navigation between tabs and updates the list on show based on the entity selected.
The result views themselves are fairly normal, they only real special thing they have is a kind of infinite scroll functionality, whereby results are rendered in chunks rather than 500 at a time.
This provides an interface for the filtering of search results.
Searches return facets(filters) the user may select a facet and further refine the search. All facets returned by the search will still be displayed to allow for ORing of facets within the same group
If a user has selected a facet(s) and then instigates a search, the system then checks the selected facet(s) and reapplies them if they are available in the search result.
The data processing occurs in lib/SolrSearch/data
Each entity has a filter collection which in turn has FilterGroupCollections for each facet group
These hold the available filters
Each entity also has a SelectedFilterCollection which stores the selected filters for each collection.
22.214.171.124 Search Building
The search is assembled in QueryBuilder...needs documenting
126.96.36.199 Saved search
Searches are saved as json objects in the database, in the PredefinedSearch model. This is not really cool.
the logic is in lib/SolrSearch/persist
search-saver sends an event requesting the current filters, which is picked up by the SelectedFilterCollections of the 3 entities, who send them back. these are then persisted to the database as json
apply-saved-search.js is used to send the filters out to the filter collections, this happens again via the event stream.
This would probably be better and more simply done by allowing direct access to the SelectedFilterCollections. A schema should be designed to save filters and search terms to the database.
188.8.131.52 Display Logic
this is all in lib/SolrSearch/views/filters/
filter manager decides which set of filters should be displayed.
actor, bulletin, incident -filters.js show the filters for each entity.
filter-mixins provides shared functionality
filter-elements provide the label and autocomplete widgets
date-range.js - the date range fitler
map filter - display the map with a list of locations
So this kind of does what it says, it allows users to create read update and delete stuff. It is responsible for the display of all our entities, and their edit forms.
It’s mainly views, divided into display-views(well named) and search-views. Search views were intended to be only the results of embedded search, which for the main part they are, but there are a few extras in there, like the media form and revision.
Like most sections there is a manager class that is responsible for opening and closing views based on the route. It is also responsible for managing whether the entity is displayed full screen or as a section on the right.
All form inputs have their name attribute set to match the backend model. This allows us not to worry about mapping between form html names and backend names.
All form inputs have -field as a class name. This is used to label form fields as belonging to a certain entity and facilitates nested forms events for example
There are a number of configurable widgets for forms, the fields are defined in the -form.js files, and implementations are in form-mixins.js
example: Sources - shows a drop down auto complete list and displays selected items below
containerid: dom id of the element
collection: collection instance containing the values for the auto complete
multiple: can one or more be selected
field_name: name of the model field on the backend
field_label: label to be shown
eventIdentifier: label selection triggers an event which has what is here + ‘_label’
bus: the bus to send the events on
Show a map
locationSource: event identifier for where the locations that will be displayed on the map are coming from
bus: event bus to listen on
show a jquery ui slider
sliderDiv: jquery selector
display: where the score value is displayed
formfield: model field name
these are jquery ui date picker fields
use the date time picker from jquery_time plugin
id’s for fields that will use the drop down boxes, values are currently hard coded on forms
184.108.40.206 Nested Forms:
Nested forms like events comments, have a hidden multi select object that stores the resource uris for child elements.
display templates contains all the templates for viewing the elements, search views have all the data entry and form templates.
220.127.116.11 Embedded search
Each entitiy under search-views has a an -search-field.js this is used to show the embedded search field and it’s results
Navigation contains the Router, the search box and the home button. Again it’s initialisation occurs in the main.js file
This is the main router for the application. it dispatches events based on uri’s clicked. There is a view in Tabrouter.js also that handles the class toggling for the tabs.
Listens for events that indicate the search has started and finished. Displays a spinner
Show the search drop down that has the search button, predefined searches and the save search option. Extends the Combo View from elements
It’s pretty dumb and delegates responsibility for saving and running searches to SolrSearch modules
Elements is a module used to store re-usable elements.
get the csrf cookie for forms
date-helper.js - poorly named
these are all the handlebars helpers for our templates
helpers for forms, validation for fields and tranformation from jquery serializeArray format to one more digestible by backbone models. Set’s error fields on forms also.
It would be nice to abstract form validation and widgets into a library that could be used for creating rich backbone forms
view-close.js - again poorly named, this adds a destroy method to all views, to help prevent zombie views and a build up of detached DOM elements.
CollectionViews are used to display lists of things, those defined by foreign key fields on models
Model render paging for list views( in SolrSearch ) and CRUD (embedded search)
Combo box view
used on forms for showing a date time range
used for saving searches
View for the search text field
This could probably be simplified,
This is used generally with a combo box in form views. It displays an autocomplete text field, the selected fields, and manages a hidden multiple select form field to store the selections to be saved.