Collections
A large element of the Data Hub application involves showing lists of items, be they related records or search results, and providing a mechanism to page through them list, alter the sort order or apply filters.
The behaviour of listing and navigating these lists is common throughout the application so the collections framework aims to provide a generic method of doing this.
There are 3 main variations of collections throughout the application. The primary example can be seen when navigating to an area from the top level navigation. The diagram above shows examples of different uses of collections.
The primary collection pages use all elements of collections: filters, a collection header/summary, a list of entities and pagination. Each of these sections is generated by a set of common Nunjucks macros which create their markup based on data passed to them.
There are other uses of collection pages also. When using the global search from the homepage a list of results is shown, this uses the same collection code but does not include filters. When viewing a company or contact record collections are also used to display related data. In the case of a company record, selecting the contacts left tab will show a list of contacts within the company context, the same goes for interactions. These pages again use the common collection macros without filters. The contents of these sub lists normally differ slightly from the top level collection lists, this is done by providing different data to the same macros.
The 2 key elements to collection pages are transformers and nunjuck macros. The idea behind transformers is to simply turn one data structure into another. Within collections a transformer transforms the data that represents an entity, such as a contact, into a common format the Nunjucks macros can understand and render in the card format that represents list items.
The transformer reads the result from the api and uses it to build properties such as the total count, then calls upon a function to generate the parameters needed for pagination then finally is called with a list of 1 or more 'item' transformers that it calls, to which it calls in order for each result item.
The piece that is unique to each entity is the 'item' transformer, this describes how to turn the properties that make up an entity into the format that the nunjucks macros require.
Because you can use multiple transformers, and dictate the order they are called in, it is possible to create one transformer for an entity that transforms common elements to it and then have a follow up transformer add or modify the results of the first. In the case of contacts a single common transformer is created and then when used within the context of a company a 2nd transformer strips out the company meta item, as it is not required.
http://datahub.com/contacts?term=Fred&sort_order=name:asc
# Response
{
count: 100,
page: 1,
items: [{
id: '1234',
name: 'Fred Smith',
updated: 20170102TZ19:00:32:0000,
company: {
id: '123123',
name: 'Acme Corp'
},
telephone: '0123 321 12312',
email: 'fred@acme.org'
}...]
}
# Transformed item
{
id: '1234',
type: 'contact',
name: 'Fred Smith',
subTitle: {
value: 20170102TZ19:00:32:0000,
type: 'dateTime',
label: 'Last updated',
},
meta: [
{ label: 'Company', value: { id: '123123', name: 'Acme Corp' }}
{ label: 'Telephone', value: '0123 321 12312' },
{ label: 'Email', value: 'fred@acme.org', }
]
}
If you look at a result on the page you can see all results have a title area, an optional sub-title and then fields to describe that entity, which are called meta. A meta item needs a label and value, the type is optional but can be used to describe some items that require special formatting or handling. The current supported types are:
- date: Transforms a native date value into a human readable one, e.h. 20 Jan 1983
- dateMonthYear: Transforms a native date into a format that just shows month and year, e.g. January 2017
- dateTime: Transforms a native data time into a human readable format that includes the time, e.g. 20 Jan 1989 09:00am
- fromNow: Transforms a native date format into a human readable format relative to the current date and time, e.g. Tomorrow, Yesterday
- badge: Badges are displayed on the right side of a record with additional styling to make them stand out.
A further attribute url can be specified, and another called urlLabel. If url alone is passed then the value to be shown will be wrapped in a link. If both are passed then the original value is displayed followed by a link using the url and label.
A value can either be a string, a datetime value from the api or an object that in turn has at the very least a name property.
To fully understand what needs to go into a meta item take a look at 'meta-item.njk', this is the nunjucks macro used to render a meta item and you can see how decides how to render the item.
A note about transformApiResponseToSearchCollection and transformApiResponseToCollection. These two transformers are similar, in fact the search transformers uses the collection transformer and adds to it. It introduces a few features unique to search and filtering, such as highlighting a term and handling the idea of aggregations for filters, though this is not currently being used.
A collection of Nunjucks macros take the collection data passed to the page and use it to render the page content based on that data.
The diagram below highlights the components of collections on a page
For the top level pages, such as company list a common layout is used. Once the middleware and controllers have prepared the data them simply render '_layouts/collection.njk'. This layout renders the entire page.
# apps/companies/controllers/list.js
res.render('_layouts/collection', {
sortForm,
filtersFields,
selectedFilters,
title: 'Companies',
countLabel: 'company',
highlightTerm: get(selectedFilters, 'name.valueLabel'),
actionButtons: [{
label: 'Add company',
url: '/companies/add-step-1',
}],
})
In other situations, such as the contact list within a company, the controller needs to calls a view within the contacts sub-app and that in turn calls on individual macros to build the markup
# apps/companies/views/contacts.njk
% block main_grid_right_column %}
<h2 class="heading-medium">Contacts</h2>
{{ CollectionFilters({
query: QUERY,
filtersFields: filtersFields,
action: CURRENT_PATH
}) }}
{% block xhr_content %}
<article id="xhr-outlet">
{{ CollectionContent(contactResults | assign({
countLabel: 'contact',
sortForm: assign({}, sortForm, {action: CURRENT_PATH}),
selectedFilters: selectedFilters,
query: QUERY,
action: CURRENT_PATH,
actionButtons: actionButtons
})) }}
</article>
{% endblock %}
{% endblock %}
# apps/companies/views/interactions.njk
{% block main_grid_right_column %}
<h2 class="heading-medium">Company interactions</h2>
{% block xhr_content %}
<article id="xhr-outlet">
{{
CollectionContent(results | assignCopy({
countLabel: 'interaction',
sortForm: assign({}, sortForm, {action: CURRENT_PATH}),
actionButtons: actionButtons,
query: QUERY
}))
}}
</article>
{% endblock %}
{% endblock %}
The collection filters macro accept an object describing the fields within a filter form and a query object. The macro renders the form and uses the query object to populate it's state. Additional styles are added to have the form styled as a filter form, which differs slightly from entity forms.
The collection content macro is used to present the content element of a collection, consisting of the header/summary, the list of entities and pagination. This macro is most commonly used to produce lists within an entity, e.g. a list of contacts within a company
At the top of the collection content is a header to provide a summary of the contents. The summary is used to convey the number of items in a list, any filters applied, sort options available and any action buttons or links to carry out tasks such as reset filters or add a new entity.
Within the main body of a collection there is a list of entities, such as contacts or interactions. The macros that produce this section of the page simply loop through the result items and render each item based on the data from the item transformer
Finally the collection has a pagination component, this simply allows simple paged navigation and is a generic macro.
As you dive down the levels of collections you finally get to the 'entity' level. An entity is a record, an object, a thing of interest such as a contact or company.
An entity is split into a few elements:
- Title: The title is a link generated from the name property combined with the type and id.
- Sub-title: The sub title for an entity is optional, typically it contains the updated on value for a record though can be used for other reasons, such as project id in investments.
- Badge: One or more meta items with the badge property set. These items are extracted from the meta property and displayed in the top right of the record, or above the title when viewed on phones.
- Meta data: The main content area of an entity is the meta list. This displays a list of label/value pairs. This is rendered by 'meta-list.njk' and 'meta-item.njk'. Meta items can be a simple as simple as containing a 'label' and 'value' with strings set or they can be more complex and have an 'id:name' set on a value or have other properties indicating their data type, url or set them to display as badges. More details on the types can be found in the macros or the item transformer section earlier on this page.
The job of building a collection page is split into a few key stages, with each stage being served by a middleware function. The typical routing set-up involves:
Define query defaults -> Extract query/filter parameters -> Fetch/transform collection -> Render
This looks for values for some default values such as sort order, based on properties typically defined in constants.js. If values are not sent in the page request then a url is generated with the default values set and the user forwarded to that url. The aim of this is to set-up things like default sort order.
This middleware defines what request parameters are supported for filtering and sorting and extract them into a local value that can be used to form the basis of the parameters to send to the back-end API. When adding new filters you want to add them here.
This middleware takes the parameters extracted and builds the request to the back-end API, then passes the result through the API Response transformer and associated item transformers
The render at the very least calls the render function with the properties needed to generate a collection page (tbd). Additional if the page contains filters then the call to generate the filter and sort forms are done here, along with calls to get option data to include int he filter form.