Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data layer and route-driven control [WIP] #12

wants to merge 80 commits into
base: master


Copy link

commented Aug 28, 2019

This PR is a major refactoring of frontend's Elm code.

Data Layer


  • All data fetching from the API is now handled through a central data layer, implemented in the modules Data.*. The data layer is part of the Apps global model.
  • This is in contrast to the former architecture, where each UI component handles API requests and the corresponding data in its own model.
  • Motivation for the new approach:
    • The data structure is generally not in-line with the component structure.
    • Reification of the data needs leads to a cleaner effect (i.e. Cmd) handling.
    • Caching data avoids unnecessary round trips.
    • Simpler model of the app state with a clear separation of API data, route-derived data and local component data.
    • A clear separation of API data, route-derived data and local component data leads to a simpler overall app state, which is much easier to reason about.


  • All common types used for representating the managed entities are defined in Data.Types.
  • The data layer caches all incoming data (module Data.Cache). A repeated query of the same data is thus avoided.
  • Currently the cache stores these entities: rootFolderIds, folders, subfolderIds, nodeTypes, documents, documentsPages, folderCounts.
  • These stores are (mostly) dictionaries from a key (like FolderId) to a RemoteData type representing the requested or received data (like Folder), or some error description.
  • The consuming app components have to deal with the different states that a RemoteData e a can have (NotAsked, Loading, Failure e, Success a).
  • The app registers its current data requirements in terms of the type Needs, which can currently be: NeedRootFolderIds, NeedSubfolders, NeedGenericNode, NeedDocument, NeedDocumentsPage, NeedFolderCounts, as well a set of needs (either to be handled in parallel or sequentially).
  • There is not a strict one-to-one relationship between needs and entity stores. For example, NeedSubfolders will request data that goes into three different stores: folders, subfolderIds and nodeTypes.
  • The function Data.Cache.requestNeeds is responsible for generating API requests for fulfilling the data needs that are not satisfied by the cache yet.

Route handling and URL design

  • The URL now represents most of the relevant app state (apart from the actual API data of course).

  • The route consists of a path and additional parameters.

    • The path can either represent zero, one or two node ids. A single id may reference either a folder or a document. Two ids reference a folder and a document. Zero ids reference the root folder.
    • The paramters are used to represent search settings, filters settings and pagination state.
    • Currently the parameter names are: fts-term, fts-sorting, filter-by-year, filter-by-title, offset, limit.
    • Parameters can be omitted if they have the default value (like offset=0 and limit=10)
    type alias Route =
        { path : RoutePath
        , parameters : RouteParameters
    type RoutePath
        = NoId
        | OneId NodeId
        | TwoIds NodeId NodeId
    type alias RouteParameters =
        { ftsTerm : String
        , ftsSorting : FtsSorting
        , filterByYear : Maybe (Range Int)
        , filterByTitle : Set String
        , offset : Int
        , limit : Int
  • Most of the current UI state is derived from the route, and most user actions are reflected in a changed route.

  • The type Presentation represents the current UI state. Its value is derived from the current route, using also the current cache to get data about the root nodes and the node types referenced in the path.

    type Presentation
      = GenericPresentation (Maybe ( NodeId, Maybe NodeId ))
      | CollectionPresentation FolderId
      | DocumentPresentation (Maybe FolderId) DocumentId
      | DocumentsPagePresentation Selection Window
  • The current Presentation controls the display of the Tree component as well as the corresponding Article.* component.

  • User actions in a UI compoment may result in a Navigation, that in turn leads to a modification of the current route.

  • Apart from the route and the cache there is few state that is modelled directly in the UI components, like expansion state in the tree component, or the controls for editing an attribute of a document.

Unit tests

  • We started to add unit tests for several Elm functions, using the elm-test framework and the console test runner.
  • The tests currently cover URL parsing and building, functions to define an odering on several types (that are not comparable by Elm's defaults), and some utility functions.
  • We use a combination of specific test cases and property-based testing (known as fuzzy testing in elm-test).
  • The tests can be run by npm run test or npm run test-watch.
ThomasWeiser added 30 commits Aug 8, 2019
The Route type represents only valid routes.
Possibly invalid routes should be represented as a `Maybe Route`.
Use type definitions from Data.Types
All Elm types used in the data layer are defined in the common module Data.Types
Get needed data in cache
All needs for requesting API data are represented as a value of the type Need
Add dummy test modules
This way elm-test can be used for compiling during development of the corresponding modules.
Use a recursive type for Needs
That way we have only one type (Needs) that represents both atomic needs as well as aggregation of needs.
Extend cache to handle NeedGenericNode
And also remember the nodeTypes seen with any of the requests.
Fix handling of NeedGenericNode
If the generic node is a folder we have to make sure that
all subfolders along its lineage are known.
Use Sort.Dict in Cache, enabling non-comparable keys
We use the package rtfeldman/elm-sorter-experiment.
ThomasWeiser added 30 commits Aug 16, 2019
Refactor Filter and FilterEditor
Represent a date filter as
`FilterYearWithin (Range Int)` instead of
`FilterYearWithin String String`.

Define the `Range a` type.

FilterEditor now operates on `Filter.Controls`
instead of directly on `Filter`,
Don't hide filters on DocumentPresentation
The height of the controls view shouldn't change when switching presentations.
Rename Article.Empty to Article.Generic
This article shall be used for GenericPresentation,
which in turn is used if the given node ids are not in the cache yet,
or if the nodes are of inappropriate type.
Cache may return derived RemoteData
where the derivation may cause errors that are described as strings.
Fix and optimize caching of subfolderIds
- Bugfix: If querying subfolders result in an empty list of subfolder ids then we have to write that result in cache's subfolderId store. Previously only non-emtpy lists of subfolder ids were written.
- Optimization: If querying a folder results in a folder with a subfolder count of 0 then we should save that info in cache's subfolderId store as an empty list of subfolder ids.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
1 participant
You can’t perform that action at this time.