1.0.0-beta.72 Big bada boom in data loading API

Updated data and metadata loading logic

The data loading logic in Discovery.js has been modified to consistently expect raw data and separate meta info, if provided. Previously, the Discovery.js app was designed to work exclusively with data generated by @discoveryjs/cli (either via a server or builder), which wrapped the data in an object { name, createdAt, elapsed, data: /* actual data */ } (combining actual data with meta information). As Discovery.js evolved, it gained the ability to load data from various data sources like url, file, stream, raw data, etc., which do not use the same wrapper. The differences in data structure made it challenging to create versatile apps that could load data from diverse sources in various ways. This issue became particularly significant with the addition of the embed API.

Starting with the latest release of @discoveryjs/cli, data is no longer wrapped, and meta info is provided differently. As a result, the Discovery.js app now handles data loading uniformly, regardless of the data source type.

To ensure compatibility with data using the old @discoveryjs/cli wrapper, code has been added to detect and unwrap the data during the transition period, allowing for a seamless experience.

Uniform load data methods

This release uniforms API shape responsible for data loading. This related to loadDataFrom* functions as well as App#loadDataFrom* methods. With the release now functions and methods have similar semantics, with a single exception for loadDataFromPush() which has no subject to load from:

• loadDataFromUrl(url, options)
• loadDataFromEvent(event, options)
• loadDataFromFile(file, options)
• loadDataFromStream(stream, options)
• loadDataFromPush(options)

The options optional parameter an object which structure depends on a method. However, there are one common option resource, which specify metadata related to a loading dataset.

Changes

• Made internal navigation using anchors (<a> tags) work without router extension enabled. Use ignore-href class on an anchor to prevent navigation by href attribute if needed
• Added permalink action to generate current state of an app (absolute page URL by default), it can be overridden by embedding host
• Changed Widget#setPageHash() to make leading # optional, i.e. Widget#setPageHash('#report') and Widget#setPageHash('report') give the same result now
• Embed API:
  • Reworked internals for more efficient data transfer, e.g. transfer a data stream intead of tranfering data with chunks
  • Replaced App#loadData with App#uploadData(source, extractResourceMetadata) method, which takes source instead of a function returning a source
  • Added setRouterPreventLocationUpdate() method for preloader and app to prevent location updates (and browser's history as well) on internal navigation
  • Added setPageHash() method for preloader embed API
• Reworked loading data API and internals:
  • Added support for loading binary encoded data (jsonxl snapshot 9)
  • Added Widget#datasets and #.datasets to store loaded datasets (data itself and its meta data as an object). Current APIs assume to work with a single dataset (data), so this list contains zero or one element for now. However, future version will allow to load and work with multiple datasets (still need to think about the details)
  • Added dataset option in Widget#setData() method options to provide dataset's attributes, which is optional and a dataset object will has data field only if the option is not provided
  • Added start(resource) method to loadFromPush() API
  • Changed loading data logic to always expect raw data and meta info specified aside (via options) if provided (see release notes for details)
  • Changed preloader to pass loadDataOptions into all the loading methods as options
  • Changed Widget#setDataProgress(data, context, progressbar) signature into setDataProgress(data, context, options), where options is an object { dataset?, progressbar? }
  • Removed #.name and #.createdAt, use #.datasets[].resource | { name, createdAt } instead
  • Removed data from Windget#context, however, data is still available in a render context as data of default dataset, i.e. #.data which the same as #.datasets[].data
  • Removed Widget#dataLoaded and #.dataLoaded flags, use Widget#hasDatasets() method or #.datasets in jora queries
  • Changes in loadDataFromUrl() method:
    • Changed getContentSize option to not take url parameter anymore
    • Changed the default getContentSize handler to prefer X-File-Size header over Content-Length header
    • Added getContentCreatedAt option to specify a function to retrieve createdAt from a fetch's response (by default, retrieve a value from the X-File-Created-At header if present, or Last-Modified header otherwise)
    • Added getContentEncodedSize option
    • Fixed a potential crash on JSON parse for a bad response with application/json content type
    • Removed dataField option
    • Removed support for loading raw data using this function
• Removed Widget#lib