Data Components Standard for Enterprise scale

Context

This document describes data handling method which needs to be applied on @fundamental-ngx/platform level in order to simplify development cycle for accessing the enterprise resources (data).

There could be many component on enterprise level that works with large set of data such as:

• Datatables
• InputSearch
• ComboBox (Autocomplete)
• Tree
• List
• and much more...

and for these we need to unify the way we access a data. Currenlty we leave most of the complex data access or data manipulation on application developer which leads to different implementation approaches and allot of code duplication.

It is common practice when working with different data input sources we try to apply some Design patterns e.g. Adapter Pattern that provides us with guidelines how different interfaces can talk to each other and from here we can derive another pattern so called Datasource pattern which is high-level architectural pattern to abstract data and internal resources access (DB calls, Rest calls, domain objects) with a mininal amount a code.

What we want to achive is to define abstract layer where each specific resource needs to implement and move all the related logic into reusable DataSources.

Current Solution

In normal scenario application developer usually do following e.g: :

Let's use component from a libray:

<w-autocomplete placeholder="Search..."  [list]="users" 
                                        (onSelect)="itemSelected($event)" 
                                        (onSearch)="doSearch($event)"
                                        [suggestions]="mySearchResult" 
                                        [mutliSelect]="true">

</w-autocomplete >

Plus implement all above handlers and much more:

@component({..})
class MySearchPage {

  users: User[]; 
  
   constructor(private userService: UserRestAPI) {
   }

   ngInit() {
     this.userService.load().subscribe((data)=> {this.users = data})
   }
    
   doSearch($event) {
     // another rest api call
     this.userService.find().subscribe((data)=> {this.suggestion = data})
  } 

  // much more logic here
   
}

• We are introducing a complexity into MySearchPage component to do different data manipulation.

  • On application level you don't really want to use users[] directly. * When developing our rest services we usually use rxjs to subcribe and manipulate our data but what we don't do is; in most of the cases we dont unsubscribe and release any open subsription.
  • Developers needs to start adding unessesary logic to handle selection, suggestions, fetch or any CRUD based operation.

• If we look over several view we are duplicating allot of code.

• When we use detection strategy onPush, we also have to trigger detect changes so our view is aware that our data changed and view needs to be refreshed.

* When using observable instead passing arrays directly, we let Angular handler all the updates, subsriptions for us. It also nicelly work with change detections.

Proposed Solution

Instead of duplicating all this functionality we need to aggregate and extract as much as common functionality into DataSources something we used already 15 years ago. These approches does not change over time, the only thing that can change how we implement all this.

For the following chapter I will use as example Material Design (CDK) as they took similar approach.

The whole idea is to hide as much as common logic into this Adapter like class and not to worry about this on the application level otherwise we are not simplifiying we are introducing more problems.

Let's have a common Interface we can call:

export interface DataSource<T> {
   open(): Observable<T[]>;

  close();
}

Observable<T[]> is our reactive stream, that changes its content based on what it pushed into it using .next().

Each type of data component such Autocomplete, DataTable, List,.. needs to provide its own specific implementation to handle specific usecase. Let's see this pseudo code (based on cdk)

export abstract class DataTableDataSource<T> implements DataSource<T> {
  data: T[];
  filterTerm: string;
  paginator: Paginator | null

  filterPredicate: ((data: T, filter: string) => boolean;  

  sortData: ((data: T[], sort: Sort) => T[]);

  //
  create (t: T): T;
  update (t: T): T;
  remove (t: T): T;   
   
}

// Autocomplete, choosers,..
export abstract class ChoiceDataSource<T> implements DataSource<T> {
  data: T[];
  filterTerm: string;
  resultLimit: number;

  filterPredicate: ((data: T, filter: string) => boolean;  
   
}

After we define some structure we can start creating concrete implementation based on the usage or resource

// Autocomplete, choosers,..
export abstract class RestChoiceDataSource<T> extend DataChoiceDataSource <T> {
  // We can use a registry that is able to map a Entity Type = Endpoint
  // have identical way how to fetch , search ,...   
}

Then on application level we dont really need to do all this heavy lifting and just use it like this:

<w-autocomplete placeholder="Search..."  [dataSource]="userSource">
</w-autocomplete >

All these tasks which we had to implement every time we accessed some data (search triggering, input monitoring) would happen internally (hidden) and there would be clear interface between data component and DataSource. Then our component can be simplified to:

@component({..})
class MySearchPage {

  userSource: ChoiceDataSource; 
 
   constructor(private registery: DataRegistry) {
   }
   ngInit() {
     this.userSource = registry.queryDataSource(User, DataSource.Layer.REST);
   }       
}

Maybe we don't really have to deal with dataSource in the view, let the component to get its data directly.

• Again you can have your own instance of DataSource or you would use Common one
• You could provide API for DataSource Registry, where on application level we would configure a mappings
  • If you work with Entity User -> Use -> this DataSource,
  • For Invoice -> use -> this DataSource
  • etc...

And you can end up with something like this:

<w-autocomplete placeholder="Search..."  [type]="dataType" (onSelection)="OnlyInterestedOnSelection($event)">
</w-autocomplete >

We need to ask:

• What is the minimum amount of code I need to write to get something working ?
• How to provide components which can boost productity ?