I would like to re implement the default file system based templates store from the ground up to achieve the following goals:
Hierarchy support
User can create directory hierarchies in the data folder and group entities together. Technically a directory will be also an entity which store expose and each entity will have an optional reference to a parent. The API stays flat and this will by design work in other stores without further modifications. Only the studio need to reflect this.
The entities' config.json will additionally include entitySet property to identify its kind. The directory itself can also contain config.json with additional metadata like tags or premission.
Easier assets and relative paths
The data folder can have at any place a random file like jquery.js. Such files will be interpreted as assets and visible at the proper place also in the studio UI. Such files won't have metadata like permissions because it won't have a config.json.
Referencing such file in html/pdf should be very easy with base tag extension. The developer can simply add
<script src="myCustomLibs/jquery-1.8.1/jquery.js"></script> and it should just work.
Or use the assets syntax like
<script>{#asset myCustomLibs/jquery-1.8.1/jquery.js}</script>
There was a proposal to make the paths relative to the particular template being rendered. This means that the base tag would resolve its root based on the template being rendered. This could be configurable in base extension options.
This will nicely work for fs-store. But the path resolved by base tag won't work in other stores:
<script src="file:///jsreport-app-path/myCustomLibs/jquery-1.8.1/jquery.js"></script>
We will likely have to rather serve the assets through a web server links. Until then, the assets syntax should always work, because the assets extension should be able to locate the particular content based on the hierarchy.
Support for AWS S3 and Azure blob storage
Running mongo or sql is often an overkill for few templates and in the cloud with multiple servers, you currently don't have a better option. I would like to provide bindings also for common AWS S3 and Azure Blob Storage which are basically a 0 cost and gives the very easy maintenance. This would be reached with the following architecture.
The first layer of the templates store would be in the jsreport-core and would be just an interface.
The second layer would work just in the memory on arrays and support mongo like queries using one of existing libs.
The third layer would differ for file system and AWS S3. The file system would be able to load the entities at the startup and monitor the underlying changes to keep them up to date in the memory from which the second layer servers the queries. The AWS S3 would on the other bind insert, update and remove to the remote S3 storage. The initial start would also load everything and provide in memory representation to the second layer.
This would be very easy to set up for the cloud. You just need a S3 or Azure Blob Storage endpoint. However if you want to run multiple instances on the same underlying template, you also need to provide an endpoint to AWS Pub/Sub Azure SB service. These would be used to propagate changes between servers.
We also received a request to use fully git on the behind for stored templates. I don't take this into consideration at this moment because I think that everyone can store our `data` folder into git repository and maintain commits on its own.
I try to make a prototype of this using node 8 and come back here afterwards. However it would be great if some kind of discussion happens here so we go the right track with it.
I would like to re implement the default file system based templates store from the ground up to achieve the following goals:
Hierarchy support
User can create directory hierarchies in the
datafolder and group entities together. Technically a directory will be also an entity which store expose and each entity will have an optional reference to a parent. The API stays flat and this will by design work in other stores without further modifications. Only the studio need to reflect this.The entities'
config.jsonwill additionally includeentitySetproperty to identify its kind. The directory itself can also containconfig.jsonwith additional metadata liketagsorpremission.Easier assets and relative paths
The
datafolder can have at any place a random file likejquery.js. Such files will be interpreted asassetsand visible at the proper place also in the studio UI. Such files won't have metadata like permissions because it won't have aconfig.json.Referencing such file in html/pdf should be very easy with base tag extension. The developer can simply add
<script src="myCustomLibs/jquery-1.8.1/jquery.js"></script>and it should just work.Or use the
assetssyntax like<script>{#asset myCustomLibs/jquery-1.8.1/jquery.js}</script>There was a proposal to make the paths relative to the particular template being rendered. This means that the base tag would resolve its root based on the template being rendered. This could be configurable in
baseextension options.This will nicely work for
fs-store. But the path resolved by base tag won't work in other stores:<script src="file:///jsreport-app-path/myCustomLibs/jquery-1.8.1/jquery.js"></script>We will likely have to rather serve the assets through a web server links. Until then, the assets syntax should always work, because the assets extension should be able to locate the particular content based on the hierarchy.
Support for AWS S3 and Azure blob storage
Running mongo or sql is often an overkill for few templates and in the cloud with multiple servers, you currently don't have a better option. I would like to provide bindings also for common AWS S3 and Azure Blob Storage which are basically a 0 cost and gives the very easy maintenance. This would be reached with the following architecture.
The first layer of the templates store would be in the
jsreport-coreand would be just an interface.The second layer would work just in the memory on arrays and support mongo like queries using one of existing libs.
The third layer would differ for file system and AWS S3. The file system would be able to load the entities at the startup and monitor the underlying changes to keep them up to date in the memory from which the second layer servers the queries. The AWS S3 would on the other bind
insert,updateandremoveto the remote S3 storage. The initial start would also load everything and provide in memory representation to the second layer.This would be very easy to set up for the cloud. You just need a S3 or Azure Blob Storage endpoint. However if you want to run multiple instances on the same underlying template, you also need to provide an endpoint to AWS Pub/Sub Azure SB service. These would be used to propagate changes between servers.
We also received a request to use fully git on the behind for stored templates. I don't take this into consideration at this moment because I think that everyone can store our `data` folder into git repository and maintain commits on its own.
I try to make a prototype of this using node 8 and come back here afterwards. However it would be great if some kind of discussion happens here so we go the right track with it.