How to make yang-js work with Mongo?

[quantang]

Hello,

I am trying to make Yang model work with my network topology data. As the network could be a big amount of data, I am not sure whether it is efficient or possible to load all of the data in memory. If I want to use Mongo as a store of Yang, how should I make them work together?

The yang-js supports events, I guess I can use them to write the data back to the database, such as on create/update/delete. However, we don't have something like a customized get function, which can load the data from the database when a property is required.

Do you have any idea or suggestion for this scenario?

Cheers,
Quan

[sekur]

@quantang - this is an excellent question... I've explored this possibility but it's a bit complicated. The problem is that Yang is primarily designed for schema validation for handling XML documents. This means that in order to process the XPATH filter rules over the underlying data model, you need to actually have the data at hand to perform the match logic (for leafref, must, etc.) conditionals.

For example, let's say you want to fetch a specific 'network' from a list of networks that contains a specific property value. Even if we fetch the network collection from the database, we would need to fetch all of them before we can filter out for only the matching networks. Otherwise, we would need to be able to offload the filter operation as part of the database transaction such that the results were already filtered according to the Yang conditionals before we load the result into the schema. This ultimately means that we need the interface between yang-js and a particular underlying database to become Yang aware such that we can dynamically construct the queries against the database to fulfill the Yang selector and filter logic before we fetch the result into the in-memory data model. Another challenge is that even if we were able to perform partial population of fetched data records, we would still need a mechanism to represent some form of placeholder list entries of unfetched data since we would also need to ensure that the actual count and number of element conditionals can remain satisfied. I was thinking of potential use of Promises to represent "unresolved" content but that also has its own challenges given that I'm currently making use of getter/setter operations for property access abstractions and that unfortunately doesn't handle asynchronous transactions. What that means is that we may need to convert every data model get/set operations to be async and that makes most of the typically simple interactions much more complex.

Ultimately, I believe this is achievable but it won't be quick & easy. I personally don't have the bandwidth to tackle this on my free time but open to consider taking it on if there's commercial licensing interest. Otherwise, I'd be glad to help review and support your development contributions on it. :-)

Peter

[sekur]

Another possibility is to provide support for binding async functions when you bind custom JS handlers around schema elements. Currently we can attach custom get/set synchronous functions on any arbitrary data model entity but perhaps we can extend the core engine to check if the bound function is using the async keyword (ES6) and treat those properties differently. Just a thought.