Future Plans

[rkrejci]

Here we will announce planned changes in libyang project. The discussion is locked to keep this thread clean. However, please open a separate issue if you have something to say to the announced plans. If you are interested in receiving notifications about new announcements, feel free to subscribe to the issue notification at the right panel.

[rkrejci]

libyang 2.x

You might notice that we are working on new libyang major version in libyang2 branch. I would like to clarify the reasons to rewrite libyang and how it is going to change.

Motivation

We are trying our best to fix the issues we find or which someone else reports via issue tracker. YANG is really complex language - all the ways how one part of a YANG schema can be interconnected, extended, changed or removed by another YANG schema are quite challenging for implementation. By the time many of the corner cases are being clarified by the NETMOD WG community and we are striving to reflect such changes in libyang as well as to fix or add handling of overlooked corner cases.

Unfortunately, the maintenance and bug fixing became difficult mainly because of some specific features complicating many other parts of libyang. Another observation was that libyang could be faster and code could be more simple (better maintainable) if the design of some of the structures would be different. But such changes are so complex that they cannot be done by iterative changes. So we have decided to rewrite libyang codebase. However, we are really satisfied with the overall design and various mechanisms in current libyang. We would like to keep the most of the usage approaches, so besides changing some of the parameters or structure members, it shouldn't be necessary to redesign the whole code using libyang.

Changes

I have mentioned, that there are some features we want to get rid of. We actually believe that they were not used frequently and that they can be replaced by a sequence of other steps. This is the case of removing (or disabling) modules in the context. Due to the interconnection between YANG modules, the process of removing YANG module is really complex and must be taken into consideration in many other parts of libyang. The common use case is to prepare context and then use it for work with YANG data, while no changes to the context are needed. And if it is necessary to remove some of the modules in the context, a new context with the required set of YANG modules can be created. Therefore, in libyang 2.x, it will not be possible to remove or disable a YANG module present in the context. Optionally, to save some memory space, it will not be possible to even add YANG schema to the context which will be explicitly marked as a complete, but this is just an option.

The second big change is about schema tree structures. Currently, libyang has just one type of schema tree structures used for all the purposes - for data validation and manipulation as well as for printing various schema formats. To speed up and simplify mainly the data validation and manipulation, we have decided to split schema structures into the two types - parsed and compiled schema tree. The parsed tree preserves the original schema structure from the input data. There are groupings, typedefs, augments, and so on but the references such as uses statement are not resolved (not even validated). This type of the schema tree is suitable for printing (transforming) schema tree formats and internally for resolving some of the references when the parsed schema tree is being compiled. In contrast, the compiled schema does not preserve the original structure of the schemas. There are no typedefs or groupings because they are just instantiated where they are referenced. Type chain is compiled into a single structure at the leaf or leaf-list where it is used with all the restrictions applicable to the data. So if you are using schema tree structure somehow, the API will change. On the other hand, all the relevant information should be newly available much more easily.

libyang already provides API for external plugins for data types and extensions. Both these APIs will change to allow implementation of the complex cases. The same API will be actually used even for internal implementation of YANG built-in types and selected YANG extensions (NACM or Metadata). In contrast to the current libyang, these internal plugins will be compiled as integral part of the binary instead of a standalone dynamic object files. We believe that it should simplify libyang deployment.

At the end we would like to provide the detailed description of the changes and how to migrate from libyang 1.x to 2.x. Despite we will do our best, it can be hardly complete as we don't know all the use cases you are aware of. Therefore, we will appreciate all your feedback.

[rkrejci]

I'm closing this issue because we have finally GitHub Discussions for this kind of communication :) Check #1309