Reconsider semver

[kisvegabor]

Introduce the problem

One of the main critics about LVGL is the heavy breaking changes on major version. It makes migrating UIs to the new versions difficult as there are too many changes. It's bad for the developers because they might stick with an older version and bad for us too because we need to deal with the older versions

Suggested solution

Instead of strictly following semver we can allow to have simple breaking changes in minor versions too. E.g. I was thinking about changing

lv_obj_t * lv_tabview_create(lv_obj_t * parent, lv_dir_t tab_pos, lv_coord_t tab_size)

to

lv_obj_t * lv_tabview_create(lv_obj_t * parent);
void lv_tabview_set_tab_pos(lv_obj_t * obj, lv_dir_t tab_pos);
void lv_tabview_set_tab_size(lv_obj_t * obj, lv_coord_t tab_size);

It'd break the API so now I need to wait until v9 (~1 year). However users can easily updated their UI to this change.

Or we have realized that lv_obj_remove_event_dsc is conceptually incorrect, but we can't remove it for backward compatibility.

IMO small changes like these can be communicated very clearly in the CHANGELOG and user could update they UIs easily.

Major releases should be reserved for cases when the "philosophy" of LVGL changes. So major version would still contain breaking changes but not as much because some of these changes were already out in minor releases.

What do you think?

[embeddedt]

I think there is a double-edged sword here. I was having a conversation the other day with someone who was still using v7 on an older minor version. They were hesitant to upgrade, even to release/v7, because they thought they would need to make changes to their code and they weren't interested. When I told them that release/v7 was fully compatible with existing v7 projects, they were skeptical, but then impressed at how seamless that upgrade was.

TL;DR I am concerned that breaking changes on minor versions will only increase friction in the update process, not decrease it. Having to take time away from your actual project design to handle an upstream library's changes is annoying and inconvenient when it occurs frequently enough. The better strategy seems to be to make less breaking changes overall. However, this slows down architectural progress as well, so the whole situation is a bit of a trade-off.

Or we have realized that lv_obj_remove_event_dsc is conceptually incorrect, but we can't remove it for backward compatibility.

I am tempted to suggest replacing the body of lv_obj_remove_event_dsc with an unconditional assertion, because of how dangerous it is. Any user code relying on it is not going to get consistent behavior. I would go as far as removing it entirely except for the fact that it would cause issues when linking with old binaries. If we leave a stub which always asserts, intermediate projects like bindings to link with LVGL unchanged, and they will only see errors if they actually do make use of the function.

For "broken" functions like lv_obj_remove_event_dsc, I think it is fine to violate semver rules a little bit.

[kisvegabor]

I see your point. While I was reading about "issues with semver" I came across with ferver. It's very similar with this difference:

• major version: conceptual change, major API cahnge
• minor version: minor API breaks, complex bug fixes
• patch version: new non breaking features, trivial bugfixes

What do you think about it?

Other question: what if we find a typo in an API function. E.g. lv_obj_set_siez (should be "size"). Of course, we can provide a wrapper and keep both but is it really ideal? Is it really that evil to keep only the correct version in a minor version.

Following the changes with 3-4 minor version per year seems manageable to me. But it's only my opinion.

[kisvegabor]

One more example came to my mind. I was thinking about improving the calendar widget to make the calendar and the header more connected. The idea is to have structure like

• calendar container
  • empty container for the header
  • the calendar itself

This way the calendar could notify the header if its date changes and they could move together. For me it seems acceptable to do such change in a minor version. If someone has one-two calendars in their design he can easily update them.

[embeddedt]

The following perspective is based on discussions I've had, not mine alone, just to clarify.

Of course, we can provide a wrapper and keep both but is it really ideal? Is it really that evil to keep only the correct version in a minor version.

In the specific case of the typo, it's obvious how to fix it, and it's also one find-and-replace, so that is a valid point.

Following the changes with 3-4 minor version per year seems manageable to me. But it's only my opinion.

The issue is that not everyone wants to be updating their LVGL version that frequently. They are more focused on building their product. Updates are a secondary priority, from what I see. It is a lot easier for us, as core LVGL developers, to migrate projects, since we are involved in development and we are also familiar with the general design philosophy of the API. But for a hobbyist who builds a user interface with LVGL and then makes small iterations to it, having to update every few months is an extra job.

The general complaint here seems to be stemming from the overall amount of change that occurs between releases, not the number of releases. It seems that adjusting the release schedule to be more or less frequent does not really solve the problem.

• When there are more frequent releases, users do not keep up with them because they are too frequent and the new features are irrelevant to most existing projects. Case in point: Zephyr was still on 7.8, last time I checked, despite release/v7 theoretically being 100% compatible, and I assume they are not going to adopt v8 anytime soon because it would mean significant breakage for existing projects.
• When there are less frequent releases, users do not keep up with them because they are not always a drop-in replacement for the previous version. Additionally, I have heard a couple times on the forum that users cannot always update because the project has been tested for production on one version of LVGL, and updating it would require redoing those tests.

[kisvegabor]

IMO v8 is good enough for a longer time but it's certain that some minor changes will be required here and there. So what should we do?

1. Wait with all these minor breaking changes until v9, maybe accumulate them in a dev branch? (I wouldn't like to bring the dev branch again because it's so much simpler to work on a single branch)
2. Let adding these changes in minor versions.

When there are more frequent releases, users do not keep up with them because they are too frequent and the new features are irrelevant to most existing projects. [...]

I agree that having more meaningful releases is a better strategy. I've opened an issue at Zephyr about updating to v8 but it didn't get much traction. 🙁

When there are less frequent releases, users do not keep up with them because they are not always a drop-in replacement for the previous version. [...]

It also tells me that fewer releases are better.

---

What do you think about opening a vote about this question in the forum?

[kisvegabor]

An interesting related thing:
ferver mentions cases when a bugfix actually changes the behavior because user might rely on that bug. This PR falls exactly into this category.

I have no suggestions what to do with cases like this, I just wanted to point out this potential problem.

[embeddedt]

I'm still thinking about the various combinations/possibilities. We can definitely ask for opinions/conduct some type of poll on the forum, though it would be good to have a concrete list of options for people to vote for if we are doing a poll.

ferver mentions cases when a bugfix actually changes the behavior because user might rely on that bug.

From a version readability standpoint, I like the ferver approach. It's clear when there is a breaking change and when there isn't. The problem I foresee with a versioning system where breaking changes are more frequent and acceptable is that every time that happens, there will always be a segment of users who freeze on the version before the breaking change and then request that future bugfixes be backported. This seems to have happened with all three recent major versions (6, 7, and 8).

This PR falls exactly into this category.

True, although I would also expect the new, logical behavior as a developer, and be more understanding of a change like that in a patch release.

I wouldn't like to bring the dev branch again because it's so much simpler to work on a single branch

Yes, otherwise there is a whole headache with merging, especially when forgetting to apply bugfixes to the oldest version first.

---

Practically speaking, the only libraries which seem to have solved this problem are ones which are in maintenance mode and only receive bugfixes, not new features. 😆 Even in the web/Node.js community, major version bumps seem to be a point of friction.

[kisvegabor]

This seems to have happened with all three recent major versions (6, 7, and 8).

IMO the amount of changes is very important here. It matters if 3 functions for 2 widgets have changed compared to all style, event, align, layout declarations are broken and some widgets are removed.

---

What I learn so far from our discussion is that it doesn't really matter how we assign version numbers (assuming it's consistent) but 2 thing matter instead:

1. how often we add breaking changes
2. how many APIs we break then

An there are 2 options

1. semver: break rarely but a lot (at least in our case)
2. ferver: break less but more often (more often means 3-4 times in a year)

Do you think these can be the 2 options for a poll?

[embeddedt]

It matters if 3 functions for 2 widgets have changed compared to all style, event, align, layout declarations are broken and some widgets are removed.

I completely agree, although I'm not sure that all users make this distinction, particularly if it's not a project they spend a lot of time on.

What I learn so far from our discussion is that it doesn't really matter how we assign version numbers (assuming it's consistent) but 2 things matter instead

I agree.

And there are 2 options

There is one other angle we haven't touched upon yet: the number of versions we end up supporting.

1. semver - suppose we release v8.0, 8.1-8.3 are nearly identical, and a year later we release v9. This means we only need to remain familiar with v8.3 and v9 APIs for the following year. Providing v8.3 information to someone using v8.0 will work most of the time, unless we reference a feature which didn't exist in 8.0.
2. ferver - we release v8.0, v8.1, v8.2, and v8.3, each with minor API tweaks. That is a worst-case scenario, but bear with me. We then release v9 with at least one subsystem majorly changed. There are now 5 variations of APIs to remember.

In the second case, should users be told that they will receive v8.3 information, and be expected to backport it to previous minor versions themselves if needed?

Also, the current minor version system allows bugfixes from 8.1 to be cherry-picked to 8.0 automatically in most cases. With API renames, there will be more conflicts and manual applying required.

I can see why a lot of projects end up using an LTS system and not providing much support for intermediary versions.

[kisvegabor]

Phew, gets complicated. 😆

ferver - we release v8.0, v8.1, v8.2, and v8.3, each with minor API tweaks. That is a worst-case scenario, but bear with me. We then release v9 with at least one subsystem majorly changed. There are now 5 variations of APIs to remember.

You are right, it'd be a hell.

Let me bring an other practical example again: the calendar header issue mentioned here. What shall we do with that? There is a widget with "strange" and buggy structure. There is a good way to fix it but it's slightly change how the calendar is used. Waiting for 1 year for fixes like this doesn't seem ideal. Fixing it also causes problems.

What if make only 2 minor releases in a year (like Ubuntu)? One of them can be LTS?
If the time is there we release major version instead of minor.

[lvgl-bot]

This issue is stale because it has been open 14 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[embeddedt]

What if make only 2 minor releases in a year (like Ubuntu)? One of them can be LTS?
If the time is there we release major version instead of minor.

It's a tough decision. That's essentially what we did with v6 (only 6.0 and 6.1, followed by 7.0). A lot of users ended up staying on v6 for quite a while after v7 was released.

Another option is to try and version different parts of LVGL separately, e.g. have multiple versions of the calendar widget that are each compatible with a certain core LVGL version. However, I suspect this would just create a bigger nightmare when it comes to releases and CI.

[kisvegabor]

Makes sense.

Another option is to try and version different parts of LVGL separately, e.g. have multiple versions of the calendar widget that are each compatible with a certain core LVGL version. However, I suspect this would just create a bigger nightmare when it comes to releases and C

Would it be something like this?

#define LV_API_COMPATIBILITY 80000 //8.0.0
#define LV_API_COMPATIBILITY 80317 //8.3.17

#if LV_API_COMPATIBILITY >= 80000 && LV_API_COMPATIBILITY < 80099  /*8.0.0 - 8.0.99*/
lv_obj_t * lv_calendar_create(lv_obj_t * calendar);
#if LV_API_COMPATIBILITY >= 80100 && LV_API_COMPATIBILITY < 80199  /*8.1.0 - 8.1.99*/
lv_obj_t * lv_calendar_create(lv_obj_t * calendar, char * param);
#else /*Latest*/
lv_obj_t * lv_calendar_create(lv_obj_t * calendar, char * param1 , char * param2);
#endif

[embeddedt]

Not quite. I was thinking that the calendar would be versioned and released as though it's a separate, downstream project. For example, lv_calendar v2, v3, and v4 (and any minor versions) could all be compatible with LVGL 8.x. I think that would be manageable since keeping breaking changes out of the core is a lot easier. Then perhaps lv_calendar v5 would require LVGL 9.x.

I'm concerned that supporting multiple LVGL versions in one widget file would just create too much complexity and then also require a lot of regression tests to make sure all versions keep working. It'd be a similar amount of work to keeping working, identically-behaved stubs for every v7 API in LVGL v8.

The only parts of an LVGL project that seem to stay largely unchanged across versions are the display and touch drivers. 😆 Mine have only needed very minor tweaking since v6.

[kisvegabor]

For example, lv_calendar v2, v3, and v4 (and any minor versions) could all be compatible with LVGL 8.x.

Oh, I see. IMO it'd be quite messy and counter-intuitive if the widgets were versioned independently from LVGL. E.g. Calendar v3 is for lvgl v8.1, or v8.2?

I'm concerned that supporting multiple LVGL versions in one widget file would just create too much complexity and then also require a lot of regression tests to make sure all versions keep working.

Maybe we could organize the files in a reasonable way but the tests, maintenance, support, etc would be really a nightmare.

In a PR we said that it's ok to add a minor breaking changes to the Wayland driver because probably it's not used by many people and the changes were easy to follow. What could be a similar case in lvgl? We have mentioned that there is no problem with fixing minor typos. But can we break a marginal widget's marginal feature? I mean where is the limit?

The only parts of an LVGL project that seem to stay largely unchanged across versions are the display and touch drivers. laughing Mine have only needed very minor tweaking since v6.

I already have some ideas how to break it 😆
Maybe lv_disp_drv_t, lv_disp_t and lv_disp_buf_t can be joined into one structure, e.g. to have only lv_disp_t. But it's an other story and definitely not for v8.

BTW, I'm really happy with v8. There are some minor things here and there but I think we reach a good point on which LVGL is really usable and flexible.

[kisvegabor]

I came across with this CHANGELOG

So 9.5.0 had a breaking change on "Lights using MQTT discover", i.e. on one module's one feature. For me, as a developer, it seems acceptable as the change is clearly communicated and a replacement is offered.

[embeddedt]

Do you know how often they do releases? This approach definitely works if releases are infrequent. However, if they are frequent enough, a developer could end up skipping 6 releases, and they probably won't be too pleased or even bother to read through 6 changelogs to find all the breaking changes.

---

Something else to consider is that we could invest more time & resources in a proper compatibility shim. This would reduce upgrade friction and therefore the fragmentation of LVGL versions in the wild. For example, it wouldn't have been that infeasible from a technical standpoint to make a semi-automatic code transform that rewrote at least some v6 style code to work with v7 styles based on my experiments with that.

I'm sure there were other breaking changes as well (it's getting hard to remember v6 and v7 now 😆) but I think styles were probably the biggest hurdle to rapid adoption.

With shims/code transforms in place, we could make more breaking changes, as they wouldn't mean as much of a hassle for users. Thus far, it definitely seems easier to get developers to at least try the newer minor version when they can swap it in without much work on their part.

[kisvegabor]

Do you know how often they do releases?

In every ~2 month. See here

Something else to consider is that we could invest more time & resources in a proper compatibility shim.

It should definitely work for minor changes. What do you think about making an experiment with the calendar?

---

After that what do you think about releasing v8.1?
We should wait for the Rt-Thread and Bitbake updates too.

[mysterywolf]

Hi, @kisvegabor
RT-Thread community has complete the basic port, and we are pushing on some advanced features, such as RT-Thread LCD and touch screen device driver port to LVGL. You could release V8.1, and don't need to wait us. We can put this advanced features in V8.2

[kisvegabor]

Great, thank you for the update @mysterywolf .

[embeddedt]

@kisvegabor How about we try 8.1 with the calendar breaking change, since it's unlikely that there will be too many calendars in one application, and see how the community reacts?

[kisvegabor]

Sounds good, let's consider it as an experiment.

[kisvegabor]

I updated the calendar.

[kisvegabor]

What about making the release next Wednesday (3 November)?

[kisvegabor]

I've added an auto-changelog generator script. It adds only the changes since the last version tag.
6d95521

[embeddedt]

Looks good!

Sorry for my inactivity recently. I agree with releasing soon but it'd likely be best to look at the outstanding PRs (e.g. #2761 can go into v8.1, it should be a harmless change).

[kisvegabor]

No problem :)

it'd likely be best to look at the outstanding PRs

Ok, I go through them.

[kisvegabor]

I'm planning to release v8.1 tomorrow.

[kisvegabor]

Released. I'm releasing lv_drivers and updating some projects now too.

Once they are ready I'll write a forum announcement.

The autogenerated changelog is very long. We definitely need more frequent patch releases (monthly?) to keep to the number of changes digestible.