How do the SemVer releases govern API compatibility?

[akutz]

We have found, via customer issues, that Dataplane API does not adhere to semantic versioning, which dictates that no backward incompatible changes (including breaking changes regarding behavior) are allowed inside a major version. We shipped a product in late 2020/early 2021 with integration with Dataplane API v2.1.0 and in the last several days have noted:

• Breaking Change Querying a collection with a filter no longer returns HTTP 400 and has not since 382bf43.
• Breaking Change v2.6.0 deprecated the use of backend and frontend for filtering the results when querying server and bind collections.

We understand it is necessary to make changes to an API and its implementation, but that is what MAJOR version changes allow. As it is, we're in a tight spot because we are now unable to claim support for anythin newer than Dataplane API 2.1.0. And if we update our client bindings, which would require customers to upgrade the product we shipped that uses these bindings, will those same customers still be able to use older Dataplane API endpoints without also upgrading them?

I would humbly request:

• Someone from HAProxy to please comment on the official stance regarding Dataplane API's schema compatibility with respect to its versions?
• To consider reverting 382bf43 to return the behavior to what it was originally.
• To not remove features marked as deprecated until a subsequent, MAJOR version change.

Thank you!

[mjuraga]

Hi @akutz, thank you for your question. Since version 2.7 we're been following HAProxy versioning which is not governed by semantic versioning. Each release of the HAProxy will get an accompanying version of the Data Plane API.

That being said, we are very careful not to break backwards compatibility unless we consider a change to be a significant bug fix. This commit you are referring to is exactly that, but we missed marking it as a BUG/MAJOR in the commit message and with that we made a mistake and did not mention it as a breaking change.

As you correctly assumed, when we make a breaking change we try to deprecate things and warn about those in the Release Notes. Those deprecated changes will be removed in the next major release which will be 3.0, and following this cadence it will be in a year's time.

Moving forward we'll keep all deprecated changes for a minimum of the next 3 minor versions and then remove them.

Regarding your question to revert the commit 382bf43 unfortunately we cannot do that now, as it was a breaking change in 2.2, this would prove to be a breaking change again for any new users since that version if we were to revert it. What we can do to help you with your issue is we can add a configuration option for the Data Plane API configuration file which when set would return 400 status codes on non-existing parents for the collection resources and keep the old behavior. We can backport this all the way back to the 2.2 version, so you can safely migrate to a newer version.

[akutz]

Hi @mjuraga,

Thank you so very much for such a detailed and thoughtful reply -- it is much appreciated. Regarding the breaking changes, I understand.

What we can do to help you with your issue is we can add a configuration option for the Data Plane API configuration file which when set would return 400 status codes on non-existing parents for the collection resources and keep the old behavior.

Would a new build of the Dataplane API be required to apply this configuration option? Or does that option exist today? Either way, thank you for the out-of-the-box thinking! If the option is available today, how would we enable it? Otherwise, what do you need from me/us to get the ball rolling on this?

FWIW, there was one other breaking change that I did not mention above, because I've yet to RCA it. I posted the log here if that is something you could possibly take a look at and help me understand what changed. Essentially it looks like a method used to create a backend started requring new data in its payload:

• Dataplane API v2.1.0

  • Request

    2023/05/23 10:03:44 
    POST /v2/services/haproxy/configuration/servers?backend=backend&transaction_id=79ecc7f8-b81b-4d2c-a939-2b8429e6a5dc HTTP/1.1
    Host: localhost:30433
    User-Agent: LB API HAProxy DataPlane API Client
    Content-Length: 19
    Accept: application/json
    Authorization: Basic Y2xpZW50OmNlcnQ=
    Content-Type: application/json
    Accept-Encoding: gzip
    
    {"name":"backend"}
    

  • Response

    2023/05/23 10:03:44 
    HTTP/1.1 202 Accepted
    Content-Length: 43
    Content-Type: application/json
    Date: Tue, 23 May 2023 15:03:44 GMT
    Vary: Origin
    
    {"name":"backend","proxy-v2-options":null}
    

• Dataplane API master

  • Request

    2023/05/18 15:11:50 
    POST /v2/services/haproxy/configuration/servers?backend=backend&transaction_id=72c45798-a199-4857-b506-2b53604b2abf HTTP/1.1
    Host: localhost:15847
    User-Agent: LB API HAProxy DataPlane API Client
    Content-Length: 19
    Accept: application/json
    Authorization: Basic Y2xpZW50OmNlcnQ=
    Content-Type: application/json
    Accept-Encoding: gzip
    
    {"name":"backend"}
    

  • Response

    2023/05/18 15:11:50 
    HTTP/1.1 422 Unprocessable Entity
    Content-Length: 52
    Configuration-Version: 1
    Content-Type: application/json
    Date: Thu, 18 May 2023 20:11:50 GMT
    Vary: Origin
    
    {"code":602,"message":"address in body is required"}
    

This works fine with DP API 2.1, so something changed somewhere. Can you please help us understand if this is something a configuration option could also assuage, or if the change is more impactful than that? Thanks again!