Large framework agreements with many lots and many winners are not manageable (technically and professionally)

[JanEmde]

A german awarding authority has awarded a framework contract with 34 lots. Each lot was awarded to around 100 bidders. A total of 159 organizations (including the contracting authority and review organization) were involved in the procedure. Most bidders have applied for (almost) all lots. As a result, approximately 3500 tenders had to be included in the result notice. Due to the huge number of elements (and references) the notice is simply not manageable (technically and professionally).

Further such notices will be awarded in the future.

Any ideas to fix this?

[julemari]

Good afternoon,
The eSenders team has already replied to your email regarding this issue. For ease of reference, please see our reply here as well:

We understand the challenges you're facing when submitting large Result notices involving multiple lots, organisations, and tenderers. We’ve since carried out a detailed analysis of the validation process for such large Result notices (subtype 30), and improvements are being implemented in the next SDK release (version 1.14).
These improvements specifically target performance bottlenecks related to rule evaluation in large submissions with many elements. While the changes won’t affect all notice types, we expect a noticeable reduction in validation time for subtype 30 notices like the ones you’ve provided.
Regarding the question on splitting notices: while this is not a legally required approach, it is a technical recommendation aimed at reducing processing complexity and ensuring timely submission. The applicable regulation does not prohibit splitting Result notices by lots, even when the awards occur simultaneously. This method is already used in practice when contracts are awarded at different times.
We will continue to monitor performance and remain committed to improving the stability of large submissions.
We hope this update is helpful and appreciate your continued cooperation.

[JanEmde]

Thank you for your response. I would like to clarify that the challenges do not arise during submitting, but during acceptance by TED.

If the SDK 1.14. does not bring the necessary improvements, would it be an option to split the submitted notices directly at TED, validate them and then merge them again? Otherwise, clear instructions would be needed on how and on the basis of which parameters the notices must be split up. Trial and error for the submitting body is unacceptable in our view.

[ferrakaop]

The API to submit notices also calls the CVS to validation the notice because it can be accepted for publication on TED - so the timeout is happening at validation, even if notice came through the submit API.

We are talking about exceptional cases, which perhaps happen only a dozen times per year (out of 800000 notices) when particularly complicated procedures are awarded. We do not advise public buyers on how they build their procedures and there is nothing to stop them from carrying out the same procurement in more than one procedure (and therefore using more than one notice).

If they nevertheless choose to publish a big competition with all lots all in the same CN notice, then they may need to prepare to award the competition in more than one Result notice.

The simplest option is to just publish one CAN per lot - this will already be pretty big for anyone to understand who won the lot (in this extreme case I understand there are around 100 winners per lot). This option has always existed (even before eForms made notices bigger and with more rules) and it's also the normal way to work if lots happen to be awarded at different moments in time. There is no trial and error and there should be no surprises for the systems or for the business users.

[mdewinne]

The simplest option is to just publish one CAN per lot - this will already be pretty big for anyone to understand who won the lot (in this extreme case I understand there are around 100 winners per lot). This option has always existed (even before eForms made notices bigger and with more rules) and it's also the normal way to work if lots happen to be awarded at different moments in time. There is no trial and error and there should be no surprises for the systems or for the business users.

@ferrakaop Does this mean that award notices do not have to reference all the lots foreseen in the original contract notice? E.g. can one publish a contract notice with lots LOT-0001 and LOT-0002 and subsequently publish a contract award notice that only references e.g. LOT-0002 in both the Lot and LotResult sections, while LOT-0001 is completely absent from that award notice? Will the future dynamic validation rules accept this?

Until now, we always figured each CAN should reference all the lots of the CN (and this is also the way our national platform is set up). But this also requires the buyer to add a LotResult section for each of those lots. And then - with the example given above of publishing mutliple CANs to award different lots - one would be blocked by the current possible values for BT-142-LotResult.

Say one has already published a CAN "A" in which LOT-0001 was awarded, but LOT-0002 was marked as open-nw. Then, when you publish a CAN "B" to award (only) LOT-0002, you still have to give a LotResult for LOT-0001. But at that point in time, neither clos-nw nor open-nw are still correct for LOT-0001, because it got awarded in CAN "A", which is already published. However, selec-w isn't a viable option either, since it will force you to register all TPA, TEN and CON for that lot - which is exactly what you were trying to avoid by splitting your award into different CANs.

Sorry for the elaborate example, but it would really be good to have some clarity on this point, as we too have noticed that "large" CANs are difficult to handle, both for humans and machines (performance). So offering our users the possibility to create separate CANs per lot would be a viable option, but we can only do this if we are certain that the dynamic validation will continue to allow this in the future.

Edit: another option would be to add a 4th option to BT-142-LotResult, which would allow to indicate that the lot was already awarded in a previous CAN of the same procedure, e.g. clos-awa.

[JanEmde]

@ferrakaop @julemari I would like to take this discussion up again following our update to SDK 1.14.

We have repeated our tests with SDK 1.14 and, unfortunately, have not yet noticed any significant improvements in the duration of validation.

Are there plans to further improve performance in the future?

Also, I would like to know if there is an answer to @mdewinne s question?

Thank you in advance.

[julemari]

Dear @JanEmde, dear @mdewinne,

Regarding Michael's question: Indeed, in a CAN it is not necessary to refer to all the lots that were mentioned in the competition notice. So, if there are multiple lots in a competition notice, say with different deadlines, it is indeed possible to publish multiple CANs each just referring to the one lot that was recently awarded. We can confirm that we will not implement any dynamic rules checking that all lots referred to in a contract notice are referred to in every related CAN.

Jan, could you maybe give us some examples where validation takes too long?

[mdewinne]

Regarding the reasons why validation (at least with EFX) takes long, see #1259

[julemari]

To clarify my previous information, in the Result notice only the necessary information is expected.
The Lots for which information must be published:

• Lots for which competition is over since the previous Result publication, and
• Lots for which competition is ongoing when it is usual to publish this information

Each Lot of the competition must have at least one associated LotResult (more than one LotResult being only possible for a lot with a Framework Agreement or a DPS).
Regarding the tenders, the winning tenders for the awards of the result notice are mandatory, the non-winning tenders are optional.

[JanEmde]

@julemari Two sample notices:

XML_Form.xml
EForm_30_a48b7c94-dff1-4002-b872-91e1dc5b88ed.xml