Deprecate eligibilityCriteria and add exclusionGrounds

[jpmckinney]

Background

In OCDS, selection criteria (for example, the criteria used to qualify bidders in a selective procedure in the sense of the GPA) are often mapped to eligibilityCriteria.

However, selection criteria are not the same as eligibility criteria. For example, the UNDP has qualified but ineligible bidders, e.g. due to suspension or debarment:

… all vendors must be qualified, as well as eligible.

Eligible vendors are qualified vendors that have not been temporarily suspended or debarred by UNDP or another UN agency, fund or programme.

eligibilityCriteria was added in #139, and is described as:

A description of any eligibility criteria for potential suppliers.

At the time, awardCriteria was incorrectly termed selectionCriteria, which was fixed in #162; the confusion might have started in #140.

Proposal

An effective solution depends on how frequently eligibilityCriteria is used to disclose selection criteria (a very common concept in procurement) versus eligibility criteria (rare).

If the field is mainly used for selection criteria, then adding a new selectionCriteria field would create a situation in which there are two terms used to disclose the same concept.

If the field is rarely used, or only used for eligibility criteria, then we can add a new selectionCriteria field, and clearly explain the difference between the two.

We should also avoid the terms 'eligible' and 'eligibility' in all related guidance and extensions, unless that is actually what we mean.

[ColinMaudry]

In a nutshell:

• selection criteria are about selecting the suppliers that have:
  • the skills (e.g. with staff resumes)
  • the best proposal (this is evaluation/award criteria)
  • the experience to complete the contract (past projects)
  • certifications
  • minimum size (capital, revenue, number of employees)
• eligibility criteria are about allowing or forbidding suppliers to tender based on inherent characteristics (instead of their potential):
  • bad financial health
  • presence in some blacklist (national, international, specific to the procuring entity)
  • the lack of certifications
  • etc.

Eligible selected = qualified

Updated according to the next comment.

[jpmckinney]

I think size tends to fall into selection criteria.

For financial health, it depends. If it's something like "declared bankruptcy", it's an exclusion ground / eligibility criterion. If it's something like "minimum $1M revenue" (as a proxy for long-term stability), it's a selection criterion.

[ColinMaudry]

If I get it right, the purpose of this issue is to:

1. scan the docs and schema for misuses of "eligibility" and "selection"
2. assess the use of eligibilityCriteria for selection criteria

More generally, in 1.2 we must pave the way towards straight definitions and consistent structures (xxxCriteriaDetails + structured array of criteria) for all criteria in 2.0.

I'll start with 1. and we can discuss how to tackle 2. I understood the data team has a copy of all the OCDS data in the universe. We should be able to extract samples per publisher and analyse them.

[ColinMaudry]

In the schema, I believe the description of awardCriteria is problematic. It refers both to award criteria (assessment of the goods/services ✔️ ) and selection criteria (assessment of the tendering party ❌):

Any detailed or further information on the award or selection criteria

@jpmckinney Is removing "or selection" a breaking change?

[ColinMaudry]

I found no misuse of "eligibility" in the docs nor in the schema. It's actually mentioned only two times:

• in the description of the tender.eligibilityCriteria field
• in the description of the eligibilityCriteria document type

[jpmckinney]

I created #1071 for awardCriteriaDetails.

For this issue, let's add detail to the definition, with some examples, e.g. "For example, a supplier might be ineligible if they have been temporarily suspended or debarred by the buyer."

The issue also proposes adding a tender.selectionCriteria field.

To assess the use of eligibilityCriteria, please ask the Helpdesk to query the database to assess how many publishers use it, and to share some example values. I think @pindec might be best to do this.

[pindec]

I've shared Helpdesk's previous research on the topic with @ColinMaudry and will update with some more recent examples.

[ColinMaudry]

Analysis of the usage of eligibilityCriteria by @pindec: https://docs.google.com/spreadsheets/d/1WgXKTfaHQKrIhMdUMQuK_TAPx19T8DXhaLKTykSoeL0/edit?ts=5f71ec09#gid=386792540

My understanding is that eligibilityCriteria is somewhat used, but not much.

• Honduras seems to use it as a procurementMethodDetails
• Czech republic seems to use it as selection criteria respectively
• Taïwan: not clear
• Mexico INAI: line 82 is not clear
• Poland crams a lot in eligibilityCriteria, some if it fits, some doesn't
• France: mostly OK, but some of it seems to be selection criteria
• Spain: same

The other 11 are OK.

Conclusion: the publishers that use eligibility generally use it correctly.

Based on this analysis, in the scope of the 1.2 update, I suggest:

• we keep eligibilityCriteria semantics as-is
• we create a selectionCriteria text field

In 2.0 we I guess we would rename these text fields xxxDetails and add array fields.

[jpmckinney]

@ColinMaudry Sounds good! I suggest working on each of the two fields in separate PRs, to make them easier to merge.

[JachymHercher]

An effective solution depends on how frequently eligibilityCriteria is used to disclose selection criteria (a very common concept in procurement) versus eligibility criteria (rare).
[...] it's an exclusion ground / eligibility criterion. [...]

Three comments:

1. Assuming from the discussion above that eligibility criteria would in the EU be the same as the exclusion grounds, then those are very common - every procedure must have them. However, since they were (accidentally) not part of the TED 2.0.9 forms, data about them often wasn't published (and is instead stored only in the ESPD).

2. Within this issue, shall we also define eligibilityCriteria and selectionCriteria? I think we should, as "eligibility" is not clear enough on its own - it doesn't appear in the GPA, EU law, nor UNCITRAL model law.

   As far as I remember, the conceptual difference between "exclusion grounds" and "selection criteria" is that selection criteria are linked to the supplier's ability to perform the contract (turnover, experience, etc.), while exclusion grounds are not (e.g. money laundering, tax evasion, labor standards). I would base the definition on that. @ec-mcs could probably remind us whether that is correct.

   (There are also two "corollaries" stemming from the above:

   • Exclusion grounds are not set by the buyer, but by a higher administrative power (EU-level and state/regional level). Selection criteria are set per procedure and the buyer has full discretion in setting them.
   • Selection criteria are often used for the selection of a limited number of participants for the second stage, while exclusion grounds are not.

   For both of the corollaries, I think the EU has some exceptions though (i.e. in some cases there are buyer-specific exclusion grounds; and in some cases you can choose a limited number of candidates based on exclusion grounds), but I would have a tendency to treat those as jurisdiction-specific anomalies and not reflect them in OCDS.)

3. What about "conditions for participation" that, both conceptually and in EU law, don't really fit neatly in the exclusion grounds nor selection criteria category? In eForms, these are in Other Requirements (BG-705) and include "participation reserved for sheltered workshops / organisations pursuing a public service mission", "participation requires security clearance" as well as the, slightly different, "tenderer must disclose names and qualifications of staff".

   My tendency would be to just assume they fit in selectionCriteria and, if necessary, deal with them another day :-). (E.g. once the ESPD team figures out where they belong :-) .)

[jpmckinney]

Another option is, given the poor semantics and infrequent terminology of eligibilityCriteria, we simply deprecate it, and introduce exclusionGrounds and selectionCriteria with proper semantics (the latter is added in #1072).

As for conditions for participation, we similarly have an Other Requirements extension, which can change once it becomes clearer where these fit conceptually.

[JachymHercher]

Sounds good. GPA does mention "may exclude [...] on grounds" as one of the general "conditions for participation". (UNCITRAL talks only about (qualification) "criteria".)

[jpmckinney]

https://www.wto.org/english/docs_e/legal_e/rev-gpr-94_01_e.htm

Where there is supporting evidence, a Party, including its procuring entities, may exclude a supplier on grounds such as:
bankruptcy;

• false declarations;
• significant or persistent deficiencies in performance of any substantive requirement or obligation under a prior contract or contracts;
• final judgments in respect of serious crimes or other serious offences;
• professional misconduct or acts or omissions that adversely reflect on the commercial integrity of the supplier; or
• failure to pay taxes

[jpmckinney]

Some other language for inspiration: https://www.procurementjourney.scot/route-3/develop-documents/exclusion-selection-and-award-criteria/exclusion-criteria

[jpmckinney]

I'm splitting this issue as I suggested in #901 (comment) so that selection criteria are dealt with separately in #1120

This issue is about the fate of eligibilityCriteria: whether to change its description or deprecate it in favor of exclusionGrounds. The comments starting from #901 (comment) offer options or what language to use in the description.

---

Regarding "conditions for participation," the GPA article with that title discusses both selection criteria (e.g. legal and financial capacities, commercial and technical abilities) and exclusion grounds.

UNCITRAL Article 9. Qualifications of suppliers and contractors also has a mix of the two:

(a) That they have the necessary professional, technical and environmental qualifications, professional and technical competence, financial resources, equipment and other physical facilities, managerial capability, reliability, experience and personnel to perform the procurement contract;
(b) That they meet ethical and other standards applicable in this State;
(c) That they have the legal capacity to enter into the procurement contract;
(d) That they are not insolvent, in receivership, bankrupt or being wound up, their affairs are not being administered by a court or a judicial officer, their business activities have not been suspended and they are not the subject of legal proceedings for any of the foregoing;
(e) That they have fulfilled their obligations to pay taxes and social security contributions in this State;

Since the two are mixed in these documents, there's a possibility that some jurisdictions also mix them. The EU separates them into selection criteria and exclusion grounds, but I don't know that this distinction exists everywhere.

As such, we might need a common model, e.g. an array of participationConditions or participationCriteria, where there's the option to classify/categorize the criterion in order to preserve the distinction where one exists.

So, we should first decide on that last question, and if we chose to have separate fields, then we can continue with the above approach before the horizontal line.

cc @ColinMaudry

[ColinMaudry]

In https://www.procurementjourney.scot/route-3/develop-documents/exclusion-selection-and-award-criteria/exclusion-criteria, bankruptcy is in the "may ask" column, thus less typical than "criminal convictions". I'll update my proposal.

[ColinMaudry]

Closed with #1296

[JachymHercher]

Regarding document types, I think we should:

• deprecate "eligibilityCriteria" in favor of "exclusionGrounds", mirroring the fate of Tender.eligibilityCriteria
• after assessing its actual usage, possibly updating the description of "evaluationReports" to explicitly mention the exclusion and selection phases (+ the evaluation of the bids?).
• create a "selectionCriteria" document type

@ColinMaudry I don't see this being resolved as part of the PR closing this issue, has this been done somewhere else? Just to make sure it doesn't fall through the cracks.

[ColinMaudry]

@JachymHercher Right, I forgot to take care of the document types! Thanks!

[JachymHercher]

The PR is ready.

1. The following seems straightforward:

   deprecate "eligibilityCriteria" in favor of "exclusionGrounds", mirroring the fate of Tender.eligibilityCriteria
   create a "selectionCriteria" document type

   In both cases I have reused the wording used in exclusionGrounds and selectionCriteria.

   (Note: selectionCriteria speaks about "tenderers", but should probably be about "potential tenderers"?)

2. The description of selectionCriteria is

   The minimum requirements for tenderers to participate in the contracting process. Selection criteria ensure that a tenderer has the legal and financial capacities and the technical and professional abilities to perform the contract to be awarded. More structured information can be provided using the selection criteria extension.

   while we also have codes for

   'economicSelectionCriteria'

   Documentation that describes the economic selection criteria.

   and 'technicalSelectionCriteria'.

   Documentation that describes the technical selection criteria.

   I am a bit surprised that someone would have documents discussing these separately. Are these fields used? If yes, I would suggest minor rewording to bring them in line with the description of selectionCriteria.

   Documentation that describes the minimum economic, financial and legal requirements for potential tenderers to participate in the contracting process.

   Documentation that describes the minimum technical and professional requirements for potential tenderers to participate in the contracting process.

   (Alternatively, we could break this into more (e.g. four) subtypes of documents, but I would do that only once there is specific user demand.)

3. after assessing its actual usage, possibly updating the description of "evaluationReports" to explicitly mention the exclusion and selection phases (+ the evaluation of the bids?).

   This could be more complicated, so I've opened up a new issue in documentType: clarify 'evaluationCriteria' and 'evaluationReports', add new codes #1463.

[jpmckinney]

I am a bit surprised that someone would have documents discussing these separately. Are these fields used? If yes, I would suggest minor rewording to bring them in line with the description of selectionCriteria.

You might be right. This is based on the EU standard forms having checkboxes for "Selection criteria as stated in the procurement documents" and "Selection criteria as stated in the procurement documents". https://standard.open-contracting.org/profiles/eu/latest/en/forms/F02/#section-iii

I imagine most buyers just have one big PDF, but I guess in principle that PDF can be broken up into separate documents.

We could omit them, but there is no downside to their inclusion. We can wait for peer review to accept/reject them.

[JachymHercher]

If these checkboxes are the only reason to include them, then I would drop them. They were added to the forms at a later stage than the other fields and sections around them and their point was just to allow the buyer to declare "I have fulfilled my legal responsibility". I don't think it was motivated by actual procurement documents being split this way. (We can wait for peer review to reject them, but we could also just wait for someone to propose it in case they have the need :). )

[jpmckinney]

Okay, we can remove them in the same PR and create an issue on https://github.com/open-contracting-extensions/european-union/issues to update the guidance there.

[JachymHercher]

Done in 41fb2cd and open-contracting-extensions/european-union#95.

I'm not 100% sure I've marked this correctly in the change-log.

• Since the codes were never part of live OCDS, I removed them, not deprecated them.
• We could also just remove their mention from the changelog altogether (because we added and remove them inbetween versions), which I propose in 8525dcd. Please reject / accept.