New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Collecting additional structured information on bids and assessment process (Discussion) #223
Comments
@myroslav: When we discussed this you spoke of potentially storing the additional data as attached documents. Is there a reason for not storing it within the JSON structure of a release itself? Then the schemas could be simple JSON-Patch documents (much like an OCDS extension), and the calculation would be able to operate over a single JSON tree. Each JSON-Patch document for structured data could specify which user role was responsible for providing that data, and could potentially introduce a field for determining whether the contents of any given field should be published openly. It might be useful as well if we can construct a very simple example case. For e.g. can you give an example of one or two extra fields a bidder would be asked to supply, and a few that the procuring agency would use, and then a calculation over them? That would let us start working up some example JSON for different approaches. |
Schema have to be stored in Tender itself. If it is document, then it is in Tender' document contents. If it is not in document contents, then it should be
After that every bid should have
The JSON-Patch has following drawbacks:
All these items above does not make JSON-Patch an elegant solution. "JSON schema" approach solves all of the issues above, but still has following issues to resolve:
|
The most significant example fields in case of vacuum cleaner purchase are:
I.e. I do have minimal level of "air intake" power and minimal "guarantee period", but I can buy more expensive vacuum cleaner if it is more powerful, has longer warranty period or is more energy efficient. |
Ok. Let's construct this as a worked example, and explore the options. Here's my attempt at a worked example. Have I missed any key steps etc? Step 1: A buyer issues a tender for a vacuum cleanerIn the tender release they specify the extra information that bids should contain, in addition to supplier information and price. In this worked example we will ask for air intake and guarantee period. Step 2: The bidders submit their bids including this additional informationThe current bid submission approach used in ProZorro is here. Step 3: The system validates this additional information and stores itIt should be able to reject incorrectly formed additional information. Step 4: The system makes calculations based on this additional information (optional)For example, calculating a score based on air intake, guarantee period and price. Step 5: The system is able to re-publish this additional informationAt such point as bids are made open. |
Looking at this, I wonder if an alternative approach to the idea of pushing a JSON Schema out at Step 1 for the additional information, and then having potentially arbitrary data structures to store in the system at future stages, would be to abstract out all this additional information as I could envisage a "items":[
{
"id": "0001",
"description": "string",
"classification": {...}
"quantity": "08",
"unit": {...}
"features":[
{
"code":"OCDS-123454-AIR",
"title":"Air Intake",
"description":"The air intake of the vacuum cleaner",
"type":"integer",
"pattern":"[0-9]*",
},
{
"code":"OCDS-123454-GUARANTEE",
"title":"Guarantee Period (months)",
"description":"The guarantee period for the vacuum cleaner in months.",
"type":"integer",
"pattern":"[0-9]*",
},
{
"code":"OCDS-123454-GUARANTEE_TERMS",
"title":"Guarantee Terms",
"description":"An explanation of any specific guarantee terms that may apply.",
"type":"string",
"pattern":"",
},
]
}
] Here I've use the JSON schema Then, when submitting their data, a bid object could look something like this: {
"data": {
"tenderers": [...],
"value": {
"amount": 480000
},
"features":[
{
"code":"OCDS-123454-AIR",
"value":100
},
{
"code":"OCDS-123454-GUARANTEE",
"value":24
},
{
"code":"OCDS-123454-GUARANTEE_TERMS",
"value":"Products must be returned to the depot for assessment and repair."
}
]
}
} ReflectionsThis approach is perhaps less flexible than arbitrary JSON schema - but could have a number of advantages:
LimitationsIn this suggestion I've not tried working out how calculation might work - and there are lots more details to look at in terms of how values etc. should be modelled, but would welcome feedback on this alternative approach. |
"Features" is elegant idea. But I'm afraid it but doesn't cover "Features" of provider itself. Not sure why but one can ask for something like "years on a market" since features of the provider itself is used to mean a lot when evaluating proposals. Not sure if it is good or bad, but it is as it is. This can be embedded in item, but that not the most elegant workaround. |
So - two options here before we discount features. (1) Attach features to the tender/bid level (rather than item)But introduce new elements to them for something like This would make for something like: "features":[
{
"code":"OCDS-123454-AIR",
"featureOf":"item",
"relatedItem":"0001"
"title":"Air Intake",
"description":"The air intake of the vacuum cleaner",
"type":"integer",
"pattern":"[0-9]*",
},
{
"code":"OCDS-123454-YEARS",
"featureOf":"tenderer"
"title":"Years trading",
"description":"The number of years the tendering organisation has been trading",
"type":"integer",
"pattern":"[0-9]*",
}
] the tender release, and then: {
"data": {
"tenderers": [...],
"value": {
"amount": 480000
},
"features":[
{
"code":"OCDS-123454-AIR",
"value":100,
"releatedItem":"0001"
},
{
"code":"OCDS-123454-YEARS",
"value":5,
"relatedTenderer":"13306232"
}
]
}
} in the actual bid object. (2) Allow features to be attached in a number of places - to both tenderers and itemsAlthough I'm not sure where we would attach the features for a tenderer in the Step 1 tender release. It might be that we need a general (a) To items via (b) To tender via |
Aside: The other element that features could include would be a |
The case (1) with The |
What about source for the data about the Feature? I.e. difference between Bidder and Assessment Committee? Should we add |
Should we establish separate field for |
I was thinking the So - I think That would make it something like: "features":[
{
"id":"air",
"source":"tenderer",
"code":"OCDS-123454-AIR",
"featureOf":"item",
"relatedItem":"0001"
"title":"Air Intake",
"description":"The air intake of the vacuum cleaner",
"type":"integer",
"pattern":"[0-9]*",
},
{
"id":"air-2",
"code":"OCDS-123454-AIR",
"source":"tenderer",
"featureOf":"item",
"relatedItem":"0002"
"title":"Air Intake",
"description":"The air intake of the vacuum cleaner",
"type":"integer",
"pattern":"[0-9]*",
},
{
"id":"trading",
"source":"tenderer",
"code":"OCDS-123454-YEARS",
"featureOf":"tenderer"
"title":"Years trading",
"description":"The number of years the tendering organisation has been trading",
"type":"integer",
"pattern":"[0-9]*",
}
] and for submissions: {
"data": {
"tenderers": [...],
"value": {
"amount": 480000
},
"features":[
{
"id":"air"
"value":100,
"releatedItem":"0001"
},
{
"id":"air-2"
"value":100,
"releatedItem":"0002"
},
{
"id":"trading",
"value":5,
"relatedTenderer":"13306232"
}
]
}
} |
One (and only) of the types of values we'll have is the "choices" one. I.e. Feature is expected to list all possible numerical choices with their textual explanation (tile, and possibly also a description). JSONschema does have 'enum' and 'oneOf' for the goal. Can you suggest something similar for Feature? |
So, from looking at So, we could simple add If If If If The objects in How would that work? It avoids us needing to introduce oneOf, and I think is consistent with JSON Schema's use of the terms. |
In our case, the primary and only "enum": [
{
"title": "До 1000 Вт",
"value": 0.1
},
{
"title": "Більше 1000 Вт",
"value": 0.15
}
], I'm not sure that ID is necessary, since that can lead to some manipulations in following event sequence:
This would mean that "copying" value from "enum" into Bid is proper way from legal standpoint. |
Makes sense: so we can leave the redundancy as recommended, but not specify in depth what should go within the enum lists - allowing different systems to use arbitrary objects. Shall I work up an extension on this basis? |
On the issue of assessment processes, just spotted http://compranetfacil.com/plataforma-nacional-de-transparencia-segunda-etapa which presents structured information on the bid process. May be relevant to this thread, or to related extensions. |
From research into the kinds of information contractors find useful at debrief state:
|
Notes from modelling an example from the GDS Digital Outcomes and Specialists Framework using a mock up of Features.
|
This thread seems to be bringing together quite a few things. Find some comments below, to see whether I've understood it correctly. By the way, wouldn't all of this be better as an "open codelist"? (One for uses 1 & 2 below, another one for use 3.)
Here we try to structure the description of the object of the contract. If I understand it correctly, that is the main goal of the "features" extension - adding it to tenders and bids, including minimal and maximal values.
Here we try to link the structure to quantifiable quality award criteria. Same as above, these award criteria should have a minimal/maximal acceptable value for the tender to be evaluated. (This type of things is very important for eAuctions, where competition can be on anything which is quantifiable.)
|
@JachymHercher Sorry for the slow reply - but I can confirm from your notes above that you have understood correctly what we're aiming to do with the features extension - covering a number of use cases with a flexible data model. |
To check my understanding in return: ESPD is the 'European Single Procurement Document' and represents essentially a form that suppliers would have to fill in to describe themselves, and whether they meet the criteria that a government entity has set out for the economics operators who might want to bid on an opportunity. We could consider in developing the features extension how ESPD data would map into features. |
@timgdavies Concerning the ESPD yes, that's it. Concerning the "features" extension, you are not approaching it as a codelist, right? You want to just describe the features, not classify them? |
Correct. An implementation of the features extension could use codes from an external codelist to describe the kind of features covered, but we are not proposing any codelists at this point in time. |
I've just been looking at the EU's Core Criterion and Core Evidence Vocabulary (CCCEV) which has recently issued Draft 4 for consultation. The CCCEV has a model based on specifying requirements (like our 'features' concept) using The use of Responses to a criterion requirement are given by a There are also some interesting modelling steps in relation to the way in which CriterionRequirements (ranges etc.) can be specified, and CCCEV work may support development of re-usable criterion identifiers in future. @duncandewhurst this would definitely be worth exploring in your work on features extension. |
I've expanded slightly on the vacuum cleaner example given above and used the CCCEV approach and language of This gives the following structure for specifying requirements at the tender stage: "tender": {
"criteria": [
{
"id": "0001",
"title": "Air intake",
"description": "The vacuum cleaner air intake must be at least 100W",
"source": "tenderer",
"code": "OCDS-123454-AIR",
"featureOf": "item",
"relatedItem": "0001",
"requirementGroups": [
{
"id": "0001-001",
"description": "The vacuum cleaner air intake must be at least 100W",
"requirements": [
{
"id": "0001-001-01",
"title": "Air intake",
"description": "Power of vacuum cleaner air intake in W",
"dataType": "integer",
"pattern": "[0-9]*",
"minValue": 100
}
]
}
]
},
{
"id": "0002",
"title": "Warranty",
"description": "The vacuum cleaner must have warranty support options for at least 36 months",
"source": "tenderer",
"code": "OCDS-123454-WARRANTY",
"featureOf": "item",
"relatedItem": "0001",
"requirementGroups": [
{
"id": "0002-001",
"description": "The standard warranty period for the vacuum cleaner must be at least 36 months",
"requirements": [
{
"id": "0002-001-01",
"title": "Standard warranty period",
"description": "Length of the vacuum cleaner standard warranty period in months",
"dataType": "integer",
"pattern": "[0-9]*",
"minValue": 36
}
]
},
{
"id": "0002-002",
"description": "The standard warranty period for the vacuum cleaner must be at least 12 months with an option to extend to 36 months",
"requirements": [
{
"id": "0002-002-01",
"title": "Standard warranty period",
"description": "Length of the vacuum cleaner standard warranty period in months",
"dataType": "integer",
"pattern": "[0-9]*",
"minValue": 12
},
{
"id": "0002-002-02",
"title": "Extended warranty option",
"description": "There is an extended warranty option for at least 36 months",
"dataType": "boolean",
"expectedValue": true
}
]
}
]
},
{
"id": "0003",
"title": "Years trading",
"description": "Number of years the bidder has been trading",
"source": "tenderer",
"code": "OCDS-123454-YEARS",
"featureOf": "tenderer",
"requirementGroups": [
{
"id": "0003-001",
"description": "Number of years the bidder has been trading",
"requirements": [
{
"id": "0003-001-01",
"title": "Years trading",
"description": "Number of years the bidder has been trading",
"dataType": "integer",
"pattern": "[0-9]*",
"minValue": 3
}
]
}
]
}
]
} And for responses: "bids": [
{
"requirementResponses": [
{
"id": "air",
"value": 125,
"requirement": "0001-001-01",
"relatedItem": "0001"
},
{
"id": "warranty",
"value": 36,
"requirement": "0002-001-01",
"relatedItem": "0001"
},
{
"id": "years",
"value": 10,
"requirement": "0003-001-01"
}
]
}
] In this approach the requirement.id would need to be unique within the OCID to allow cross referencing from the bids in which case we may be able to remove the relatedItem property of requirementResponse. @myroslav - does this meet your requirements? There are a couple of areas in the CCCEV model which differ from the OCDS approach:
CCCEV has some additional detail which we currently expect to sit outside of the proposed features/requirements extension, although existing OCDS building blocks could be reused for most of these concepts:
Feedback and thoughts on the above very welcome. We're also unsure of the wider applicability of the enum approach discussed in previous comments so would appreciate views on this from the community. We're considering whether this could be expressed as semi-structured data using the requirements.description field. |
The CCCEV-inspired proposal of @duncandewhurst is too verbose for ProZorro and MEAT (as outlined via Ukrainian Public Procurement Law). Example of Formula that is weighting the price and parameters of every feature is hard-coded in the Public Procurement Law and perfectly aligns with several aspects of public procurement:
More about weighting at http://openprocurement.org/en/nonprice-criteria.html |
Thanks @myroslav: is the issue solely verbosity? I.e. is it possible to model the information from your features in terms of a single criteria and requirementsGroup? Or are there semantic issues we need to consider here? |
The version of this extension for review is now located at https://github.com/open-contracting/ocds_requirements_extension |
We are currently updating the ESPD data model (https://github.com/ESPD/ESPD-EDM) which is behind the CCEV. The aim is to have the ESPD data model updated end of April. After that the idea is to CCEV accordingly. When do you want to have your new version ready? |
@ec-mcs We're aiming to release OCDS 1.1 by end of April, which I think would be formally pinned to a version of our ocds_requirements_extension as it stands then, and also ocds_requirements_extension will be integrated into the Public Private Partnerships profile of OCDS which is due for release around the same time. However, the extension can continue to evolve further beyond that for early adopters. Where would we look to get a sense of any substantial changes anticipated to EPSD / CCEV data structures? |
My comment from the 1.1 peer review:
|
A features extension is under consideration for the 1.1 version of OCDS.
This issue has been updated to reflect the current state of proposal development.
The issue
There are scenarios where:
In a distributed system, where bidders are submitting data through different portals, an approach is required that can:
The current approach being explored is providing a schema for this extra information which systems can use to provide additional structured data.
Specific requirements
Any approach should be able to:
Proposal
See #223 (comment) for a draft proposal which involves:
features
array in tender, bids, award and contractsFurther work to develop a fully worked up features extension will take place in coming weeks.
Engagement
Do you support the development of a features extension? Do the proposals in this thread head in the right direction?
The text was updated successfully, but these errors were encountered: