Unit of Measure - additional fields and codelist

[timgdavies]

This issue is under consideration for the 1.1 milestone.

This builds on prior discussions in #145 and #177

The problem

In OCDS it is possible to specify item.quantity and item.unit, and to indicate the value/cost per unit.

However, unit of measure (unit.name) is currently a string. This means there is no consistent way to provide a unit of measure. This impacts comparing units across contracts or within the same contract.

Proposed solution

The unit building block will be revised to include four new properties:

• scheme - an identifier for the codelist from which units of measure are taken
• id - the identifier from the codelist referenced in scheme
• symbol - an optional user-friendly symbol for the unit of measurement
• uri - an optional URI to machine-readable data on this unit of measure

A open codelist will be created of unitClassificationSchemes, with initially a single entry for:

Code	Name	Description
UNCEFACT	UN/CEFACT Recommendation 20	Use the commmon code from the latest version of the UN/CEFACT Recommendation 20 list of "Codes for Units of Measure Used in International Trade"

UNCEFACT is preferred based on it's use in CEN BII, and it's inclusion of informative units.

Example

To describe an order for 100Gb of storage at £0.50 per GB:

"items": [{
    "quantity":100,
    "unit": {
        "name":"gigabyte",
        "id":"E34",
        "symbol":"GB",
        "scheme":"UNCEFACT",
        "value": {
                "currency":"GBP",
                "amount":0.5
        }
    }
}]

To describe an order for 100 cars at EUR 8,000 per car using the UNCEFACT code 'EA' for 'Each item'.

"items": [{
    "description":"Cars for use by the police force",
    "classification":{
        "scheme":"CPV",
        "id":"34114200"
    },
    "quantity":100,
    "unit": {
        "name":"Each",
        "id":"EA",
        "scheme":"UNCEFACT",
        "value": {
                "currency":"EUR",
                "amount":8000
        }
    }
}]

Engagement

Please indicate support or opposition for this proposal using the +1 / -1 buttons or a comment. If opposing the proposal, please give clear justifications, and where possible, make an alternative proposals.

Remainting detail questions

• What is the update process for the UNCEFACT Reccomendation 20 list? Are codes ever removed? What is the risk in referencing 'the latest version' as opposed to a specific version of this codelist?
• Should we maintain a list of common units or examples extracted from UNCEFACT in the documentation? What should be on this list if so?

[akuckartz]

This does not seem to be immediately useable as Linked Data.

[timgdavies]

The proposal includes a URI option. However, in the absence of URIs for entries from UNCEFACT, I wasn't able to include URIs in the example.

If a third-party provided URIs for the UNCEFACT list, would this meet criteria to be usable as linked data?

The other codelist considered in #177 was QUDT but this was ruled out as it lacks key units like 'Each'.

[akuckartz]

If a third-party provided URIs for the UNCEFACT list, would this meet criteria to be usable as linked data?

Decentralisation is a feature of Linked Data !

I only have a few minutes time now, but there could be other problematic aspects but they might be solvable quite easily.

The other codelist considered in #177 was QUDT but this was ruled out as it lacks key units like 'Each'.

It should be possible to extend QUDT. I will attempt to have a look.

[LindseyAM]

Unit prices came up today. If we would like to consider examples from other countries, apparently some attempt at unit price is in the Georgian procurement system data and apparently well done in the Italian CONSIP system.

[timgdavies]

@LindseyAM If you can share pointers to Georgian or CONSIP example data, or signpost to people who might be able to help there that would be great.

[LindseyAM]

I sent contacts for Georgia over email.. and for Italy, maybe @ekoner can take a look at the CONSIP data? Bibhusan and Anjesh are also working on this in Moldova .... but I dont know how to tag them in the issue...

[timgdavies]

Further exploration:

UBL Measures

Discussions relating to EU Procurement has pointed us towards the UBL concept of a 'Measure'.

UBL has multiple kinds of measure, such as NetWeightMeasure, GrossTonnageMeasure, AltitudeMeasure etc.

Each measure should have a unitCode, the 'usage rules' for which states 'Reference UNECE Rec. 20 and X12 355'.

UNECE Rec 20. is what we are currently proposing to use.

X12 355 appears to be from US Customs Electronic Data Interchange formats, and may equate to this list here.

I found an example of measures in a UBL sample file here.

UNECE Rec 20.

The Excel files from UNECE with their codelist need careful reading alongside the recommendation itself to note that:

• The lists contain many deprecated and removed codes (indicated by an X in the status column for Annex II and III of 'normative equivalent' and 'informative' units).
• The Annexe I codes are based on ISO SI units

With deprecated and removed codes taken out, the Annex II and III codelist still contains around 1700 codes, far too many for a standard interface to display in a drop-down box etc. - so we might want to include guidance to the effect that any data input systems based on OCDS make an editorial choice about which units to display to users.

We could also look to see if there is a dataset using UNECE Rec. 20. units that we could use to sort the list by frequency of use, and so help users identify which units they are most likely to want to offer in search / selection boxes.

[timgdavies]

A question has been raised as to whether we should include symbol as a field in the enhanced unit of measure section.

My thinking on this was that, as an optional field, this would help data users understand many quantities at a glance, without having to resort to a lookup in the UNECE Rec 20. list.

For example, looking through a spreadsheet, seeing KMK may not be that intuitive, but if it is paired with 'km²' then it is easier for the user to interpret.

However, I was thinking of symbol as something that would always be added by a tool based on the codelist, not entered by the user. The question then is whether attaching the symbol to the code should be carried out on the publisher or the user side.

Recognising that the definition proposed above doesn't make this clear, perhaps it should be updated to something along the lines of:

• symbol - an optional user-friendly symbol for the unit of measurement, provided by the scheme in use, and automatically included in the data based on the selected unit.

and accompanied by additional guidance.

Alternatively - if the view is that the effort here should be on the user side (as unless this is made a mandatory field, they may have to make that effort for some publishers anyway), then symbol could just be dropped from the proposal.

[timgdavies]

From conversations around use-cases for unit-pricing:

For us, the main use case are unit prices to be able to benchmark.

[timgdavies]

An extension, excluding 'symbol', is staged for inclusion here: https://github.com/open-contracting/ocds_upgrade_366

[timgdavies]

In peer-review, feedback noted that:

The way the documentation is worded it seems like "UNCEFACT" is the only scheme supported. More than one scheme should be allowed, my preference would be to use QUDT ( http://qudt.org/1.1/schema/qudt) as it is more close to Linked Data. Hence all possible schemes should be added under the "http://standard.open-contracting.org/1.1-dev/en/schema/codelists/#unit-classification-scheme".