Double check implemented matching rules

[fjuniorr]

• integrity issues with mappings where two FK are not PK in junction table
• cas numbers match conditions (eg. zync)
• match for https://commonchemistry.cas.org/results?q=Metaldehyde

[fjuniorr]

@cmutel here are a more detailed notes on the topics above.

integrity issues with mappings where two FK are not PK in junction table

I was not able to reproduce this. In the process of investigating I've added a useful summary of the mappings cardinalities in 7f9fdb5:

from flowmapper.utils import read_field_mapping, read_flowlist
from flowmapper.flowmap import Flowmap
from flowmapper.flow import Flow

fields = read_field_mapping('config/simapro-ecoinvent.py')
source_flows = [Flow.from_dict(flow, fields['source']) for flow in read_flowlist('data/agribalyse-3.1.1-biosphere.json')]
target_flows = [Flow.from_dict(flow, fields['target']) for flow in read_flowlist('data/ecoinvent-3.7-biosphere.json')]

flowmap = Flowmap(source_flows, target_flows)
flowmap.mappings
flowmap.statistics()

5667 source flows...
4329 target flows...
3422 mappings (59.75% of total).
Mappings cardinalities: {'N:1': 1753, '1:1': 1598, 'N:M': 48, '1:N': 23}

cas numbers match conditions (eg. zync)

In Match - Validated CAS numbers.ipynb CAS numbers that are not valid are discarded. However, in match_identical_cas_numbers if the CAS number and the context are equal, it's considered a match. Is this behaviour problematic?

match for https://commonchemistry.cas.org/results?q=Metaldehyde

To be honest I don't remember how or why I saw this because I'm not doing this type of verification. We have the match:

  {
    "source": {
      "name": "Metaldehyde (tetramer)",
      "categories": [
        "Emissions to soil",
        "agricultural"
      ],
      "unit": "kg"
    },
    "target": {
      "uuid": "597847df-518f-4bd6-ae50-1de01f2761a4",
      "name": "Metaldehyde",
      "context": "soil/agricultural",
      "unit": "kg"
    },
    "conversion_factor": 1.0,
    "comment": "Minor random name differences"
  }

From SimaProv9.4.csv I saw that the CAS for Metaldehyde (tetramer) is 000108-62-3 while for Metaldehyde (597847df-518f-4bd6-ae50-1de01f2761a4) it's 009002-91-9 which would indicate they are different flows? If that's the case does it make sense to enrich Simapro flows with the CAS number before doing matching with ecoinvent to find similar problems?

[fjuniorr]

@cmutel if we don't need to have 100% correct mappings maybe I can close this until it becomes a problem?

[cmutel]

In Match - Validated CAS numbers.ipynb CAS numbers that are not valid are discarded. However, in match_identical_cas_numbers if the CAS number and the context are equal, it's considered a match. Is this behaviour problematic?

Honestly, humans, get you shit together... I am fine with however this is currently handled, this should be undefined behaviour. Do two wrongs make a right?

[cmutel]

If that's the case does it make sense to enrich Simapro flows with the CAS number before doing matching with ecoinvent to find similar problems?

This is an excellent idea, let's make this an option in the config.

[fjuniorr]

Honestly, humans, get you shit together... I am fine with however this is currently handled, this should be undefined behaviour. Do two wrongs make a right?

😆

I will leave this as is then (ie. invalid CAS numbers that are equal it's considered a match)

If that's the case does it make sense to enrich Simapro flows with the CAS number before doing matching with ecoinvent to find similar problems?

This is an excellent idea, let's make this an option in the config.

I've created #74 to track this and I'm closing this issue.