Conversions are used for two different things in our model

[dennisquintel]

Since a long time we have (mis)used Conversions in our model for two different purposes.

With RDR and KIE, this is unfortunate, since Use 1 is generic and does not depend on the area/year of the dataset, but Use 2 does depend on the area/year of the dataset.

Use 1: Converter specs

For example the energy_power_nuclear_gen3_uranium_oxide converts uranium_oxide into electricity and loss:

         +---------+
<--elec--|         |
   (64%) | NUCLEAR |
         |  POWER  |<-uranium--
         |  PLANT  |  (100%)
<--loss--|         |
   (36%) +---------+

And, for example the households_space_heater_heatpump_air_water_electricity converts ambient_heat and electricity into usuable_heat, cooling and loss:

            +--------+
<--heating--|        |<-elec----------
  (100%)    |  HEAT  |  (22.22%)
<--cooling--|  PUMP  |
    (0%)    |        |<-ambient_heat--
<--loss-----|        |  (77.78%)
    (0%)    +--------+

Use 2: Different distributions/shares

Conversions can be used to designate a different share of inputs that are used by a Converter/Node. Typically, this differs per country/year.

This happens currently for two type of Nodes: Mixers and 'pseudo-mixers'.

Mixers:

And example of a Mixer is the energy_production_biogas, where manure and corn are mixed to form biogas

Pseudo-Mixer:

Some Converters is the model take on a number of different inputs, such as industry_other_metals_burner_network_gas (See also #326), and that 'mix' can differ per area/year.

Possible Solutions

I think the current situation is unfortunate. It is the same technical solution for two different purposes (and hence different data origins). In the short-term we need to find a solution how to support both features.

My suggestions:

1. SubClass for Mixers:

We could make a special class for the MixerNodes and let them take on a 'query' (or something that is area-dependent.

Pro of this solution is that is is unnecessary to adapt ETEngine.

2. Use links with shares

We could also use shares of a link directly. Currently shares are work within a Slot (and a Slot is a group of links with the same carrier).

Con of this solution is that is is necessary to adapt ETEngine.

I would like to hear the ideas of @antw in this.

Note

Just to not cause confusion: In both uses, the user should (still) be able to adept this in his/her scenario. That is not the point.

including @AlexanderWirtz, @ChaelKruip, @StijnDellaert and @wmeyers.

[AlexanderWirtz]

As I understand this problem:
We want to define global converter attributes in .ad files. The conversions are such attributes that we want to be global and because of the inconsistent use of conversions this is currently not possible.

This means we need to get rid of the 'area/year specific use of conversions.

Correct?

[ChaelKruip]

@AlexanderWirtz you are right! 😃

This means we need to get rid of the 'area/year specific use of conversions.

Differently put: things that need to change with year/area should not be called the same thing as the global/static conversions anymore.

[dennisquintel]

Differently put: things that need to change with year/area should not be called the same thing as the global/static conversions anymore.

Amen.

[antw]

I think the current situation is unfortunate. It is the same technical solution for two different purposes (and hence different data origins). In the short-term we need to find a solution how to support both features.

I'm not sure why this is such a bad thing. From my perspective, both uses of "conversion" are basically the same; it's a way to aggregate multiple edges by their carrier, and adjust their shares simply, instead of having to adjust each share manually. The data may come from different sources, and some may be static (entered as a raw number) or variable (set with a query), but the end result is the same. ETEngine, as far as I know, makes no distinction between the two uses.

Solution 2: Use links with shares

It's unfortunate that slots have to exist at all, considering their only significant purpose is the "conversion" attribute, but I think getting rid of them (which is what I understand "solution 2" to be) would make everyone's life harder. We could implement some "ad-hoc" slot feature which works directly on edge shares – perhaps with the use of Osmosis – but I don't think the added complexity in ETEngine and ETSource would be worth it.

MAGIC!(OUT_EDGES(:some_node, :heat),        :share, 0.4)
MAGIC!(OUT_EDGES(:some_node, :electricity), :share, 0.6)

Urghh.

Solution 1: SubClass for Mixers

I would favour this approach, since it means we only have to write new code in the project which needs it. Major changes to ETEngine – which aren't driven by the need for a new feature (such as removing slots) – should really be restricted to cases where it's absolutely necessary, and instead added to the wish list for ETEngine.next.

An added bonus is that it should be pretty easy to implement. We need to change ActiveDocument to support multiple queries so that we can set multiple conversions (or a demand + one or more conversions), but that's not too hard. The hard thing is deciding how the syntax should look in the ".ad" files. 😃

Maybe something like this?

- yes = no
- up  = down

= output.electricity
AWESOME_DATA(a, b, c)

= output.heat
AWESOME_DATA(x, y, z)

[dennisquintel]

I would favour this approach, since it means we only have to write new code in the project which needs it. Major changes to ETEngine – which aren't driven by the need for a new feature (such as removing slots) – should really be restricted to cases where it's absolutely necessary, and instead added to the wish list for ETEngine.next.

Agree!, so solution 1 it is.

I think supporting multiple queries is a good thing, and that shouldn't be too hard. We can also make it explicit, comparable to what we do with inputs (max_value versus max_value_gql).

- yes = no
- up  = down
- output.electricity_query = AWESOME_DATA(a, b, c)
- output.heat_query = AWESOME_DATA(x, y, z)

Coming to think of it, I think we can improve on the ActiveDocument format... but that's another Issue.

[dennisquintel]

Summing up the conclusions of this ticket:

• @ChaelKruip is going to make sure they are no 'pseudo'-mixers (see definition above) anymore in the graph. See quintel/inputexcel#300
  UPDATE: I (@ChaelKruip) don't think we want / need to remove the remaining pseudo-mixers
• @dennisschoenmakers (or @antw when delivery starts) will create a new subclass for the MixerNode. See Create MixerNode atlas#3
• @dennisschoenmakers (or @antw when delivery starts) will add support for multiple queries for the See ActiveDocument should support multiple queries atlas#4

[wmeyers]

Is my input still required here? My most important question is: Am I going to notice any changes I need to be aware of?

[dennisquintel]

Is my input still required here?

No.

Am I going to notice any changes I need to be aware of?

@ChaelKruip will investigate how many pseudo-mixers exist in the graph and how many new Mixers will have to be added to the Graph/InputExcel.

This might change the structure of the graph a bit, which might impact the analyses. I would watch that ticket quintel/inputexcel#300 closely.

[antw]

Since we decided against MixerNode (quintel/atlas#3), and added the ability to use queries on any attribute (including slot conversions), I think this can be considered closed. Reopen if you disagree. 😃