Seperate Orkney Islands from Great Britain #1615

systemcatch · 2018-09-27T15:29:17Z

Things to do:

Compute a default carbon intensity for the islands.
Find capacity information.
Generate shapes from a combination of subunits & states.
Check it all works by running the mockserver.

systemcatch · 2018-10-03T08:47:49Z

@corradio so in terms of what needs to happen here, I can add the capacities & compute a carbon intensity. But I'll need someone else to alter to code so we can make shapes from a combination of subunits and states.

corradio · 2018-10-03T09:10:22Z

@systemcatch I'm quite busy right now so not sure I'll be able to do it. However you should be able to edit the existing function that filters geometries. Once you have a list, you can simply call geomerge on them to merge them all.

jarek · 2018-10-03T10:21:25Z

Hey,

It looks like the UK sub-shapes might be able to be created off the gu_a3 field. Its values are SCT, ENG, NIR, and WLS. Then we'd subtract Shetland (adm1_code=GBR-2747) and Orkneys (GBR-2744) from Scotland, and use them as separate zones instead.

I might be able to make a dry attempt at the required code, but my Node environment is unfortunately still non-functional and I won't have much time to set it up, so I'd have to send over the changes and have someone else test them... Sorry.

systemcatch · 2018-10-03T11:13:18Z

Hey @jarek that would be really cool if you'd help with that, I'd be happy to test by pulling from one of your branches.

Regarding your node environment could #1322 help?

corradio · 2018-10-05T09:36:07Z

@maxbellec maybe you can help here if you have time?

maxbellec · 2018-10-09T06:25:05Z

I can help, had a look at the code, this won't be a problem, but I need a chat with @systemcatch to exactly understand what we want to do with shapes. Let's talk on Slack.

maxbellec · 2018-10-09T14:00:14Z

maxbellec · 2018-10-09T14:00:57Z

I've also separated Shetland islands, since @systemcatch tells me they're not connected to GB. We do not have a source yet, so it will stay grey for the moment

systemcatch · 2018-10-09T14:34:28Z

Very nice @maxbellec! I'm trying to build the map but keep getting the following error. You could try running the parser and seeing if it displays ok.

(EM-env) chris@ThinkPad:~/electricitymap$ docker-compose run --rm web npm run watch
ERROR: Version in "./docker-compose.yml" is unsupported. You might be seeing this error because you're using the wrong Compose file version. Either specify a supported version (e.g "2.2" or "3.3") and place your service definitions under the `services` key, or omit the `version` key and place your service definitions at the root of the file to use version 1.
For more on the Compose file format versions, see https://docs.docker.com/compose/compose-file/

Docker version 18.06.1-ce, build e68fc7a
docker-compose version 1.16.1, build 6d1ac21

maxbellec · 2018-10-09T16:32:02Z

maxbellec · 2018-10-09T16:32:44Z

I have docker-compose==1.21.2 and Docker version 1.13.1, build 092cba3. I guess it should work for you if you get the same version as mine.

maxbellec · 2018-10-09T16:36:17Z

web/generate-geometries.js

  }
 });

+// special case for GB, as we separate two islands from it


@corradio it's the only zone for which we do manual manipulation. Do you think that code is OK? At least it works fine.

let's move the code up to L236 then!

how would I move that code within an Object definition?

ah yes indeed. Maybe creating a method would make more sense, and call it from the object def

systemcatch · 2018-10-09T17:28:27Z

@maxbellec got it working (old docker-compose) and everything looks ok. Are you happy about the line between England and Scotland?

jarek · 2018-10-09T17:41:23Z

Great job @maxbellec , thanks for sorting out the working code.

Though it is kind of strange that line is there between England and Scotland and not for example between England and Wales, I don't really understand why. We should be able to geomerge zones that have already been geomerged, I mean geomerge(geomerge(...scotland_parts), geomerge(england, wales))... is something not working there?

maxbellec · 2018-10-09T20:00:59Z

hmm good point about the line, I'll try to understand why that happens

maxbellec · 2018-10-10T08:58:13Z

I don't know how to remove the line. I also have it when doing a single geomerge. @corradio any idea here?

zones['GB'] = geomerge(...allGeos.filter(d => (
  d.properties.subid === 'ENG'
  || d.properties.subid === 'WLS'
  || ((d.properties.gu_a3 === 'SCT') && (d.properties.adm1_code !== 'GBR-2747') && (d.properties.adm1_code !== 'GBR-2744'))
  )));

corradio · 2018-10-11T05:17:38Z

geomerge simple does a union of shapes, so I think it will always create a line.
For example, SPP is the union of a certain number of states, and it shows lines between states:

Is there a previous situation where we merge items without any border? If yes, then we should look at how we did it for this case.

corradio · 2018-10-11T05:13:39Z

parsers/GB_ORK.py

+DATETIME_LINK = 'https://www.ssen.co.uk/ANM/'
+GENERATION_LINK = 'https://www.ssen.co.uk/Sse_Components/Views/Controls/FormControls/Handlers/ActiveNetworkManagementHandler.ashx?action=graph&contentId=14973&_=1537467858726'
+
+# TODO map correctly


Not anymore I'll remove it, are you happy with the carbon intensity I've assigned to the Orkney's?

@brunolajoie should answer that, he's more qualified than I am

corradio · 2018-10-11T05:14:35Z

web/generate-geometries.js

  }
 });

+// special case for GB, as we separate two islands from it


let's move the code up to L236 then!

…ricitymap into orkney-islands

maxbellec · 2018-10-11T16:58:52Z

🎉
Now waiting for the Queen's phone call to thank me for saving peace in the UK.

maxbellec · 2018-10-11T17:01:00Z

FYI the script I used to remove all GeoJSON data is SCT above a certain latitude:

coords = sct['geometry']['coordinates']

new_coords = []
for coord in coords:
    assert len(coord) == 1
    coord = coord[0]
    
    new_coord = []
    append = True
    for x, y in coord:
        if y > 58.7:
            append = False
            new_coord.append([x, y])
    
    if append:
        new_coords.append([coord])

sct['geometry']['coordinates'] = new_coords

sct['properties']['subid'] = 'SCT-no-islands'

import json

with open('/home/m/temp/sct-no-islands.json', 'w') as f:
    f.write(json.dumps(sct))

where sct is a json.loads of the line where properties.subid == 'SCT' from tmp_countries.json

maxbellec · 2018-10-11T17:02:35Z

I think we're just waiting for @brunolajoie to confirm carbon intensity

corradio · 2018-10-12T06:34:56Z

Nice hack @maxbellec ! However I don't think this is the way to go, because having a 3rd party geom for SCT will be problematic as soon as we will split GB (we will have to regenerate the files, keep knowledge of your script etc..).
I don't think it's a problem that the line is there TBH. It will motivate people even more to open data up about GB (because we know there's data!)

maxbellec · 2018-10-12T08:10:16Z

We'd still use sct-no-islands instead of SCT after the split, wouldn't we?

brunolajoie · 2018-10-12T08:35:21Z

@systemcatch what does the parser returns as production mix? Only the total renewable generation? Is there any backup system ?

systemcatch · 2018-10-12T09:26:57Z

@brunolajoie it returns the sum of renewable generation and extra generation attributable to their active network management (ANM) system. There are no backup systems of any real size since the Orkney's are connected to the mainland via cable.

However rereading https://www.ssen.co.uk/ANM/ maybe the ANM system should not be added to generation. The way they present it is confusing, what do you think?

edit: http://www.oref.co.uk/orkneys-energy/innovations-2/ suggests it should be added.

brunolajoie · 2018-10-14T19:07:08Z

@systemcatch Yeah, looking at the live bar chart right now, and comparing with the load, it seems that we double count if we add ANM + Renewable generation, no?
Or are they exporting about as much as their load, right now? Can you check comparing API load / export / generation results?

jarek · 2018-10-14T19:28:04Z

Right now (site says 14 October 2018 20:24:49, my time 21:26...) I see the following:

Orkney Demand - Live Demand 16.88 MW
Orkney Generation - Orkney ANM 2.983 MW
Orkney Generation - Non-ANM Renewable Generation 7.391 MW
Orkney Demand - Winter Peak Demand 35.70 MW
Orkney Generation - Total Renewable Capacity 57.10 MW

So current generation would be 10.374 MW and demand 16.88 MW, with balance being imported.

Note that the chart includes peak demand and total production capacity, it has thrown me off more than once as well.

We don't have a separate source for import/export, it is calculated from demand minus generation.

brunolajoie · 2018-10-14T19:33:58Z

Ok so clearly we need to add both ANM and non-ANM-renewable indeed as it is coded right now.
But we need to find a way to understand what fuel is the ANM? Could it be a storage system?

jarek · 2018-10-15T06:25:36Z

My understanding was that ANM is "managed" renewable generation, in 2010 I guess they'd call it a "smart grid". There are also some renewable generators which are not managed in this system (grandfathered in, perhaps?) and these form the "non-ANM generation". https://www.ssen.co.uk/ANMGeneration/ and https://www.ssen.co.uk/ANMFAQ/ might shed some light on it...

As a further example, right now we'd have an export of around 2.5 MW:

Orkney Demand - Live Demand 16.65 MW
Orkney Generation - Orkney ANM 6.904 MW
Orkney Generation - Non-ANM Renewable Generation 12.105 MW
Orkney Demand - Winter Peak Demand 35.70 MW
Orkney Generation - Total Renewable Capacity 57.10 MW
Last updated at 15 October 2018 07:13:22

brunolajoie · 2018-10-15T08:21:27Z

Ok, thanks, so our parser seems to give the right amount of total generation. And both "ANM" and "non-ANM" are renewable capacities. I'll merge now then!

systemcatch requested review from jarek, corradio and alixunderplatz September 27, 2018 15:29

systemcatch added 9 commits October 8, 2018 14:46

Create parser for Orkney Islands

c4c90d5

Add GB-ORK to readme

7d92483

Add GB-ORK to exchanges

ee90d32

Add GB-ORK to en locale

50b2512

Add GB-ORK to zones

95a34f2

Retrieve states for GB

95a50e3

Initial steps for creating Orkney & Shetland shapes

47295c7

Capacity information for GB-ORK

0a73ee7

Set default CO2 emissions for GB-ORK

0d2b644

systemcatch force-pushed the orkney-islands branch from 89792ff to 0d2b644 Compare October 8, 2018 13:48

GB without islands

6474953

update state

7f94749

maxbellec reviewed Oct 9, 2018

View reviewed changes

corradio reviewed Oct 11, 2018

View reviewed changes

systemcatch and others added 3 commits October 11, 2018 10:27

Remove todo

739b0f6

sct in third party

eb25800

Merge branch 'orkney-islands' of https://github.com/systemcatch/elect…

0a93628

…ricitymap into orkney-islands

add source in generate-geometries and sct-no-islands.json

dda9a31

corradio approved these changes Oct 12, 2018

View reviewed changes

corradio assigned brunolajoie Oct 12, 2018

corradio requested a review from brunolajoie October 12, 2018 14:11

brunolajoie merged commit 784fd2d into electricitymaps:master Oct 15, 2018

systemcatch deleted the orkney-islands branch March 4, 2019 10:56

systemcatch mentioned this pull request Aug 2, 2019

Seminole Electric Cooperative (Florida) #1713

Closed

Seperate Orkney Islands from Great Britain #1615

Seperate Orkney Islands from Great Britain #1615

Conversation

systemcatch commented Sep 27, 2018 • edited Loading

systemcatch commented Oct 3, 2018

corradio commented Oct 3, 2018

jarek commented Oct 3, 2018

systemcatch commented Oct 3, 2018

corradio commented Oct 5, 2018

maxbellec commented Oct 9, 2018 • edited Loading

maxbellec commented Oct 9, 2018

maxbellec commented Oct 9, 2018

systemcatch commented Oct 9, 2018

maxbellec commented Oct 9, 2018

maxbellec commented Oct 9, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

systemcatch commented Oct 9, 2018

jarek commented Oct 9, 2018

maxbellec commented Oct 9, 2018

maxbellec commented Oct 10, 2018

corradio commented Oct 11, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

maxbellec commented Oct 11, 2018

maxbellec commented Oct 11, 2018

maxbellec commented Oct 11, 2018

corradio commented Oct 12, 2018

maxbellec commented Oct 12, 2018

brunolajoie commented Oct 12, 2018

systemcatch commented Oct 12, 2018 • edited Loading

brunolajoie commented Oct 14, 2018 • edited Loading

jarek commented Oct 14, 2018

brunolajoie commented Oct 14, 2018 • edited Loading

jarek commented Oct 15, 2018 • edited Loading

brunolajoie commented Oct 15, 2018 • edited Loading

systemcatch commented Sep 27, 2018 •

edited

Loading

maxbellec commented Oct 9, 2018 •

edited

Loading

maxbellec commented Oct 9, 2018 •

edited

Loading

systemcatch commented Oct 12, 2018 •

edited

Loading

brunolajoie commented Oct 14, 2018 •

edited

Loading

brunolajoie commented Oct 14, 2018 •

edited

Loading

jarek commented Oct 15, 2018 •

edited

Loading

brunolajoie commented Oct 15, 2018 •

edited

Loading