Calculate CO₂ emissions of itineraries by Antiik91 · Pull Request #5278 · opentripplanner/OpenTripPlanner

Antiik91 · 2023-08-07T06:58:57Z

Summary

CO2 emissions information required for public transport itineraries and car journeys.
The data is loaded from a file called emissions.txt located in the gtfs package and saved to the graph during the build phase. A filter adds the emissions data to the itineraries using a service when itineraries are searched.

Issue

closes #4940

Unit tests

Tests added in a new file EmissionsServiceTest.java
Tests for:

Getting results for itinerary that has emissions data (non zero)
Getting results for itinerary that has zero emissions in the data
Not getting emission results for a feed that does not have emissions data
Getting results for a car journey

Manual testing was also performed.

Documentation

Modified:

BuildConfiguration.md
Configuration.md
Added:
DigitransitEmissions.md

Changelog

From title

…ule data fetching since the utility method was removed. Also initial unit tests added

optionsome · 2023-08-07T09:00:48Z

+
+Emissions data should be in json array with objects that have the following properties:
+
+`db`: database it refers to or CAR when referring to personal travel by car. For Example Linkki, HSL


Maybe rename this to feed_id. Although, does it necessarily match to the feed id we use for these data sets in OTP? I think we could omit this, agency_id and agency_name fields for CAR travel.

optionsome · 2023-08-07T09:08:19Z

+Emissions data is loaded from the provided source and embedded in graph building process
+as part of the itinerary.


Suggested change

Emissions data is loaded from the provided source and embedded in graph building process

as part of the itinerary.

Emissions data is loaded from the provided source and embedded into the graph during the build process.

optionsome · 2023-08-07T09:10:54Z

+
+### OTP 2.4
+
+- Initial implementation of Emissions calculation


Suggested change

- Initial implementation of Emissions calculation

- Initial implementation of the emissions calculation

optionsome · 2023-08-07T09:23:46Z

Maybe we should have a package structure like package org.opentripplanner.ext.emissions.digitransit; and then these classes shouldn't have digitransit in the name. This would allow people to add other emission implementations inside the emissions sandbox. Other option would be to name this package as digitransitemissions.

optionsome · 2023-08-07T09:26:56Z

+import org.opentripplanner.framework.lang.Sandbox;
+
+@Sandbox
+public class DigitransitEmissions implements Serializable {


I don't think we should load the json data into objects as such but parse it into a more sensible structure (and with more java friendly naming conventions).

optionsome · 2023-08-07T10:30:52Z

+  public List<Itinerary> filter(List<Itinerary> itineraries) {
+    return itineraries
+      .stream()
+      .peek(i -> {


Maybe forEach instead, I'm not too familiar with this peek method.

A old fasion loop and a return statment would make it mush easier to read - also, do not create a new list - just return the list passed into this filter. It is modifying the itineraries not to list.

optionsome · 2023-08-07T10:32:44Z

+public interface EmissionsService extends Serializable {
+  HashMap<String, DigitransitEmissionsAgency> getEmissionByAgency();
+
+  Float getEmissionsForRoute(Itinerary itinerary);


I think this should be called getEmissionsForItinerary

optionsome · 2023-08-07T10:33:14Z

+    return null;
+  }
+
+  public Float getEmissionsForRoute(Itinerary itinerary) {


This should be the only public method in this class.

optionsome · 2023-08-07T10:50:00Z

+@Sandbox
+public class DigitransitEmissionsService implements EmissionsService {
+
+  private HashMap<String, DigitransitEmissionsAgency> emissionByAgency;


The information we need to decide what the emissions are for a leg are agency id and transport mode. I think the key therefore could be a string of something like <feedscoped agency id>_<transport mode> and the value should be a float/double/int whatever we decide is appropriate.

Then, for car, the key could just be CAR because the agency is not relevant. Although, it could also make sense to have a special field here to store the car emission as we don't at least yet need to support emissions for other street modes.

optionsome · 2023-08-07T10:50:18Z

+
+  private HashMap<String, DigitransitEmissionsAgency> emissionByAgency;
+
+  public DigitransitEmissionsService(DigitransitEmissions[] emissions) {


This could just take in the Map as the constructor parameter.

optionsome · 2023-08-07T11:26:38Z

+/**
+ * This class is responsible for mapping OSM configuration into OSM parameters.
+ */


Left over from copy paste, update.

t2gran · 2023-08-15T14:12:07Z

+  public EmissionsService getEmissionsService() {
+    return emissionsService;
+  }
+
+  public void setEmissionsService(EmissionsService emissionsService) {
+    this.emissionsService = emissionsService;
+  }
+


Sorry, we do not pass "stuff" around in the graph any more, you need to inject it using Dagger.

Thanks for putting time into reviewing this draft! There are alot of changes coming soon, but you have pointed out some things I hadn't considered. I shall fix these as well before updating the pr.

Unfortunately, I'm having slight issues with the injection, as I am not sure where exactly it should be done. As far as I understand, we need to add the service/filter to the builder in RouteRequestToFilterChainMapper or ItineraryListChainBuilder, but we cannot inject anything in those. Could you maybe clarify how exactly a filter should be added without the graph? Thanks!

Take a look at the OtpServerRequestContext, witch is available where the ItineraryListFilterChainBuilder is created. You should let Dagger create your service. Then inject it in your graph-builder-module, the SerializedGraphObject and the OtpServerRequestContext. You need to inject it in the SerializedGraphObject for the data to be serialized.

I'm afraid you might have to spell this out for me, as I'm failing to see how to inject the service into the SerializedGraphObject without doing something strange. The SerializedGraphObject is initialised in the OTPMain only and I doubt you meant the emissionsservice is supposed to added to the SerializedGraphObject in the OPTMain. Also, I managed to inject and get the service from the OtpServerRequestContext, but of course it is a new instance and not the one handled during the building phase that contains the emissions data. So, I'm also failing to see how the service instance (that should be) injected into the SerializedGraphObject, will end up being the same instance that is gained from the context in the RouteRequestFilterChainMapper (as opposed to the brand new singleton without emissions data that I get now). Does there happen to be any existing service that has been implemented in the manner you are suggesting that I could maybe look at? I was only able to find the ones made in the old way and one that doesn't require the build phase at all (rideHailingService). Thanks!

I don't think there is a full example for a service, so you're breaking new ground, but the worldEnveloperRepository and dataImportIssueSummary combine data from various graph building phases and put that into the SerializedGraphObject.

OpenTripPlanner/src/main/java/org/opentripplanner/standalone/OTPMain.java

Lines 148 to 155 in bf6e4be

new SerializedGraphObject(

app.graph(),

app.transitModel(),

app.worldEnvelopeRepository(),

config.buildConfig(),

config.routerConfig(),

DataImportIssueSummary.combine(graphBuilder.issueSummary(), app.dataImportIssueSummary())

)

t2gran · 2023-08-15T14:23:54Z

+  @Sandbox
+  private EmissionsService emissionsService;


I would prefer that you set the ItineraryListFilter emissionsFilter here, not the service, so that the ItineraryListFilterChainBuilder has does not need to depend on the org.opentripplanner.ext.emissions package. I know we have a few other dependencies to the sandbox in here - because they relay on the config, but the emission seems to not have any config.

t2gran · 2023-08-21T14:43:25Z

+
+  private String name;
+  private float avg;
+  private int p_avg;


Underscores(C style) are not allowed in any package, class, field or variable names according to our coding guideline.

leonardehrenfried · 2023-09-01T11:25:10Z

You need to rebase on or merge the dev-2.x branch in order for the compilation to succeed.

optionsome

The issue store entries shouldn't have too generic labels as then it's hard to figure what the errors are related to when browsing through error types.

optionsome · 2023-10-31T14:01:25Z

+    try (InputStream feedInfoStream = new FileInputStream(directory + "/feed_info.txt")) {
+      feedId = readFeedId(feedInfoStream);
+    } catch (IOException e) {
+      issueStore.add("InvalidData", "Reading feed_info.txt failed.");


Suggested change

issueStore.add("InvalidData", "Reading feed_info.txt failed.");

issueStore.add("InvalidEmissionData", "Reading feed_info.txt failed.");

optionsome · 2023-10-31T14:01:34Z

+    try (InputStream stream = new FileInputStream(directory + "/emissions.txt")) {
+      emissionsData = readEmissions(stream, feedId);
+    } catch (IOException e) {
+      issueStore.add("InvalidData", "Reading emissions.txt failed.");


Suggested change

issueStore.add("InvalidData", "Reading emissions.txt failed.");

issueStore.add("InvalidEmissionData", "Reading emissions.txt failed.");

optionsome · 2023-10-31T14:02:17Z

+      zipFile.close();
+      return emissionsData;
+    } catch (IOException e) {
+      issueStore.add("InvalidData", "Reading emissions data failed.");


Suggested change

issueStore.add("InvalidData", "Reading emissions data failed.");

issueStore.add("InvalidEmissionData", "Reading emissions data failed.");

I think in these scenarios where the whole thing fails, we can also add a log line.

optionsome · 2023-10-31T14:02:37Z

+
+      if (avgCo2PerVehiclePerKmString.isEmpty()) {
+        issueStore.add(
+          "InvalidData",


Suggested change

"InvalidData",

"InvalidEmissionData",

optionsome · 2023-10-31T14:02:52Z

+      }
+      if (avgPassengerCountString.isEmpty()) {
+        issueStore.add(
+          "InvalidData",


Suggested change

"InvalidData",

"InvalidEmissionData",

optionsome · 2023-10-31T14:04:00Z

+      reader.readRecord();
+      return reader.get("feed_id");
+    } catch (IOException e) {
+      issueStore.add("InvalidData", "Reading emissions data failed.");


Suggested change

issueStore.add("InvalidData", "Reading emissions data failed.");

issueStore.add("InvalidEmissionData", "Reading emissions data failed.");

optionsome · 2023-10-31T14:04:13Z

+    }
+    if (avgPassengerCount <= 0 || avgCo2PerVehiclePerMeter < 0) {
+      issueStore.add(
+        "InvalidData",


Suggested change

"InvalidData",

"InvalidEmissionData",

leonardehrenfried · 2023-10-31T14:26:38Z

There must be an error in graphql-codegen.yml.

sharhio · 2023-10-31T14:43:38Z

There must be an error in graphql-codegen.yml.

The issue with grams missing? I think I just forgot to run the generation.

leonardehrenfried · 2023-10-31T14:50:47Z

No, I think the import for Grams is wrong.

sharhio · 2023-11-01T06:28:36Z

No, I think the import for Grams is wrong.

You're right, the package was wrong for some reason.

optionsome · 2023-11-01T20:57:17Z

There is a conflict here again. While you solve that, you can also address #5278 (comment) but it's not really required.

t2gran

There are a few things is left to fix.

We only want the root level doc element for a feature to be present in out main doc, there rest should be in the Sandbox doc. There are several examples to follow here.
If a Sandbox feature is not enabled then GraphBuilder module, services, models and repositories should not be created.

Co-authored-by: Thomas Gran <t2gran@gmail.com>

leonardehrenfried · 2023-11-07T10:39:18Z

I checked that all of @t2gran's requests are fulfilled and therefore I'm merging this.

Congratulations, @sharhio for landing this large pull request!

Antiik91 and others added 12 commits June 16, 2023 11:49

load co2 data for digitransit

1faf8a0

remove sout

689d98c

calculate co2 emissions

46c479b

Merge conflict fix

9f27798

updated configuration from placeholder values. Adjusted emissions mod…

b7cd138

…ule data fetching since the utility method was removed. Also initial unit tests added

initial documentation

06f7f79

improved docs

2df7d32

Fixed typo

72c03b1

Fixed typos from tests and docs

9af1ba5

Merge remote-tracking branch 'otp/dev-2.x' into co2

2b0d663

Emissions logic refined, more tests + fixes

a075ffa

generated documentation fixed

740e0c8

optionsome reviewed Aug 7, 2023

View reviewed changes

t2gran added this to the 2.4 (next release) milestone Aug 15, 2023

t2gran reviewed Aug 15, 2023

View reviewed changes

t2gran reviewed Aug 21, 2023

View reviewed changes

Implementation using dagger injection

4f56079

sharhio added 3 commits September 4, 2023 09:15

Merge remote-tracking branch 'otp/dev-2.x' into co2

af88c7d

test fixed

7207521

emissions added to datafetchers

ab6efe8

t2gran modified the milestones: 2.4, 2.5 (next release) Sep 13, 2023

sharhio added 5 commits September 14, 2023 13:11

emissions data from gtfs.zip

571757e

Merge remote-tracking branch 'otp/dev-2.x' into co2

241e784

data should use comma separator

7ec48e4

documentations fixed, comment removed

bd7285b

also read nonzipped gtfs

ccecb7e

optionsome requested changes Oct 31, 2023

View reviewed changes

Emissions issssue type and import fixes

8eb34b5

leonardehrenfried added the +Bump Serialization Id Add this label if you want the serialization id automatically bumped after merging the PR label Nov 1, 2023

leonardehrenfried previously approved these changes Nov 1, 2023

View reviewed changes

optionsome previously approved these changes Nov 1, 2023

View reviewed changes

More logging and Merge remote-tracking branch 'otp/dev-2.x' into co2

3ab8b67

sharhio dismissed stale reviews from optionsome and leonardehrenfried via 3ab8b67 November 2, 2023 09:11

leonardehrenfried previously approved these changes Nov 2, 2023

View reviewed changes

optionsome previously approved these changes Nov 2, 2023

View reviewed changes

t2gran reviewed Nov 2, 2023

View reviewed changes

Update src/main/java/org/opentripplanner/model/plan/Itinerary.java

aa5f235

Co-authored-by: Thomas Gran <t2gran@gmail.com>

sharhio dismissed stale reviews from optionsome and leonardehrenfried via aa5f235 November 2, 2023 14:12

sharhio added 2 commits November 6, 2023 13:13

Emissions model config and doc generation fix

0f331e9

Merge branch 'co2' of github.com:HSLdevcom/OpenTripPlanner into co2

95e5532

optionsome approved these changes Nov 6, 2023

View reviewed changes

optionsome requested review from leonardehrenfried and t2gran November 6, 2023 11:37

leonardehrenfried approved these changes Nov 6, 2023

View reviewed changes

leonardehrenfried merged commit 08c76e1 into opentripplanner:dev-2.x Nov 7, 2023

t2gran pushed a commit that referenced this pull request Nov 7, 2023

Add changelog entry for #5278 [ci skip]

13fe69f

t2gran pushed a commit that referenced this pull request Nov 7, 2023

Bump serialization version id for #5278

70bf18a

optionsome mentioned this pull request Dec 11, 2023

Carbon Savings Counter #134

Closed


		Emissions data should be in json array with objects that have the following properties:

		`db`: database it refers to or CAR when referring to personal travel by car. For Example Linkki, HSL

		Emissions data is loaded from the provided source and embedded in graph building process
		as part of the itinerary.


		### OTP 2.4

		- Initial implementation of Emissions calculation

	- Initial implementation of Emissions calculation
	- Initial implementation of the emissions calculation


		private HashMap<String, DigitransitEmissionsAgency> emissionByAgency;

		public DigitransitEmissionsService(DigitransitEmissions[] emissions) {

	new SerializedGraphObject(
	app.graph(),
	app.transitModel(),
	app.worldEnvelopeRepository(),
	config.buildConfig(),
	config.routerConfig(),
	DataImportIssueSummary.combine(graphBuilder.issueSummary(), app.dataImportIssueSummary())
	)

	issueStore.add("InvalidData", "Reading feed_info.txt failed.");
	issueStore.add("InvalidEmissionData", "Reading feed_info.txt failed.");

	issueStore.add("InvalidData", "Reading emissions.txt failed.");
	issueStore.add("InvalidEmissionData", "Reading emissions.txt failed.");

Conversation

Antiik91 commented Aug 7, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Issue

Unit tests

Documentation

Changelog

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

t2gran Aug 21, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

leonardehrenfried Aug 24, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

leonardehrenfried commented Sep 1, 2023

Uh oh!

optionsome left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

leonardehrenfried commented Oct 31, 2023

Uh oh!

sharhio commented Oct 31, 2023

Antiik91 commented Aug 7, 2023 •

edited

Loading

t2gran Aug 21, 2023 •

edited

Loading

leonardehrenfried Aug 24, 2023 •

edited

Loading