Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow xml write/read of additional attributes #118

Merged
merged 39 commits into from
Jul 14, 2022
Merged

Conversation

KasiaKoz
Copy link
Collaborator

@KasiaKoz KasiaKoz commented Jun 8, 2022

Until now we only allowed additional attributes for links. They exist in the following format in the link attribute dictionaries:

link_data = {
    'id': 'link_id',
    ...
    'attributes': {
        'additional_attrib': {'name': 'additional_attrib', 'class': 'java.lang.String', 'text': 'attrib_value'}
    }
}

and are saved to xml in the following format:

    ....
    <links capperiod="01:00:00" effectivecellsize="7.5" effectivelanewidth="3.75">
        <link id="link_id" ... >
            <attributes>
                <attribute name="additional_attrib" class="java.lang.String">attrib_value</attribute>
            </attributes>

This PR extends this to other elements:

  • network nodes
  • schedule stops
  • schedule routes
  • schedule services

which follow the same format as links: 'attributes': {'additional_attrib': {'name': 'additional_attrib', 'class': 'java.lang.String', 'text': 'attrib_value'}}, and

  • global network level attributes
  • global schedule level attributes

Example

This is what a network with extra attributes looks like:

<network>
    <attributes>
        <attribute name="crs" class="java.lang.String">epsg:27700</attribute>
    </attributes>
    <nodes>
        <node id="21667818" x="528504.1342843144" y="182155.7435136598"/>
            <attributes>
                <attribute name="attrib" class="java.lang.String">hello</attribute>
            </attributes>
        <node id="25508485" x="528489.467895946" y="182206.20303669578"/>
    </nodes>
    <links capperiod="01:00:00" effectivecellsize="7.5" effectivelanewidth="3.75">
        <link id="1" from="25508485" to="21667818" freespeed="4.166666666666667" capacity="600.0" permlanes="1.0" oneway="1" modes="subway,car,walk,metro" length="52.765151087870265">
            <attributes>
                <attribute name="osm:way:access" class="java.lang.String">permissive</attribute>
            </attributes>
        </link>
    </links>
</network>

This is an example of a schedule with additional attributes:

<transitSchedule>
    <attributes>
	<attribute name="crs" class="java.lang.String">epsg:27700</attribute>
        <attribute name="additional_attrib" class="java.lang.String">attrib_value</attribute>
    </attributes>
    <transitStops>
        <stopFacility id="s1" x="1.0" y="1.0" name=""/>
            <attributes>
                <attribute name="stop_attrib" class="java.lang.String">attrib_value</attribute>
            </attributes>
        <stopFacility id="s2" x="1.0" y="1.0" name=""/>
    </transitStops>
    <transitLine id="s1" name="r1">
        <attributes>
            <attribute name="service_attrib" class="java.lang.String">attrib_value</attribute>
        </attributes>
        <transitRoute id="r1">
            <attributes>
                <attribute name="route_attrib" class="java.lang.String">attrib_value</attribute>
            </attributes>
            <transportMode>bus</transportMode>
            <routeProfile>
                <stop refId="s1" departureOffset="00:00:00"/>
                <stop refId="s2" arrivalOffset="00:01:00"/>
            </routeProfile>
            <departures>
                <departure id="r1_07:00:00" departureTime="07:00:00" vehicleRefId="veh_bus_r1_07:00:00"/>
                <departure id="r1_07:20:00" departureTime="07:20:00" vehicleRefId="veh_bus_r1_07:20:00"/>
                <departure id="r1_07:40:00" departureTime="07:40:00" vehicleRefId="veh_bus_r1_07:40:00"/>
                <departure id="r1_08:00:00" departureTime="08:00:00" vehicleRefId="veh_bus_r1_08:00:00"/>
            </departures>
        </transitRoute>
    </transitLine>
</transitSchedule>

Caveats

There are a few caveats, I think I cleaned it up so it makes sense, but if you have better ideas, do let me know!

C1

Attributes for Schedule elements (Routes, Services, Stops) can be changed using the dedicated apply_attributes_to_X methods in the Schedule in which case you apply the changes in the same way as network nodes and links:

{'id': {'attributes': {'additional_attrib': {'name': 'additional_attrib', 'class': 'java.lang.String', 'text': 'attrib_value'}}}

and that is logged into the schedule's change_log, BUT, those elements can also be accessed and a class instance of e.g. Route will have an attributes attribute that can be accessed and changed, but that will not be recorded in the changelog:

>>> some_route = schedule.route('id')
>>> some_route.attributes

{'additional_attrib': {'name': 'additional_attrib', 'class': 'java.lang.String', 'text': 'attrib_value'}}

>>> some_route.attributes['another_attrib': {'name': 'another_attrib', 'class': 'java.lang.String', 'text': 'another'}]
>>> some_route.attributes

{'additional_attrib': {'name': 'additional_attrib', 'class': 'java.lang.String', 'text': 'attrib_value'}, 'another_attrib': {'name': 'another_attrib', 'class': 'java.lang.String', 'text': 'another'}}

Services, Routes and Stops may or may not have the attributes attribute and that can be checked via the x.has_attrib('attributes') method.

C2

Network and Schedule attributes can only be accessed via the attributes attribute and that always exists and at minimum holds the projection for the element, and for most our networks will have the 'simplified True' attribute. Stuff can be added and changed, and it is not recorded

>>> network.attributes

{'crs': {'class': 'java.lang.String', 'name': 'crs', 'text': 'epsg:27700'}}

C3

There is another element for MATSim that is not supported: trip departures can also have additional attributes. But the way we store information for trips does not make it easy for us to allow this. I doubt we'd make much use of it though.

Future Work

FW 1

Now that network and schedule elements always have a crs attribute we should make the epsg parameter in the matsim read functions optional. One thing that needs to happen also is for PUMA to always save these attributes too.

FW 2 [SOLVED, see #124]

I absolutely hate this format:

'attributes': {
        'additional_attrib': {'name': 'additional_attrib', 'class': 'java.lang.String', 'text': 'attrib_value'}
    }

We should implement something that has less repetition, at least:

'attributes': {
        'additional_attrib': {'class': 'java.lang.String', 'text': 'attrib_value'}
    }

or maybe even

'attributes': {
        'additional_attrib': 'attrib_value' 
    }

which default to the right java class when you save it to matsim...

@KasiaKoz KasiaKoz requested review from mfitz and ana-kop June 8, 2022 13:42
Copy link
Contributor

@mfitz mfitz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the README need updating to describe this new functionality?

genet/core.py Show resolved Hide resolved
genet/core.py Outdated Show resolved Hide resolved
genet/output/matsim_xml_writer.py Outdated Show resolved Hide resolved
tests/test_core_network.py Show resolved Hide resolved
tests/test_core_schedule.py Show resolved Hide resolved
tests/test_input_matsim_reader.py Outdated Show resolved Hide resolved
KasiaKoz and others added 3 commits June 28, 2022 09:51
* add assuming java types for additional attributes when writing to xml

* add assuming python types for additional java-typed attributes when reading from xml

* fix lingering dependence on long form attributes

* update notebooks with simple form attributes

* extra fixes, add dtype param to road pricing to match with OSM ID dtypes

* increase code coverage

* address PR comments

* PR update: enable forcing long form when reading matsim networks

* lint
# Conflicts:
#	scripts/add_elevation_to_network.py
#	tests/test_core_network.py
@KasiaKoz KasiaKoz requested a review from mfitz June 30, 2022 16:41
Copy link
Contributor

@mfitz mfitz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to go!

KasiaKoz and others added 2 commits July 1, 2022 16:17
# Conflicts:
#	genet/outputs_handler/matsim_xml_writer.py deleted in HEAD and modified in master.
* add example script for snapping pt stops to other parts of the network

* warn of unsnapped stops if distance threshold reached prematurely

* relax condition

* add handling multiple network modes and teleport (access only) modes
@KasiaKoz KasiaKoz merged commit c1e69a5 into master Jul 14, 2022
@KasiaKoz KasiaKoz deleted the arbitrary-attribs branch July 14, 2022 09:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants