Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Agent simulation development #244

Merged
merged 43 commits into from
Mar 10, 2021
Merged

Agent simulation development #244

merged 43 commits into from
Mar 10, 2021

Conversation

zptro
Copy link
Collaborator

@zptro zptro commented Dec 2, 2020 •
edited

Agent simulation option added to dev-config.json. Requires at least 12 GB of RAM to run with full population.

Major improvements of agent simulation

  • bugs fixed
  • recycled tour objects and stable choices over iterations
  • much more efficient choice draws, with cumulative probability vectors shared among tours
  • printing of results
  • minor general refactoring of demand side

For 2018 scenario, the agent model is as fast as the aggregate model

  • more overhead in tour generation, destination and mode choice
  • secondary destination choice only for OD pairs with demand makes it much faster

West Jens and others added 21 commits December 2, 2020 09:04
Allow age groups share sum to go slightly above 1
6b77dee
Added agent demand fraction for simulation
a81212f
Added expansion factor to departure time model
7d046ca 
Simplified departure time model for agents
Split agent creation and car usership
018542e
Merge branch 'master' into feat/agents
6e0922d
Fix sec dest probability bug
2e6da5d
Optimizations of agent model (#246)
3538620 
* Split primary and secondary destination choice

* Calc cumulative probs in advance to save calculation time in agent simulation

* Do sec dest calculations more efficiently

* Refactor position variable

* Calc cumulative probs in advance for sec dests

* Simplify zone index lookup

* Change mode into implicit property

* Use defaultdict for sec_dest_tours

* Make car passenger separate property

* Calc cumulative probs in advance for tour combinations
Recycle tours
d2957d3
Stabilize choices
26d068b
Make resultdata.printdata more failsafe, add printing of agent
367638b 
mode shares and attractions
Print second destination attraction
61e3549
Add agent model functionality to helmet.py
4166ff5
Make agent model multi-threaded
ad8a159
Simplify and speed-up division of population
311de85
Fix how RUN_AGENT_SIMULATION is taken from config
d4f034b
Fix assignment result printing
c2f6d79
Restructured multithreading
5bed782
Simplify sec dest impedance handling
a2b2ce1
Simplify and optimize division of zones for multithreading
88dece1
Rename variable
d8edba7
Move references to parameters, remove unnecessary script
f0f13e9 
Simplify sum initialization using defaultdict
@zptro zptro linked an issue Jan 20, 2021 that may be closed by this pull request
transform.demand_transformer is not used #253
Closed
@zptro zptro marked this pull request as ready for review January 25, 2021 10:55
attesn
attesn requested changes Feb 9, 2021
View reviewed changes
Copy link
Collaborator

@attesn attesn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested few agent model runs with MockAssignment and results seems to quite stable (mode share varies 0.0-0.3 %). Results seem to be reasonable compared to main as well. However, in some zones there are significant changes in mode specific trip sums. Not yet sure if these are due to printing or other issues, so would fix result printing before merging. I think I could also try agent model with Emme and check network volume differences.

Some issues:

  • Zone specific mode sums printed only for transit and car. See comment below.
  • Result impedance_ratio is printed only for time, because car_cost array is zeros. Didn't find out where this is different from to main.
  • Results aggregated_demand.txt and trips_lengths.txt are printed only for surrounding municipalities. Could we move printing of these files to purpose.print_data?

Also some minor comments listed below.

Scripts/demand/trips.py Outdated Show resolved Hide resolved
Scripts/datatypes/person.py Show resolved Hide resolved
Scripts/models/logit.py Show resolved Hide resolved
Scripts/datatypes/tour.py Show resolved Hide resolved
Scripts/modelsystem.py Show resolved Hide resolved
@zptro
Copy link
Collaborator Author

zptro commented Feb 11, 2021 •
edited

  • Result impedance_ratio is printed only for time, because car_cost array is zeros. Didn't find out where this is different from to main.

Yes, need to dig into this.

  • Results aggregated_demand.txt and trips_lengths.txt are printed only for surrounding municipalities. Could we move printing of these files to purpose.print_data?

The problem is that these results are aggregated from the demand matrix, and in agent simulation we have no purpose-specific demand matrix, only set of Tour objects. We need to implement another solution for these results.

@zptro
Copy link
Collaborator Author

zptro commented Feb 15, 2021

  • Result impedance_ratio is printed only for time, because car_cost array is zeros. Didn't find out where this is different from to main.

This seems to be another inconsistency in MockAssignment. Car cost is actually defined differently in demand calculation and in CBA. In demand calculation, distance cost is included, but not in CBA. Because MockAssignment takes matrices which are meant for CBA, they do not produce the correct impedance ratios, and not the correct demand in general. Created issue for this: #262

  • Results aggregated_demand.txt and trips_lengths.txt are printed only for surrounding municipalities. Could we move printing of these files to purpose.print_data?

Solved.

Jens West added 2 commits February 15, 2021 13:10
Income imputation (#260)
c9f1e17 
* First draft of income model

* Add parameters

* Refactoring of income model and simplification of `LinearModel`

* Delete obsole import

* Add random term to income model

* Avoid division by zero when no trips generated for zone
Merge branch 'master' into feat/agents
b4dc407
@zptro zptro requested a review from attesn February 15, 2021 11:17
attesn
attesn approved these changes Feb 18, 2021
View reviewed changes
Copy link
Collaborator

@attesn attesn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Run with full model system and results seem to be ok. Some minor things listed below.

There might still be some minor issues, but I think I could fix them when adding output scripts.

Scripts/modelsystem.py Outdated Show resolved Hide resolved
Scripts/datatypes/person.py Show resolved Hide resolved
@zptro zptro added this to the v4.0.4 milestone Mar 5, 2021
@zptro zptro merged commit 872ace6 into master Mar 10, 2021
@zptro zptro deleted the feat/agents branch March 10, 2021 13:44
@attesn attesn mentioned this pull request Mar 16, 2021
Closed
@zptro zptro mentioned this pull request Mar 15, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@attesn attesn attesn approved these changes

@samulikyytsonen samulikyytsonen Awaiting requested review from samulikyytsonen

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
v4.0.4
Development

Successfully merging this pull request may close these issues.

transform.demand_transformer is not used
2 participants
@zptro @attesn