Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Envision near real time (#785) * Added frame_time in envision State, frame_time is the elapsed simulation time * Added "near real-time" and "uncapped" mode in Envision * add optimization to get frame_time from frame in server.py by using ijson instead of json loading the entire frame Co-authored-by: Stephen Hockema <stephen.hockema1@huawei.com> * Zoo manager spawned as a subprocess (#814) * Envision fixes (#810) * Fixed race condition with seek flushing. * formatting * also limit stateQueue size while I'm at it. * More options for buffer overflow. * added comment * Quiet Envision logs (#820) * Quiet error received when initially unable to connect to Envision. * output a warning on the first failed connection only. * formatting * added to CHANGELOG * Quiet pybullet import msgs (#821) * quiet "pybullet build time: ..." messages on stdout when pybullet is imported. * added CHANGELOG entry * formatting * typo fix in CHANGELOG * fixed typo in filename * fixed backwards polarity on dones check (#824) * fix rounding cf timestp_sec to ensure we always keep enough digits (#825) * ULTRA Update callbacks (#818) * update episode callback * Apply suggestions from code review Co-authored-by: christianjans <49890535+christianjans@users.noreply.github.com> Co-authored-by: christianjans <49890535+christianjans@users.noreply.github.com> * Added a new pypi instructions file to update general user installation instructions * Ignore tune test in ULTRA's heavy CI tests (#811) * Make naming consistent for task 2's levels (#806) * Fixed typo in instructions * Cleaned up the instructions to only include the pip install instructions * Added further instructions about the modules involved in the package and how to use them to create environments and scenarios * updated CHANGELOG.md * updated README_pypi.md based on changes requested * fixed a few more minor typos * added command to remove all .coverageXX files after the tests commands exit * added a separate recipe to remove all the coverage files * minor bug fix * remove __init__.py files from scenarios and examples modules so that they are removed from python installation package or pip * Revert "remove __init__.py files from scenarios and examples modules so that they are removed from python installation package or pip" This reverts commit bdf80e0 * remove __init__.py file from scenarios module so that it is removed from python installation package or pip * rename the __init__.py file to argument_parser.py for examples folder * renamed README_pypi.md * Add social vehicles controlled by custom policy through bubbles (#793) * Add ability to include bubbles in scenarios * Add 'bubbles' to task configs * Add stops test * Add bubbles test * Refactor stops implementation in generate_scenarios.py * Refactor stops generation * Add type hints and docstrings * Remove stops from default bubble scenarios * Add __init__.py to tests/tune/ (#840) * Fix README typos in ULTRA (#764) * Fix typos in ultra/README.md * Fix typo in the citation * Fix headless argument behaviour in ULTRA (#737) * Fix headless argument in ULTRA scripts * Change headless flag to behave like SMARTS * Control number of episodes in ULTRA tests (rllib and train) (#739) * Make evaluations run in parallel to training * Add rollout_fragment_length and max_episode_steps to rllib * Add new flags to documentation * Fix flag in example * Ensure evaluation is not run in test_train.py * Check if log_dir doesn't exist in test_train.py * Fix typo in argument and set eval_episodes to 0 * Check if not exists for paths * Fix formatting * Update evaluate and train to evaluate per episode (#639) * Update evaluate and train to eval per episode * Fix tests and change normalization * Fix formatting * Record episode for train/eval at the same time * Train performance averages last eval_rate episodes * Add gap_mode and calculate generalization gap * Enable explore during eval for better comparison * Fix formatting * Fix averaging in record_episode * Only average scalars in record_episode * Check for scalar when calculating gap * Fix formatting * Fix bugs introduced by merge * Fix calculate_gap * Fix averaging over episodes * Fix formatting * Check eval_episodes is 0 before evaluation * Evaluate on train and test without exploration * Make train/test occur in one ray.get * Remove episode_count * Remove episode_count from tests * Remove averaging over training episodes * Restructure evaluation_check with eval_mode = False * Fix formatting * try split ci test method * Check for valid eval_episodes inside evaluation_check * Removed averaging feature of record_episode * Fix formatting Co-authored-by: Jenish Patel <jenish@ualberta.ca> * Merge with ultra-rllib-speedup-tests * Rename evaluation_task_id for training * Simplify argument to ray.wait(...) * Remove print statements * Fix defaults * Fix eval-rate in docs * Initial changes (#744) Co-authored-by: christianjans <cjjans@ualberta.ca> Co-authored-by: Jenish Patel <jenish@ualberta.ca> Co-authored-by: christianjans <49890535+christianjans@users.noreply.github.com> * Bump 0.4.15 -> 0.4.16 * Make format * Update missing version numbers * switched import statement to relative import * bug fix * made a new bin folder and moved gen_header.sh and header_test.sh to that folder * made a new etc folder and moved the third party folder into it * moved tools folder to etc folder * moved extras folder to etc * moved tools folder to examples folder * moved benchmark folder to etc folder * moved benchmark folder to cases folder and updated import statements * updated Makefile with proposed changes * moved install_deps.sh to etc/setup * updated Makefile based on changes proposed * moved docker files to etc/docker folder * fixed script paths in Makefile * fixed script paths in ci files * bug fix * bug fix * added recipe to remove coverageXXX and coverage.XXX files in Makefile * added README.pypi.md to etc/setup * fixed paths for dockerfile and made path changes in README.md * updated CHANGELOG.md * added __init__.py to scenarios and examples and removed them from pip installation in setup.py * updated based on proposed changes * revert back to absolute paths * added __init__.py * update CHANGELOG.md * new Imitation action space (#854) * first pass at new Imitation action space also fixed bug for non-dynamic vehicles not getting actions * Only do non-dynamic actions for agents not owned by a Provider * use actual mass from chassis instead of hardcoding * added hack to allow setting the initial vehicle speed * don't use Sumo here by default * added CHANGELOG entries * update from review: normalize by 4-wheel torque * ULTRA Improve high density traffic (#783) * experiment with different begin time functions * fixed map position in gif * added legends * WIP: experimenting with different combinations of traffic settings * save begin time settings * added more maps to high-density task, constrict stopwatcher begin time and t-intersection distribution improved from high-density traffic * formatted * clean up * final traffic settings saved * added mid-density to compare, modified high-density distribution * simplified the high density traffic to lessen congestion * more tuning * more tuning * added video analysis test * clean up * Apply suggestions from code review Co-authored-by: christianjans <49890535+christianjans@users.noreply.github.com> * changes from code review * comment changes * slight mod to traffic distribution & relaxed stopwatcher begin times * Update begin_time_init_funcs.py Co-authored-by: christianjans <49890535+christianjans@users.noreply.github.com> * remove debugging levels * small fix Co-authored-by: christianjans <49890535+christianjans@users.noreply.github.com> * Remove extra comma in ULTRA citation (#849) * Remove extra comma in README citation * Use period instead of comma * revert back the file to prevent ci-test from failing for now * bug fix * reproducing bug * reproducing bug * bug fix * Fixed the bug by turning off the wrong_way event when vehicle goes into an intersection. Updated CHANGELOG.md. * Traffic History Dones (#827) * better handling of Dones for traffic history - detect vehicles dropping out of history - added ExitGoal for imitation learning agents * Updated CHANGELOG. * back out deprecation fix * allow for collisions to be reported in Events even if goal reached * Changed ExitGoal to TraverseGoal. Backed out semantic change to done Events. (feedback from dicussions) * type hints * updates from review & formatting * merge * Multiple replacement of history vehicles (#836) * First pass at multi-agent IL example * choose only overlapping vehicles to replace also, refactored TrafficHistory to clean up database queries * moved TraffiCHistory class to its own file * minor none check for replaced history vehicles * refactored overlapping_choice into TrafficHistory * updates from review * allow -k as a shortcut for --replacements-per-episode * formatting * added some type hints * more type hints * updates from review * updates from review * formatting * more type hints * fixed weird bug in last PR * updated based on changes proposed * updated based on changes proposed * updated based on changes proposed * removed the additional files to reproduce the bug * refactoring * reformatting sensors.py * reformatted sensors.py through black * updated CHANGELOG.md * added max-steps arg (#861) * Fix permission error. * moved the script etc/setup folder * ULTRA Simple level (#857) * added simple level * classified simple levels, position offsets Classified simple levels into v0, v1, v2. Move the position offsets to route distribution * small modification * pos_offset not mandatory * tweaked distribution flags & formatted * moved cross intersection types to test section in levels * bubbles flags added * renamed simple distribution, added a more general simple level * reinstated blocks distribution * Apply suggestions from code review Co-authored-by: christianjans <49890535+christianjans@users.noreply.github.com> * moved debugging levels from task1 -> taskX Co-authored-by: christianjans <49890535+christianjans@users.noreply.github.com> * update based on proposed changes * path fix * reformatted file using black * renamed etc folder to conf, moved README.pypi.md to utils/setup and supervisord.conf to examples * renamed cases to baselines * removed CONTRIBUTING.md from docs and updated the top level one * fixed CI test path * os path fix * reformatting file using black * Bugfix memory issues (#852) * Cap threadpoolexecutor virtual memory growth * Fix sumo memory growth * Make format * Apply suggestions * Update reset of sumo to every 50 * Add fixes to changelog * TODO to remove fix when memory growth fixed * revert back to 4 grpc connection workers * Make format * Bugfix remove unused `make` pview (#845) * Remove pview for unused .egg files * Update changelog * refactored repeating code to a class method * updated CHANGELOG.md * reformatted code * removed unit test file from another branch * ULTRA Update existing maps (#868) * update existing maps * formatted * changes from review * only observe cars for imitation learning (#874) * only observe cars for imitation learning data collection * added type hint * revert to absolute paths * prevent pybullet from updating during setup with python3.8 * updated CONTRIBUTING.md * resolve merge issue * ULTRA Add medium level (#869) * added medium level * adjusted traffic distr. in medium level * removed curvy_t intersection type * updated simple level moved existing simple level into taskX * cleaned up taskX config * changed level's total * Update ultra/ultra/scenarios/taskX/config.yaml Co-authored-by: christianjans <49890535+christianjans@users.noreply.github.com> * updated simple level Co-authored-by: christianjans <49890535+christianjans@users.noreply.github.com> * updated CHANGELOG.md * removed unwanted packages * updated requirements.txt * updated CHANGELOG.md * Changes based on reviews * Created a seperate setup.py for benchmark to remove opencv-python from SMARTS in general. * updated requirements.txt * updated requirements.txt * updated requirements.txt * examples and bug fixes for imitation learning (#878) * examples and bug fixes for imitation learning * added PR number to CHANGELOG * took out debugging hack * use BoxChassis (kinematics only, no physics) by default for Imitation action space * modified placeholder agent to replay from saved pkl files. +-get starting speed for mission, bug fixes * formatting * remove unnecessary dependencies * better rounding for history time query * do collision processing for non-dynamic action spaces too * fixed overzelous removal of provider type check + updates from review * added a few more type hints * reverted more over-zealousness re provider merging * fixed minor bug for when speed==0. * updates from review: better timestep_sec rounding * backed out Observation change + misc updates from review * re-compute traffic history speeds based on actual positions * Moved speed re-computation to genhistories to get out of step loop. - use next pos instead of prev pos in speed computation Fixed minor bug with obs_time hack. * don't check position on first step * enforce vehicle history limit to cars * fix for isclose() near 0 * move filtering by position to the end of genhistories (so all are used in smoothing stuff for edge effects) * change back to scalar acceleration for Imitation action space * formatting * use current and next pos to compute history speed * record and save the first observation (prior to step) too * fixed history replay 1-off timestep issue also added mission start adjustment hack for front bumper * ULTRA add `--durations` flag to ULTRA CI tests (#885) * added durations flag * test pypi ultra test * minor name fix * history agent vehicle sizing (#886) * allow traffic history agent vehicles' sizes to be set * simplified controller calculations (should not change them) * oops * use angular_velocity for position updates too * use current angular_velocity instead of next * reverted angular_velocity correction to position in controller also fixed bug to prevent stepping pybullet twice * Refactor Waypoints (#872) * waypoints/lanepoits refactor: first pass * incremental * move lanepoints into sumo_road_network * moved Waypoints (back) into MissionPlanner * added some type hints * minor cleanup * updated comment * added WaypointsCache and other minor performance optimizations * debugged waypoints cache * removed no-longer used method * updates from review * fixed typos * reduced lane-following lookahead to 10 * revert lane_following_controller lookahead reduction * added CHANGELOG entry * update per review: fixed hash for LinkedLanePoints * added more type hints per review * remove start/end offset from simple level (#891) * Proper shutdown of RemoteAgentBuffer after code crash (#822) * Removed atexit destroy in RemoteAgentBuffer. * fix internal holes in sumo road network * Updated CHANGELOG.md * code reformatting * code reformatting * comments * fixed all holes that were at extremely sharp turn (almost perpendicular). * traffic-history data speed and heading tweaks. (#896) * re-calculate speeds and headings based on position changes for both NGSIM and INTERACTION datasets * comment * ULTRA Update `docs/setup.md` (#903) * updated setup.md * remove manual black installation step * added install_deps.sh into ultra * Update ultra/docs/setup.md Co-authored-by: christianjans <49890535+christianjans@users.noreply.github.com> * empty commit Co-authored-by: christianjans <49890535+christianjans@users.noreply.github.com> * added an experiment utility file to replace context of supervisord.conf * reformatting * removed supervisord from dependencies and requirements * Ensured unique deque objects were used in framestack. (#894) * Updated README.md * updated CHANGELOG.md * updated requirements.txt * updated requirements.txt * updated requirements.txt * resolving merge conflict * resolving merge conflict * resolving merge conflict * path fixes * path fixes * Fix args count error caused by `websocket` (#906) * Fix vargs error sometimes caused by websocket * Change vargs to 2 additional defaults * Add fix to changelog * ignore options not defined in the command * reformatting * open new web browser tab automatically * reformatting * added scl zoo command to installl agents at the specified paths * Updated CHANGELOG.md * update tensorflow for Rl_Agent * Update getting_started.md (#925) * update tensorflow for Rl_Agent * added test file for rl-agent * updated the wheel file for rl-agent installation * updated based on reviews * added another option to change port * Small fix to docs/getting_started.md (#931) * cleanup * reformatting and more detail to error output * reformatting and more detail to error output * bump up the version number, update scenarios for rl-agent * updated wheel file for rl-agent * updated CHANGELOG.md * updated README.md and cleanup the environment before installing the agents. * updated README.md and cleanup the environment before installing the agents. * Add flag to disable mission shuffling when generating scenarios in ULTRA (#927) * Add option to generate scenarios without shuffling missions * Add comments to mission shuffling tests * Fix comments' flag names * Warning if port passed without starting envision. * Warning if port passed without starting envision. * remove argument parser (#935) * refactoring * revert changes * bug fix * bug fix * ULTRA Migrate from step-based to episode-based sampling (#932) * change agent_itr to episode.index * formatted * recording data episodically in tune.py and train.py * change recording_step inside evaluate_saved_model() * added episode indices test * Update ultra/tests/test_evaluate.py Co-authored-by: christianjans <49890535+christianjans@users.noreply.github.com> * Update ultra/tests/test_evaluate.py Co-authored-by: christianjans <49890535+christianjans@users.noreply.github.com> Co-authored-by: christianjans <49890535+christianjans@users.noreply.github.com> * replace the panda3d.Quat.xform method to remove dependency from panda3D package * wrap the import statements and ShowBaseInstance class in try block and warn user if they have not installed the [render] dependencies. * make sure the CI tests install dependencies before rendering camera observations * remove panda3d from dependencies in setup.py * Updated CHANGELOG.md * ULTRA Run tests in forked subprocesses (#937) * forked train test * remove backslash * try forking evaluate test * small fix * small fix * small fix * comment added * update documentation about rendering and camera based observations * moved helper functions to utils/math * reformat the try block to raise RendererException with warning message. Comments. * rename the dependency name from render to camera-obs * Multi-agent FrameStack wrapper (#916) * Added multi-agent FrameStack wrapper and test. * cleaner way to make renderer optional * formatting * Refactor ULTRA adapter to support benchmarking (#871) * Initial ideas for refactored adapter * Define the reward and observation adapters * Finish basic adapter types in adapter.py * Get PPO baseline to run * Get SAC baseline to run * Get TD3 baseline to run * Get RLlib training to work * Get DQN baseline to run * Get BDQN baseline to run * Get ULTRA Tune to run * Pass ULTRA CI tests * Format the code * Move adapters to own directory * Remove original adapter and state preprocessors * Add discrete action Gym space * Implement default image observation adapter function * Simplify required_interface_fromtypes function * Initialize ultra_scores from enum type * Comment constants in default vector and reward * Change adapter CI test to use new adapters * Use seed from parameters for social vehicle config in RLlib * Add docstrings to adapters * Convert TODO regarding ULTRA score to NOTE in ULTRA environments * Fix small logic errors in default_observation_image_adapter * Apply small changes from review * Create deepcopy of observation before performing vector adaptation * Add initial documentation for default adapters * Add reason for sorting social vehicles by distance in adapters.md * Keep consistent line length in adapters.md * Add comments to image adapter constants * Edit docstring of image adapter function * Change image observation adapter to produce a 3D array * Test adapter's Gym space in adapter tests * Change gray-scale weighting to (0.1, 0.8, 0.1) * Fix 'enironment' typo and describe the_environment_reward in docs/adapters.md * Keep variable name of the rewards obtained from the environment consistent * User assertIn instead of assertTrue for testing containing in test_adapter.py * Add default info adapter to remove reward adapter's NeighborhoodVehicles dependency (#902) * Initial changes * Format the code * Remove ultra_scores from the ULTRA environment * Add info adapter documentation * Add info adapter test * Move shared adapter constants a common file * Make trimming of extra social vehiccles explicit * Replace RLlib environment info adaptation with default info adapter in agent * minor bug fix. Reformatting * Say which tests are run in ULTRA CI tests rather than use `--ignore` (#941) * Explicitly say which tests are run in heavy and light * Fix 'analysys' spelling mistake * ULTRA Minimalize `train.py` script (#904) * Initial ideas for refactored adapter * Define the reward and observation adapters * Finish basic adapter types in adapter.py * Get PPO baseline to run * Get SAC baseline to run * Get TD3 baseline to run * Get RLlib training to work * Get DQN baseline to run * Get BDQN baseline to run * Get ULTRA Tune to run * Pass ULTRA CI tests * Format the code * Move adapters to own directory * Remove original adapter and state preprocessors * Add discrete action Gym space * Implement default image observation adapter function * Simplify required_interface_fromtypes function * Initialize ultra_scores from enum type * Comment constants in default vector and reward * Change adapter CI test to use new adapters * Use seed from parameters for social vehicle config in RLlib * Add docstrings to adapters * Convert TODO regarding ULTRA score to NOTE in ULTRA environments * Fix small logic errors in default_observation_image_adapter * Apply small changes from review * Create deepcopy of observation before performing vector adaptation * Add initial documentation for default adapters * save work * moved parsing outside of main * made build_agent() public * empty commit * added max_steps args * empty commit * small fix * changes from code review * small fix * bypass hard coded file name * clean up Co-authored-by: christianjans <cjjans@ualberta.ca> * Add trajectory interpolation provider. * Update scenario building documentation and arguments in ULTRA (#944) * Fix scl scenario build command in docs * added default value to pool_dir arg * make path relative to base ultra dir * update scenario README * add logic clause to allow build_scenarios() function calls to not mandate using pool_dir argument * Change pool_dir initialization in build_scenarios Co-authored-by: Jenish Patel <jenish@ualberta.ca> * Add frame stacking to ULTRA (#948) * Initial changes * Add initial environment documentation * Fix ULTRA environment and adapter test to pass tests * Clean up the ULTRA environment and environment test * Add note to RLlib training about SMARTS observations adaptation * Update ULTRA environment documentation * update MANIFEST.in (#956) * fix multi-display envision * prevent envision to reconnect to a simulation every 3 second * prevent envision to reconnect to a simulation every 3 second * prevent envision to reconnect to a simulation every 3 second. bug fixes. * Updated CHANGELOG.md * Build and Update the envision distribution based on changes. * Bug fixes. Major reformatting for .js files. * build and update map * fix envision to check for a new simulation every 3 seconds and not redirect when switched to a multi-instance display. * build and update the distribution. Reformatting. * typo fix * changes based on reviews * fork all light base tests and observe duration (#967) * ULTRA Evaluate specific models (#915) * refactor existing evaluate_saved_models * fixed bug * updated evaluation process, fixed tune test issue * Update getting_started.md * updated getting_started.md documentation * clean up * added exceptions * small fix Co-authored-by: christianjans <49890535+christianjans@users.noreply.github.com> Co-authored-by: christianjans <cjjans@ualberta.ca> Co-authored-by: christianjans <49890535+christianjans@users.noreply.github.com> * ULTRA baselines with new adapters (#958) * Initial ideas for refactored adapter * Define the reward and observation adapters * Finish basic adapter types in adapter.py * Get PPO baseline to run * Get SAC baseline to run * Get TD3 baseline to run * Get RLlib training to work * Get DQN baseline to run * Get BDQN baseline to run * Get ULTRA Tune to run * Pass ULTRA CI tests * Format the code * Move adapters to own directory * Remove original adapter and state preprocessors * Add discrete action Gym space * Implement default image observation adapter function * Simplify required_interface_fromtypes function * Initialize ultra_scores from enum type * Comment constants in default vector and reward * Change adapter CI test to use new adapters * Use seed from parameters for social vehicle config in RLlib * Add docstrings to adapters * Convert TODO regarding ULTRA score to NOTE in ULTRA environments * Fix small logic errors in default_observation_image_adapter * Apply small changes from review * Create deepcopy of observation before performing vector adaptation * Add initial documentation for default adapters * Add reason for sorting social vehicles by distance in adapters.md * Keep consistent line length in adapters.md * Add comments to image adapter constants * Edit docstring of image adapter function * Change image observation adapter to produce a 3D array * Initial changes * Fix bug in image replay buffer dataset * Add initial replay buffer test * Clean up replay buffer test * Make CNNs exclusively process images * Remove BDQN's explore.py and network.py * Remove extra import of DQNWithSocialEncoder * ULTRA Sanity check on baselines (#965) * added test to train all baselines * formatted * added evaluation to baseline tests * fix agent interface numbering (#978) Co-authored-by: Rutvik Gupta <RutvikGupta@users.noreply.github.com> * fixed genhistories bug introduced in PR #896. * Graceful shutdown of zoo when daemon process terminates abruptly (#924) * Update version number Co-authored-by: Jeffer Jingfei Peng <36685292+JingfeiPeng@users.noreply.github.com> Co-authored-by: Stephen Hockema <stephen.hockema1@huawei.com> Co-authored-by: adai <adaickalavan@gmail.com> Co-authored-by: sah-huawei <80282957+sah-huawei@users.noreply.github.com> Co-authored-by: JenishPatel99 <55423380+JenishPatel99@users.noreply.github.com> Co-authored-by: christianjans <49890535+christianjans@users.noreply.github.com> Co-authored-by: Rutvik Gupta <RutvikGupta@users.noreply.github.com> Co-authored-by: Rutvik Gupta <47334253+RutvikGupta@users.noreply.github.com> Co-authored-by: Alex Lewandowski <lewandowskialexp@gmail.com> Co-authored-by: christianjans <cjjans@ualberta.ca> Co-authored-by: Jenish Patel <jenish@ualberta.ca> Co-authored-by: Adaickalavan Meiyappan 84166141 <adaickalavan.meiyappan@huawei.com> Co-authored-by: guptarut <https://markus.cs.teach.edu/git/csc209-2020-01/guptarut> Co-authored-by: Valax Kong <valaxkong@163.com>
- Loading branch information
14 people
committed
Jul 2, 2021
1 parent
c5a1608
commit 2c1979e