Skip to content

Commit

Permalink
Merge master into v0.3 (#536)
Browse files Browse the repository at this point in the history 
* update docker hub init (#367)

* update docker hub init

* replace personal account with maro-team

* update hello files for CIM

* update docker repository name

* update docker file name

* fix bugs in notebook, rectify docs

* fix doc build issue

* remove docs from playground; fix citibike lp example Event issue

* update the exampel for vector env

* update vector env example

* update README due to PR comments

* add link to playground above MARO installation in README

* fix some typos

Co-authored-by: Jinyu Wang <Wang.Jinyu@microsoft.com>

* update package version

* update README for package description

* update image links for pypi package description

* update image links for pypi package description

* change the input topology schema for CIM real data mode (#372)

* change the input topology schema for CIM real data mode

* remove unused importing

* update test config file correspondingly

* add Exception for env test

* add cost factors to cim data dump

* update CimDataCollection field name

* update field name of data collection related code

* update package version

* adjust interface to reflect actual signature (#374)

Co-authored-by: Jeremy Reynolds <jeremr@microsoft.com>

* update dataclasses requirement to setup

* fix: fixing spelling grammarr

* fix: fix typo spelling code commented and data_model.rst

* Fix Geo vis IP address & SQL logic bugs. (#383)

Fix Geo vis IP address & SQL logic bugs (issue [352](#352) and [314](#314)).

* Fix the "Wrong future stop tick predictions" bug (#386)

* Propose my new solution

Refine to the pre-process version

.

* Optimize import

* Fix reset random seed bug (#387)

* update the reset interface of Env and BE

* Try to fix reset routes generation seed issue

* Refine random related logics.

* Minor refinement

* Test check

* Minor

* Remove unused functions so far

* Minor

Co-authored-by: Jinyu Wang <jinywan@microsoft.com>

* update package version

* Add _init_vessel_plans in business_engine.reset (#388)

* update package version

* change the default solver used in Citibike OnlineLP example, from GLPK to CBC (#391)

Co-authored-by: Jinyu Wang <Wang.Jinyu@microsoft.com>

* Refine `event_buffer/` module (#389)

* Core & Business Engine code refinement (#392)

* First version

* Optimize imports

* Add typehint

* Lint check

* Lint check

* add higher python version (#398)

* add higher python version

* update pytorch version

* update torchvision version

Co-authored-by: Jinyu Wang <Wang.Jinyu@microsoft.com>

* CIM scenario refinement (#400)

* Cim scenario refinement (#394)

* CIM refinement

* Fix lint error

* Fix lint error

* Cim test coverage (#395)

* Enrich tests

* Refactor CimDataGenerator

* Refactor CIM parsers

* Minor refinement

* Fix lint error

* Fix lint error

* Fix lint error

* Minor refactor

* Type

* Add two test file folders. Make a slight change to CIM BE.

* Lint error

* Lint error

* Remove unnecessary public interfaces of CIM BE

* Cim disable auto action type detection (#399)

* Haven't been tested

* Modify document

* Add ActionType checking

* Minor

* Lint error

* Action quantity should be a position number

* Modify related docs & notebooks

* Minor

* Change test file name. Prepare to merge into master.

* .

* Minor test patch

* Add `clear()` function to class `SimRandom` (#401)

* Add SimRandom.clear()

* Minor

* Remove commented codes

* Lint error

* update package version

* add branch v0.3 to github workflow

* update github test workflow

* Update requirements.dev.txt (#444)

Added the versions of dependencies and resolve some conflicts occurs when installing. By adding these version number it will tell you the exact.

* Bump ipython from 7.10.1 to 7.16.3 in /notebooks (#460)

Bumps [ipython](https://github.com/ipython/ipython) from 7.10.1 to 7.16.3.
- [Release notes](https://github.com/ipython/ipython/releases)
- [Commits](ipython/ipython@7.10.1...7.16.3)

---
updated-dependencies:
- dependency-name: ipython
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add & sort requirements.dev.txt

Co-authored-by: Jinyu-W <53509467+Jinyu-W@users.noreply.github.com>
Co-authored-by: Jinyu Wang <Wang.Jinyu@microsoft.com>
Co-authored-by: Jinyu Wang <jinywan@microsoft.com>
Co-authored-by: Jeremy Reynolds <jeremy.reynolds@microsoft.com>
Co-authored-by: Jeremy Reynolds <jeremr@microsoft.com>
Co-authored-by: slowy07 <slowy.arfy@gmail.com>
Co-authored-by: solosilence <abhishekkr23rs@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
  • Loading branch information
@lihuoran @Jinyu-W
@jreynolds01 @slowy07 @itsabhishekhere @dependabot
9 people committed Jun 9, 2022
1 parent 05ab922 commit 0cb98cd
Show file tree
Hide file tree
Showing 6 changed files with 183 additions and 105 deletions.
6 changes: 3 additions & 3 deletions .github/workflows/test.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,17 @@ name: test

on:
push:
branches: [ master, v0.1, v0.2 ]
branches: [ master, v0.1, v0.2, v0.3 ]
pull_request:
branches: [ master, v0.1, v0.2 ]
branches: [ master, v0.1, v0.2, v0.3 ]
workflow_dispatch:

jobs:
test:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-16.04, ubuntu-18.04, windows-latest, macos-latest]
os: [ubuntu-18.04, windows-latest, macos-latest]
python-version: [3.6, 3.7, 3.8, 3.9]

steps:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/vulnerability_scan.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ name: "vulnerability scan"

on:
push:
branches: [master, v0.1, v0.2]
branches: [master, v0.1, v0.2, v0.3]
pull_request:
# The branches below must be a subset of the branches above
branches: [master]
Expand Down
199 changes: 131 additions & 68 deletions maro/README.rst
View file
Original file line number Diff line number Diff line change
@@ -1,43 +1,43 @@


.. image:: https://img.shields.io/pypi/l/pymaro
:target: https://github.com/microsoft/maro/blob/master/LICENSE
:alt: License
:target: https://github.com/microsoft/maro/blob/master/LICENSE
:alt: License


.. image:: https://raw.githubusercontent.com/microsoft/maro/master/docs/source/images/badges/platform.svg
:target: https://pypi.org/project/pymaro/
:alt: Platform
:target: https://pypi.org/project/pymaro/
:alt: Platform


.. image:: https://img.shields.io/pypi/pyversions/pymaro.svg?logo=python&logoColor=white
:target: https://pypi.org/project/pymaro/#files
:alt: Python Versions
:target: https://pypi.org/project/pymaro/#files
:alt: Python Versions


.. image:: https://img.shields.io/github/languages/code-size/microsoft/maro
:target: https://github.com/microsoft/maro
:alt: Code Size
:target: https://github.com/microsoft/maro
:alt: Code Size


.. image:: https://img.shields.io/docker/image-size/maro2020/maro
:target: https://hub.docker.com/repository/docker/maro2020/maro/tags?page=1
:alt: Docker Size
:target: https://hub.docker.com/repository/docker/maro2020/maro/tags?page=1
:alt: Docker Size


.. image:: https://img.shields.io/github/issues/microsoft/maro
:target: https://github.com/microsoft/maro/issues
:alt: Issues
:target: https://github.com/microsoft/maro/issues
:alt: Issues


.. image:: https://img.shields.io/github/issues-pr/microsoft/maro
:target: https://github.com/microsoft/maro/pulls
:alt: Pull Requests
:target: https://github.com/microsoft/maro/pulls
:alt: Pull Requests


.. image:: https://img.shields.io/librariesio/github/microsoft/maro
:target: https://libraries.io/pypi/pymaro
:alt: Dependencies
:target: https://libraries.io/pypi/pymaro
:alt: Dependencies


.. image:: https://github.com/microsoft/maro/workflows/test/badge.svg
Expand All @@ -51,8 +51,8 @@


.. image:: https://github.com/microsoft/maro/workflows/docker/badge.svg
:target: https://hub.docker.com/repository/docker/maro2020/maro
:alt: docker
:target: https://hub.docker.com/repository/docker/maro2020/maro
:alt: docker


.. image:: https://readthedocs.org/projects/maro/badge/?version=latest
Expand All @@ -61,23 +61,86 @@


.. image:: https://img.shields.io/pypi/v/pymaro
:target: https://pypi.org/project/pymaro/#files
:alt: PypI Versions
:target: https://pypi.org/project/pymaro/#files
:alt: PypI Versions


.. image:: https://img.shields.io/pypi/wheel/pymaro
:target: https://pypi.org/project/pymaro/#files
:alt: Wheel
:target: https://pypi.org/project/pymaro/#files
:alt: Wheel


.. image:: https://raw.githubusercontent.com/microsoft/maro/master/docs/source/images/badges/citi_bike.svg
:target: https://maro.readthedocs.io/en/latest/scenarios/citi_bike.html
:alt: Citi Bike
:target: https://maro.readthedocs.io/en/latest/scenarios/citi_bike.html
:alt: Citi Bike


.. image:: https://raw.githubusercontent.com/microsoft/maro/master/docs/source/images/badges/cim.svg
:target: https://maro.readthedocs.io/en/latest/scenarios/container_inventory_management.html
:alt: CIM
:target: https://maro.readthedocs.io/en/latest/scenarios/container_inventory_management.html
:alt: CIM


.. image:: https://raw.githubusercontent.com/microsoft/maro/master/docs/source/images/badges/vm_scheduling.svg
:target: https://maro.readthedocs.io/en/latest/scenarios/vm_scheduling.html
:alt: VM Scheduling


.. image:: https://img.shields.io/gitter/room/microsoft/maro
:target: https://gitter.im/Microsoft/MARO#
:alt: Gitter


.. image:: https://raw.githubusercontent.com/microsoft/maro/master/docs/source/images/badges/stack_overflow.svg
:target: https://stackoverflow.com/questions/ask?tags=maro
:alt: Stack Overflow


.. image:: https://img.shields.io/github/release-date-pre/microsoft/maro
:target: https://github.com/microsoft/maro/releases
:alt: Releases


.. image:: https://img.shields.io/github/commits-since/microsoft/maro/latest/master
:target: https://github.com/microsoft/maro/commits/master
:alt: Commits


.. image:: https://github.com/microsoft/maro/workflows/vulnerability%20scan/badge.svg
:target: https://github.com/microsoft/maro/actions?query=workflow%3A%22vulnerability+scan%22
:alt: Vulnerability Scan


.. image:: https://github.com/microsoft/maro/workflows/lint/badge.svg
:target: https://github.com/microsoft/maro/actions?query=workflow%3Alint
:alt: Lint


.. image:: https://img.shields.io/codecov/c/github/microsoft/maro
:target: https://codecov.io/gh/microsoft/maro
:alt: Coverage


.. image:: https://img.shields.io/pypi/dm/pymaro
:target: https://pypi.org/project/pymaro/#files
:alt: Downloads


.. image:: https://img.shields.io/docker/pulls/maro2020/maro
:target: https://hub.docker.com/repository/docker/maro2020/maro
:alt: Docker Pulls


.. image:: https://raw.githubusercontent.com/microsoft/maro/master/docs/source/images/badges/play_with_maro.svg
:target: https://hub.docker.com/r/maro2020/maro
:alt: Play with MARO



.. image:: https://github.com/microsoft/maro/blob/master/docs/source/images/logo.svg
:target: https://maro.readthedocs.io/en/latest/
:alt: MARO LOGO

================================================================================================================


.. image:: https://raw.githubusercontent.com/microsoft/maro/master/docs/source/images/badges/vm_scheduling.svg
Expand Down Expand Up @@ -164,8 +227,8 @@ Key Components of MARO:


.. image:: https://github.com/microsoft/maro/blob/master/docs/source/images/maro_overview.svg
:target: https://github.com/microsoft/maro/blob/master/docs/source/images/maro_overview.svg
:alt: MARO Key Components
:target: https://github.com/microsoft/maro/blob/master/docs/source/images/maro_overview.svg
:alt: MARO Key Components


Contents
Expand Down Expand Up @@ -244,22 +307,22 @@ Install MARO from Source

.. code-block:: powershell
# If your environment is not clean, create a virtual environment firstly.
python -m venv maro_venv
# If your environment is not clean, create a virtual environment firstly.
python -m venv maro_venv
# You may need this for SecurityError in PowerShell.
Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy Unrestricted
# Activate the virtual environment.
.\maro_venv\Scripts\activate
# Activate the virtual environment.
.\maro_venv\Scripts\activate
*
Install MARO

.. code-block:: sh
# Git Clone the whole source code.
git clone https://github.com/microsoft/maro.git
# Git Clone the whole source code.
git clone https://github.com/microsoft/maro.git
*
Expand All @@ -286,14 +349,14 @@ Install MARO from Source

.. code-block:: sh
export PYTHONPATH=PATH-TO-MARO
export PYTHONPATH=PATH-TO-MARO
* Windows

.. code-block:: powershell
$Env:PYTHONPATH=PATH-TO-MARO
$Env:PYTHONPATH=PATH-TO-MARO
Quick Example
-------------
Expand All @@ -316,15 +379,15 @@ Quick Example

.. code-block:: sh
# Enable environment dump feature, when initializing the environment instance
env = Env(scenario="cim",
topology="toy.5p_ssddd_l0.0",
start_tick=0,
durations=100,
options={"enable-dump-snapshot": "./dump_data"})
# Enable environment dump feature, when initializing the environment instance
env = Env(scenario="cim",
topology="toy.5p_ssddd_l0.0",
start_tick=0,
durations=100,
options={"enable-dump-snapshot": "./dump_data"})
# Inspect environment with the dump data
maro inspector dashboard --source_path ./dump_data/YOUR_SNAPSHOT_DUMP_FOLDER
# Inspect environment with the dump data
maro inspector dashboard --source_path ./dump_data/YOUR_SNAPSHOT_DUMP_FOLDER
Show Cases
^^^^^^^^^^
Expand All @@ -334,26 +397,26 @@ Show Cases
Case I - Container Inventory Management

.. image:: https://github.com/microsoft/maro/blob/master/docs/source/images/visualization/dashboard/cim_inter_epoch.gif
:target: https://github.com/microsoft/maro/blob/master/docs/source/images/visualization/dashboard/cim_inter_epoch.gif
:alt: CIM Inter Epoch
:target: https://github.com/microsoft/maro/blob/master/docs/source/images/visualization/dashboard/cim_inter_epoch.gif
:alt: CIM Inter Epoch


.. image:: https://github.com/microsoft/maro/blob/master/docs/source/images/visualization/dashboard/cim_intra_epoch_by_ports.gif
:target: https://github.com/microsoft/maro/blob/master/docs/source/images/visualization/dashboard/cim_intra_epoch_by_ports.gif
:alt: CIM Intra Epoch
:target: https://github.com/microsoft/maro/blob/master/docs/source/images/visualization/dashboard/cim_intra_epoch_by_ports.gif
:alt: CIM Intra Epoch


*
Case II - Citi Bike

.. image:: https://github.com/microsoft/maro/blob/master/docs/source/images/visualization/dashboard/citi_bike_inter_epoch.gif
:target: https://github.com/microsoft/maro/blob/master/docs/source/images/visualization/dashboard/citi_bike_inter_epoch.gif
:alt: Citi Bike Inter Epoch
:target: https://github.com/microsoft/maro/blob/master/docs/source/images/visualization/dashboard/citi_bike_inter_epoch.gif
:alt: Citi Bike Inter Epoch


.. image:: https://github.com/microsoft/maro/blob/master/docs/source/images/visualization/dashboard/citi_bike_intra_epoch_by_station.gif
:target: https://github.com/microsoft/maro/blob/master/docs/source/images/visualization/dashboard/citi_bike_intra_epoch_by_station.gif
:alt: Citi Bike Intra Epoch
:target: https://github.com/microsoft/maro/blob/master/docs/source/images/visualization/dashboard/citi_bike_intra_epoch_by_station.gif
:alt: Citi Bike Intra Epoch


Run Playground
Expand All @@ -365,13 +428,13 @@ Run Playground

.. code-block:: sh
# Pull the docker image from docker hub
docker pull maro2020/playground
# Pull the docker image from docker hub
docker pull maro2020/playground
# Run playground container.
# Redis commander (GUI for redis) -> http://127.0.0.1:40009
# Jupyter lab with maro -> http://127.0.0.1:40010
docker run -p 40009:40009 -p 40010:40010 maro2020/playground
# Run playground container.
# Redis commander (GUI for redis) -> http://127.0.0.1:40009
# Jupyter lab with maro -> http://127.0.0.1:40010
docker run -p 40009:40009 -p 40010:40010 maro2020/playground
*
Build from source
Expand All @@ -385,10 +448,10 @@ Run Playground
# Build playground image.
bash ./scripts/build_playground.sh
# Run playground container.
# Redis commander (GUI for redis) -> http://127.0.0.1:40009
# Jupyter lab with maro -> http://127.0.0.1:40010
docker run -p 40009:40009 -p 40010:40010 maro2020/playground
# Run playground container.
# Redis commander (GUI for redis) -> http://127.0.0.1:40009
# Jupyter lab with maro -> http://127.0.0.1:40010
docker run -p 40009:40009 -p 40010:40010 maro2020/playground
*
Windows
Expand All @@ -398,10 +461,10 @@ Run Playground
# Build playground image.
.\scripts\build_playground.bat
# Run playground container.
# Redis commander (GUI for redis) -> http://127.0.0.1:40009
# Jupyter lab with maro -> http://127.0.0.1:40010
docker run -p 40009:40009 -p 40010:40010 maro2020/playground
# Run playground container.
# Redis commander (GUI for redis) -> http://127.0.0.1:40009
# Jupyter lab with maro -> http://127.0.0.1:40010
docker run -p 40009:40009 -p 40010:40010 maro2020/playground
Contributing
------------
Expand Down Expand Up @@ -431,8 +494,8 @@ Related Papers


.. image:: https://github.com/microsoft/maro/blob/master/docs/source/images/scenario/cim_vis.gif
:target: https://github.com/microsoft/maro/blob/master/docs/source/images/scenario/cim_vis.gif
:alt: CIM Vis
:target: https://github.com/microsoft/maro/blob/master/docs/source/images/scenario/cim_vis.gif
:alt: CIM Vis


Wenlei Shi, Xinran Wei, Jia Zhang, Xiaoyuan Ni, Arthur Jiang, Jiang Bian, Tie-Yan Liu. "\ `Cooperative Policy Learning with Pre-trained Heterogeneous Observation Representations <https://arxiv.org/abs/2012.13099>`_\ ". AAMAS 2021
Expand Down
7 changes: 4 additions & 3 deletions maro/simulator/scenarios/abs_business_engine.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ def snapshots(self) -> SnapshotList:
def scenario_name(self) -> str:
return self._scenario_name

@abstractmethod
def get_agent_idx_list(self) -> List[int]:
"""Get a list of agent index."""
pass
Expand Down Expand Up @@ -173,9 +174,9 @@ def reset(self, keep_seed: bool = False) -> None:
"""Reset states business engine."""
pass

# @abstractmethod
# def set_seed(self, seed: int) -> None:
# raise NotImplementedError
@abstractmethod
def set_seed(self, seed: int) -> None:
raise NotImplementedError

def post_step(self, tick: int) -> bool:
"""This method will be called at the end of each tick, used to post-process for each tick,
Expand Down

0 comments on commit 0cb98cd

Please sign in to comment.