Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge Grid, SingleGrid, and MultiGrid into one class #625

Closed
wants to merge 16 commits into from
Closed

Merge Grid, SingleGrid, and MultiGrid into one class #625

wants to merge 16 commits into from

Conversation

kumom
Copy link
Contributor

@kumom kumom commented Jan 16, 2019

This is some follow-up work for reasons explained in #623. I basically merge three classes (Grid, SingleGrid, MultiGrid) into one, and use multigrid as a boolean attribute to specify the behavior of the class. I haven't modified the test files, so it will for sure fail since I changed some method signatures. In addition, I notice HexGrid is a subtype of Grid, and this modification will cause some of its behaviors to fail potentially.

What I intend to do next, if the current hierarchy looks good to you, is to change multigrid into grid_type. Allowed values of grid_type would be single, multi, or hex, so that HexGrid would be "merged" to Grid class as well. Just a side node, I adopt this style of code because I am somewhat influenced by the style of sklearn.

I also had this question as I read the source code. The only reason I can think of is probably to avoid concurrency issues/code optimization done by compiler. Is there any good reason to write the code this way?

Copy link
Contributor

@Corvince Corvince left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great PR! However, as the changes would break every model with a Grid, we have to be clever on how to to the transition.

I think calls to SingleGrid and MultiGrid should call the appropriate version of Grid and issue a Depreciation warning. Same for some of the removed functions.

I added a bunch of code comments, sometimes they are specific to your PR, but some a more general remarks on the grid functions.

I also opened a topic on the mailing list to discuss how to proceed with you proposed changes

mesa/space.py Outdated Show resolved Hide resolved
mesa/space.py Outdated

def neighbors(self, pos, moore=True, radius=1, get_agents=False, include_empty=False):
"""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary line and explanation of the method is missing

mesa/space.py Outdated Show resolved Hide resolved
mesa/space.py Outdated Show resolved Hide resolved
mesa/space.py Outdated
self.empties.remove(pos)
else:
if replace:
old_agent.pos = None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this will fail for multigrid? As old_agent would be a set. It is also not clear how "replace" should work for a multigrid. Should it replace a single agent? All agents? Or can't it be used with MultiGrid?

mesa/space.py Outdated Show resolved Hide resolved
mesa/space.py Outdated
"Cell already occupied by agent {}".format(old_agent.unique_id))

def _pick_random_position(self, agent, pos):
if pos == ("random", "random") and self.exists_empty_cells():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the function should also work with other sequences, e.g. ["random", "random"]

mesa/space.py Outdated
else:
empties = set()
if pos[0] == "random":
# We pick a random cell in a specified row
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment and code got me confused. You don't pick a random cell here, you just make a set (why not list?) of empty cells and then randomly select one further down

mesa/space.py Outdated
@@ -322,21 +273,26 @@ def remove_agent(self, agent):
def _remove_agent(self, pos, agent):
""" Remove the agent from the given location. """
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The agents pos attribute should also be set to None

@kumom
Copy link
Contributor Author

kumom commented Jan 18, 2019

Thanks a lot for the comments! I was rushing a bit to show the hierarchy. I was also a bit conservative due to backward compatibility problem like you said. I haven't really taken a good look at any other classes yet, so I am hesitant about merging HexGrid as well. I am implementing a model involving network and I will try to use NetworkGrid to test out the functions there. I will look into the code more carefully next week and push new code to the branch.

@kumom
Copy link
Contributor Author

kumom commented Jan 18, 2019

btw, is there any motivation to name a network NetworkGrid? I was a bit confused by the name, but since I did not really look at the code and for backward compatibility I left this class intact.

@Corvince
Copy link
Contributor

Really quick, so you get this info if you want to work on this on the weekend:

I tested your branch and there are several severe bugs. I suggest adding

class SingleGrid(Grid):
    """Depreciated class."""
    def __init__(self, width, height, torus):
        super().__init__(width, height, torus, multigrid=False)


class MultiGrid(Grid):
    """Depreciated class."""
    def __init__(self, width, height, torus):
        super().__init__(width, height, torus, multigrid=True)

to your file and then run the tests to see a lot of errors (not just related to missing functions). I already made some changes. I wanted to PR my changes into your repo, but since you did not fork from projectmesa, I wasn't able to tell GitHub where to PR. So here is a gist of my file based on your changes.

@jackiekazil jackiekazil self-requested a review January 21, 2019 03:34
@Corvince
Copy link
Contributor

Corvince commented Jan 21, 2019

As noted already there are several bugs in this implementation. See this PR to your repo for which ones and how I solved them.

I would suggest the following approach: Make sure the current tests work (by adding in deprecated functions and/or resolve bugs). Then add new tests based on the current implementation.

Again, I really like your changes/new structure but its not ready for a merge yet. I would also like to see a return of the get_neighborhood function (including center).

@kumom
Copy link
Contributor Author

kumom commented Jan 21, 2019

As noted already there are several bugs in this implementation. See this PR to your repo for which ones and how I solved them.

I would suggest the following approach: Make sure the current tests work (by adding in deprecated functions and/or resolve bugs). Then add new tests based on the current implementation.

Again, I really like your changes/new structure but its not ready for a merge yet. I would also like to see a return of the get_neighborhood function (including center).

Thanks a million for the efforts! I know the current code was committed under rush (sorry for that) and I intended to work on it this week.

@kumom
Copy link
Contributor Author

kumom commented Jan 23, 2019

I don't understand why I pass the tests locally but fail in Travis...I really don't want to push a lot to annoy you, but I am not sure why this happens.

tests/test_batchrunner.py ....                                           [  4%]
tests/test_datacollector.py ....                                         [  9%]
tests/test_examples.py .                                                 [ 10%]
tests/test_grid.py ...............................                       [ 45%]
tests/test_lifespan.py ..                                                [ 47%]
tests/test_main.py .                                                     [ 48%]
tests/test_scaffold.py F                                                 [ 50%]
tests/test_space.py .............................                        [ 82%]
tests/test_time.py ........                                              [ 92%]
tests/test_usersettableparam.py ....                                     [ 96%]
tests/test_visualization.py ...                                          [100%]Coverage.py warning: No data was collected. (no-data-collected)

In addition, the failure is

=================================== FAILURES ===================================
________________ ScaffoldTest.test_scaffold_creates_project_dir ________________

self = <test_scaffold.ScaffoldTest testMethod=test_scaffold_creates_project_dir>

    def test_scaffold_creates_project_dir(self):
        with self.runner.isolated_filesystem():
            assert not os.path.isdir("example_project")
            self.runner.invoke(cli, ['startproject', '--no-input'])
>           assert os.path.isdir("example_project")
E           AssertionError: assert False
E            +  where False = <function isdir at 0x103c989d8>('example_project')
E            +    where <function isdir at 0x103c989d8> = <module 'posixpath' from '/Users/kumom/anaconda3/lib/python3.6/posixpath.py'>.isdir
E            +      where <module 'posixpath' from '/Users/kumom/anaconda3/lib/python3.6/posixpath.py'> = os.path

tests/test_scaffold.py:21: AssertionError
----------------------------- Captured stdout call -----------------------------
A valid repository for "/Users/kumom/anaconda3/lib/python3.6/site-packages/mesa/cookiecutter-mesa" could not be found in the following locations:
/Users/kumom/anaconda3/lib/python3.6/site-packages/mesa/cookiecutter-mesa
/Users/kumom/anaconda3/lib/python3.6/site-packages/mesa/cookiecutter-mesa

However, I have passed the local tests when I did the PR #622, and I haven't updated/downgraded the package in any way. Why would this error suddenly pops up?

@Corvince
Copy link
Contributor

Exciting new commits! I will review your changes and give you more feedback in the next days.

RE tests: Maybe try to clean your local .pytest_cache folders or something like this? Also not sure which tests you pass locally

@jackiekazil
Copy link
Member

@kumom this is a great PR. At first glance / skim, I have to say I agree with Corvince. If you aren't a part of the dev group -- can you join us there for the discussion? (Otherwise, we will report back here.) https://groups.google.com/forum/#!topic/projectmesa-dev/H7IUeol8iRM

@kumom
Copy link
Contributor Author

kumom commented Jan 24, 2019

@kumom this is a great PR. At first glance / skim, I have to say I agree with Corvince. If you aren't a part of the dev group -- can you join us there for the discussion? (Otherwise, we will report back here.) https://groups.google.com/forum/#!topic/projectmesa-dev/H7IUeol8iRM

I am in the group already. Thanks a lot for the comments.

@kumom
Copy link
Contributor Author

kumom commented Jan 24, 2019

As noted already there are several bugs in this implementation. See this PR to your repo for which ones and how I solved them.

I would suggest the following approach: Make sure the current tests work (by adding in deprecated functions and/or resolve bugs). Then add new tests based on the current implementation.

Again, I really like your changes/new structure but its not ready for a merge yet. I would also like to see a return of the get_neighborhood function (including center).

After cleaning the cache, I passes all the tests (with the buggy code)...

============================= test session starts ==============================
platform darwin -- Python 3.6.5, pytest-4.0.2, py-1.5.3, pluggy-0.8.1
rootdir: /Users/kumom/Documents/Projects/mesa, inifile:
plugins: remotedata-0.2.1, openfiles-0.3.0, doctestplus-0.1.3, cov-2.6.1, arraydiff-0.2
collected 88 items

tests/test_batchrunner.py ....                                           [  4%]
tests/test_datacollector.py ....                                         [  9%]
tests/test_examples.py .                                                 [ 10%]
tests/test_grid.py ...............................                       [ 45%]
tests/test_lifespan.py ..                                                [ 47%]
tests/test_main.py .                                                     [ 48%]
tests/test_scaffold.py .                                                 [ 50%]
tests/test_space.py .............................                        [ 82%]
tests/test_time.py ........                                              [ 92%]
tests/test_usersettableparam.py ....                                     [ 96%]
tests/test_visualization.py ...                                          [100%]Coverage.py warning: No data was collected. (no-data-collected)


---------- coverage: platform darwin, python 3.6.5-final-0 -----------
Name                                                    Stmts   Miss  Cover
---------------------------------------------------------------------------
mesa/__init__.py                                            9      9     0%
mesa/agent.py                                              10     10     0%
mesa/batchrunner.py                                       152    152     0%
mesa/datacollection.py                                     78     78     0%
mesa/main.py                                               25     25     0%
mesa/model.py                                              28     28     0%
mesa/space.py                                             326    326     0%
mesa/time.py                                               61     61     0%
mesa/visualization/ModularVisualization.py                120    120     0%
mesa/visualization/TextVisualization.py                    40     40     0%
mesa/visualization/UserParam.py                            50     50     0%
mesa/visualization/__init__.py                              1      1     0%
mesa/visualization/modules/CanvasGridVisualization.py      25     25     0%
mesa/visualization/modules/ChartVisualization.py           25     25     0%
mesa/visualization/modules/HexGridVisualization.py         28     28     0%
mesa/visualization/modules/NetworkVisualization.py         16     16     0%
mesa/visualization/modules/TextVisualization.py             5      5     0%
mesa/visualization/modules/__init__.py                      6      6     0%
---------------------------------------------------------------------------
TOTAL                                                    1005   1005     0%

@Corvince
Copy link
Contributor

Here is something that's up to a discussion:

You have removed the get_neighbors and get_neighborhood functions and provide a single neighbors function. Although you can kind of get around with the get_agents and include_empty arguments, I think the former functions are much clearer.
This might come down to personal taste, but my personal expectations would be:
neighborhood: A list of cells that surround an agent (and, per default, include the own cell). Might or might not return agents
neighbors: A list of agents that are within my neighborhood (and in a multigrid possibly on my own cell)

I think the use cases for both functions are quite distinct and I would like to keep them separate. Also your function defaults would return a list of cell coordinates, which are not empty. I think that is kind of odd.

But as I said this is just my personal opinion/expectation. Feel free to discuss with me

@Corvince
Copy link
Contributor

After cleaning the cache, I passes all the tests (with the buggy code)...

Thats strange. When I run your changes locally the same tests fail as on travis

@dmasad dmasad assigned dmasad and unassigned dmasad Feb 8, 2019
@dmasad dmasad self-requested a review February 8, 2019 02:30
@dmasad
Copy link
Member

dmasad commented Feb 9, 2019

@kumom Thanks for working on this -- my memory is that the split into single/multi-grid was a path-dependent artifact and not a well-thought-out design choice, so this is probably long overdue.

I agree with @Corvince that it makes sense to have separate methods for getting neighbors (agents) and neighborhood (coordinates). Even if one is just a thin wrapper around the other, it's one less thing for a modeler to need to implement.

I'm also seeing the same test failures on my end as Travis is showing.

@kumom
Copy link
Contributor Author

kumom commented Feb 16, 2019

I am still debugging the issue with pytest, and I think I have figured out why all tests are passed - because mesa installed via pip was imported. However, after I ran pip3 uninstall mesa and did pytest --cov=mesa tests/ again, I received the following error

__________________ ERROR collecting tests/test_batchrunner.py __________________
ImportError while importing test module '/Users/kumom/Documents/Projects/mesa/tests/test_batchrunner.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/test_batchrunner.py:8: in <module>
    from mesa import Agent, Model
E   ImportError: cannot import name 'Agent'

I ran this under a new folder cloned from projectmesa/mesa. Why would this happen? Am I doing something wrong with pytest?

@Corvince
Copy link
Contributor

Pytest ist fine, but you still have to install Mesa. Instead of pip install Mesa, which pulls it from pypi, try to install it in the root dir (projectmesa/Mesa) with pip install -e .[dev]

@kumom
Copy link
Contributor Author

kumom commented Feb 17, 2019

Pytest ist fine, but you still have to install Mesa. Instead of pip install Mesa, which pulls it from pypi, try to install it in the root dir (projectmesa/Mesa) with pip install -e .[dev]

Thanks a lot! Problem solved already

@jackiekazil jackiekazil added the example Changes the examples or adds to them. label Apr 5, 2019
@jackiekazil jackiekazil deleted the branch projectmesa:master March 14, 2021 05:26
@EwoutH
Copy link
Member

EwoutH commented Dec 6, 2023

@Corvince At some point in developing the PropertyGrid in #1898 I also considered merging them. Instead of a binary "single" or "multi", I considered a capacity keyword which could be 1 (equivalent to Single), inf (equivalent to Multi) or any integer in between (new functionality).

Do you think ideally we should pursue merging these classes? If so, do you think it's worth it?

The reasons I didn't where 1) backwards compatibility and 2) performance (some functions are faster if there can be only 1 agent anywhere or if there can be infinite somewhere).

@Corvince
Copy link
Contributor

Corvince commented Dec 6, 2023

I think that's a pretty solid idea and I think it would be possible to do this in a backwards compatible way. I would guess the devil will be in the details, for example for grid access do you return a single agent or a list of a single agent? The latter would be more consistent but possibly confusing (why return a list if there can only be one agent)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
example Changes the examples or adds to them.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants