Too much RAM needed for Squadratinhos

[pstorch]

I encounter several crashes while using geo-activity-playground.

Menu Explorer:

See #24.

Menu Squadratinhos:

2023-11-26 12:10:40 geo_activity_playground.webui.app ERROR Exception on /explorer/17 [GET]
Traceback (most recent call last):
  File "/home/peter/.cache/pypoetry/virtualenvs/geo-activity-playground-uIcfKzN9-py3.11/lib/python3.11/site-packages/flask/app.py", line 1455, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/peter/.cache/pypoetry/virtualenvs/geo-activity-playground-uIcfKzN9-py3.11/lib/python3.11/site-packages/flask/app.py", line 869, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/peter/.cache/pypoetry/virtualenvs/geo-activity-playground-uIcfKzN9-py3.11/lib/python3.11/site-packages/flask/app.py", line 867, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/peter/.cache/pypoetry/virtualenvs/geo-activity-playground-uIcfKzN9-py3.11/lib/python3.11/site-packages/flask/app.py", line 852, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/peter/git/geo-activity-playground/geo_activity_playground/webui/app.py", line 49, in explorer
    "explorer.html.j2", **explorer_controller.render(int(zoom))
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/peter/git/geo-activity-playground/geo_activity_playground/webui/explorer_controller.py", line 26, in render
    explored = get_three_color_tiles(tiles, self._repository, zoom)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/peter/git/geo-activity-playground/geo_activity_playground/explorer/grid_file.py", line 24, in get_three_color_tiles
    a = np.zeros((2**zoom, 2**zoom), dtype=np.int8)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
numpy.core._exceptions._ArrayMemoryError: Unable to allocate 16.0 GiB for an array with shape (131072, 131072) and data type int8
2023-11-26 12:10:40 werkzeug INFO 127.0.0.1 - - [26/Nov/2023 12:10:40] "GET /explorer/17 HTTP/1.1" 500 -

Menu Equipment:

See: #25

[martin-ueding]

Thanks for the report! I've split them up into three tickets.

Regarding the Squadratinhos it seems like you are using an older version. I've changed the algorithm in version 0.7.0. Can you try this again with the latest version? Do a git pull to get the latest version.

[pstorch]

I get the same error. Is a poetry install after git pull enough to update the app?

 peter  manjaro  ~  git  geo-activity-playground  master  %  git pull
remote: Enumerating objects: 29, done.
remote: Counting objects: 100% (20/20), done.
remote: Total 29 (delta 20), reused 20 (delta 20), pack-reused 9
Unpacking objects: 100% (29/29), 2.44 KiB | 155.00 KiB/s, done.
From https://github.com/martin-ueding/geo-activity-playground
   57caee1..ea3e642  master     -> origin/master
   14ccbc4..03a9b0b  gh-pages   -> origin/gh-pages
 * [new tag]         0.8.2      -> 0.8.2
Updating 57caee1..ea3e642
Fast-forward
 docs/getting-started/starting-the-webserver.md   | 2 +-
 docs/reference/changelog.md                      | 5 +++++
 geo_activity_playground/core/activity_parsers.py | 4 ++--
 pyproject.toml                                   | 2 +-
 4 files changed, 9 insertions(+), 4 deletions(-)
 peter  manjaro  ~  git  geo-activity-playground  master  %  poetry install                                                                   
Installing dependencies from lock file
Warning: poetry.lock is not consistent with pyproject.toml. You may be getting improper dependencies. Run `poetry lock [--no-update]` to fix it.

No dependencies to install or update

Installing the current project: geo-activity-playground (0.8.2)
 peter  manjaro  ~  git  geo-activity-playground  master  %  poetry run geo-activity-playground --basedir ~/Documents/ActivityPlayground serve
2023-11-26 14:12:01 geo_activity_playground.core.config WARNING Missing a config, some features might be missing.
2023-11-26 14:12:01 geo_activity_playground.importers.directory INFO Loading metadata file …
 * Serving Flask app 'geo_activity_playground.webui.app'
 * Debug mode: off
2023-11-26 14:12:01 werkzeug INFO WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on http://127.0.0.1:5000
2023-11-26 14:12:01 werkzeug INFO Press CTRL+C to quit
Timestamp('2023-11-19 12:18:34.400000+0000', tz='UTC')
Timestamp('2023-10-26 13:12:06.176590+0000', tz='UTC')
2023-11-26 14:12:06 werkzeug INFO 127.0.0.1 - - [26/Nov/2023 14:12:06] "GET / HTTP/1.1" 200 -
2023-11-26 14:12:08 geo_activity_playground.explorer.converters INFO Building explorer tile history from all activities …
2023-11-26 14:12:08 geo_activity_playground.explorer.converters INFO Consolidating explorer tile history …
2023-11-26 14:12:08 geo_activity_playground.explorer.converters INFO Store explorer tile history to cache file …
2023-11-26 14:12:08 geo_activity_playground.webui.app ERROR Exception on /explorer/17 [GET]
Traceback (most recent call last):
  File "/home/peter/.cache/pypoetry/virtualenvs/geo-activity-playground-uIcfKzN9-py3.11/lib/python3.11/site-packages/flask/app.py", line 1455, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/peter/.cache/pypoetry/virtualenvs/geo-activity-playground-uIcfKzN9-py3.11/lib/python3.11/site-packages/flask/app.py", line 869, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/peter/.cache/pypoetry/virtualenvs/geo-activity-playground-uIcfKzN9-py3.11/lib/python3.11/site-packages/flask/app.py", line 867, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/peter/.cache/pypoetry/virtualenvs/geo-activity-playground-uIcfKzN9-py3.11/lib/python3.11/site-packages/flask/app.py", line 852, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/peter/git/geo-activity-playground/geo_activity_playground/webui/app.py", line 49, in explorer
    "explorer.html.j2", **explorer_controller.render(int(zoom))
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/peter/git/geo-activity-playground/geo_activity_playground/webui/explorer_controller.py", line 26, in render
    explored = get_three_color_tiles(tiles, self._repository, zoom)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/peter/git/geo-activity-playground/geo_activity_playground/explorer/grid_file.py", line 24, in get_three_color_tiles
    a = np.zeros((2**zoom, 2**zoom), dtype=np.int8)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
numpy.core._exceptions._ArrayMemoryError: Unable to allocate 16.0 GiB for an array with shape (131072, 131072) and data type int8
2023-11-26 14:12:08 werkzeug INFO 127.0.0.1 - - [26/Nov/2023 14:12:08] "GET /explorer/17 HTTP/1.1" 500 -

[martin-ueding]

You don't even need to do poetry install, that is only necessary when requirements have changed. It doesn't hurt, but shouldn't be necessary in this case.

The problem was that I didn't delete the allocation of the array. I have no idea why it worked on my laptop fine now. Either way, I've removed those two lines now. When you do another git pull, you can try it out again.

[pstorch]

It works now, great.