Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to compute all data (heatmap, explorer/14/17) at once #58

Closed
Natenom opened this issue Dec 10, 2023 · 2 comments
Closed

Ability to compute all data (heatmap, explorer/14/17) at once #58

Natenom opened this issue Dec 10, 2023 · 2 comments
Labels
type: performance It works, but it could be faster

Comments

@Natenom
Copy link

Natenom commented Dec 10, 2023

Workflow now:

  • Add new tracks
  • run with serve
  • open website explorer/14 and explorer/17 and wait very long with no information
  • open heatmap and wait very long with no information

New workflow:

  • Add new tracks
  • run geo-activity-playground --comput-all-data
  • drink a tea
  • open local website and look around. :)
@martin-ueding
Copy link
Owner

I am also not sure what the best approach is. I'd call them eager and lazy. Currently I have implemented the lazy style for some parts and the eager style for others. At the time it wasn't so clear which would be the best approach.

Now it becomes clearer and I think I will move more things to the eager evaluation.

The thing is that some things are easy to generate and aren't requested often:

  • Individual heatmap tiles can be rendered on-demand and are cached. It is infeasible to compute them all in advance as there are just way too many tiles. Each tile takes 64 kiB and there are 274,877,906,944 tiles on level 19, so that would be 16 PiB of data.
  • Share pictures for each activity are also quickly rendered.
  • Downloading pixel tiles from Open Street Map.

But there are other things which we should compute in advance:

  • Extract the time series data from GPX, FIT and TCX files.
  • Take each activity and compute the tiles that it has explored.
  • For each tile compute which activity has explored it, especially noting the first and last one.

I will try to shift the balance more towards the eager part. Perhaps it then also makes sense to add the progress bars back. With lazy evaluation the progress bars didn't make so much sense.

@martin-ueding martin-ueding added type: enhancement New feature or request type: performance It works, but it could be faster and removed type: enhancement New feature or request labels Dec 10, 2023
martin-ueding added a commit that referenced this issue Dec 11, 2023
martin-ueding added a commit that referenced this issue Dec 11, 2023
martin-ueding added a commit that referenced this issue Dec 11, 2023
martin-ueding added a commit that referenced this issue Dec 11, 2023
@martin-ueding
Copy link
Owner

I've changed how this works and converted more things to this eager mode. The changes are on master. They work for me, so I hope that they also work for you. Could you please try it out?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: performance It works, but it could be faster
Projects
None yet
Development

No branches or pull requests

2 participants