Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Updated] Add page to explore all changes of gyms over time #331

Merged
merged 57 commits into from Feb 1, 2018

Conversation

Projects
None yet
6 participants
@michikrug
Copy link
Collaborator

michikrug commented Dec 2, 2017

Updated version of PR #283

Description

Since I was always curious of what is happening in the gyms in my town, but no historical data is tracked by RocketMap, I have added a new SQL table that does so.
Using the MySQL scheduler, every 20secs the data from the "gym" table is added to the new "gym_history" one.
This data is then nicely represented on a separate new page. /gymhistory
The data is searchable, sortable and filterable.

Motivation and Context

Now you are able to see, when gyms are leveled up or down and which Pokémon are placed in or thrown out.
(This will also show you all those losers, who are using multiple accounts to shave their spot into high level gyms. There is a separate branch that extracts such behavior.)

How Has This Been Tested?

Tested in my own Worldopole online instance.
Does not interfere with any other functionality.

It is necessary to execute some SQL statements that I have documented in readme.gymhistory.md to create the new table and the update events.
Furthermore, you have to enable the MySQL event scheduler. This is also documented in the readme for a running server, but you should also update your server configuration to enable the scheduler on start.

Remark: it does not include any localization so far -> english only

Screenshots (if appropriate):

bildschirmfoto 2017-05-08 um 12 37 48

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

TODO

  • Localization
@Sttug
Copy link
Contributor

Sttug left a comment

for some reason y need do this.
SET GLOBAL sql_mode=(SELECT REPLACE(@@sql_mode,'ONLY_FULL_GROUP_BY',''));
and

DELIMITER // CREATE EVENT IF NOT EXISTS gymhistory_insert ON SCHEDULE EVERY 30 SECOND DO BEGIN INSERT INTO gymhistory (SELECT NULL, g.gym_id, g.team_id, g.guard_pokemon_id, g.total_cp, g.last_modified, g.last_modified as last_updated, (SELECT GROUP_CONCAT(DISTINCT pokemon_uid ORDER BY deployment_time SEPARATOR ',') FROM gymmember AS gm WHERE gm.gym_id = g.gym_id GROUP BY gym_id) AS pokemon_uids, (SELECT COUNT(DISTINCT pokemon_uid) FROM gymmember AS gm WHERE gm.gym_id = g.gym_id) AS pokemon_count FROM gym AS g WHERE g.last_modified > (SELECT MAX(last_modified) FROM gymhistory)); END // DELIMITER ;

CREATE EVENT IF NOT EXISTS gymhistory_update ON SCHEDULE EVERY 30 SECOND DO UPDATE gymhistory AS gh JOIN (SELECT gym_id, MAX(last_modified) as max_last_modified FROM gymhistory GROUP BY gym_id) AS gg ON gh.gym_id = gg.gym_id AND gh.last_modified = gg.max_last_modified JOIN (SELECT gym_id, last_scanned, GROUP_CONCAT(DISTINCT pokemon_uid ORDER BY deployment_time SEPARATOR ',') AS pokemon_uids, COUNT(DISTINCT pokemon_uid) as pokemon_count FROM gymmember AS gm GROUP BY gym_id) AS gm ON gh.gym_id = gm.gym_id SET gh.last_updated = gm.last_scanned, gh.pokemon_uids = gm.pokemon_uids, gh.pokemon_count = gm.pokemon_count WHERE gh.last_updated < gm.last_scanned; END // DELIMITER ;

now work fine (mysql 5.7)

@Obihoernchen

This comment has been minimized.

Copy link
Collaborator

Obihoernchen commented Dec 8, 2017

Ahh again ONLY_FULL_GROUP_BY. This was introduced with mysql 5.7 as a default.
@michikrug can you have a look at this please?

@Obihoernchen Obihoernchen added the WIP label Dec 8, 2017

@michikrug

This comment has been minimized.

Copy link
Collaborator Author

michikrug commented Dec 8, 2017

Sure

@javisabalete

This comment has been minimized.

Copy link
Contributor

javisabalete commented Dec 10, 2017

Tested with mysql5.6 and it's working as expected

@michikrug michikrug force-pushed the michikrug:PR-gymhistory branch 2 times, most recently from 965b8c7 to cdff5b5 Dec 12, 2017

@javisabalete

This comment has been minimized.

Copy link
Contributor

javisabalete commented Dec 12, 2017

Tested new commits with gymhistory_hide_cp_changes option, works 👍

@Obihoernchen

This comment has been minimized.

Copy link
Collaborator

Obihoernchen commented Dec 17, 2017

Does it work with new mysql now?

@blakewenloe

This comment has been minimized.

Copy link

blakewenloe commented Dec 18, 2017

@michikrug Everything seems to be working fine except for the URL and redirects.

When I load /gymhistory/ I get a 404.
When I load the URL as https://mywebsite.com/?page=gymhistory it loads the correct page, but when it loads, the URL changes to https://mywebsite.com/gymhistory?name=. Navigating to that page and/or refreshing it throws the 404 again. I have to use the ?page=gymhistory to get back to the page.

@michikrug

This comment has been minimized.

Copy link
Collaborator Author

michikrug commented Dec 18, 2017

@blakewenloe this seems like a configuration problem of your web server

@blakewenloe

This comment has been minimized.

Copy link

blakewenloe commented Dec 18, 2017

@blakewenloe

This comment has been minimized.

Copy link

blakewenloe commented Dec 19, 2017

For anyone else running into the issue with the /gymhistory/ URI on Nginx, you can add this to /etc/nginx/sites-available/your-site/:

rewrite ^/gymhistory/?$ /index.php?page=gymhistory&name= break;

That resolved it for me.

@blakewenloe
Copy link

blakewenloe left a comment

Works beautifully.

@darkelement1987

This comment has been minimized.

Copy link

darkelement1987 commented Jan 21, 2018

i had to rename "htaccess" to ".htaccess" (including the dot) to make the url in the navbar work.

Also i think the PR needs Responsive design. the page doesn't fit on mobile screens and you're unable to scroll sideways. (Huawei P10 Lite / Chrome app)

Works fine in Desktop-mode and on PC.

image

@michikrug

This comment has been minimized.

Copy link
Collaborator Author

michikrug commented Jan 21, 2018

It is responsive and there is nothing to side scroll, since the Pokemon are hidden on mobile.
The detailed entries will only show the changed ones.

@Obihoernchen

This comment has been minimized.

Copy link
Collaborator

Obihoernchen commented Jan 21, 2018

is this ready to be merged?

@michikrug

This comment has been minimized.

Copy link
Collaborator Author

michikrug commented Jan 27, 2018

I would say yes :)

@Obihoernchen Obihoernchen removed the WIP label Jan 28, 2018

@javisabalete

This comment has been minimized.

Copy link
Contributor

javisabalete commented Jan 30, 2018

I just tested this PR with #353 and json_encode returns error 5 (JSON_ERROR_UTF8)

@darkelement1987

This comment has been minimized.

Copy link

darkelement1987 commented Jan 31, 2018

I'm using this for a week or something without problems so my review will be positive.

@Obihoernchen
Copy link
Collaborator

Obihoernchen left a comment

looks fine

@Obihoernchen Obihoernchen merged commit 172c83b into brusselopole:master Feb 1, 2018

1 check failed

Scrutinizer Analysis: Failure condition met – Tests: passed
Details

rahciach pushed a commit to rahciach/Worldopole that referenced this pull request Feb 3, 2018

Add page to explore all changes of gyms over time (brusselopole#331)
* add gymhistory

* add readme for required SQL stuff

* make scrutinizer happier

* fix copy&paste error introduced with last commit

* correct inital SQL statement in readme

* fix last history entry not showing differences

* do not show undefined pokemon in empty gyms

* minor style change

* css indentation

* fix another mistake in the SQL statement for table creation

* refactor hiding of gym history tables

* fix window.history handling, refactor duplicated code

* add separate gym history loader, fix window.history handling finally

* minor style improvements

* update SQL statement for event creation, since it missed the delimiter part
(sorry folks, I should more often test my own stuff)

* change delimiter sign, make keyword uppercase

* add htaccess entry for gymhistory url

* comply with rocketmaps latest db eoncoding changes

* update charset to utf8mb4 as well

* filter out duplicated gym pokemon

* update sql statements

* adapt gymhistory to recent changes

* Merge remote-tracking branch 'brusselopole/master' into PR-gymhistory

* fix pokemon order

* fix pokemon order for history entries

* add option to hide only cp changes in gym history, add menu entry

* restore pokemon list, always show "load more" button since we may filter out entries on server side

* do filtering on client-side

* add history link

* fix ONLY_FULL_GROUP_BY incompatibility

* add gymhistory

* add readme for required SQL stuff

* make scrutinizer happier

* fix copy&paste error introduced with last commit

* correct inital SQL statement in readme

* fix last history entry not showing differences

* do not show undefined pokemon in empty gyms

* minor style change

* css indentation

* fix another mistake in the SQL statement for table creation

* refactor hiding of gym history tables

* fix window.history handling, refactor duplicated code

* add separate gym history loader, fix window.history handling finally

* minor style improvements

* update SQL statement for event creation, since it missed the delimiter part
(sorry folks, I should more often test my own stuff)

* change delimiter sign, make keyword uppercase

* add htaccess entry for gymhistory url

* comply with rocketmaps latest db eoncoding changes

* update charset to utf8mb4 as well

* filter out duplicated gym pokemon

* update sql statements

* adapt gymhistory to recent changes

* Merge remote-tracking branch 'brusselopole/master' into PR-gymhistory

* fix pokemon order

* fix pokemon order for history entries

* minor style fix

rahciach pushed a commit to rahciach/Worldopole that referenced this pull request Feb 5, 2018

Add page to explore all changes of gyms over time (brusselopole#331)
* add gymhistory

* add readme for required SQL stuff

* make scrutinizer happier

* fix copy&paste error introduced with last commit

* correct inital SQL statement in readme

* fix last history entry not showing differences

* do not show undefined pokemon in empty gyms

* minor style change

* css indentation

* fix another mistake in the SQL statement for table creation

* refactor hiding of gym history tables

* fix window.history handling, refactor duplicated code

* add separate gym history loader, fix window.history handling finally

* minor style improvements

* update SQL statement for event creation, since it missed the delimiter part
(sorry folks, I should more often test my own stuff)

* change delimiter sign, make keyword uppercase

* add htaccess entry for gymhistory url

* comply with rocketmaps latest db eoncoding changes

* update charset to utf8mb4 as well

* filter out duplicated gym pokemon

* update sql statements

* adapt gymhistory to recent changes

* Merge remote-tracking branch 'brusselopole/master' into PR-gymhistory

* fix pokemon order

* fix pokemon order for history entries

* add option to hide only cp changes in gym history, add menu entry

* restore pokemon list, always show "load more" button since we may filter out entries on server side

* do filtering on client-side

* add history link

* fix ONLY_FULL_GROUP_BY incompatibility

* add gymhistory

* add readme for required SQL stuff

* make scrutinizer happier

* fix copy&paste error introduced with last commit

* correct inital SQL statement in readme

* fix last history entry not showing differences

* do not show undefined pokemon in empty gyms

* minor style change

* css indentation

* fix another mistake in the SQL statement for table creation

* refactor hiding of gym history tables

* fix window.history handling, refactor duplicated code

* add separate gym history loader, fix window.history handling finally

* minor style improvements

* update SQL statement for event creation, since it missed the delimiter part
(sorry folks, I should more often test my own stuff)

* change delimiter sign, make keyword uppercase

* add htaccess entry for gymhistory url

* comply with rocketmaps latest db eoncoding changes

* update charset to utf8mb4 as well

* filter out duplicated gym pokemon

* update sql statements

* adapt gymhistory to recent changes

* Merge remote-tracking branch 'brusselopole/master' into PR-gymhistory

* fix pokemon order

* fix pokemon order for history entries

* minor style fix
@darkelement1987

This comment has been minimized.

Copy link

darkelement1987 commented Feb 11, 2018

I just realised my page doesnt work the way it should at all. If i click a gym no history is shown at all.

image

I followed all the steps from the gymhistory readme.

Ran all the sql stuff, added the scheduler to my.ini

Windows 7
Mariadb
Latest Worldopole build

@javisabalete

This comment has been minimized.

Copy link
Contributor

javisabalete commented Feb 11, 2018

Maybe SET GLOBAL event_scheduler = ON; ?

@darkelement1987

This comment has been minimized.

Copy link

darkelement1987 commented Feb 12, 2018

Never mind. After hours of trying to fix it, it was just the wrong db i was in -_- this can be closed again. I'm sorry

Only 1 more problem:

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.