Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor build system, simplify templates and escape macros in code #229

Merged
merged 16 commits into from Dec 5, 2019

Conversation

@johnd0e
Copy link
Contributor

johnd0e commented Jul 30, 2019

  • Refactor build system into several modules (can be used as cli utilities). Implement watch mode to auto-rebuild on source changes.
  • Escape macros in code in order to keep js-validity, fix #50.
    Use template from settings instead of url harcoding (closes #150).
  • Simplify userscripts source template, rename plugin sources: *.user.js -> *.js, rearrange source directories.
  • Remove timestamp component from version of Release scripts (closes #99).
  • Additional dev tools: web_meta_gen.py, web_server_local.py, tampermonkey_stubs_gen.py.
@johnd0e

This comment has been minimized.

Copy link
Contributor Author

johnd0e commented Jul 30, 2019

.

@johnd0e johnd0e mentioned this pull request Jul 30, 2019
@johnd0e johnd0e added this to In progress in General development issues via automation Aug 6, 2019
@johnd0e johnd0e added the development label Aug 6, 2019
@johnd0e johnd0e force-pushed the johnd0e:simplify-plugin-template branch from f705f43 to e22623e Aug 6, 2019
build.py Outdated Show resolved Hide resolved
@johnd0e johnd0e force-pushed the johnd0e:simplify-plugin-template branch from 4028a1b to 72e2bcc Aug 8, 2019
@johnd0e

This comment has been minimized.

Copy link
Contributor Author

johnd0e commented Aug 13, 2019

I am also going to enhance build script to be able to build single plugin (even 3rd-party).

@johnd0e johnd0e added the WIP label Aug 28, 2019
@johnd0e johnd0e force-pushed the johnd0e:simplify-plugin-template branch from 72e2bcc to f12b8ee Oct 24, 2019
@johnd0e johnd0e mentioned this pull request Oct 24, 2019
@johnd0e johnd0e force-pushed the johnd0e:simplify-plugin-template branch from f12b8ee to 9355aaa Oct 25, 2019
Copy link
Contributor

modos189 left a comment

It's pretty cool on the whole. I left a few questions in the comments

code/boot.js Outdated Show resolved Hide resolved
plugins/debug-raw-portal-data.user.js Outdated Show resolved Hide resolved
plugins/periodic-refresh.user.js Outdated Show resolved Hide resolved
total-conversion-build.js Outdated Show resolved Hide resolved
plugins/fix-china-map-offset.user.js Outdated Show resolved Hide resolved
build_plugin.py Outdated Show resolved Hide resolved
build_plugin.py Outdated Show resolved Hide resolved
plugins/basemap-gaode.user.js Outdated Show resolved Hide resolved
plugins/basemap-kartverket.user.js Outdated Show resolved Hide resolved
plugins/link-show-direction.user.js Outdated Show resolved Hide resolved
johnd0e added a commit to johnd0e/ingress-intel-total-conversion that referenced this pull request Oct 28, 2019
At the moment IITC-Mobile for iOS affected

Note: without GM_info IITC does not have much of plugins info (especially of 3rd-party plugins).
Such info is used About IITC dialog, and in console error messages,
and may be used even for more purposes in the future.

---
Another change:
remove version/description extra processing, as it is much clearer after IITC-CE#229.
johnd0e added a commit to johnd0e/ingress-intel-total-conversion that referenced this pull request Oct 29, 2019
At the moment IITC-Mobile for iOS affected

Note: without GM_info IITC does not have much of plugins info (especially of 3rd-party plugins).
Such info is used in 'About IITC' dialog and in console error messages
(and may be used even for more purposes in the future).

---
Another change:
remove description extra processing, as it is much clearer after IITC-CE#229.
@johnd0e johnd0e force-pushed the johnd0e:simplify-plugin-template branch from 709ff59 to c96348a Oct 29, 2019
@johnd0e johnd0e changed the title Simplify userscripts source template Refactor build system, simplify templates and escape macros in code Oct 29, 2019
@johnd0e johnd0e removed the WIP label Oct 29, 2019
@johnd0e johnd0e marked this pull request as ready for review Oct 29, 2019
@johnd0e johnd0e removed this from In progress in General development issues Oct 29, 2019
@johnd0e johnd0e added this to Priority in Track features for next release via automation Oct 29, 2019
@johnd0e johnd0e moved this from Priority to Testing in Track features for next release Oct 29, 2019
@johnd0e johnd0e force-pushed the johnd0e:simplify-plugin-template branch from c96348a to 74f338f Oct 29, 2019
@modos189 modos189 mentioned this pull request Oct 29, 2019
@johnd0e johnd0e dismissed modos189’s stale review Oct 30, 2019

outdated

@johnd0e johnd0e force-pushed the johnd0e:simplify-plugin-template branch from 74f338f to d865e06 Oct 30, 2019
@johnd0e johnd0e mentioned this pull request Oct 30, 2019
9 of 9 tasks complete
@johnd0e johnd0e force-pushed the johnd0e:simplify-plugin-template branch from d865e06 to a939ee4 Oct 31, 2019
@johnd0e johnd0e force-pushed the johnd0e:simplify-plugin-template branch 2 times, most recently from 7032adf to f88f967 Nov 30, 2019
johnd0e added 16 commits Oct 29, 2019
It's unfunctional since underlying webkit API removal (several years ago)
Refactor build system into several modules:
- `settings.py`: to provide common settings (from `buildsettings.py` / `localbuildsettings.py`)
- `build_plugin.py`: to build single plugin
  formats userscript metablock, preprocesses sources - makes templates substitutions,
  inlines dependencies js/css/img - and wraps into wrapper (from `pluginwrapper.py`)
  - `pluginwrapper_noinject.py` - alternative wrapper, useful for debugging scripts.
     Sample usage in `buildsettings.py`, for build name 'tmdev'.
- `build_mobile.py`: to build android apk (embedding scripts from given directory)
- `build.py`: to build all: main IITC script, all plugins, and (optionally) Android apk.
  - `--watch` mode (auto-rebuild on sources changes)

Each module can be used independently as cli utility.

Escape macros in code in order to keep js-validity, fix #50.
Simplify userscripts source template, completely get rid of these ugly substitutions:
- `@@metainfo@@`
- `@@PLUGINSTART@@`
- `@@PLUGINEND@@`
- `.@@DATETIMEVERSION@@`
- `[@@Buildname@@-@@BUILDDATE@@]`

Close #150: use template from settings instead of url harcoding.

Close #99: remove timestamp component from `version` of Release scripts.
From now Python at least version 3.6 required
*.js for raw sources
*.user.js for ready-to-use scripts
external -> /core/external/,        /plugins/external/
images   -> /core/images/,          /plugins/images/
         -> /core/external/images/, /plugins/external/images/
…n.py

Watch mode is now properly implemented:
it considers exact source dependencies
(instead of reacting on just any directory change)
…tead of cwd)

Can be overriden in localbuildsettings.py
Descriptions source: plugins/categories.json
(from site's repo: https://github.com/IITC-CE/iitc.modos189.ru/blob/master/box.py)

Generated meta.json now used for:
- site generation
- IITC-Button
Do not try to track changes of localbuildsettings.
Anyway it had no effect without settings reloading.
to make Tampermonkey read userscripts from local disk.
@johnd0e johnd0e force-pushed the johnd0e:simplify-plugin-template branch from bd17206 to 063fb8d Dec 5, 2019
@johnd0e johnd0e merged commit f718d42 into IITC-CE:master Dec 5, 2019
Track features for next release automation moved this from Testing to Merged Dec 5, 2019
@johnd0e johnd0e deleted the johnd0e:simplify-plugin-template branch Dec 5, 2019
johnd0e added a commit to johnd0e/ingress-intel-total-conversion that referenced this pull request Dec 5, 2019
- Update most upstream sources, and significantly enhance several plugins IITC-CE#134 (a lot of changes, follow the link to see)
- Fix performance drop when displaying a large number of ornaments IITC-CE#181
- Improve RegionScoreboard IITC-CE#179 © McBen + some fixes
- Remove confusing message from artifacts dialog IITC-CE#216
- Portal info: display coordinates without angled brackets IITC-CE#233
- Leaflet controls: prevent mobile style on desktop IITC-CE#189
- Increase map renderer padding (and make value customizable) IITC-CE#201
- Portal info: `shielding` tooltip: round mitigation `excess` to cope with lack of js float precision IITC-CE#279

- Fix miscellaneous bugs in core and plugins IITC-CE#157
- Change default intel url (to match stock) IITC-CE#267

- Implement plugins priority control IITC-CE#205
- Stop console 'spam' using customizable logger IITC-CE#235
- Consistent errors throwing IITC-CE#248
- API for local files (up|down)loading IITC-CE#243
- Remove some unused stuff from window object IITC-CE#101
- Refactor to use common function `window.makePermalink` (utils_misc.js) IITC-CE#198
- Isolate every core module in separate IIFE IITC-CE#234
- Move wrapper template to separate file IITC-CE#238
- DEFAULT_ZOOM = 15 (to match stock intel) IITC-CE#281
- IITC-CE#229:
  - Refactor build system into several modules (can be used as cli utilities).
    Implement `watch` mode to auto-rebuild on source changes.
  - Escape macros in code in order to keep js-validity, fix IITC-CE#50.
    Use template from settings instead of url harcoding (closes IITC-CE#150).
  - Simplify userscripts source template, rename plugin sources:  `*.user.js` -> `*.js`, rearrange source directories.
  - Remove timestamp component from `version` of Release scripts (closes IITC-CE#99).
  - Additional dev tools: `web_meta_gen.py`, `web_server_local.py`, `tampermonkey_stubs_gen.py`.
- Other
  - see IITC-CE#134 (solve IITC-CE#70: Manage upstream externals)
  - Update docs IITC-CE#264, remove outdated (info transferred to [wiki](https://github.com/IITC-CE/ingress-intel-total-conversion/wiki))

- Improve scale-bar and scoreboard plugins IITC-CE#158
- Improve styles of text labels (portal-level-numbers, portal-names, portal-names, regions, keys-on-map) IITC-CE#104
- Other
  - draw-tools, bookmarks: (make use of IITC-CE#243) support file import/export on desktop too
  - see IITC-CE#134: significantly enhance several plugins : overlay-kml, minimap, basemap-bing, basemap-yandex, tidy-links, pan-control, etc (see commit messages)

- sync: fix and update IITC-CE#212
- draw-tools: fix mobile-related bugs; add Circle again IITC-CE#175
- bookmarks: fix iitc download url IITC-CE#150
- portals-list: filter out portal without real data (bug in intel) IITC-CE#265
- Other: see IITC-CE#157 (cache-portals-on-map, layer-count, draw-tools, bookmarks, missions)

- 'Support DeX desktop mode' option IITC-CE#161
- Add 'Plugins' shortcut to main menu IITC-CE#266

- fix bugs on login page IITC-CE#164
- fix some bugs in plugins handling IITC-CE#245
- fix 'Send screenshot' function IITC-CE#169
- fix error that made it difficult to pick plugin file IITC-CE#239
- fix plugin initialization bug IITC-CE#270

- better handling of uploads IITC-CE#250
johnd0e added a commit to johnd0e/ingress-intel-total-conversion that referenced this pull request Dec 5, 2019
# Release notes

## IITC main script

### enhancements
- Update most upstream sources, and significantly enhance several plugins IITC-CE#134 (a lot of changes, follow the link to see)
- Fix performance drop when displaying a large number of ornaments IITC-CE#181
- Improve RegionScoreboard IITC-CE#179 © McBen + some fixes
- Remove confusing message from artifacts dialog IITC-CE#216
- Portal info: display coordinates without angled brackets IITC-CE#233
- Leaflet controls: prevent mobile style on desktop IITC-CE#189
- Increase map renderer padding (and make value customizable) IITC-CE#201
- Portal info: `shielding` tooltip: round mitigation `excess` to cope with lack of js float precision IITC-CE#279

### bugfixes
- Fix miscellaneous bugs in core and plugins IITC-CE#157
- Change default intel url (to match stock) IITC-CE#267

### development
- Implement plugins priority control IITC-CE#205
- Stop console 'spam' using customizable logger IITC-CE#235
- Consistent errors throwing IITC-CE#248
- API for local files (up|down)loading IITC-CE#243
- Remove some unused stuff from window object IITC-CE#101
- Refactor to use common function `window.makePermalink` (utils_misc.js) IITC-CE#198
- Isolate every core module in separate IIFE IITC-CE#234
- Move wrapper template to separate file IITC-CE#238
- DEFAULT_ZOOM = 15 (to match stock intel) IITC-CE#281
- IITC-CE#229:
  - Refactor build system into several modules (can be used as cli utilities).
    Implement `watch` mode to auto-rebuild on source changes.
  - Escape macros in code in order to keep js-validity, fix IITC-CE#50.
    Use template from settings instead of url harcoding (closes IITC-CE#150).
  - Simplify userscripts source template, rename plugin sources:  `*.user.js` -> `*.js`, rearrange source directories.
  - Remove timestamp component from `version` of Release scripts (closes IITC-CE#99).
  - Additional dev tools: `web_meta_gen.py`, `web_server_local.py`, `tampermonkey_stubs_gen.py`.
- Other
  - see IITC-CE#134 (solve IITC-CE#70: Manage upstream externals)
  - Update docs IITC-CE#264, remove outdated (info transferred to [wiki](https://github.com/IITC-CE/ingress-intel-total-conversion/wiki))

## Plugins

### enhancements
- Improve scale-bar and scoreboard plugins IITC-CE#158
- Improve styles of text labels (portal-level-numbers, portal-names, portal-names, regions, keys-on-map) IITC-CE#104
- Other
  - draw-tools, bookmarks: (make use of IITC-CE#243) support file import/export on desktop too
  - see IITC-CE#134: significantly enhance several plugins : overlay-kml, minimap, basemap-bing, basemap-yandex, tidy-links, pan-control, etc (see commit messages)

### bugfixes
- sync: fix and update IITC-CE#212
- draw-tools: fix mobile-related bugs; add Circle again IITC-CE#175
- bookmarks: fix iitc download url IITC-CE#150
- portals-list: filter out portal without real data (bug in intel) IITC-CE#265
- Other: see IITC-CE#157 (cache-portals-on-map, layer-count, draw-tools, bookmarks, missions)

## IITC-Mobile app

### enhancements
- 'Support DeX desktop mode' option IITC-CE#161
- Add 'Plugins' shortcut to main menu IITC-CE#266

### bugfixes
- fix bugs on login page IITC-CE#164
- fix some bugs in plugins handling IITC-CE#245
- fix 'Send screenshot' function IITC-CE#169
- fix error that made it difficult to pick plugin file IITC-CE#239
- fix plugin initialization bug IITC-CE#270

### development
- better handling of uploads IITC-CE#250
@johnd0e johnd0e mentioned this pull request Dec 5, 2019
johnd0e added a commit that referenced this pull request Dec 6, 2019
prevent exception when cmd is callable
johnd0e added a commit that referenced this pull request Dec 7, 2019
bump version to 0.30

# Release notes

## IITC main script

### enhancements
- Update most upstream sources, and significantly enhance several plugins #134 (a lot of changes, follow the link to see)
- Fix performance drop when displaying a large number of ornaments #181
- Improve RegionScoreboard #179 © McBen + some fixes
- Remove confusing message from artifacts dialog #216
- Portal info: display coordinates without angled brackets #233
- Leaflet controls: prevent mobile style on desktop #189
- Increase map renderer padding (and make value customizable) #201
- Portal info: `shielding` tooltip: round mitigation `excess` to cope with lack of js float precision #279

### bugfixes
- Fix miscellaneous bugs in core and plugins #157
- Change default intel url (to match stock) #267

### development
- Implement plugins priority control #205
- Stop console 'spam' using customizable logger #235
- Consistent errors throwing #248
- API for local files (up|down)loading #243
- Remove some unused stuff from window object #101
- Refactor to use common function `window.makePermalink` (utils_misc.js) #198
- Isolate every core module in separate IIFE #234
- Move wrapper template to separate file #238
- DEFAULT_ZOOM = 15 (to match stock intel) #281
- #229:
  - Refactor build system into several modules (can be used as cli utilities).
    Implement `watch` mode to auto-rebuild on source changes.
  - Escape macros in code in order to keep js-validity, fix #50.
    Use template from settings instead of url harcoding (closes #150).
  - Simplify userscripts source template, rename plugin sources:  `*.user.js` -> `*.js`, rearrange source directories.
  - Remove timestamp component from `version` of Release scripts (closes #99).
  - Additional dev tools: `web_meta_gen.py`, `web_server_local.py`, `tampermonkey_stubs_gen.py`.
- Other
  - see #134 (solve #70: Manage upstream externals)
  - Update docs #264, remove outdated (info transferred to [wiki](https://github.com/IITC-CE/ingress-intel-total-conversion/wiki))

## Plugins

### enhancements
- Improve scale-bar and scoreboard plugins #158
- Improve styles of text labels (portal-level-numbers, portal-names, portal-names, regions, keys-on-map) #104
- Other
  - draw-tools, bookmarks: (make use of #243) support file import/export on desktop too
  - see #134: significantly enhance several plugins : overlay-kml, minimap, basemap-bing, basemap-yandex, tidy-links, pan-control, etc (see commit messages)

### bugfixes
- sync: fix and update #212
- draw-tools: fix mobile-related bugs; add Circle again #175
- bookmarks: fix iitc download url #150
- portals-list: filter out portal without real data (bug in intel) #265
- Other: see #157 (cache-portals-on-map, layer-count, draw-tools, bookmarks, missions)

## IITC-Mobile app

### enhancements
- 'Support DeX desktop mode' option #161
- Add 'Plugins' shortcut to main menu #266
- Adapt to changes in Google authorization #283

### bugfixes
- fix bugs on login page #164
- fix some bugs in plugins handling #245
- fix 'Send screenshot' function #169
- fix error that made it difficult to pick plugin file #239
- fix plugin initialization bug #270

### development
- better handling of uploads #250
@modos189 modos189 moved this from Merged to Old release in Track features for next release Dec 22, 2019
@modos189 modos189 moved this from Old release to Merged in Track features for next release Dec 22, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
2 participants
You can’t perform that action at this time.