Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor build system, simplify templates and escape macros in code #229

Merged
merged 16 commits into from Dec 5, 2019
Merged

Refactor build system, simplify templates and escape macros in code #229

merged 16 commits into from Dec 5, 2019

Conversation

johnd0e
Copy link
Contributor

@johnd0e johnd0e commented Jul 30, 2019 •
edited

  • Refactor build system into several modules (can be used as cli utilities). Implement watch mode to auto-rebuild on source changes.
  • Escape macros in code in order to keep js-validity, fix #50.
    Use template from settings instead of url harcoding (closes #150).
  • Simplify userscripts source template, rename plugin sources: *.user.js -> *.js, rearrange source directories.
  • Remove timestamp component from version of Release scripts (closes #99).
  • Additional dev tools: web_meta_gen.py, web_server_local.py, tampermonkey_stubs_gen.py.

@johnd0e
Copy link
Contributor Author

@johnd0e johnd0e commented Jul 30, 2019 •
edited

.

@johnd0e johnd0e mentioned this pull request Jul 30, 2019
Closed
@johnd0e johnd0e added this to In progress in General development issues via automation Aug 6, 2019
@johnd0e johnd0e added the development label Aug 6, 2019
@johnd0e johnd0e force-pushed the simplify-plugin-template branch from f705f43 to e22623e Compare Aug 6, 2019
modos189
modos189 reviewed Aug 6, 2019
View changes
build.py Outdated Show resolved Hide resolved
@johnd0e johnd0e force-pushed the simplify-plugin-template branch from 4028a1b to 72e2bcc Compare Aug 8, 2019
@johnd0e
Copy link
Contributor Author

@johnd0e johnd0e commented Aug 13, 2019

I am also going to enhance build script to be able to build single plugin (even 3rd-party).

@johnd0e johnd0e added the WIP label Aug 28, 2019
@johnd0e johnd0e force-pushed the simplify-plugin-template branch from 72e2bcc to f12b8ee Compare Oct 24, 2019
@johnd0e johnd0e mentioned this pull request Oct 24, 2019
Closed
@johnd0e johnd0e force-pushed the simplify-plugin-template branch from f12b8ee to 9355aaa Compare Oct 25, 2019
modos189
modos189 previously requested changes Oct 25, 2019
View changes
Copy link
Contributor

@modos189 modos189 left a comment

It's pretty cool on the whole. I left a few questions in the comments

code/boot.js Outdated Show resolved Hide resolved
plugins/debug-raw-portal-data.user.js Outdated Show resolved Hide resolved
plugins/periodic-refresh.user.js Outdated Show resolved Hide resolved
total-conversion-build.js Outdated Show resolved Hide resolved
plugins/fix-china-map-offset.user.js Outdated Show resolved Hide resolved
build_plugin.py Outdated Show resolved Hide resolved
build_plugin.py Outdated Show resolved Hide resolved
plugins/basemap-gaode.user.js Outdated Show resolved Hide resolved
plugins/basemap-kartverket.user.js Outdated Show resolved Hide resolved
plugins/link-show-direction.user.js Outdated Show resolved Hide resolved
johnd0e added a commit to johnd0e/ingress-intel-total-conversion that referenced this issue Oct 28, 2019
@johnd0e
return workarounds for userscript managers not providing GM_info
82ba43b 
At the moment IITC-Mobile for iOS affected

Note: without GM_info IITC does not have much of plugins info (especially of 3rd-party plugins).
Such info is used About IITC dialog, and in console error messages,
and may be used even for more purposes in the future.

---
Another change:
remove version/description extra processing, as it is much clearer after IITC-CE#229.
johnd0e added a commit to johnd0e/ingress-intel-total-conversion that referenced this issue Oct 29, 2019
@johnd0e
return workarounds for userscript managers not providing GM_info
1ca7f3a 
At the moment IITC-Mobile for iOS affected

Note: without GM_info IITC does not have much of plugins info (especially of 3rd-party plugins).
Such info is used in 'About IITC' dialog and in console error messages
(and may be used even for more purposes in the future).

---
Another change:
remove description extra processing, as it is much clearer after IITC-CE#229.
@johnd0e johnd0e mentioned this pull request Oct 29, 2019
Merged
@johnd0e johnd0e force-pushed the simplify-plugin-template branch from 709ff59 to c96348a Compare Oct 29, 2019
@johnd0e johnd0e changed the title Simplify userscripts source template Refactor build system, simplify templates and escape macros in code Oct 29, 2019
@johnd0e johnd0e removed the WIP label Oct 29, 2019
@johnd0e johnd0e marked this pull request as ready for review Oct 29, 2019
@johnd0e johnd0e removed this from In progress in General development issues Oct 29, 2019
@johnd0e johnd0e added this to Priority in Track features for next release via automation Oct 29, 2019
@johnd0e johnd0e moved this from Priority to Testing in Track features for next release Oct 29, 2019
@johnd0e johnd0e force-pushed the simplify-plugin-template branch from c96348a to 74f338f Compare Oct 29, 2019
@modos189 modos189 mentioned this pull request Oct 29, 2019
Closed
@johnd0e johnd0e force-pushed the simplify-plugin-template branch from 74f338f to d865e06 Compare Oct 30, 2019
@johnd0e johnd0e mentioned this pull request Oct 30, 2019
Closed
9 tasks
@johnd0e johnd0e force-pushed the simplify-plugin-template branch from d865e06 to a939ee4 Compare Oct 31, 2019
@johnd0e johnd0e mentioned this pull request Oct 31, 2019
Closed
@johnd0e johnd0e force-pushed the simplify-plugin-template branch 2 times, most recently from 7032adf to f88f967 Compare Nov 30, 2019
johnd0e added 16 commits Dec 5, 2019
@johnd0e
plugins/deleted: remove old stubs
2368109
@johnd0e
speech-search -> deleted/
d89befa 
It's unfunctional since underlying webkit API removal (several years ago)
@johnd0e
Refactor build system, simplify templates and escape macros in code
bd15226 
Refactor build system into several modules:
- `settings.py`: to provide common settings (from `buildsettings.py` / `localbuildsettings.py`)
- `build_plugin.py`: to build single plugin
  formats userscript metablock, preprocesses sources - makes templates substitutions,
  inlines dependencies js/css/img - and wraps into wrapper (from `pluginwrapper.py`)
  - `pluginwrapper_noinject.py` - alternative wrapper, useful for debugging scripts.
     Sample usage in `buildsettings.py`, for build name 'tmdev'.
- `build_mobile.py`: to build android apk (embedding scripts from given directory)
- `build.py`: to build all: main IITC script, all plugins, and (optionally) Android apk.
  - `--watch` mode (auto-rebuild on sources changes)

Each module can be used independently as cli utility.

Escape macros in code in order to keep js-validity, fix IITC-CE#50.
Simplify userscripts source template, completely get rid of these ugly substitutions:
- `@@metainfo@@`
- `@@PLUGINSTART@@`
- `@@PLUGINEND@@`
- `.@@DATETIMEVERSION@@`
- `[@@Buildname@@-@@BUILDDATE@@]`

Close IITC-CE#150: use template from settings instead of url harcoding.

Close IITC-CE#99: remove timestamp component from `version` of Release scripts.
@johnd0e
clear excessive permissions
92774c4
@johnd0e
plugins: *.user.js -> *.js
41820dd 
*.js for raw sources
*.user.js for ready-to-use scripts
@johnd0e
smartphone.css: move from mobile/ to root
be2fdce
@johnd0e
web_meta_gen.py: utility to generate meta.json (for IITC-Button)
0c31cec
@johnd0e
web_server_local.py: utility to start local webserver for specified b…
ee66df2 
…uild name
@johnd0e
build scripts: drop Python 2.7 support
3a26cc4 
From now Python at least version 3.6 required
@johnd0e
sources rearrangement
0eda847 
external -> /core/external/,        /plugins/external/
images   -> /core/images/,          /plugins/images/
         -> /core/external/images/, /plugins/external/images/
@johnd0e
update .travis.yml
b2da247
@johnd0e
build.py: improve watch mode, and add same possibility to build_plugi…
4162de9 
…n.py

Watch mode is now properly implemented:
it considers exact source dependencies
(instead of reacting on just any directory change)
@johnd0e
settings.py: tie source/target directories to script's directory (ins…
d3c803a 
…tead of cwd)

Can be overriden in localbuildsettings.py
@johnd0e
web_meta_gen.py: support (multilingual) descriptions for categories
0ba88f3 
Descriptions source: plugins/categories.json
(from site's repo: https://github.com/IITC-CE/iitc.modos189.ru/blob/master/box.py)

Generated meta.json now used for:
- site generation
- IITC-Button
@johnd0e
build.py: minor correction of watch mode
ff655d8 
Do not try to track changes of localbuildsettings.
Anyway it had no effect without settings reloading.
@johnd0e
tampermonkey_stubs_gen.py: new helper utility
063fb8d 
to make Tampermonkey read userscripts from local disk.
@johnd0e johnd0e force-pushed the simplify-plugin-template branch from bd17206 to 063fb8d Compare Dec 5, 2019
@johnd0e johnd0e merged commit f718d42 into IITC-CE:master Dec 5, 2019
Track features for next release automation moved this from Testing to Merged Dec 5, 2019
@johnd0e johnd0e deleted the simplify-plugin-template branch Dec 5, 2019
johnd0e added a commit to johnd0e/ingress-intel-total-conversion that referenced this issue Dec 5, 2019
@johnd0e
bump version to 0.30
80e4451 
- Update most upstream sources, and significantly enhance several plugins IITC-CE#134 (a lot of changes, follow the link to see)
- Fix performance drop when displaying a large number of ornaments IITC-CE#181
- Improve RegionScoreboard IITC-CE#179 © McBen + some fixes
- Remove confusing message from artifacts dialog IITC-CE#216
- Portal info: display coordinates without angled brackets IITC-CE#233
- Leaflet controls: prevent mobile style on desktop IITC-CE#189
- Increase map renderer padding (and make value customizable) IITC-CE#201
- Portal info: `shielding` tooltip: round mitigation `excess` to cope with lack of js float precision IITC-CE#279

- Fix miscellaneous bugs in core and plugins IITC-CE#157
- Change default intel url (to match stock) IITC-CE#267

- Implement plugins priority control IITC-CE#205
- Stop console 'spam' using customizable logger IITC-CE#235
- Consistent errors throwing IITC-CE#248
- API for local files (up|down)loading IITC-CE#243
- Remove some unused stuff from window object IITC-CE#101
- Refactor to use common function `window.makePermalink` (utils_misc.js) IITC-CE#198
- Isolate every core module in separate IIFE IITC-CE#234
- Move wrapper template to separate file IITC-CE#238
- DEFAULT_ZOOM = 15 (to match stock intel) IITC-CE#281
- IITC-CE#229:
  - Refactor build system into several modules (can be used as cli utilities).
    Implement `watch` mode to auto-rebuild on source changes.
  - Escape macros in code in order to keep js-validity, fix IITC-CE#50.
    Use template from settings instead of url harcoding (closes IITC-CE#150).
  - Simplify userscripts source template, rename plugin sources:  `*.user.js` -> `*.js`, rearrange source directories.
  - Remove timestamp component from `version` of Release scripts (closes IITC-CE#99).
  - Additional dev tools: `web_meta_gen.py`, `web_server_local.py`, `tampermonkey_stubs_gen.py`.
- Other
  - see IITC-CE#134 (solve IITC-CE#70: Manage upstream externals)
  - Update docs IITC-CE#264, remove outdated (info transferred to [wiki](https://github.com/IITC-CE/ingress-intel-total-conversion/wiki))

- Improve scale-bar and scoreboard plugins IITC-CE#158
- Improve styles of text labels (portal-level-numbers, portal-names, portal-names, regions, keys-on-map) IITC-CE#104
- Other
  - draw-tools, bookmarks: (make use of IITC-CE#243) support file import/export on desktop too
  - see IITC-CE#134: significantly enhance several plugins : overlay-kml, minimap, basemap-bing, basemap-yandex, tidy-links, pan-control, etc (see commit messages)

- sync: fix and update IITC-CE#212
- draw-tools: fix mobile-related bugs; add Circle again IITC-CE#175
- bookmarks: fix iitc download url IITC-CE#150
- portals-list: filter out portal without real data (bug in intel) IITC-CE#265
- Other: see IITC-CE#157 (cache-portals-on-map, layer-count, draw-tools, bookmarks, missions)

- 'Support DeX desktop mode' option IITC-CE#161
- Add 'Plugins' shortcut to main menu IITC-CE#266

- fix bugs on login page IITC-CE#164
- fix some bugs in plugins handling IITC-CE#245
- fix 'Send screenshot' function IITC-CE#169
- fix error that made it difficult to pick plugin file IITC-CE#239
- fix plugin initialization bug IITC-CE#270

- better handling of uploads IITC-CE#250
johnd0e added a commit to johnd0e/ingress-intel-total-conversion that referenced this issue Dec 5, 2019
@johnd0e
bump version to 0.30
bb0bbf2 
# Release notes

## IITC main script

### enhancements
- Update most upstream sources, and significantly enhance several plugins IITC-CE#134 (a lot of changes, follow the link to see)
- Fix performance drop when displaying a large number of ornaments IITC-CE#181
- Improve RegionScoreboard IITC-CE#179 © McBen + some fixes
- Remove confusing message from artifacts dialog IITC-CE#216
- Portal info: display coordinates without angled brackets IITC-CE#233
- Leaflet controls: prevent mobile style on desktop IITC-CE#189
- Increase map renderer padding (and make value customizable) IITC-CE#201
- Portal info: `shielding` tooltip: round mitigation `excess` to cope with lack of js float precision IITC-CE#279

### bugfixes
- Fix miscellaneous bugs in core and plugins IITC-CE#157
- Change default intel url (to match stock) IITC-CE#267

### development
- Implement plugins priority control IITC-CE#205
- Stop console 'spam' using customizable logger IITC-CE#235
- Consistent errors throwing IITC-CE#248
- API for local files (up|down)loading IITC-CE#243
- Remove some unused stuff from window object IITC-CE#101
- Refactor to use common function `window.makePermalink` (utils_misc.js) IITC-CE#198
- Isolate every core module in separate IIFE IITC-CE#234
- Move wrapper template to separate file IITC-CE#238
- DEFAULT_ZOOM = 15 (to match stock intel) IITC-CE#281
- IITC-CE#229:
  - Refactor build system into several modules (can be used as cli utilities).
    Implement `watch` mode to auto-rebuild on source changes.
  - Escape macros in code in order to keep js-validity, fix IITC-CE#50.
    Use template from settings instead of url harcoding (closes IITC-CE#150).
  - Simplify userscripts source template, rename plugin sources:  `*.user.js` -> `*.js`, rearrange source directories.
  - Remove timestamp component from `version` of Release scripts (closes IITC-CE#99).
  - Additional dev tools: `web_meta_gen.py`, `web_server_local.py`, `tampermonkey_stubs_gen.py`.
- Other
  - see IITC-CE#134 (solve IITC-CE#70: Manage upstream externals)
  - Update docs IITC-CE#264, remove outdated (info transferred to [wiki](https://github.com/IITC-CE/ingress-intel-total-conversion/wiki))

## Plugins

### enhancements
- Improve scale-bar and scoreboard plugins IITC-CE#158
- Improve styles of text labels (portal-level-numbers, portal-names, portal-names, regions, keys-on-map) IITC-CE#104
- Other
  - draw-tools, bookmarks: (make use of IITC-CE#243) support file import/export on desktop too
  - see IITC-CE#134: significantly enhance several plugins : overlay-kml, minimap, basemap-bing, basemap-yandex, tidy-links, pan-control, etc (see commit messages)

### bugfixes
- sync: fix and update IITC-CE#212
- draw-tools: fix mobile-related bugs; add Circle again IITC-CE#175
- bookmarks: fix iitc download url IITC-CE#150
- portals-list: filter out portal without real data (bug in intel) IITC-CE#265
- Other: see IITC-CE#157 (cache-portals-on-map, layer-count, draw-tools, bookmarks, missions)

## IITC-Mobile app

### enhancements
- 'Support DeX desktop mode' option IITC-CE#161
- Add 'Plugins' shortcut to main menu IITC-CE#266

### bugfixes
- fix bugs on login page IITC-CE#164
- fix some bugs in plugins handling IITC-CE#245
- fix 'Send screenshot' function IITC-CE#169
- fix error that made it difficult to pick plugin file IITC-CE#239
- fix plugin initialization bug IITC-CE#270

### development
- better handling of uploads IITC-CE#250
@johnd0e johnd0e mentioned this pull request Dec 5, 2019
Merged
johnd0e added a commit that referenced this issue Dec 6, 2019
@johnd0e
build.py: fixup for #229
474b623 
prevent exception when cmd is callable
johnd0e added a commit that referenced this issue Dec 7, 2019
@johnd0e
Merge pull request #282 from johnd0e/0.30
dca8b6e 
bump version to 0.30

# Release notes

## IITC main script

### enhancements
- Update most upstream sources, and significantly enhance several plugins #134 (a lot of changes, follow the link to see)
- Fix performance drop when displaying a large number of ornaments #181
- Improve RegionScoreboard #179 © McBen + some fixes
- Remove confusing message from artifacts dialog #216
- Portal info: display coordinates without angled brackets #233
- Leaflet controls: prevent mobile style on desktop #189
- Increase map renderer padding (and make value customizable) #201
- Portal info: `shielding` tooltip: round mitigation `excess` to cope with lack of js float precision #279

### bugfixes
- Fix miscellaneous bugs in core and plugins #157
- Change default intel url (to match stock) #267

### development
- Implement plugins priority control #205
- Stop console 'spam' using customizable logger #235
- Consistent errors throwing #248
- API for local files (up|down)loading #243
- Remove some unused stuff from window object #101
- Refactor to use common function `window.makePermalink` (utils_misc.js) #198
- Isolate every core module in separate IIFE #234
- Move wrapper template to separate file #238
- DEFAULT_ZOOM = 15 (to match stock intel) #281
- #229:
  - Refactor build system into several modules (can be used as cli utilities).
    Implement `watch` mode to auto-rebuild on source changes.
  - Escape macros in code in order to keep js-validity, fix #50.
    Use template from settings instead of url harcoding (closes #150).
  - Simplify userscripts source template, rename plugin sources:  `*.user.js` -> `*.js`, rearrange source directories.
  - Remove timestamp component from `version` of Release scripts (closes #99).
  - Additional dev tools: `web_meta_gen.py`, `web_server_local.py`, `tampermonkey_stubs_gen.py`.
- Other
  - see #134 (solve #70: Manage upstream externals)
  - Update docs #264, remove outdated (info transferred to [wiki](https://github.com/IITC-CE/ingress-intel-total-conversion/wiki))

## Plugins

### enhancements
- Improve scale-bar and scoreboard plugins #158
- Improve styles of text labels (portal-level-numbers, portal-names, portal-names, regions, keys-on-map) #104
- Other
  - draw-tools, bookmarks: (make use of #243) support file import/export on desktop too
  - see #134: significantly enhance several plugins : overlay-kml, minimap, basemap-bing, basemap-yandex, tidy-links, pan-control, etc (see commit messages)

### bugfixes
- sync: fix and update #212
- draw-tools: fix mobile-related bugs; add Circle again #175
- bookmarks: fix iitc download url #150
- portals-list: filter out portal without real data (bug in intel) #265
- Other: see #157 (cache-portals-on-map, layer-count, draw-tools, bookmarks, missions)

## IITC-Mobile app

### enhancements
- 'Support DeX desktop mode' option #161
- Add 'Plugins' shortcut to main menu #266
- Adapt to changes in Google authorization #283

### bugfixes
- fix bugs on login page #164
- fix some bugs in plugins handling #245
- fix 'Send screenshot' function #169
- fix error that made it difficult to pick plugin file #239
- fix plugin initialization bug #270

### development
- better handling of uploads #250
@modos189 modos189 moved this from Merged to Old release in Track features for next release Dec 22, 2019
@modos189 modos189 moved this from Old release to Merged in Track features for next release Dec 22, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@modos189 modos189

Assignees
No one assigned
Labels
development
Projects
Track features for next release
  
Merged
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Scripts versioning
@@-substitutions break validity of plugins' sources
2 participants
@johnd0e @modos189