-
Notifications
You must be signed in to change notification settings - Fork 0
Home
-> Go back to README
-> Jump to JSON Dictionary
-> List of QC Steps
- a GitHub Wiki is a repository as well
clone withgit clone git@github.com:rankingdigitalrights/datalab.wiki.git
(The development environment. Clockwise: VSCode+clasp; Google Apps Script; GitHub; Spreadsheet Output on Drive)
(top ↥)
To provide contributors with a cognitive model of this project, it is helpful to distinguish between
- the development environment
-
collaboration & version control: source code shared on
git
, edited locally with an IDE -
clasp
for deploying to Google Apps Script,
-
collaboration & version control: source code shared on
- the parameters & configuration:
JSON
files for Indicators, Companies, Research Steps, and global Index settings- the indicator & element language process has been stream-lined further with the introduction of the "indicator-factory" (cf. in @datapipe repo)
- the execution / runtime environment (Google Apps Script online editor),
- the output environment (Google Drive), and
- the UI/frontend (currently Google Spreadsheets).
- the data pipeline (in
@datapipe
repository), which is based on R, draws the data from the data stores, and produces all machine-readable outputs incl. curating the data for the Metabase SQL DB
(top ↥)
The basic structure is now modular, with
- 1 Master controller
00_mainController.js
in/*
(root) - all core modules being in
/src/*
and beginning with1x
(DC),2x
(SC), etc., and - most parameters stored in
/json/*
(companies, indicators, research steps, global Index parameters)
With the exception of the Input Spreadsheets Module, all core modules have been split into ...
- a main high-level module controller (
x0
) - a module interface (
x1
) - lower-level helper submodules (
x2 - xN
)
... in order to provide a single execution interface, to improve Code readability and re-use, and to enable affordable maintenance.
After modules had been introduced and eventually refactored, most modules and methods operate on the same centrally defined set of parameters (the JSON files) and the same set of helper functions (i.e. defining named Ranges). Therefore, if you make structural changes to e.g. researchSteps.step.component.labelShort
in JSON_researchSteps.js
while working on 10_inputInterface.js
, you need to apply this structural change to all modules concerned (incl. Helpers) as it might impact e.g. 21_scoringInterface.js
.
current best practice is to
- stick to Clean Code principles (i.e. explicit and precise naming of variables / functions
- use global search & replace with case-by-case replacement decisions
(top ↥)
currently stored in the Git repository as JSON/JS Objects (./json/JSON*.js)
TODO: should be offered either as clean JSON files (i.e. with fetch(JSON).parse()) or be imported from a central human-editable spreadsheet or AirTable config table
- research steps
- indicators
- new: elemsMetadata for richtext formatting / optional hyperlinking of Company Feedback forms
- companies
- config
JSON_researchSteps.js
TODO: currently, SSOT for indicator- and element-level **language **is in the
@datapipe
repository (specs-ssot/indicators.html). But SSOT for meta-**specifics **such as scoring scopes or legacy subcomponent relations are only defined in this repository
Therefore see Notion/Specs for current editing workflow.
TODO: proposal - if Language isn't going to be relocated to a central spreadsheet, then Language should be relocated to this repository and then both, Specs (json/JSON_Indicators.js) and Language being imported with a fetch() by @datapipe
-
JSON_indicators.js
(Specs) -
JSON_elemsMetadata.js
(Tags / Links for element-level language)
TODO: eventually merge both into 1 file and adjust @datapipe parsers
(top ↥)
caution: make sure that service children are listed in order of previous Index' listing (i.e. prepaid, postpaid, broadband)
-
Guidelines
-
Baseline: use 1-letter category
prefix
for companies and 2-letterprefix
for services - Authoritative: Dictionary
- use a GUI JSON editor for validation of JSON syntax (i.e. VSCode + JSON Editor
plugin
, or JSON Editor Online)
-
Baseline: use 1-letter category
(Screenshot: VSCode with JSON Editor Plugin: JSON file is opened on the left, the GUI editor on the right.)
(top ↥)
defines Index, methodology / sets, companies, folders, and maybe maps editors to companies (should be done from central spreadsheet for convenience for research team)
(top ↥)
Motivational quote:
The GAS documentation is not complete now and many features are discoverable experimentally." Source
- Documentation GAS
- GAS Best practices
- SO/google-apps-script
- SO/corey-g (former GAS engineer)
(top ↥)
- if you update existing files, named ranges are not removed by sheet.clear()
- max. runtime is 30 minutes / 1800 seconds. If your script breaks due to runtime error, you can see the log (Ctrl+Enter in the online editor) to see which company was processed last, delete the output for this company, and re-run the script with a subset, starting with the unfinished company.
- also, you can run multiple scripts in parallel… For this,
slice(1,4)
the companies vector in multiple parts in00_mainController.js
- file name ambiguity --> use fileID where possible
- only owner can delete files for real
- cell-based operations are costly, with formatting being the most expensive ones. Try to use array-based methods (2D range [][]) as much as possible.
- ...
(top ↥)
-
Collaboration:
"If you are working on a script project with other developers, you can collaborate on Apps Script projects with shared drives. Files in a shared drive are owned by the group, rather than individuals. This makes development and maintenance of the project easier." GAS Best practices
-
Refactoring:
- start with VSCode refactoring
- see array vs cell-based operations
- arrays in GAS in general