Skip to content

Latest commit

 

History

History
799 lines (631 loc) · 36.2 KB

DEVELOPERS.md

File metadata and controls

799 lines (631 loc) · 36.2 KB
Raw

Developers

Table of contents {ignore=true}

Quick start

Get running on your host machine quickly with:

npm install
npm start

(See below to get setup with Docker instead)

Docker

WVUI comes with a docker configuration for local development.

Using Docker is not necessary, but strongly suggested. See quick start for developing without Docker. Containerizing WVUI with Docker makes it easy to have a standard, shared environment for local development among developers, as well as integration with automated CI pipelines.

To get started:

  1. Install Docker and Docker Compose.
  2. Build Docker images
docker-compose build --build-arg UID=$(id -u) --build-arg GID=$(id -g) --build-arg HOST=$(uname -s)
# Build arguments needed so that we own Docker generated files
  1. Install npm packages from host machine
npm install
  1. Startup containers
docker-compose up node storybook

Container Configuration

WVUI's docker compose configuration produces 3 services. The startup command above produces two separate docker containers each with their own service: node and storybook. The rationale behind 2 containers is for separation of concerns, so that each container is responsible for one service only. A third docker container release exists strictly for publishing WVUI releases.

storybook
On container startup, storybook will be accessible on localhost:3003. This container is intended for local development with Storybook.

node
On container startup, node is by default stopped. This service is for mounting project files. Execute any ad-hoc commands inside the container ( e.g. any NPM scripts by running:

docker-compose run --rm [node|storybook] npm run [script name]

If you need to install additional dependencies after container creation (e.g. adding any modules to package.json), make sure you run docker-compose up again for the changes to take affect.

I/O performance on macOS

Docker containers run via Docker Desktop for Mac interact with the host's filesystem via a Hyperkit hypervisor running in a LinuxKit Virtual Machine. The hypervisor and VM are hidden from the user but they quickly become visible when performing I/O intensive operations like npm i. For example, an unscientific benchmark has docker run --rm node npm install taking over 19 minutes.

Fortunately, Docker Desktop for Mac supports NFS volumes. Jeff Geerling wrote an excellent summary of this issue along with a guide to sharing folders via NFS for use with Docker Desktop for Mac. Briefly:

  1. echo "nfs.server.mount.require_resv_port = 0" | sudo tee --append /etc/nfs.conf
  2. echo "${PWD} -alldirs -mapall=$(id -u):$(id -g) 127.0.0.1" | sudo tee --append /etc/exports
  3. Create docker-compose.override.yaml and add the following:
version: "3.8"

services:
    node:
        volumes:
            - "nfsmount:/app"

volumes:
    nfsmount:
        driver: local
        driver_opts:
            type: nfs
            o: addr=host.docker.internal,rw,nolock,hard,nointr,nfsvers=3
            device: ":${PWD}"
  1. Rebuild the node container (see Docker)

With the above done, the unscientific benchmark above takes a little over five minutes.

Blubber

WVUI contains a blubber.yaml file, for use by the tool Blubber. Blubber is developed and used by Wikimedia as an abstraction layer between a project and the creation of the Docker images that will build, test, and deploy the project. When WVUI goes through Wikimedia's Jenkins CI pipeline, Blubber will read the blubber.yaml, generate a Dockerfile, create the image per the blubber configuation, and execute the command specified in the blubber.yaml command attribute. The blubber.yaml file should be modified if you use Blubber in your CI pipeline. Otherwise, it can be ignored.

NPM scripts

  • install / i: install project dependencies.
  • start: run Storybook development workflow.
  • test / t: build the project and execute all tests. Anything that can be validated automatically before publishing runs through this command. See testing.
  • run test:unit: run the unit tests. Pass -u to update all Jest snapshots.
  • run format: apply lint fixes automatically where available.
  • run build: compile source inputs to bundle outputs under dist/.
  • run doc: generate all documentation under doc/.
  • version: increment the version. See versioning.
  • publish: publish the version to NPM. See versioning.

Scripts containing : delimiters in their names are sub-scripts. They are invoked by the outermost delimited name (and possibly other scripts). For example, test:unit is executed by test.

Undocumented scripts are considered internal utilities and not expressly supported workflows.

💡 Tips:

  • Add -- to pass arguments to the script command. For example, npm run test:unit -- -u to update snapshots or npm run build -- -dw to automatically rebuild a development output.
  • Add -s to silence verbose command echoing. For example, npm -s i or npm -s run format.

NVM is recommended to configure the Node.js version used.

# Install the project's recommended Node.js version. This is a one-time installation command and
# does not need to be run again except when the project's .nvmrc is revised. `nvm use` will print an
# error message if this command needs to be run again.
nvm install "$(<.nvmrc)"

# Configure the current shell's environment to use the recommended Node.js version. This command
# should be run whenever opening a new shell to work on the project _prior_ to executing any of the
# project's NPM scripts, especially `npm install`.
nvm use

# Install the project's development and production dependencies. This is a one-time installation
# command and does not need to be run again except when the project's package.json `dependencies` or
# `devDependencies` are revised.
npm install

# All dependencies are now available. Execute any project scripts as wanted.

Storybook workflow

As the primary development flow WVUI uses Storybook which allows developing UI components in isolation without worrying about specific dependencies and requirements. Storybook uses so called stories. For each SFC (single file component) its story should be placed in the same directory:

|-- src
    |-- components
        |-- your-component
            |-- YourComponent.vue
            |-- YourComponent.stories.ts

Each story represents a single visual state of a component.

WVUI uses different Storybook addons, namely:

  • Controls that allow you to edit component props dynamically.
  • Actions to retrieve data from event handlers.
  • Docs to automatically generate documentation from component definitions.
  • a11y to analyze accessibility issues.
  • links which allows a developer to create links that navigate between different stories.
  • backgrounds to change background colors inside the preview
  • viewport to display UI components in different sizes and layouts
  • storysource to show story source in Storybook.

To start developing with Storybook, simply run npm start command (see NPM scripts). This command will open Storybook in your browser.

Vue.js

Vue.js Single File Components are used for all runtime components. The Vue.js template explorer is useful for debugging.

Reusable code

WVUI is using the Composition API plugin for Vue 2, in order to take advantage of this new feature before the library is migrated to Vue 3. Component-agnostic, reusable code can be stored as "composables," written in TypeScript, in src/composables.

See the plugin documentation for usage details.

Conventions

The Vue.js Style Guide is adhered to where possible.

  • PascalCase multi-word component names are used per the Vue.js Style Guide. Since every component is prefixed with Mw, all components are multi-word just by keeping that pattern. Examples:
    • ✓ Use MwFoo with a lowercase "w".
    • ✗ Do not use MWFoo with a capital "W". This breaks kebab-cased HTML in templates.
  • Avoid making primitive base components complex. Make new components instead.

Templates

Conventions

  • Static CSS class names should be included directly in the template while dynamic class names should come from a computed property that returns an object (not an array). This computed property should be named rootClasses for the outermost element.
  • If an element has both static and dynamic class names, the static classes should be listed first, then the dynamic classes should be included via v-bind on the next line.

TypeScript

TypeScript is used for all runtime sources. The TypeScript playground is useful for debugging.

Conventions

  • All top-level file symbols should be fully typed. Seams should not have their types inferred because they are most likely to have subtle flaws.
  • All named functions and methods should have inputs and output typed. When functions are fully typed, their contents usually can be inferred.
  • Favor type inference for locals rather than explicit typing. Locals are unlikely to have incorrect typing assumptions and the verbosity of typing is usually a hindrance.
  • Use TypeScript typing where available, JSDoc typing where not. Avoid typing both as this is verbose and the docs may be incorrect.

Imports

  • TypeScript supports import. For example, import Vue from 'vue';.
  • Destructuring is supported. For example, import { PropType } from 'vue';. Destructuring can be combined with default imports. For example, import Vue, { PropType } from 'vue';.
  • According to the TypeScript paths and Webpack alias configurations, @ references paths relative the source root (src) directory. For example, import WvuiButton from '../../src/components/button/Button.vue may be equivalent to import WvuiButton from '@/components/button/Button.vue.
  • Vue imports terminate in .vue. TypeScript imports are extensionless. A compilation error will occur otherwise.

Less & CSS

Less is used for all runtime styles. The Less playground is useful for debugging.

Conventions

  • BEM naming conventions are adhered to where possible.
  • Components are consistently rendered across browsers, orienting on normalize.css and documented with “Support [affected browsers]: Normalize by …”. We can't expect component normalization being available in all places using the library. This may lead to minimal rule duplication, depending on application, but that's the lesser evil.
  • All components use a box-sizing of border-box.
  • Each component should be entirely independent and usable in any context. Parents can specify the presentation of their children (for example, display: flex) but no component should expect to only exist in a given container.
  • WVUI uses stylelint-order to order CSS/Less properties for quicker orientation in all style rules.
  • Storybook-specific styles are prefixed with sb-.
  • Storybook-specific styles have their own Less files that end in .stories.less.

Imports

Several import options are available. The two most relevant are:

  • once: the default. If no option is specified, the once option is implied. Use with care as this bundles one full copy of the specified file into the bundle. References are always preferred. For example, @import "./foo.less";.
  • reference: When only symbols or mixins are necessary for Less to CSS compilation, use a reference import. Only the compiled output ships, not the definitions themselves or dead code. For example, @import ( reference ) "./foo.less";.

Import paths are resolved using less-loader:

  • Relative paths are used for project files. For example, @import ( reference ) './Foo.less';.
  • Prepend @/ for paths relative the source root (src) directory. For example, @import ( reference ) '@/themes/wikimedia-ui.less';.
  • Prepend a single ~ for NPM dependency files. For example, @import ( reference ) '~wikimedia-ui-base/wikimedia-ui-base.less';.

Testing

To run tests, use npm test command (see NPM scripts).

Unit tests

  • WVUI uses Vue Test Utils, the official unit testing utility library for Vue.js.
  • WVUI uses Jest as a test runner.
  • Tests for every component should be colocated with the component itself:
|-- src
    |-- components
        |-- your-component
            |-- YourComponent.vue
            |-- YourComponent.test.ts
  • WVUI uses snapshot testing, snapshot files are colocated with components as well:
|-- src
    |-- components
        |-- your-component
            |-- YourComponent.vue      <-- Functional code and test subject
            |-- YourComponent.test.ts  <-- Unit tests
            |-- YourComponent.snap.ts  <-- Jest snapshot rendered component HTML
  • WVUI uses jest-fetch-mock to mock API calls. Mocks can be disabled and run against live servers by setting the environment variable TEST_LIVE_REQUESTS=true.

Coverage

Coverage reports are generated automatically in the docs/coverage directory whenever unit tests are executed.

Coverage thresholds are configured under .jest/jest.config.json. These are lower limits for the entire repo and, as a convention, the number is rounded down to the nearest 10%. For example, if the actual repository coverage is 89%, the threshold is configured to 80%. See Jest documentation for details.

⚠️ ./src/entries/*.ts is excluded from the coverage report and expected to be side-effect free.

Git strategy

  • Authors should revise the changelog each commit so this work is not postponed to release.
  • Operating system and editor-specific files are not considered.
  • The Git configuration should be precise and accurate like any other part of the codebase. The .gitignore file, for instance, should not become cluttered or vague.

OS and editor-specific files {#files}

Different programmers use different editors and IDEs. WVUI will attempt to facilitate different workflows, especially in the form of documentation, but will avoid making changes specific to them such as ignoring Vim swap files.

OS-specific files such as .DS_Store and Thumbs.db should be excluded by the user's global Git configuration as they're unwanted in every repository and not specific to WVUI. See gitignore documentation for details.

Example:

  1. Add a global exclusions file by executing git config --global core.excludesfile '~/.gitignore' or updating your ~/.gitconfig manually:
excludesfile = ~/.gitignore
  1. Always ignore .DS_Store files by executing echo .DS_Store >> ~/.gitignore or updating your ~/.gitignore manually:
.DS_Store

Integrated development workflow

Example: I want to see my local WVUI library changes live in my app or MediaWiki skin.

Package linking is the primary integrated development workflow for use when isolated development is impractical. Tight coupling of WVUI to a specific implementation is strongly discouraged. Nevertheless, it is often the case that changes tested live in the context of a particular use case are wanted prior to publishing. For example, perhaps a bug only manifests easily in one target.

The steps are:

  1. Clone the WVUI repository if you haven't already.
  2. Enter the WVUI directory.
  3. Install the WVUI dependencies if you haven't already (see NPM scripts).
  4. Note WVUI's directory. For example, wvuiDir="$PWD".
  5. Enter your integration project's directory. For example, if you are integrating WVUI into Vector, the command might be cd ~/dev/mediawiki/skins/Vector. This location should contain a package.json with a @wikimedia/wvui dependency (either dependency, devDependency, or peerDependency).
  6. Symbolically link the development WVUI into the integration project via npm link "$wvuiDir" where $wvuiDir is the location of WVUI. This swaps the published production WVUI library for a link to your local development copy.
  7. Verify the link is correct by seeing where that it resolves to WVUI's location. For example, readlink -m node_modules/@wikimedia/wvui should match $wvuiDir.
  8. Watch for changes and produce development build file outputs by executing npm run build -- -dw.
  9. Perform all development and iteration wanted in WVUI and integration project.
  10. Unlink the development WVUI via npm unlink @wikimedia/wvui. This deletes the symlink to your development copy of WVUI.

The above process seems a little clumsy because it is initially. However, it's quite practical and becomes easy with practice.

Changing dependencies

  • Always configure your environment with NVM prior to un/installing dependencies (not necessary when using Docker) as these operations modify the NPM lockfile. See NPM scripts for example usage.
  • Obviously, carefully consider any proposed new dependencies. Runtime dependencies that increase the bandwidth consumption should be given especial care and implicit dependencies should be avoided.
  • When adding or revising NPM dependencies, pin dependencies and devDependencies to exact patch versions for the same reasons pinning WVUI itself to patch version is recommended. See Installation and version history for details.
  • Dependencies are not transpiled and must be ES5. Additionally, dependencies must only use supported browser APIs.

Linting and formatting

WVUI uses several linters and formatters. The former identify functional issues and the latter identify nonfunctional presentational inconsistencies such as incorrect indentation. Both support some measure of fixing or "formatting" problems automatically by executing npm run format.

  • Prettier: Markdown, JSON, and YAML files are formatted by Prettier. When it comes to generating beautiful and extremely consistently styled code, Prettier's ability to accept utter garbage code in and automatically apply formatting changes is exceptional, far superior to ESLint, and may even change the way you write code. For example, the indentation of braceless loops is never misleading once prettified. However, Prettier can never replace ESLint as it doesn't support any functional linting, only nonfunctional formatting. ESLint integration and additional languages such as TypeScript and JavaScript are supported but currently unused in WVUI. See .prettierrc.json and .prettierignore for configuration.
  • ESLint: ESLint is used for linting and formatting JavaScript, TypeScript, and Vue.js files. A hierarchy of overrides is used so that extends and rules can be separated. See .eslintrc.json and .eslintignore for details and configuration. An additional configuration is present in dist/.eslintrc.json for validating that only ES5 is shipped.
  • Stylelint: Stylelint is used for linting and formatting Less and Vue.js files. See .stylelintrc.json and .stylelintignore for configuration.

Editor and IDE support

Great workflows often require great tooling and those tools need to be configured. This section describes how to optimize your editor or IDE for optimal usage.

Visual Studio Code

  • Configure your line length to 100. For example, add common widths: "editor.rulers": [ 80, 100 ].

Recommended extensions

Known issues

  • Vue.extend() is used for the type inference of components. This is anticipated to be replaced by defineComponent() in the Vue v3 Composition API.
  • Storybook is incompatible with Vue Devtools. Tap "Open canvas in a new tab" as a workaround.
  • "Download the React DevTools…" is printed to the browser console when running Storybook.
  • If Storybook encounters an error when booting, it does not launch even after the error is resolved.
  • Code that is executed but never used (e.g. JavaScript configuration files or unused exports) is considered dead and is shaken out by Webpack on compile. As a result, dead code will not be type checked when building the library. All types can be tested manually via npx --no-install tsc --noEmit --incremental false.
  • The linter doesn't enforce tabs in TypeScript enumerations or module declarations.
  • Renaming test files may cause Jest to still try to open the old file name. In that case consider clearing the cache via npm -s run test:unit -- --clearCache.

Compatibility

Browserslist

WVUI uses Browserslist to help support and enforce browser compatibility. Supported targets are configured in .browserslistrc and extends browserslist-config-wikimedia/modern according to MediaWiki modern browsers compatibility. To see the current list, execute npx --no-install browserslist.

JavaScript

JavaScript build products are linted for ES5 compatibility.

Less

Less inputs are linted for compatibility and automatically prefixed for browser vendors according to the Browserslist config via the PostCSS plugin. The current configuration only adds vendor prefixes like -webkit-transition:all 1s; transition:all 1s, not polyfills. #rgba color syntax, like #0000 for transparent, are also replaced as needed by cssnano. The prefixes used can be seen by executing npx --no-install autoprefixer --info.

Versioning

Production release

It is highly recommended to perform all releases from the release Docker image. See the docker section if you have not already built the image.

You will also need to create a new ssh key pair specifcially for WVUI deploys. Expand for details...
  1. Execute ssh-keygen and save your keys to ~/.ssh/wvui-deploy. This creates a set of ssh keys specific for WVUI releases.
  2. Add your ssh public key at ~/.ssh/wvui-deploy.pub to your Gerrit account.

To publish a new release:

  1. Checkout the latest master branch: git checkout master && git pull.
  2. Update the changelog with release notes.
  3. Commit the changelog.
  4. Remove existing node modules and re-install through Docker
  5. Execute docker-compose run --rm release TYPE=<patch|minor|major> bin/release-prod.
  6. Perform a rolling development release.

Example commands:

# Checkout the latest master branch.
git checkout master && git pull

# Review the changes since the last release. For example,
# `git log "$(git describe --tags --abbrev=0)..@" --oneline`.

# Document a new feature and a couple bug fixes since the last release. (Emacs can also be used to
# edit the changelog.)
vim CHANGELOG.md

# Stage the changelog.
git add CHANGELOG.md

# Commit the changelog.
git commit -m '[docs][changelog] prepare release notes'

# Remove existing node modules and re-install
rm -rf node_modules
docker-compose run --rm release npm install

# Version, build, and test a release.
docker-compose run --rm release TYPE=patch bin/release-prod

The NPM scripts are configured to help ensure that only tested artifacts are published on Gerrit and npmjs.com.

By executing npm version, the following scripts are invoked in this order:

  1. preversion: test that the workspace contains no uncommitted changes.
  2. version: increment the version, clean, build, and test the candidate, commit, and tag the change.

In detail, version is a built-in NPM script that increases the package.json's version property (patch, minor, or major) as specified, commits the result to version control, and adds a Git tag. Prior to committing the version bump, clean, build, and test the candidate artifact. See npm help version for further details.

The preversion NPM script, which runs prior to version, is defined to test that Git's version control state is clean before that happens. No uncommitted changes are allowed! For example, imagine if a superfluous file containing a password was unintentionally in the workspace and published to npmjs.com.

By executing npm publish, the following scripts are invoked in this order:

  1. prepublishOnly: push the Git tag to the remote.
  2. publish: push the artifacts to npmjs.com as per usual.

Before publish is executed, prepublishOnly pushes the current commit and tag to the Git remote. If the push or publish fail due to connectivity, you should probably call npm publish directly which will re-push the tag and archive as needed.

Finally, the publish script is executed which releases the raw files built into the wild at the npm registry. See npm help publish for further details.

The intended result is:

  • Uncommitted changes (both modifications and untracked files) are forbidden.
  • Only clean and tested packages are published.
  • Git tags are available for prerelease and production releases.
  • Git tags pushed and NPM artifacts published are always in sync.
  • NPM's @latest tag points to the current stable release and @next points to the latest commit.

See also:

Pre-release (alpha, beta, or release candidate)

To publish a new alpha, beta, or release candidate, execute docker-compose run --rm release TYPE=<prerelease|prepatch|preminor|premajor> PRE_ID=<alpha|beta|rc> bin/release-pre. This will create a new version commit on the current branch.

prerelease is the safest choice. It always bumps the metadata number and only bumps the patch number if a stable version exists. For example, given the current version is a stable v1.2.3, TYPE=prerelease PRE_ID=alpha bin/release-pre will create v1.2.4-alpha.0. Note that both the patch is bumped and metadata is added. If executed again, note that only the metadata number is bumped and the patch number stays the same: v1.2.4-alpha.1.

prerelease can be slightly incorrect if the next release is known to be a minor or major release. In those cases, the correct initial alpha release would be TYPE=preminor PRE_ID=alpha bin/release-pre (or premajor) which would create v1.3.0-alpha.0. The subsequent alpha release would then be TYPE=prerelease PRE_ID=alpha bin/release-pre (note the command TYPE changes to prerelease) which creates v1.3.0-alpha.1.

Rolling development release

To publish the current master HEAD, execute docker-compose run --rm release ./bin/release-dev.

  • You may need to create a project-specific git config, e.g. your username and email address. If executing docker-compose run --rm release cat /app/.git/.config returns nothing, set config values from inside the root wvui repo directory on your host machine

Development releases can be installed by consumers via npm install @wikimedia/wvui@next. These releases are useful for integration testing and development as well as for early adopters who don't wish to build the WVUI library themselves.

Performance

Bundle composition and source maps

The contents of each bundle generated can be evaluated through its source map. source-map-explorer and Webpack Bundle Analyzer are used to generate reports for minified and minified + gzipped bundle breakdowns. The reports are similar but crosschecking may be useful.

Bundle size

WVUI uses Webpack for bundling different library entry points into distinct build products or "bundles". All JavaScript and CSS build product bandwidth performances are tracked and tested with bundlesize and versioned in bundlesize.config.json. Reports are generated under docs/minGzipBundleSize.txt.

The rule of thumb is: identical data generally compresses well. It is recommended to evaluate performance using the minified gzipped outputs. For example, some CSS selectors are distant but have identical rules. This creates a large uncompressed CSS bundle when compiled. However, the compressed size may be negligible. Use the bundlesize tests to evaluate gzipped sizes before making optimizations that impede readability.

Manual evaluation:

If a second opinion is wanted, consider using the gzip CLI:

# Individual chunk sizes (min / min+gz).
ls -1 dist/*.{js,css}|
sort|
while IFS= read filename; do
	printf \
		'%s: %sB / %sB\n' \
		"$filename" \
		"$(wc -c < "$filename"|numfmt --to=iec-i)" \
		"$(gzip -c "$filename"|wc -c|numfmt --to=iec-i)"
done

# All chunks concatenated (allows maximum possible compression). This makes sense if a request to
# ResourceLoader will depend on multiple chunks.
printf \
	'%s: %sB / %sB\n' \
	"Total" \
	"$(cat dist/*.{js,css}|wc -c|numfmt --to=iec-i)" \
	"$(cat dist/*.{js,css}|gzip -c|wc -c|numfmt --to=iec-i)"

bundlesize configuration

When changing the bundlesize configuration:

  • The values in the configuration are upper limits. As a convention, the number is rounded up to the nearest tenth of a kibibyte. For example, a new file added of size 4.15 KB would have its initial limit set at 4.2 KB. Whenever intentional changes causes its limit to increase or decrease beyond a tenth of a kibibyte boundary, the size should be revised.
  • bundlesize internally uses Bytes utility which only supports base-2 units. Case-insensitive decimal JEDEC notation is used in the config. This means 1.5 KB or 1.5 kb is 1536 bytes, not 1500 bytes.
  • ⚠️ Warning: values that cannot be parsed are silently ignored! When making changes, verify that a comparison of two values is printed like 2.54KB < maxSize 2.6KB (gzip). If only one number is shown (e.g., 2.54KB (gzip)), the number has been entered incorrectly.
  • ⚠️ Warning: values entered must have a leading units position specified. Sub-one sizes like .5 KB must be written with a leading zero like 0.5 KB or they will not be parsed.
  • The bundlesize thresholds specify minified gzipped maximums. Outputs are minified as part of the build process and gzip is the most common HTTP compression.