Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue/webpack 5 upgrade #106

Merged
merged 40 commits into from
Oct 24, 2023
Merged

Conversation

cayb0rg
Copy link
Collaborator

@cayb0rg cayb0rg commented May 25, 2023

MWDK Upgrade

  • Upgrades all of the packages in the Materia Widget Development Kit to their latest versions and removes deprecated packages.
  • Server is now created in express.js and uses webpack-dev-middleware to compile and view assets.
  • Adds a loading screen for installing widgets to local Materia
  • Adds a scorescreen page. Custom scorescreens are supported if the author uploads sample score data from Materia.

Requirements:

  • Node v16 or later (including v18)
  • Yarn

Local setup

### Materia The current Materia widget-dependencies-package does not support the MWDK. UPDATE: the MWD package has been updated. **These steps are no longer needed**. 1. Checkout this [react/widget-dependencies-package-part-three branch](https://github.com/cayb0rg/Materia/tree/react/widget-dependencies-package-part-three) of Materia. This is required to load the player and creator. 2. Install Materia if you have not. 3. Run ``yarn build`` inside Materia's root folder. 4. Run ``yarn link`` inside Materia/public/dist. 5. Start Materia (run ``docker-compose up`` in Materia/docker and ``yarn dev`` in Materia/src)

Widget

Each widget may need a few small changes to support Webpack v5. Here is an incomprehensive list of the possible changes:

  1. New webpack.config.js entry rules:
  • Include only entries that have HTML files (except for markdown files), as these will be used by HtmlWebpackPlugin. Include the HTML file as the first element, then include any necessary JS and CSS in the array.
  • There should be no extension in the entry's name.
  • CSS and Markdown guides should not have their own entries

It should look similar to:

const entries = {
	'player': [
		path.join(srcPath, 'player.html'),
		path.join(srcPath, 'player.js'),
		path.join(srcPath, 'player.scss')
	],
	'creator': [
		path.join(srcPath, 'creator.html'),
		path.join(srcPath, 'creator.js'),
		path.join(srcPath, 'creator.scss'),
	],
	'scoreScreen': [
		path.join(srcPath, 'scoreScreen.html'),
		path.join(srcPath, 'scoreScreen.js'),
		path.join(srcPath, 'scoreScreen.scss'),
	]
}
  1. Either include CSS inside your JS files for MiniCssExtractPlugin to recognize them, or include them as elements in the entry arrays (see above).
  2. Use absolute urls for assets like loading fonts
image
  1. Remove the following rules from moduleRules, if applicable: rules.loadAndCompileMarkdown andrules.loadAndPrefixCSS
  2. Update the NPM scripts inside package.json to:
"start": "mwdk-start",
"build": "mwdk-build-prod",
"build-dev": "mwdk-build-dev"
  • mwdk-start: This starts the Express server and sets the webpack configuration to development mode.
  • mwdk-build-prod: This runs webpack in production mode. See the output in /build folder.
  • mwdk-build-dev: This runs webpack in development mode. See the output in /build folder.

Look at an example for the changes that need to be made (ignore the images).

AngularJS Widgets

AngularJS widgets may encounter an angular injection error. This usually only occurs with widgets that have directives, services, and controllers loaded in separate files. There are two ways to fix this. One way is to do what This or That does, and set up the directives, controllers, and services in a single file (for example, see creator.js. The other, more difficult way I've found is to manually bootstrap the angular modules.

  1. Remove ng-app from body or HTML tags.
  2. Move the main scripts such as player.js and creator.js to the end of the body tag
  3. Inside the files declaring the angular modules, remove the variable declaration when initially declaring the module. For example, the matching widget would look like:
    image
  4. At the very end of the last file in the respective webpack entry, manually start the module with the command: angular.bootstrap(document, ['your-module-name-here']). The example above would look like:
    image

Look at an example for the changes that need to be made.

Running this MWDK PR

  1. Checkout this PR.
  2. Run yarn install && yarn build && yarn linkinside the Materia Widget Development Kit. You may also use bun install.
  3. Run rm -rf node_modules inside your widget.
  4. Enter the git url for MWDK dependency: "materia-widget-development-kit": "https://github.com/cayb0rg/Materia-Widget-Dev-Kit.git#issue/webpack-5-upgrade"
  5. Run yarn install or bun install
  6. (Optional) Run yarn link materia-widget-development-kit. This will allow you to make changes to the MWDK without having to reinstall it in your widget.
  7. Run yarn start or bun start

Known Bugs / To-Do

  • Angular widgets do not work. (see above)
  • Not updating the title of a widget may cause an error when trying to preview it.
  • Question importer does not work. (fixed)
  • MWDK does not run independently of widgets. (fixed)
  • URLs for assets could be cleaned up and made clearer.
  • Media importer creates incorrect urls in creator (client issue)
  • Custom scorescreen support
    • Attempting to call each widget's score_module class functions may be more hassle than it's worth
    • Alternative solution: have the user upload a .json file of sample score data (pulled from the widget_instance_play_scores_get in Materia)
  • Issues resolving babel-loader for React widgets (client issue)
  • Getting a Compilation.assets deprecation error in console (fixed)
  • Screenshot annotation tool (fixed)
  • Trying to load localhost:8118 before webpack has finished building will not load the demo instance
  • Update all of the widgets' MWDK path
  • Icomoon fonts get a 404 once installed in Materia

Updated Widgets (PRs made)

Copy link
Member

@clpetersonucf clpetersonucf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really great work here so far. With limited issues I was able to get one of your widget PRs running locally and installed to dev Materia.

Some high-level comments:

  • As commented, MWD has been updated to hopefully (?) include the most recent/missing dependencies. It's worth confirming 0.2.0 works out of the box, and replace the reference to it in the package.json when possible.
  • I love the little progress bar when installing to local Materia. That's a nice touch.
  • I haven't tested the score screen feature, but the instructions could be a little clearer as to what's required. If the score screen options aren't ready for prime-time, we could also potentially disable them until they have more time in the oven.
  • Previously the MWDK would allow you to save draft/publish a qset to test in the player. Right now it looks like the demo is the only qset that works with the player. Is that the case? Are there significant challenges to re-implementing cached qsets?
  • Would it be possible to render Player and Creator guides, if they exist, within the MWDK? It looks like we already have view templates to support them.

Otherwise, just a little bit of polish based on feedback items, and making sure all current changes on this branch are final, required, and complete, and it looks like it might be good to go.

qsets/.gitkeep Outdated Show resolved Hide resolved
express.js Outdated
// 2. filter for materia-web image and named xxxx_phpfpm_1 name
// 3. pick the first line
// 4. pick the container name
let targetImage = execSync('docker ps -a --format "{{.Image}} {{.Names}}" | grep -e ".*materia:.* docker_app_.*" | head -n 1 | cut -d" " -f2');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line continues to give us problems. Some versions of docker-compose use hyphens, others use underscores, and the server does not handle a search failure well. We can modify the search pattern grep is using to: ".*materia:.* docker[-_]app[-_].*" which will catch either option.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also it'd be nice if the server gave a more well-handled response if the docker install process runs into an issue

.vscode/settings.json Outdated Show resolved Hide resolved
package.json Outdated Show resolved Hide resolved
webpack-widget.js Outdated Show resolved Hide resolved
@cayb0rg
Copy link
Collaborator Author

cayb0rg commented Oct 10, 2023

Thanks for the feedback!

New changes:

  • Updated materia-widget-dependencies package version to 0.2.0
  • Fixed link to guides.js in guide pages
  • Updated scorescreen instructions
  • The kit should support demo and one additional instance. I've added the ability to have multiple widget instances.
  • Made the widget installation process more verbose so that all errors are forwarded to the user as they are received

clpetersonucf
clpetersonucf previously approved these changes Oct 20, 2023
Copy link
Member

@clpetersonucf clpetersonucf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There might be some edge case issues we haven't encountered yet, but nothing show-stopping. By all accounts the new dev kit does the job required and works great.

Really excellent work. This was a significant undertaking and you knocked it out of the park.

Copy link
Contributor

@FrenjaminBanklin FrenjaminBanklin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super minor nitpick with some inline spacing, but nothing I'd hold up a release for.

I couldn't get the score screen working with valid JSON, but considering the latest version of the MWDK doesn't have a functional score screen either I'm not sure I'd hold up a release for that, either.

Everything looks like it's working well otherwise.

express.js Outdated
Comment on lines 1 to 20
const path = require('path');
const fs = require('fs');
const express = require('express')
const qsets = path.join(__dirname, 'qsets');
const yaml = require('yamljs');
const { execSync } = require('child_process');
const waitUntil = require('wait-until-promise').default
const { v4: uuidv4 } = require('uuid')
const sharp = require('sharp')
const util = require('util');
const cors = require('cors')
const hbs = require('hbs');

// common paths used here
const srcPath = path.join(process.cwd(), 'src') + path.sep
const outputPath = path.join(process.cwd(), 'build') + path.sep

// Webpack middleware setup
const webpack = require('webpack');
const webpackDevMiddleware = require('webpack-dev-middleware');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't really matter, but stick to spaces for inline alignment.

Would prefer to remove all the extraneous white space, personally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants