Omnibus of new features and fixes

[pvspain]

Apologies for the size!

I think the only outstanding TODO is to improve accessibility.
Changes in reverse chronological order:

Fix: Relocate keybinding to precede gridView construction
Fix: Revert cell id to '(x,y)'
Relabel API buttons in demo: Test -> Check
Add documentation and samples for .ipuz puzzle format.
Fix: setCellAnswer() - don't clobber existing
Add documentation of puzzle format .puz
Fix: Improve demo load time
Package: Add script to bootstrap project and tools from a freshly cloned git repository in a Linux environment.
Add test sample and doc for grid resizing.
Feature: Markdown support in clueText.
Refactor controller construction. Improve documentation.
Add error handling for CrosswordDefinition conversion. Refined crossword domain documentation
Fix: Change usage of event.keyCode to event.key Event.keyCode is deprecated! https://developer.mozilla.org/en-US/docs/Web/API/KeyboardEvent/keyCode
Doc: Miscellaneous small tweaks
Feature: Customisable keyboard shortcuts and actions.
Fix: conditionally set currentCell in event:focus
Doc: Improve samples section (crossword-domain.md)
Add default keybindings
Feature: (Re)set crossword file on live controller.
Package: Add project bootstrap script for Linux.
Simplify controller property names.
Refactor: Restructure source: separate files for gridview and cluesview
Refactor: Extract methods and helpers from Controller to reduce complexity.
Refactor: Extract functions to reduce complexity
Package: Add package script "update". Update nvm, npm, node LTS and packages to latest versions. Sort package scripts.
Fix: definition for clueModel.lengthText
Revise clue model regexes to allow whitespace. Extend testing. Extend doc and modify for changes.
Doc: Refine docs for clueModel and crosswordDefinition
Improve crossword validation
Feature: Dynamically load new puzzle
Feature: Expose controller.setCell() to programmatically set grid cell text
Doc: tweak description of TAB keystroke.
Refactor: Refactor CellMap to optimise performance modified: src/cell-map.mjs modified: src/crossword-controller.mjs
Refactor: Extract memoized functions
Refactor: Extract method #delayPublish Remove method context from assert() calls - given in stack dump.
Fix: Processing of (focus,click) event sequence
Fix: Add moveBackCell() - direction of movement now based on current clue.
Refactor: Renaming: anchor:non-anchor -> head:tail
Doc: Improve domain concepts documentation. Change code to match.
Fix: Handle absent clue.solution
Feature: Add 'incomplete' events for clue and crossword testing.

[pvspain]

Hi Misha @mmkal,

I have included your changes from PR38, but they have been relocated with refactoring of src/crossword-controller.mjs to reduce complexity. The CSS vars are now set inside newCrosswordGridView() in src/crossword-gridview.mjs.

[mmkal]

This looks great from eyeballing it! A small comment about yaml being in dependencies so far.

I can help test this, but I'll need the css variables change to go in first or be included in this branch - I'm relying on that for styling in my app.

[mmkal]

This is a big dependency, and potentially controversial - on an old team we found js-yaml more performant and smaller.

I think it should be left as an "exercise to the reader" to parse yaml into a json object. That way end users could install their preferred yaml library before passing a plain old JavaScript object to crosswords-js.

No objections to it being a dev dependency for the sake of the demo site.

[pvspain]

Hi Misha!

This is a big dependency, and potentially controversial - on an old team we found js-yaml more performant and smaller.

I've had a quick look at js-yaml - great tip - thanks!

I think it should be left as an "exercise to the reader" to parse yaml into a json object. That way end users could install their preferred yaml library before passing a plain old JavaScript object to crosswords-js.

No objections to it being a dev dependency for the sake of the demo site.

I understand your concern. Our points of differentiation to date are no dependencies, a small footprint, and universal usage (pure JS).

YAML makes manual document editing feasible. Looking at the JSON in ipuz, for example - there's little chance people would regularly construct a 15x15 puzzle by hand!

Simplicity is the third of @dwmkerr Dave's design goals. I think the puzzle format plays into this. I would like to support YAML for the sake of puzzle makers. Perhaps not in the core repo? I've spoken to @dwmkerr about managing framework samples as separate repos, perhaps another repo for useful utilities, like YAML imports?

Once we move beyond one repo, we should move to an organization account on GitHub, as is the pattern. GitHub handles link forwarding, so we wouldn't lose anyone in the transition.

[dwmkerr]

If the goal is that the puzzles could be edited by human users without technical knowledge, I think yaml will be more useful than JSON! I love the idea of a YAML editor on the left side of a screen with a puzzle on the right, for live interactive editing. In terms of which library to use, I'm fairly flex, if we use bundlers that do tree-shaking it may matter less - at this stage I'd say keep libraries to the minimum number needed, but don't worry too much about size, if overall size becomes a problem we can optimise later. For now I'd focus on dev experience and user experience!

[mmkal]

Here's how I see the best of all worlds being achieved:

• keep all dependencies including yaml out of crosswords-js - which could become the core crosswords renderer package. I am using it (and a big fan!) on a site I'm building now. I don't need yaml because the main use case is crossword playing, not crossword setting. The setters are uploading .puz files which I'm parsing using another library. Yaml would be baggage in this package in my opinion
• add yaml and maybe some other opinionated dependencies in a package called something like @crosswords-js/set which is designed to help people write new crosswords - this should be separate because it's a whole different set of concerns, I think
• if there are more spin-off packages for slightly different use cases, those could also go under the (for now non existent) @crosswords-js/ npm scope.

That may sound like scope creep for this project but I would say if that's the case then it'd be better to keep yaml out for now. When we switched to js-yaml on my old team we still had problems keeping it out of our client bundle.

The above idea could either be implemented with another repo like @pvspain suggested or by converting this into a multi-package monorepo - monorepo tooling these days is very good for npm packages. My recommendation would be to do that and use turbo.build. I'd be happy to help set this up if interested.

[pvspain]

This looks great from eyeballing it! A small comment about yaml being in dependencies so far.

I can help test this, but I'll need the css variables change to go in first or be included in this branch - I'm relying on that for styling in my app.

If you want to test these mods, I've written a gist on processing pull requests locally. This method enables you to evaluate changes without interrupting your ongoing work. I use the process regularly before submitting PRs to ensure they go in smoothly.

[dwmkerr]

This looks great - I'd say merge and fix forward for any issues, as this is such a large one. Want me to hold off until we've got more clarity on the YAML side of things?

[pvspain]

I vote we merge now and refactor later. Let's keep the velocity up. We don't want a backlog of PR's, as it makes merging later PRs more difficult.

To that end, how about a policy of only submitting PRs that are ready to merge? The PR must merge smoothly into dmwkerr/main, as @dwmkerr has requested previously. The onus is on the submitter to ensure that by test-merging offline. To assist that, I've written a gist to work through the PR locally, which I use for every PR, and also for merging changes back upstream. There are several checkpoints to test and inspect the changes.

Lets have another channel for discussing the roadmap and technology changes? We don't want wasted effort in PR's - nobody's getting paid here! @dwmkerr's original TODO is complete with this PR, apart from improving accessibility; tabbing out of the gridview, for example, is impossible with the default keybindings. I imagine there's prior art for that problem.
Do you want to set up another channel Dave? @dwmkerr

I'm happy with segregation into different modules, a multi-package mono-repo - just no more complicated than necessary right now. We can always refactor. Knock yourself out @mmkal! I have no issue with keeping this core repo dedicated to compiling crossword definitions (JS Objects) and rendering the different views - grid, clues. With the proviso that our hello world remains simple, as does adding YAML support. I think we're all agreed it's a better format for human editing.

On a side note, I've switched to js-yaml from yaml on an upstream branch and its painless. The example yaml puzzles all convert happily. I see the major task with extracting YAML is rewriting the docs, as there are a lot of links and references to YAML now. It will be easier if we have another repo set up before all that editing begins.

[mmkal]

That's ok with me - maybe as a simpler follow-up to this I could submit a pull request which switches js-yaml to be an optional peer dependency instead. I agree that end users should be encouraged to write yaml over json, I'm just nit-picking implementation details from the perspective of a developer-user rather than a webapp-user.

(And of course, I am just a user of this package giving feedback, so things shouldn't be held up on my account!)

[dwmkerr]

Love it - I'm going to merge

[dwmkerr]

This is amazing work - @pvspain and @mmkal should we start some kind of design doc we can work on offline on how we want to structure things going forwards? I like the idea of an org for this, and maybe modules like:

• Crossword format loader (responsible for parsing, error-checking and validating json/yaml xword definitions) - able to turn them into the in-memory model. Allows anyone to use 'our' crossword format
• Crossword builder: takes a loaded crossword model and creates a usable in-memory structure that can power interaction, either server side or client side (e.g. holds the clues, the letters entered, the state of answers, etc)
• 3x samples - a 'plain old javascript sample' a 'react native sample' then maybe a vue sample or even a server side command line one
• then a geeky one for me - I'd like a terminal based crossword tool! so I can load a crossword file and play in in the terminal :)

[dwmkerr]

We could set a timeline / set or priorities, modularise a bit and then divide and conquer based on interests, what do you think? Could then have code owners per module, e.g. we all own the format/foundational stuff, but divide and conquer on samples?

[dwmkerr]

It also democratises the ownership a bit and lets you both move quickly, with my workload I don't want to be a blocker, but would love to give input (and maybe own the dorky commandline version)