Roadmap beyond 4.0.0: platform refactor, admin UX, translations and theming

[ismisepaul]

With 4.0.0 getting close, we've started thinking about what comes after it. This is a draft plan for the 4.x releases and 5.0, posted to collect feedback before we turn any of it into issues. Nothing here is part of 4.0.0.

There are four areas we want to improve:

1. Translations are hard to maintain, incomplete, and it isn't clear how to contribute one.
2. The backend is tangled: the database layer, request handling and HTML output are mixed together, with a lot of repeated code.
3. The front end and back end are tightly coupled.
4. The UX, especially the admin side. Setting up a class should be quick and easy, the menus should make sense, and the look should be easy to change, starting with dark and light mode.

Where things stand

Translations

• There are 7 languages, but coverage against the 1,283 English keys is low: Chinese 33%, Russian 19%, Irish 13%, Hindi and Portuguese 7%, French 6%, Spanish 4%.
• Translated text is looked up in two different ways: JSTL tags in the main pages, and direct ResourceBundle calls in the modules.
• The 111 bundle files are mostly named by module hash, non-Latin text is stored as \u escapes, and there's no guide for translators.
• Most admin pages are hardcoded English, and the servlets write English HTML.
• Module translations don't display at all right now because of Validate.validateLanguage rejects every two-letter language code, so module i18n always falls back to en_GB #937, which is in 4.0.0. Merging the translation files (Merge i18n Files together where possible #517) has been open since 2019.

Backend

• The platform code, excluding training content, is about 16k lines. Getter.java is 2,643 lines and Setter.java is 1,507.
• There are 59 platform servlets, and almost all of them repeat the same session, CSRF and logging code by hand.
• 55 of them build HTML strings, and the database layer builds HTML menus itself.
• Splitting Getter/Setter into DAOs is already filed as Refactor Getter.java and Setter.java into DAO pattern #815 and Convert ResultSet-returning Getter methods to bounded collections (DAO refactor follow-up) #839.

Front end

• index.jsp is 1,225 lines of mixed JSP and JavaScript, and the browser posts to servlets that return HTML fragments.
• Only 4 endpoints return JSON. The jQuery upgrade (Upgrade vendored jQuery 1.7.1 / jQuery UI 1.8 + 1.11 and other front-end libraries #957) is in 4.0.0.

Admin UX and theming

• The admin menu is 22 items in a flat list.
• Players are added one form at a time, with no bulk import. Setting up a class means creating it, adding each student, assigning them, setting the default class, then opening modules, all on separate pages.
• Colours are hardcoded in the stylesheets, with no CSS variables, and the JSPs contain over 300 inline styles.
• Class-specific admin (Class Specific Admin Only Menu with Select Class #224) and reskinning (Admin to Customise Shepherd CSS via UI #138) have both been open since 2015.

Proposed order

Rank	Area	Why this position
1	Backend restructure (platform only)	The other areas depend on it, and many current bugs are in it
2	Admin and class UX	The biggest user-visible gain, for instructors running Shepherd
3	Translation workflow	The workflow fixes are cheap and independent; filling the gaps is community work over time
4	Theming and dark mode	CSS variables and dark mode are a quick win; non-technical reskinning comes later
5	Front/back decoupling	Falls out of 1 and 2 rather than being a project of its own

Ground rules

• Training content stays as it is. Lessons and challenges are deliberately vulnerable. They stay server-rendered inside frames, and the decoupling applies to the dashboard, admin pages and login only. Before any change touches a module's files, we add a test that the module can still be solved.
• Migrate one area at a time instead of rewriting. For each area: DAO, then service, then JSON endpoint, then new page, then delete the old code.
• Keep the front end build-free. Every Maven dependency is PGP-verified, and an npm toolchain would bring in hundreds of packages with no equivalent check. Native ES modules plus a few vendored libraries pinned by hash should be enough.
• Build each pattern once, then open the rest up. Once one example of each migration exists, the remaining pages become good first issues.

Release plan

4.1: Foundations and quick wins

• Backend: one shared filter or base servlet for session, CSRF, logging and error handling. Start the DAO split (Refactor Getter.java and Setter.java into DAO pattern #815) with users, classes and settings.
• Translations:
  • Store properties as plain UTF-8, so translators type native text instead of \u escapes.
  • Merge the files (Merge i18n Files together where possible #517) and add an index from module hash to module name.
  • Set up a hosted translation platform.
  • Add a CI report of missing keys per language, and show each language's coverage in the picker.
  • Write a "Translating Shepherd" guide.
• Theming: turn the colours into CSS variables, add dark and light mode (following the OS setting, with a toggle), and remove inline styles from the main pages.

4.2: Classroom

• JSON endpoints for users and classes.
• Admin menu regrouped by task: Classes, Players, Modules, Event, System.
• A class setup wizard:
  1. Name the class.
  2. Add players by paste or CSV, by generating accounts with a printable sheet, or with a join code.
  3. Pick modules.
  4. Choose the event settings.
  5. Land on a class dashboard.
• Bulk actions on player lists replace the single-purpose user pages. This also covers Class Specific Admin Only Menu with Select Class #224, [Enhancement] See user email in Admin -> User Management -> Reset Password + Upgrade Player to Admin + Assign Players to Class + Add / Deduct Player Points #683 and [Enhancement] Order level by level name in Admin -> Module Management -> Open / Close Modules (currently ordered by moduleID) - [ ] in fact make this UI better - [ ] drag and drop maybe? #681.
• Admin pages get translated as they're rebuilt.

4.3: Player dashboard on the API

• Menu, scoreboard and result submission move to JSON.
• The database layer stops building HTML, and the Java code leaves index.jsp.
• Modules stay in frames.

5.0: Remove the old code, reskin without code

• Remove the old endpoints that return HTML. That breaks anyone scripting against them, hence the major version.
• Admin theme settings: colour pickers and logo upload, stored as validated colour values. Never raw CSS (Admin to Customise Shepherd CSS via UI #138), because admin-supplied CSS is an injection risk.
• The first step towards reskinning already lands in 4.1: a single documented theme file that someone can edit without knowing the code.

Open questions

We'd especially like input on these:

1. Front end: build-free ES modules, as proposed above, or a framework? If a framework, how would we keep the same supply-chain guarantees?
2. Translation platform: Weblate or Crowdin? Both are free for open source projects. Translators, what have you used and liked?
3. Old admin URLs: may they disappear in 4.x minor releases, or should they stay until 5.0?
4. Instructors: what slows you down most when setting up a class or event today?

Comments on the order, the scope, or anything missing are welcome. Once the plan settles, we'll create milestones and issues for each release.

[ismisepaul]

Here are the notes from our meeting in Dublin on 15 October 2026, matched against the current issues and PRs. Some of them overlap with this roadmap, and some are areas it doesn't cover yet.

✅ done · 🔄 tracked or in progress · 🆕 not tracked yet · 🤔 needs a decision

General

• ✅ Meme Level - Apples + Bananas: done around 2019, when the troll faces were replaced because they were dated.
• 🔄 Admin password reset: #578, #325. Related bugs: #571 (PR #918), #973 (PR #981).
• 🔄 profile / Badges: badges are covered by #679, #680 and #335. 🆕 There's no issue for a player profile page.
• 🤔 UI - Server/Frontend - React: #914 (PR #920), #457. This roadmap proposes build-free ES modules rather than React, so we need to decide.
• 🔄 Translations: #517, #937 (PR #1014), #452, #243, #178, #166, plus the translation section of this roadmap.
• ✅ RCE - log4j: the log4j CVE is fixed (PR #963).
• 🆕 Reach out to owasp for docker image management: publishing to Docker Hub works (#1025, #1054, and PR #1060 for #865), but the conversation with OWASP isn't tracked.
• 🔄 XXE level: #548 (XXE Challenge 2), #654 (PR #1000).
• ✅ Worktrees: documented in AGENTS.md.
• ✅ Jakarta Enterprise SSO: #861 is merged, and SSO now runs on pac4j-saml (PR #939). Follow-ups: #948 (PR #983), #952 (PR #984), #820 (PR #985), #879.
• 🤔 paul will Renew Apis: turn the backend into APIs that the frontend consumes. This roadmap covers it (the DAO split in #815, then JSON endpoints in 4.2 and 4.3), but the approach isn't settled.
• ✅ Sean to test SSO: tested, and merged with the Jakarta switch.
• 🆕 Skill files in different folders: put agent skills in folders across the project, so an agent working in a directory knows what to do there.

①

• 🤔 Migrate Apis: the same work as "Renew Apis" above.
• 🔄 Dependencies: #955 is done. #957, #958, #960 and #1022 are open.
• 🔄 Test mobile: #896, #922–#930.
• ✅ RCE - Latest dependencies: the log4j CVE is fixed (PR #963) and #955 is closed.
• 🔄 Documentation: #1020 (PR #1021), #989.

②

• 🤔 PostgreSQL: probably not worth doing. We may stay on MariaDB.
• 🔄 UI/Server separation: #914, plus this roadmap.
• 🔄 Custom themes: #138, plus this roadmap.
• 🆕 LLM levels

③

• 🆕 Shepherd AI chatbot
• 🆕 Shepherd cert
• 🔄 Gamification: the #653 epic.
• 🔄 Achievements + Badges: #679, #680.
• 🆕 class chat
• 🆕 Slack Announcements
• 🔄 Containerization + cloud deployment: #389, #332, #814, #1046. No cloud deployment work has started yet.

Not in this roadmap yet

• New levels: LLM levels, XXE 2
• Gamification, achievements and a player profile
• AI chatbot, cert, class chat, Slack announcements
• Cloud deployment
• Mobile

Older backlog items worth adding to the plan

• #824: brute-force lockout. The stored procedure exists but is never used.
• #576: tag levels with ASVS and CWE. This fits with the cert idea.
• #372 / #208: end-to-end browser tests, before the UI rework.
• #501: player teams.

[SeanDuggan]

Mobile: the unified Android app is targeted for 4.0.0

The app is being merged as a series of smaller PRs so each part is reviewable. The order is module IDs and cheatsheets with a database upgrade path, then the server API, then the app shell, then the per-category modules.

The server side is deliberately thin. The Mobile* servlets are JSON endpoints for login, registration, module list, module start and flag submit. Scoring goes through the normal result path. Per-user flags use the existing HMAC scheme, so mobile players appear on the same scoreboard as web players.

Client-side injection has been verified end-to-end on an emulator, including the scoreboard update. The other categories still need the same pass.

Mobile categories

Category
Client-Side Injection
Insecure Data Storage
Poor Authentication
Insecure Authorization
Insufficient Cryptography
Input Validation
Insecure Communication
Security Misconfiguration
Privacy Controls
Supply Chain
Reverse Engineering

What 4.0.0 needs from it

1. Merge order: all the split PRs landing before the release cut. The first two are the foundation, and the app shell and category modules depend on them.
2. Solvability gate: every shipped mobile module is solved end-to-end on an emulator before release. This is the mobile version of the "a test must confirm the module is still solvable" rule. Anything that fails is held back, not shipped broken.
3. Existing installs: 4.0.0 is a major upgrade, so the database path matters. The module consolidation comes with a re-runnable "Update mobile modules" setup option, and it also fixes a broken classpath lookup that made the existing v3.0 "upgrade" option fail. An existing install that deploys the new WAR without running it hits a MissingResourceException when listing challenges, so the upgrade steps need to be in the release notes. The upgrade also removes three consolidated modules and the progress recorded against them, and that should be called out too.
4. Release mechanics: decide how the APK is built and published alongside the WAR and Docker images, and whether it is signed and verified like the images.
5. Docs: a short "Mobile app" section covering installing the APK, pointing it at a server and which categories are included. The running-a-CTF guide should mention that mobile players share the scoreboard.
6. Support policy: Android only, and the minimum API level.

How it fits the later releases

• 4.1 base servlet/filter: the Mobile* servlets repeat session checks. I have pulled that into a small MobileSession.isAuthenticated helper, which can be folded into the shared filter once it exists. Until then, leaving the mobile servlets separate is fine.
• 4.2/4.3 JSON endpoints: the mobile API is already a small JSON API over the same DB layer, so it can be a reference for the dashboard endpoints, and the app can reuse them as they land.
• Translations: the mobile modules now have "Mobile ..." names, and 7 dead module-name keys were removed across the Russian, Chinese and Irish bundles. They will benefit from the module-hash-to-name index planned for 4.1.
• Level tagging: the categories follow the OWASP Mobile Top 10, so they map naturally to MASVS as well as CWE, if levels get ASVS/CWE tags.
• End-to-end tests: the emulator flow is a candidate for the end-to-end suite planned before the UI rework.
• Theming and dark mode: the app has its own theme, so it should follow the same colour tokens once they exist.

Questions for Mobile

• Is the plan to cut 4.0.0 once all mobile PRs are merged, or to ship it with whichever categories pass the solvability gate at that point?
• Should the APK ship with 4.0.0 or follow in a 4.0.x release if the app is ready later than the server?
• Should mobile modules get their own "Mobile" group in the class wizard's module picker later?