chore: test Academy exercises #2097

honzajavorek · 2025-11-18T13:51:02Z

This is a proof of concept how we could test exercises in the academy:

Exercises target real world websites, so they can easily break without us knowing
We could test them e.g. weekly, so it's not too noisy
The solution should be as simple as possible, and capable running both JavaScript and Python
While working on the PoC I actually discovered one exercise which is broken 95% of time due to aggressive anti-scraping protections, so I changed it
The test can be executed with bats -r --print-output-on-failure .
Bats is a simple testing framework based on Bash, which allows to run arbitrary programs and evaluate their output - https://github.com/bats-core/bats-core Not sure yet how to get it inside the CI, but on macOS it's just brew install bats

Todo:

Note

Adds a monthly GitHub Action and Bats-based suite to run Academy JS/Python exercise solutions, embeds solutions into lessons, and updates docs/ignore files.

CI:
- Add monthly and manual workflow .github/workflows/test-academy.yml to run Academy exercises via Node (npm) and Python (uv).
Testing:
- Introduce Bats-based tests for Academy exercises: sources/academy/**/exercises/test.bats (JS & Python) executing solutions and asserting outputs.
- New npm script test:academy and dev dependency bats.
Docs/Academy content:
- Embed executable exercise solutions using CodeBlock + !!raw-loader across JS/Python lessons (04–12), replacing inline snippets.
- Update several exercises (e.g., switch example from AliExpress to LEGO) and add multiple new solution files (Cheerio/BeautifulSoup/Crawlee).
Repo:
- Update .gitignore to exclude exercise artifacts (storage, node_modules, package*.json, dataset.json).
- Document testing process in CONTRIBUTING.md (broken links check + Academy exercises CI).

^{Written by Cursor Bugbot for commit 118a550. Configure here.}

apify-service-account · 2025-11-21T16:38:21Z

Preview for this PR was built for commit 05099c1 and is ready at https://pr-2097.preview.docs.apify.com!

tomnosek · 2025-11-24T10:28:50Z

From my perspective, we should design them in a way that they run automatically and do not require manual start-up. Otherwise, we'll forget about it eventually.

honzajavorek · 2025-11-24T12:20:06Z

Sure, I'd make a GitHub Action, which runs like once a week or once a month (depends on our ability to fix the exercises, doesn't make sense to run them too often).

honzajavorek · 2025-11-24T12:20:47Z

(Creating such GitHub Action is a matter of a few lines and I'll add it to this PR)

apify-service-account · 2025-11-24T13:25:15Z

Preview for this PR was built for commit cf913593 and is ready at https://pr-2097.preview.docs.apify.com!

apify-service-account · 2025-11-24T13:43:39Z

Preview for this PR was built for commit cb4959b2 and is ready at https://pr-2097.preview.docs.apify.com!

apify-service-account · 2025-11-24T13:47:22Z

Preview for this PR was built for commit 5e78a4db and is ready at https://pr-2097.preview.docs.apify.com!

apify-service-account · 2025-11-24T13:51:35Z

Preview for this PR was built for commit 463457b9 and is ready at https://pr-2097.preview.docs.apify.com!

apify-service-account · 2025-11-24T13:57:30Z

Preview for this PR was built for commit 0214184a and is ready at https://pr-2097.preview.docs.apify.com!

apify-service-account · 2025-11-24T17:21:28Z

Preview for this PR was built for commit b930d1f and is ready at https://pr-2097.preview.docs.apify.com!

apify-service-account · 2025-11-25T09:31:41Z

Preview for this PR was built for commit 4fccbf45 and is ready at https://pr-2097.preview.docs.apify.com!

apify-service-account · 2025-11-25T12:02:32Z

Preview for this PR was built for commit 96ae391 and is ready at https://pr-2097.preview.docs.apify.com!

apify-service-account · 2025-11-25T12:06:49Z

Preview for this PR was built for commit 44d7b1a and is ready at https://pr-2097.preview.docs.apify.com!

apify-service-account · 2025-11-25T13:58:04Z

Preview for this PR was built for commit ddf09129 and is ready at https://pr-2097.preview.docs.apify.com!

apify-service-account · 2025-11-25T14:43:03Z

Preview for this PR was built for commit 6c5f05bb and is ready at https://pr-2097.preview.docs.apify.com!

…atory comment

…ity of uv options

#2112 )

apify-service-account · 2025-11-25T14:49:50Z

Preview for this PR was built for commit 118a550 and is ready at https://pr-2097.preview.docs.apify.com!

honzajavorek · 2025-11-25T14:49:51Z

I think this should be ready now. The Bats tests currently do fail, because there are failing exercises, so that's expected. But the testing infrastructure is solid and this PR is about the infrastructure. I've set the frequency of the tests to monthly and let's see. I didn't want to snowball this PR, so I recorded the failures and other issues separately and I'll work on them in subsequent PRs:

honzajavorek · 2025-11-25T14:51:31Z

CONTRIBUTING.md

    - Run `vale sync` to download styles
    - Configure exceptions in `accepts.txt`

+### Testing


@TC-MO Can you take a look at this README change, please? Does it make sense this way?

honzajavorek · 2025-11-25T14:52:08Z

package.json

    "dependencies": {
-        "@apify/ui-library": "^1.97.2",
        "@apify/ui-icons": "^1.19.0",
+        "@apify/ui-library": "^1.97.2",


Not my change, npm re-ordered this on its own 👀

honzajavorek · 2025-11-25T14:54:56Z

sources/academy/webscraping/scraping_basics_javascript/04_downloading_html.md

 <Exercises />

-### Scrape AliExpress
+### Scrape LEGO


This is the only I fixed. After this one I decided fixing the exercises should be in separate PRs, not in this one: #2113

cursor

This PR is being reviewed by Cursor Bugbot

Details

Your team is on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle for each member of your team.

To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

Comment @cursor review or bugbot run to trigger another review on this PR

sources/academy/webscraping/scraping_basics_python/exercises/process_products_json.py

sources/academy/webscraping/scraping_basics_javascript/exercises/crawlee_netflix_ratings.mjs

sources/academy/webscraping/scraping_basics_python/exercises/test.bats

sources/academy/webscraping/scraping_basics_javascript/exercises/test.bats

apify-service-account · 2025-11-25T15:04:12Z

Preview for this PR was built for commit 8c875d1 and is ready at https://pr-2097.preview.docs.apify.com!

honzajavorek added the t-academy Issues related to Web Scraping and Apify academies. label Nov 18, 2025

honzajavorek changed the title ~~Test Academy exercises~~ chore: test Academy exercises Nov 18, 2025

honzajavorek force-pushed the honzajavorek/test-exercises branch from ebf76a7 to 05099c1 Compare November 21, 2025 16:35

honzajavorek force-pushed the honzajavorek/test-exercises branch from 0214184 to b930d1f Compare November 24, 2025 17:18

honzajavorek force-pushed the honzajavorek/test-exercises branch from 4fccbf4 to 96ae391 Compare November 25, 2025 11:59

honzajavorek mentioned this pull request Nov 25, 2025

Fix broken exercises #2113

Open

4 tasks

honzajavorek added 9 commits November 25, 2025 15:45

wip

fb9e3bc

feat: keep exercises as separate files, include them to Markdown

819462b

chore: implement testing of JavaScript exercises

10a0743

refactor: use shorter names

6d50689

chore: add GitHub Action to run tests automatically

401cc92

chore: ouch, wrong branch

e296be1

chore: one does not simply npm install

e673e9f

style: make linter happy

846602c

chore: make sure there is no schedule until we merge this, add explan…

a674604

…atory comment

honzajavorek added 16 commits November 25, 2025 15:45

refactor: simplify the tests

a72090d

docs: document lychee and academy testing

db19ee5

refactor: make exercises testable

fc05b0d

fix: avoid the yes option, fix crawlee installation, improve readabil…

6936de9

…ity of uv options

chore: make the tests more meaningful

cd5deaa

chore: improve the JS test suite

857d3a3

style: make the code linter happy

9901553

style: condense and fix the solutions markup

c056906

chore: setup and teardown for Python

ddb9a1b

chore: fix the JS test suite not to rely on npx --package

87b813b

fix: improve the Python test suite and fix solutions using Crawlee (see

cdc8023

#2112 )

style: fix markup

d006397

chore: fix typo

dfe9b28

chore: enable only as a cron

b324b20

chore: run monthly

d4e0643

style: fix markup

118a550

honzajavorek force-pushed the honzajavorek/test-exercises branch from 6c5f05b to 118a550 Compare November 25, 2025 14:46

honzajavorek marked this pull request as ready for review November 25, 2025 14:50

honzajavorek requested a review from B4nan November 25, 2025 14:50

honzajavorek commented Nov 25, 2025

View reviewed changes

cursor bot reviewed Nov 25, 2025

View reviewed changes

fix: address bugbot comments

8c875d1

chore: test Academy exercises #2097

Are you sure you want to change the base?

chore: test Academy exercises #2097

Uh oh!

Conversation

honzajavorek commented Nov 18, 2025 • edited by cursor bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

apify-service-account commented Nov 21, 2025

Uh oh!

tomnosek commented Nov 24, 2025

Uh oh!

honzajavorek commented Nov 24, 2025

Uh oh!

honzajavorek commented Nov 24, 2025

Uh oh!

apify-service-account commented Nov 24, 2025

Uh oh!

apify-service-account commented Nov 24, 2025

Uh oh!

apify-service-account commented Nov 24, 2025

Uh oh!

apify-service-account commented Nov 24, 2025

Uh oh!

apify-service-account commented Nov 24, 2025

Uh oh!

apify-service-account commented Nov 24, 2025

Uh oh!

apify-service-account commented Nov 25, 2025

Uh oh!

apify-service-account commented Nov 25, 2025

Uh oh!

apify-service-account commented Nov 25, 2025

Uh oh!

apify-service-account commented Nov 25, 2025

Uh oh!

apify-service-account commented Nov 25, 2025

Uh oh!

apify-service-account commented Nov 25, 2025

Uh oh!

honzajavorek commented Nov 25, 2025

Uh oh!

honzajavorek Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

honzajavorek Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

honzajavorek Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

This PR is being reviewed by Cursor Bugbot

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

apify-service-account commented Nov 25, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

honzajavorek commented Nov 18, 2025 •

edited by cursor bot

Loading

honzajavorek Nov 25, 2025 •

edited

Loading