How to test Panel data apps

[MarcSkovMadsen]

Closes #4243

You can preview the how to guides here

• https://github.com/holoviz/panel/blob/feature/document-test/doc/how_to/test/pytest.md
• https://github.com/holoviz/panel/blob/feature/document-test/doc/how_to/test/uitests.md
• https://github.com/holoviz/panel/blob/feature/document-test/doc/how_to/test/loadtests.md

Issues experienced or identified during testing

• Help using playwright example SvenskaSpel/locust-plugins#101 (comment)

[MarcSkovMadsen]

Assets

Some assets used in the docs that should later be uploaded to a cdn.

app.mp4

app.mp4

app.gif

uitest.mp4

uitest.mp4

uitest.gif

panel-playwright-recording.mp4

panel-playwright-recording.mp4

panel-playwright-recording.gif

panel-locust.mp4

panel-locust.mp4

panel-locust.gif

loadwright.mp4

loadwright.mp4

loadwright.gif

[MarcSkovMadsen]

Hi @philippjfr

Please make a first draft review. I need some feedback on the overall outline and content. As a minimum I would appreciate feedback on:

• Is it ok to have all this content in one "how to" guide? If not how should it be split?
• Whether the code in this how to should somehow be automatically tested to avoid regressions later on?
• What should I do additionally to bring it to a state where is should be possible to get it approved?

[philippjfr]

My initial reaction is that this is indeed a lot of content for a single guide. I'd suggest splitting it into at least two guides:

• Testing Panel UIs with Playwright
• Load testing with Locust

[codecov]

Codecov Report

Merging #4244 (9d4a950) into main (805ddd5) will decrease coverage by 0.34%.
The diff coverage is 80.48%.

@@            Coverage Diff             @@
##             main    #4244      +/-   ##
==========================================
- Coverage   83.05%   82.71%   -0.35%     
==========================================
  Files         237      237              
  Lines       34252    34285      +33     
==========================================
- Hits        28449    28358      -91     
- Misses       5803     5927     +124     

Flag	Coverage Δ
ui-tests	37.98% <50.40%> (+0.03%)	⬆️
unitexamples-tests	73.87% <80.48%> (-0.35%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
panel/tests/test_docs.py	35.00% <0.00%> (ø)
panel/tests/ui/io/test_convert.py	95.68% <ø> (ø)
panel/tests/pane/test_echart.py	33.33% <33.33%> (-4.60%)	⬇️
panel/tests/template/test_vanilla_manual.py	15.21% <50.00%> (-71.74%)	⬇️
panel/tests/pane/test_alert.py	65.62% <66.66%> (-28.13%)	⬇️
...nel/tests/template/fast/test_fast_list_template.py	79.31% <66.66%> (-13.80%)	⬇️
panel/tests/widgets/test_text_to_speech.py	77.50% <66.66%> (-15.00%)	⬇️
panel/io/state.py	70.63% <70.83%> (+0.22%)	⬆️
...nel/tests/template/fast/test_fast_grid_template.py	52.72% <75.00%> (-43.57%)	⬇️
panel/tests/widgets/test_speech_to_text.py	62.88% <75.00%> (-27.84%)	⬇️
... and 25 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[maximlt]

Documenting how to test Panel apps is a great addition!

However, I would recommend splitting this into different how-tos. I believe most users should only be concerned with simple unit tests with Pytest. In its current form the how-to can lead users into an overly complicated path, specially beginners and non developers.

[maximlt]

Isn't this too alpha to be documented?

[MarcSkovMadsen]

😄

There is no other way currently. And I also need it for other the next how to guides I plan to write on "performance optimization" and "dask". Loadwright is the only thing that can provide insights into the improved performance.

Real Load Testing of Panel data apps has really, really been missing. This is why the Bokeh/ Panel server still has performance issues. We have not been able to get insights and provide reproducible examples. This makes it possible. I've specifically pinned the version here. And its not more alpha than any new example or code contributed to Panel.

[maximlt]

I'm saying alpha as this is the disclaimer on the project's README.

Please note this project is at a very early stage an the api and functionality will change!

I would personally be hesitating when advised to depend on a project that states this, as it feels like it's still being built and that I'll have to change a lot of my code in the future.

[MarcSkovMadsen]

Thanks for all the good comments. I've refactored. Please make a second review. Thanks.

[philippjfr]

Thanks for your fixes. I've made a number of edits. One thing that would be nice is if you could re-record some of the gifs with a larger font size. You can do that any time and I'll upload the new versions.

[MarcSkovMadsen]

@philippjfr . The link https://assets.holoviz.org/panel/gifs/loadwright.gif used in the docs does not exist. Could you please upload the file? Thanks.

[philippjfr]

Done

[MarcSkovMadsen]

I've increased the fontsize of the uitest.gif and loadwright.gif files above. Please take a look and upload them if they are ok @philippjfr . Thanks.

[MarcSkovMadsen]

I've tested the how to guides manually twice and fixed any errors identified. I don't plan to do more unless told what to do :-)