Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python web app #8873

Draft
wants to merge 26 commits into
base: master
Choose a base branch
from
Draft

Python web app #8873

wants to merge 26 commits into from

Conversation

purcell
Copy link
Member

@purcell purcell commented Jan 2, 2024

The changes here will hopefully replace the Javascript single-page application version of the MELPA and MELPA Stable websites with a server-side rendered equivalent, using a small web app written in Python, which reads its data from the JSON files produced during package builds instead of maintaining any sort of database.

This will fulfill #3728, and the implementation is an alternative and (for me) preferred approach to #8584 and #8585. It should also fix #6205.

Remaining work:

  • Misc app details, see TODOs in app.py
  • Dockerfile with correct dependencies
  • Incorporate app instance(s) into docker compose
  • Reverse proxy /, /package/* and /getting-started behind nginx
  • Document how to install/run/test locally
  • Live search, ie. search-as-you-type (non-essential)
  • Sentry or similar for crash reporting

@purcell purcell mentioned this pull request Jan 2, 2024
5 tasks
@@ -0,0 +1,8 @@
FROM debian:bookworm-slim
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using Python images instead of Debian. These get updated faster and contain less programs, so smaller size (especially Alpine-based) and less possibilities to exploit.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, yeah, this was copy-pasted for now from the other Python-related dockerfile, but that's a good steer. Personally I'd build ultra-minimal docker images with Nix, but probably won't do so here.


HTML_DIR="../../html"

BuildStatus = namedtuple("BuildStatus", ['started_at', 'completed_at', 'next_at', 'duration'])
Copy link
Contributor

@KeyWeeUsr KeyWeeUsr Jan 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using @dataclass if there's not a significant performance increase by having namedtuple(), it's more readable and allows specifying types and better static checking with mypy.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just aiming for the most lightweight and concise object-instead-of-a-dict wrapper tbh, but I might consider dataclasses.

docker/webapp/app.py Outdated Show resolved Hide resolved

package_data = None
def load_package_data():
global package_data
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using @lru_cache() instead of a global variable. That way the cache will be stored in the function's object instead of in the app.py's namespace and there would be no need for the conditions. Cleaning then is as simple as load_package_data.cache_clear().

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yay, thanks, this is a great tip. I've been mostly out of the Python ecosystem for a long time since the early days when I wrote unittest.py.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I try mentioning it whenever I see singleton-like approach in Python which isn't heavy for performance. Less testing and bugs. If it were a possible bottleneck, maybe try it with timeit beforehands, though I believe the difference with the recent Python might be optimized away anyway. 😊

docker/webapp/app.py Outdated Show resolved Hide resolved
@progfolio
Copy link
Contributor

progfolio commented Jan 15, 2024

Will this change effect the availability of https://melpa.org/archive.json?
Elpaca currently downloads this file to supplement package recipes with metadata.

@milkypostman
Copy link
Member

@purcell wanted to ping and see if you saw @progfolio 's question.

I haven't had time to dig through this and understand the end state.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Input field eats keystrokes on MELPA web page
4 participants