Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Try to unify grabbers in a more generic way #26

Open
elboletaire opened this issue Jan 5, 2024 · 0 comments
Open

Try to unify grabbers in a more generic way #26

elboletaire opened this issue Jan 5, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@elboletaire
Copy link
Owner

elboletaire commented Jan 5, 2024

After adding support to tcbscans.com I realized grabbers could be created in a more generic way, rather than having grabbers almost for every single site.

The new PlainHTML grabber could be a good starting point. On the other side, the current TCB grabber could be renamed to Ajax, since all its behavior works around ajax calls.

This new PlainHTML is actually a copy of manganelo (removing ajax functionality recovered), so most of the sites from the manganelo grabber could be moved to the PlainHTML one.

I think it'll be better to add a new layer in the grabber architecture that handles the different selectors, and leave the grabbers as more generic interfaces. Doing this, in the future adding new sites should be as easy as just setting the proper grabber and define the selectors and other required identifiers for that grabber.

@elboletaire elboletaire added the enhancement New feature or request label Jan 5, 2024
elboletaire added a commit that referenced this issue Jan 5, 2024
This removes manganelo's specific grabber, adding support to more sites
via plainhtml.

- added support for mangamonks
- added support for mangapanda

refs #26
elboletaire added a commit that referenced this issue Jan 5, 2024
This removes manganelo's specific grabber, adding support to more sites
via plainhtml.

- added support for mangamonks
- added support for mangapanda
- fix support for ajax pagination in sites like mangajar

refs #26
elboletaire added a commit that referenced this issue Jan 5, 2024
This removes manganelo's specific grabber, adding support to more sites
via plainhtml.

- added support for mangamonks
- added support for mangapanda
- fix support for ajax pagination in sites like mangajar

refs #26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant