New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support headless browser and webview to login to a site #27
Comments
Ok, I have a headless browser implementation in other project, I will implement it in Kinoko. The headless browser api will look like this: let webView = new HiddenWebView({
// Replace the resource in the web site.
resourceReplacements: [{
test:'jwplayer\.js',
resource: this.loadString('my_jwplayer.js'),
mimeType: 'text/javascript',
}]
});
let cleanUp = () => {
this.webView = null;
};
webView.load(src);
// Receive message
webView.onmessage = (ev) => {
let event = ev.event;
let data = ev.data;
switch (event) {
case 'complete': {
let items = [];
for (let source of data.sources) {
items.push({
title: source.label,
url: source.file
});
}
resolve(items);
cleanUp();
break;
}
}
};
// Hold the reference otherwise the webView will be free before it callback.
this.webView = webView; |
The webview api is added in v4.1.0, I put the usage here. |
Thank you for your support, I will test the headless webview. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The v3 plugin system is great, I could add new site easily, but it is great if the core support more functions for crawling:
Some sites using encrypted script to generate content, or they are using cloudflare to prevent bot, if we could using a headless browser, we could pass the block.
Some sites protect their content behind membership (vip member), if we could open a webview for user login, after user loggedin to that site, we could share the cookie with script.
The text was updated successfully, but these errors were encountered: