How to use post to submit data.

[chunfytseng]

I need to submit data using post.

[shalipoto]

Hi, I saw @jahenkor doing a POST with the crawler. You might ask him how he did that

[chunfytseng]

thx.

[shalipoto]

@chunfytseng , when you are happy with the answers, can you close this issue please?

[chunfytseng]

hi,so far, I still don't know how to get post data, so I need your help.

[jahenkor]

If you need to login to a page, you can submit your post request using AuthInfo (BasicAuth or FormAuth), and receive cookie(s) in which you can persistent across your crawl session. However, if you want to submit and receive post data across multiple pages, I've got no clue.

[jahenkor]

@chunfytseng

[chunfytseng]

@jahenkor
hi, jahenkor
I have already used AuthInfo to login, and after logging in, I need to submit data using post.

[chunfytseng]

@Chaiavi

[chunfytseng]

please!!!

[Chaiavi]

I don't understand your scenario, what are you trying to do ?

…

On Sun, Sep 16, 2018 at 6:27 AM chunfytseng ***@***.***> wrote: please!!! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#342 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABrbW4FkW55h35c9W2Pq96QoD7Aol4UOks5ubcUhgaJpZM4WZgK4> .

[chunfytseng]

I don't understand your scenario, what are you trying to do ?
…
On Sun, Sep 16, 2018 at 6:27 AM chunfytseng @.***> wrote: please!!! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#342 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/ABrbW4FkW55h35c9W2Pq96QoD7Aol4UOks5ubcUhgaJpZM4WZgK4 .

There are scenes as follows:

1. Login system
2. Get information
3. post submit data to obtain the results.
4. There are two results in step 3. If the successful task ends, if it fails, continue to step 3..
   The above is my usage scenario.
   Now, I have been logged in successfully, and how to use post to submit data.

[chunfytseng]

@Chaiavi

[Chaiavi]

We don't support failed logins. So if the login fails you won't get any data from the site as you are not logged into it. The authentication component in c4j is very basic and not sophisticated, it was kind of a later hack added to the crawler, so it is not so flexible. You can try to hack the auth system to do as you wish but as it is coded now I don't see a way to try logging into a site, parsing the response so according to it you will be able to login with a different user/pass combination. Chaiavi.

…

On Sun, Sep 16, 2018 at 6:57 PM chunfytseng ***@***.***> wrote: @Chaiavi <https://github.com/Chaiavi> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#342 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABrbW8zHdigqj5jZoi2NhQCokKblZBILks5ubnTtgaJpZM4WZgK4> .

[chunfytseng]

hi, Chaiavi.I have 2 questions.
1.I have solved the problem of logon failure. Now, there is a question of using post to get data.
2.If my crawler task is long-running and crawls data with multi-user logins (I deal with multi-user logins myself), would you recommend that I start multiple PageFetcher instances and give different configs? Or do you have any better advice?
@Chaiavi

[Chaiavi]

No better advice. Just be sure you don't mix sessions of different users. Login with one user, crawl, logoff Login with a different user etc... Chaiavi.

…

On Tue, Sep 18, 2018 at 4:54 PM chunfytseng ***@***.***> wrote: hi, Chaiavi.I have 2 questions. 1.I have solved the problem of logon failure. Now, there is a question of using post to get data. 2.If my crawler task is long-running and crawls data with multi-user logins (I deal with multi-user logins myself), would you recommend that I start multiple PageFetcher instances and give different configs? Or do you have any better advice? @Chaiavi <https://github.com/Chaiavi> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#342 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABrbW-LRdNYyQo9p9ML-SAXCaCrOWwioks5ucPswgaJpZM4WZgK4> .

[chunfytseng]

thx.