Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Helper to easily obtain a response and its body #370

Open
mvdan opened this issue May 31, 2019 · 6 comments
Open

Helper to easily obtain a response and its body #370

mvdan opened this issue May 31, 2019 · 6 comments

Comments

@mvdan
Copy link
Contributor

mvdan commented May 31, 2019

We added ExampleRetrieveHTML, which did this with network.GetResponseBody and a dozen lines of Go code, but apparently it's not that simple. We also need to wait for network.EventLoadingFinished on top of the response's network.EventResponseReceived.

At this rate, the user would probably need over twenty lines of non-trivial Go code. This probably calls for a simple helper to deal with the simple use cases.

@marcelloh
Copy link

Perhaps, I'mn a nitwit for these solutions, but how can the document title be retrieved in the easy way that you have now, and a full source only with a dozen lines of code? How come that fill html source needs a EventResponseReceived kind of solution, while Title is just Title (I never looked what it does under the hood). I can imagine, the source needs to be in memory to have access to the title. But as I said: I'm the nitwit here ;-)

@marcelloh
Copy link

Btw. Now I did some extra digging. Title is evaluated like a javascript command. so I was looking if there is such a thing for all of the html. I found document.all to contain that information.
When I do document.all[0] or document.all[0].innerHTML I get more info in the console. Perhaps this is usable?

@mvdan
Copy link
Contributor Author

mvdan commented Jun 3, 2019

We already have InnerHTML and OuterHTML as actions you can use. This is different from obtaining the original HTML response sent by the HTTP server, though. That requires waiting for a couple of events and executing an action, so it's not as simple.

mvdan added a commit that referenced this issue Jun 10, 2019
We'll add a higher level API for this in #370. For now, just leave it
with OuterHTML.
@kenshaw
Copy link
Member

kenshaw commented Jun 12, 2019

This is doable, but the there's a reason that Chrome/ium does a secondary retrieval when doing view-source:. Will make a point of implementing this along with HAR capture ability.

@kenshaw kenshaw added this to the v0.4.0 milestone Jul 2, 2019
@mvdan mvdan removed this from the v0.5.0 milestone Nov 14, 2019
@marco-m
Copy link
Contributor

marco-m commented Apr 16, 2020

For people coming here and wondering where ExampleRetrieveHTML() is, it has been renamed to Example_retrieveHTML() in e657155 and is in file chromedp/example_test.go. It is also visible in the godoc: https://godoc.org/github.com/chromedp/chromedp#example-package--RetrieveHTML

@ianmarmour
Copy link

Would love to see this feature added having to create a listener and capture the response body is more painful then it needs to be for any XML or JSON requests this would be preferable

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants