Add support for HTTP Archive (HAR)

[JayNewstrom]

📷 Screenshots

📄 Context

This allows Chucker to share/export files with desktop http viewers such as Charles Proxy, Chrome Network Debugger, etc.
See #401

🚫 Breaking

No breaking changes.

🛠️ How to test

I added some automated tests. You can also test via the UI (see screenshots). After you have the file ready to import into another application, it should be able to be imported.

⏱️ Next steps

This handles the simple case, and allows you to view request/response in Charles. The generated file is likely not 100% accurate/to spec yet. See comments in #401 for some reasons why (ie. missing data).

I'd like to ensure this is the right direction, and get bugs/examples that fail to import and fix them as we find them.

I'd also like to add support for binary request/response bodies via base 64 encoding.

[vbuberen]

I was finally able to sit and test your PR.
So far I see the following issues with exported har file:

1. When open such file in Charles I see missing info like response code, TLS cypher info, etc. which is available in the DB. Not sure if it is supposed to be so, since I haven't seen mentions of such missing info in Add support for exporting requests to HTTP Archive format (.har) #401.

2. I can't open exported file in Chrome dev tools with the following error:

Again, not sure if you plan to address Chrome issue later or wanted to have `har` working in all tools right away.

Both issues happen for list of transactions and for a single exported transaction.

Going to do a code review shortly.

[JayNewstrom]

Thanks for taking a look.

1. I think this is more of a Charles Proxy limitation than missing information. If you export a session recorded from Charles as .har, and re import, it has the same issue.
2. I noticed that as well. I'd like to fix it, but ideally as a follow up. I couldn't find a place to get the reason why it was failing easily. (I see the error, but that didn't help me, I was hoping for much more information.)

Let me know if there's anything else I can do.

[cortinico]

Thanks for taking the time to work on this 🙏

[JayNewstrom]

@cortinico I think I've addressed all PR feedback, and fixed merge conflicts. Please give this another round of review when you get the chance.

[MiSikora]

Mentioning this because the conversation about desugaring got resolved. Please see my comment - #417 (comment).

[vbuberen]

@JayNewstrom My main concern with these issues I showed on screenshots is about value of such .har file. It seems that some essential info (like response codes) is missing and is such case what is the point of doing so many steps (export, open in Charles, etc.) for the QA team?

[cortinico]

what is the point of doing so many steps (export, open in Charles, etc.) for the QA team?

To piggyback on this: is there a tool where we can import the generated .har file?
Maybe we should investigate further why the Chrome import is broken?

[JayNewstrom]

It seems that some essential info (like response codes) is missing

Response codes are being saved/displayed in Charles. They're just not displayed in the overview page (which is likely just a Charles bug)

what is the point of doing so many steps (export, open in Charles, etc.) for the QA team?

For complex network interactions that are hard to recreate, Charles can be a life saver when trying to recreate. It also has features such as map local, which allows you to use the responses from a session like this instead of hitting a real server.

Our team has used this approach for a long time, and it's worked well for us.

To piggyback on this: is there a tool where we can import the generated .har file?

There are a lot of tools that can import it, here's another one I've used to test:
https://toolbox.googleapps.com/apps/har_analyzer/

Maybe we should investigate further why the Chrome import is broken?

I absolutely agree, and I'm willing to be the one that does this!

I think there's likely other bugs, and other programs that will have trouble importing for other various reasons, so I didn't expect this to be a one shot and done.

[JayNewstrom]

I've got some extra time this week if there's something I can do to continue to move this along.

[cortinico]

Can you provide a short status update @JayNewstrom ? I'd love to eventually pick this up or have a note if another contributor wants to work on this.

[JayNewstrom]

The current state is working. Although my previous comments still hold, meaning it's working for a very specific set of use cases. Things like binary request/responses, data missing from the database, etc.

My biggest reason for closing this is I no longer work on the project that was intended to use this, and my branch has gotten quite out of date.

If there's anything else I can do to help going forward, let me know.