mnot / redbot

This is a well-known security issue; RED should notice this condition.

RED should be able to test and report on multiple URLs. This includes:

• taking multiple URLs in URL input
• formatting report as a table, etc.

RED should have the option to output messages in JSON, to support integration with tools, etc. May
need to be disabled on a per-deployment basis (e.g., we probably don't want this on redbot.org).

RED should support compression of HTML output, e.g., using cgi_buffer.

RED should offer some mechanism to allow webmasters to tell it 'no'. Straw-man is to support
robots.txt, but not honour *; i.e., you'd need to target RED specifically.

Right now we have a connect() timeout, but no read() timeout, so a connection could in theory stall
RED for some time, consuming resources.

When a response has cookies, RED's UI should have the option to resubmit a new request with
those cookies.

RED should allow users to specify HTTP authentication credentials (Basic, Digest, and maybe
OAuth).

RED hasn't even come near IE yet; it's been tested mostly in Safari and Mozilla.

Look at cookies and note interaction with caching, etc.

parse HTML forms and submit them in a side panel

Allow contains a list of allowed methods; where possible, allow them to be used (perhaps uploading a POST or PUT body?)

There are a lot of features / problems in the code, indicated by FIXME or TODO.

RED should support HTTPS URLs as well.

This means changes in nbhttp as well as some modification of the caching logic.

Some servers behave differently using HTTP/1.0, either because chunked encoding is not used (length delimitation), or because they think that they need to send back different headers (e.g., Cache-Control).

RED should send a 1.0 request and see if there's any appreciable difference.

http://groups.google.com/group/firebug-working-group/web/http-tracing---export-format

Initially for input; maybe output too?

Currently, RED makes a request that sets Accept-Encoding: gzip first, and then shows those response headers for the 'main' (detail) view.

This is done to optimise; if the resource doesn't support negotiation, RED doesn't need to make two requests to figure this out.

However, now that we're showing the body, which is always uncompressed, showing the headers from the compressed response is confusing and not the right thing to do.

Fixing this means either introducing some logic to find the uncompressed response and promote it to the one that's shown, or to always make two requests.

Also, some wording in the messages will need to change.

Sometimes there may be a good reason to serve data with a Content-Type that seems not to match. For example, serving HTML as text/plain, because the idea is showing the source code. But some mismatches are very clearly wrong, like serving a PNG image as image/jpeg.

RED should compare the Content-Type with file(1) output (or similar) and warn about mismatches.

A real-world instance of this problem (unless it's fixed since I wrote this) may be found in the OGG links here:
http://www.mazedude.com/aa/themusic.htm
They are served as text/html despite being audio/ogg, causing browsers to show junk text.