Skip to content
This repository has been archived by the owner on Oct 15, 2022. It is now read-only.

Create Instant Answer for HTTP headers information #111

Merged
merged 6 commits into from Feb 26, 2015

Conversation

andrewalker
Copy link
Contributor

What does your Instant Answer do?

It shows information about HTTP headers.

What problem does your Instant Answer solve (Why is it better than organic links)?

When somebody searches for an HTTP header by name, what they are probably looking for is to know what it means and an example of usage, which is what this PR implements.

What is the data source for your Instant Answer? (Provide a link if possible)

Wikipedia: https://en.wikipedia.org/wiki/List_of_HTTP_header_fields

Why did you choose this data source?

Because it was mentioned as a possible data source in the idea posted to duck.co: https://duck.co/ideas/idea/4859/http-headers-description, and it seemed like a good choice.

Are there any other alternative (better) data sources?

There is an alternative: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
I don't know if it is better. It doesn't include non-standard headers (even if they are really common).

What are some example queries that trigger this Instant Answer?

  • http header content-type
  • user agent http header
  • if-modified-since

Which communities will this Instant Answer be especially useful for? (gamers, book lovers, etc)

Programmers.

Is this Instant Answer connected to a DuckDuckHack Instant Answer idea?

Yes: https://duck.co/ideas/idea/4859/http-headers-description

Which existing Instant Answers will this one supersede/overlap with?

If I type "user-agent", it actually shows my current user agent. If I type "accept", it'll show audio tracks by a band with that name. I'm sure there will be other overlaps, mostly when only the header name is typed, not followed by http header or something like that.

Are you having any problems? Do you need our help with anything?

This is my first Instant Answer. It seemed to work fine, but I'd love to have some feedback on whether it follows all best practices, works as expected, and didn't miss anything.

Where did you hear about DuckDuckHack? (For first time contributors)

DuckDuckGo's hiring page 馃槃 I had visited DuckDuckHack a few times before, and I don't know where I first heard about it, but that is what motivated me to make a first contribution.

What does the Instant Answer look like? (Provide a screenshot for new or updated Instant Answers)

I don't know! How can I test Fathead instant answers?

@MrChrisW
Copy link
Collaborator

@andrewalker Thanks for your contribution. This will be reviewed shortly.

@moollaza
Copy link
Member

Hi @andrewalker thanks a lot for making this! We really appreciate it. After a quick glance I don't see any big problems.

I don't know! How can I test Fathead instant answers?

Haha sorry about that! Currently developers can't test their Fatheads, we'll have to test it on our end and give you feedback.

@jdorweiler will review this and give you some feedback shortly 馃槂

@andrewalker
Copy link
Contributor Author

@moollaza thank you for taking a look :)

About testing, ok, no problem. I'll wait for the tests on ddg's end and @jdorweiler's review.

@jdorweiler
Copy link
Contributor

@andrewalker thanks, looks pretty good. Can you create redirect entries (R for the type field) for the hyphenated titles. That way searches for things like Access Control Allow Origin would show the entry for Access-Control-Allow-Origin. Do you have any trigger words in mind?

@andrewalker
Copy link
Contributor Author

@jdorweiler sure, I'll create redirect entries. I couldn't think of any other trigger words...

This makes it easier (for humans) to read the file and find the headers.

Note: The headers are only sorted within each section, which at least
makes it consistent accross regenerations of the file. That is,
executing parse.pl a thousand times will now produce the same output (in
the same order).
This way, ``Access Control Allow Origin`` will also match
``Access-Control-Allow-Origin``.
@andrewalker
Copy link
Contributor Author

@jdorweiler I found another issue, I'm not quite sure how to fix it.

Some headers (e.g. Content-Type, Cache-Control, Connection) will have more than one definition, because they are listed both in the Request fields and Response fields.

We could use disambiguation, I guess, though generally they look very similar.

Example:

Connection (request): Control options for the current connection and list of hop-by-hop request fields.
Connection (response): Control options for the current connection and list of hop-by-hop response fields.

I tried to look for how to create disambiguation entries, but the Fathead docs don't seem to include any information about it.

How should I proceed?

@andrewalker
Copy link
Contributor Author

Oh, and I also implemented the redirection in 632433a :)

@andrewalker
Copy link
Contributor Author

@jdorweiler, did you have a chance to take a look?

@jdorweiler
Copy link
Contributor

@andrewalker looks pretty good. You can test it out here: https://ddh10.duckduckgo.com/?q=http+content+type&ia=about

Could you link to the actual entry in to table on the wiki page?

@andrewalker
Copy link
Contributor Author

Right. So do you think it's okay to leave the ambiguous results as is?

Could you link to the actual entry in to table on the wiki page?

I'm not sure it is possible. Looking at the source code, very few 's have id's. Maybe I could edit the Wikipedia page to set the id's, though I've never edited anything there before. Another possibility is to link to the table at least.

Which way should I go?

@andrewalker
Copy link
Contributor Author

@jdorweiler also, thanks for the link! It's nice to see it is working 馃槃

@jdorweiler
Copy link
Contributor

@andrewalker You're right, I remember a similar problem linking into a wiki table on another fathead. You can just leave it linking to the main page then. I'll look into the disambiguations and get back to you. I think we only use that type for scraping mediawiki articles.

@jdorweiler
Copy link
Contributor

@andrewalker after some playing around I got a disambiguation to work. Here's an example that would add two disambiguation links:

title \t D \t \t \t \t \t \t \t \t* [[disambiguation title]],text that you want to show for this\n*[[another title]],and some more text\n\t \t \t

@andrewalker
Copy link
Contributor Author

Awesome, thank you, @jdorweiler! I'll try and implement this. If it doesn't fit well (if it really only works for mediawiki articles), we can just revert it.

@jdorweiler
Copy link
Contributor

@andrewalker the disambiguation would show up on a "meanings" tab like this https://duckduckgo.com/?q=orange&t=canonical&ia=meanings

@jdorweiler
Copy link
Contributor

@andrewalker did you get a chance to try the disambiguation out or do you have any questions?

@andrewalker
Copy link
Contributor Author

@jdorweiler, yes, I tried it :)

Sorry for the delay, my current job demanded a lot this last week. It was a little tough finding the time to finish this up.

Anyway, please take a look at be1847e. It seems a little weird to have things like:

*[[Response]],The date and time that the message was sent (in "HTTP-date" format as defined by RFC 7231).<br><pre><code>Date: Tue, 15 Nov 1994 08:12:31 GMT</code></pre>\n*[[Request]],The date and time that the message was sent (in "HTTP-date" format as defined by RFC 7231).<br><pre><code>Date: Tue, 15 Nov 1994 08:12:31 GMT</code></pre>\n

They are both very similar. I don't know how we could improve that. Maybe it's more of a UI/UX issue, it's a little out of my league :P

Also, I basically copied the same content that was in the "A" types, including HTML in the description/example, and the link to the wikipedia article. Is that OK? Should I remove it? I see in your example you didn't use HTML, and left the URL empty.

@jdorweiler
Copy link
Contributor

@andrewalker thanks! the disambiguation entries look good. The linking on the tiles is a little weird but that's a problem on my end.
https://ddh10.duckduckgo.com/?q=http+connection&ia=meanings
selection_359

A few more examples.
https://ddh10.duckduckgo.com/?q=http+dnt&ia=about
selection_358

https://ddh10.duckduckgo.com/?q=http+header+user+agent&ia=about

selection_360

@jdorweiler
Copy link
Contributor

Try it out on ddh10.duckduckgo.com and let me know what you think.

@moollaza
Copy link
Member

This is looking awesome guys, nicely done!

@andrewalker
Copy link
Contributor Author

looks great, @jdorweiler and @moollaza. Thank you so much for reviewing it!

@jdorweiler
Copy link
Contributor

@andrewalker what about adding "header" or "headers" to each title as a redirect? That way things like "http location header" would redirect to the "location" entry.

@andrewalker
Copy link
Contributor Author

@jdorweiler sure! will do.

@jdorweiler
Copy link
Contributor

@andrewalker thanks! It's ready to merge after that.

@andrewalker
Copy link
Contributor Author

@jdorweiler done! 馃槃

@jdorweiler
Copy link
Contributor

Thanks @andrewalker! 馃嵃

jdorweiler pushed a commit that referenced this pull request Feb 26, 2015
Create Instant Answer for HTTP headers information
@jdorweiler jdorweiler merged commit 7be4249 into duckduckgo:master Feb 26, 2015
@andrewalker andrewalker deleted the andrewalker/http_headers branch February 26, 2015 16:27
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants