New Cocoapods Spice #2224

soleo · 2015-10-11T18:29:31Z

What does your Instant Answer do?
Searching in CocoaPods

What problem does your Instant Answer solve (Why is it better than organic links)?
Shows more result related to the query

What is the data source for your Instant Answer? (Provide a link if possible)
CocoaPods API
http://search.cocoapods.org/api/v1/pods.flat.hash.json?query=test

Why did you choose this data source?
Very comprehensive and provide all data points we need.

Are there any other alternative (better) data sources?
More endpoints in this blog. http://blog.cocoapods.org/Search-API-Version-1/

What are some example queries that trigger this Instant Answer?
cocoapods afnetworking
cocoapods test

Which communities will this Instant Answer be especially useful for? (gamers, book lovers, etc)
Software Developers

Is this Instant Answer connected to a DuckDuckHack Instant Answer idea?
https://duck.co/ideas/idea/6307/cocoapods-library-lookup

Which existing Instant Answers will this one supersede/overlap with?
None

Are you having any problems? Do you need our help with anything?
Nope

What are the terms of use for the API? Will DuckDuckGo need specific authorization (e.g. an API key)? Are there any costs associated with API usage?
Free. No Authorization needed. No Cost.

Where did you hear about DuckDuckHack? (For first time contributors)
GitHub

What does the Instant Answer look like? (Provide a screenshot for new or updated Instant Answers)

Checklist

Please place an 'X' where appropriate.

[X] Added metadata and attribution information
[X] Wrote test file and added to t/ directory
[X] Verified that instant answer adheres to design guidelines (https://duck.co/duckduckhack/design_styleguide)
[X] Verified that instant answer adheres to code styleguide (https://duck.co/duckduckhack/code_styleguide)
[X] Tested cross-browser compatibility

    Please let us know which browsers/devices you've tested on:
    - Windows 8
        [] Google Chrome
        [] Firefox
        [] Opera
        [] IE 10

    - Windows 7
        [] Google Chrome
        [] Firefox
        [] Opera
        [] IE 8
        [] IE 9
        [] IE 10

    - Windows XP
        [] IE 7
        [] IE 8

    - Mac OSX
        [X] Google Chrome
        [X] Firefox
        [] Opera
        [X] Safari

    - iOS (iPhone)
        [] Safari Mobile
        [] Google Chrome
        [] Opera

    - iOS (iPad)
        [] Safari Mobile
        [] Google Chrome
        [] Opera

    - Android
        [] Firefox
        [] Native Browser
        [] Google Chrome
        [] Opera

IA Page: https://duck.co/ia/view/cocoapods

Enable caching for CocoaPods Spice Fixed typo in test cases Fixed favicon for CocoaPods

into cocoapods

moollaza · 2015-10-13T22:13:05Z

lib/DDG/Spice/Cocoapods.pm

@@ -0,0 +1,38 @@
+package DDG::Spice::Cocoapods;
+# ABSTRACT: Write an abstract here


@soleo you can remove these comments, they're just for you to see/read when developing your IA.

soleo · 2015-10-14T05:28:10Z

@moollaza , cleaned code and added more test cases.

moollaza · 2015-10-15T16:48:21Z

@soleo this is looking great!

I see that the version number is getting cut off though, so I think we should put that in the altSubtitle and put the platforms in the subtitle.

Also, let's make sure we join the authors names with a , and use the ddgsi-user icon like we do elsewhere to indicate it’s the dev:

JS

normalize: function(item) {
                return {
                    title: item.id,
                    altSubtitle: item.version,
                    subtitle: item.platforms.join(", ").toUpperCase(),
                    url: item.link,
                    description: item.summary,
                    authors: Object.keys(item.authors),
                }
            },

            templates: {
                group: 'list',
                detail: false,
                item_detail: false,
                variants: {
                    tileTitle: '1line',
                    tileSnippet: 'small',
                    tileFooter: '1line'
                },
                options: {
                    footer: Spice.cocoapods.footer
                }
            }

Footer

<div class="one-line">
  <span class="ddgsi ddgsi-user"></span> {{#concat authors sep=", " conj=", "}}{{this}}{{/concat}}
</div>

The above code results in:

One other approach I thought of would be to pass an array with the version and platforms to subtitle, which creates:

moollaza · 2015-10-15T16:51:28Z

@abeyang any thoughts on this?

soleo · 2015-10-15T17:01:42Z

I like the second option better. After all, platform string should not be longer than "IOS,OSX,WATCHOS". There are plenty of space left for version number.

moollaza · 2015-10-16T23:58:20Z

@soleo home important is the contributor name? I spoke with our design team and we think the best solution is to remove the contributor and put the supporter platforms as the footer. Would you be okay with that?

soleo · 2015-10-19T17:33:59Z

I'm fine with removing the authors line. I was thinking about giving the creator more credit for making dev libraries, but the users should be able to see the info after they click on those cards anyway. I'll make the change once I'm available.

soleo · 2015-10-19T18:10:36Z

Looks better now?

moollaza · 2015-10-19T22:10:06Z

@soleo much better, lets keep the platform right-aligned for consistency.

moollaza · 2015-10-21T19:19:38Z

@soleo this LGTM 👍

Thanks a lot for contributing (we really appreciate it 😃). We invite you to stick around and comment/help on other PR's & Issues or even submit an Instant Answer!

New Cocoapods Spice

soleo · 2015-10-21T22:08:35Z

@moollaza , I will. DDG is my farvorate searching engine, love to help and keep building something cool.

moollaza · 2015-10-21T23:29:40Z

@soleo By the way, we think you did a great job with this! Are you interested in helping others who build similar Instant Answers in the future?

If yes, we'll reach out and add you to a private list of mentors as well as send over some DDG merch as an additional thank you. Just email open@duckduckgo.com with the subject 'I want to help.' Please include your shipping address, phone-number (required for shipping outside USA) and t-shirt size. We'll reach out from there!

Otherwise, thank you for this amazing contribution. We hope you'll keep exploring and playing around with DuckDuckHack.

moollaza · 2016-04-22T12:54:12Z

@soleo hey there! I'm very sorry that we haven't gotten your Instant Answer online! We've been trying to understand for quite a while why it has never worked in our production environment.

Yesterday we finally figured it out! It turns out that the Cocoapods API returns a HTTP 403 to any requests made to their servers with a defined referer header!

Are you familiar with the API Maintainers? Would you be interested in reaching out to them to explain this and have them fix their API?

soleo · 2016-04-22T15:21:51Z

Haha, i see. I'll contact them about this

On Friday, April 22, 2016, Zaahir Moolla notifications@github.com wrote:

@soleo https://github.com/soleo hey there! I'm very sorry that we
haven't gotten your Instant Answer online! We've been trying to understand
for quite a while why it has never worked in our production environment.

Yesterday we finally figured it out! It turns out that the Cocoapods API
returns a HTTP 403 to any requests made to their servers with a defined
referer header!

Are you familiar with the API Maintainers? Would you be interested in
reaching out to them to explain this and have them fix their API?

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#2224 (comment)

Sent from Gmail Mobile

moollaza · 2016-04-22T17:40:03Z

Thanks!

soleo · 2016-05-19T21:15:16Z

@moollaza just got a message from the search API developer @floere.

You have to provide the Origin when Referer is passed.

for instance,

curl -X "GET" "http://search.cocoapods.org/api/v1/pods.flat.hash.json?query=AFNetworking" \
    -H "Referer: https://duckduckgo.com/something" \
    -H "Origin: https://duckduckgo.com"

would work.

But

curl -X "GET" "http://search.cocoapods.org/api/v1/pods.flat.hash.json?query=AFNetworking" \
    -H "Referer: https://duckduckgo.com/something"

would return Forbidden

moollaza · 2016-05-20T19:57:48Z

@soleo thanks for the update -- AFAIK there's no easy way for us to do that. It also seems like it wouldn't make sense for us to send it in our situations because we're not technically making a cross-origin request here (which is when the Origin header is useful)

Would it be possible for them to make an exception for us? Perhaps a new URL parameter or api-key?

soleo · 2016-05-20T21:23:15Z

@moollaza if Cocoapods add jsonp support, would it solve the current issue?

floere · 2016-05-21T12:31:52Z

Hi @moollaza! I thought I'd say hi 😊

we're not technically making a cross-origin request

Can you help me out a bit here and explain a bit more? (I am not aware of the infrastructure) Thanks!

soleo · 2016-05-21T19:44:54Z

@floere great, you're here! We can work through this here instead of emailing each other.

moollaza · 2016-05-24T21:42:37Z

@moollaza if Cocoapods add jsonp support, would it solve the current issue?

That would be great, but I don't think it will solve this problem...

Can you help me out a bit here and explain a bit more? (I am not aware of the infrastructure) Thanks!

Hi @floere thanks a lot for joining us on here and great to meet you!

Let me explain the infrastructure/data-flow

Client makes a request
DDG checks if the query is relevant to an Instant Answer and puts a <script> tag on the page
The script tag calls the DDG Endpoint for the Instant Answer: e.g. duckduckgo.com/js/spice/coacoapods/foo
Our NGINX proxies this request to the Cocoapods API, setting the query parameter to "foo"
If your API supports JSONP then we (DuckDuckGo) pass along the JSONP response to the client, if not, we simply wrap the returned JSON into a function call (spoofing JSONP)
The function is executed and uses the JSON to build and display an Instant Answer

It works this way so the client only ever speaks to DuckDuckGo and we speak to third-party APIs on their behalf. This way, the users IP and other info is only exposed to DuckDuckGo to protect their privacy.

From what I can tell, in proxying the request, we're not technically creating a cross-origin request, at least not in the typical sense, because the client isn't aware that the data is coming from another domain, it just thinks it's speaking to DuckDuckGo.

Does that make sense?

jagtalon · 2016-07-07T15:31:35Z

Hi, @floere! Is it possible to make it work without the required headers?

floere · 2016-07-08T09:26:24Z

Hi @jagtalon – thanks for pinging me again. @moollaza's answer must have gotten lost on my end. Sorry about that.

First of all, thanks @moollaza for the warm welcome and the great explanation!

Ok, so I understand why the client should think it's only talking to DDG, and all APIs are proxied through the DDG servers. That makes sense, and is helpful to know. So I think we can reduce the focus of this discussion on the call from DDG server (NGINX) to the CP API.

I agree that it is not a cross-origin request from the client. However, given that you pass a referrer from the DDG server, I'd say that it is a cross-origin request as well from the DDG server. To me, it makes sense to pass either none, or both referrer and origin. Wouldn't you agree? I assume that it would not be hard to implement either solution in NGINX and the CP API accepts either as @soleo demonstrated above – but I am open to be proven wrong, in any case.

All the best to you all and a useful & great search engine!

moollaza · 2016-07-13T15:01:28Z

To me, it makes sense to pass either none, or both referrer and origin. Wouldn't you agree?

I agree. We just made some recent changes to ensure we do send the referer, so I'll see if I can easily add support for origin or an option to remove the referer

moollaza · 2016-07-13T16:41:50Z

@soleo @floere I have good news!

We have a PR open that once merged, will allow use to unset the referer header, which should make things work as expected!

duckduckgo/duckduckgo#208

With that we'll be able to set spice headers => { Referer => '' }; and we should be good!

edgesince84 · 2016-07-13T20:15:15Z

hey @soleo thank you for your contribution to DuckDuckHack, we really appreciate it! I'm reaching out to let you know that we recently launched a new forum with the initiative of improving programming-related searches, and I hope you'll join us!

There are a lot of important tasks up for grabs that we'd like your help with, so feel free to and jump in.

floere · 2016-07-15T14:27:35Z

@moollaza This is great to hear 😊 All the best in moving forward with the CP spice! Let me know if you need something from the CP API end 🚀

soleo · 2016-07-18T23:18:01Z

@moollaza Great news indeed!

jagtalon · 2016-07-27T20:06:07Z

@soleo would you be interested in finishing this up? We'd like to improve the Swift query space (https://forum.duckduckhack.com/t/swift-search-overview/98) and this should answer a significant chunk of it. Here's the forum task: https://forum.duckduckhack.com/t/cocoapods-instant-answer/241

If you're interested in joining us, please hit me up at jag@duckduckgo.com or on the forum!

jagtalon · 2016-07-27T20:25:01Z

Made an issue for it: #2927 🐿

moollaza · 2016-07-28T00:28:05Z

@jagtalon this IA can move forward until that DDG PR is merged.

jagtalon · 2016-07-28T01:46:25Z

@moollaza oh I misunderstood! My bad.

moollaza · 2016-12-15T20:13:05Z

@soleo @floere 431 days after merge, this is finally live!

https://duckduckgo.com/?q=cocoapods+expecta

😄 🚀

Sorry for the very long delay. It took us a little while to add in support for specifying the values of individual headers.

jagtalon · 2016-12-15T20:59:58Z

Hell yes!

soleo added 3 commits October 11, 2015 18:08

Added spice for cocoapods

911ae27

Enable caching for CocoaPods Spice

5f0407e

Fixed typo in test cases

203685c

soleo changed the title ~~Cocoapods~~ New Cocoapods Spice Oct 11, 2015

soleo added 3 commits October 11, 2015 19:00

Fixed favicon for CocoaPods

1c8e3a3

Added spice for cocoapods

af8ae33

Enable caching for CocoaPods Spice Fixed typo in test cases Fixed favicon for CocoaPods

Merge branch 'cocoapods' of https://github.com/soleo/zeroclickinfo-spice

14ecb1b

into cocoapods

moollaza added New Instant Answer Status: Needs Community Review labels Oct 13, 2015

moollaza self-assigned this Oct 13, 2015

moollaza reviewed Oct 13, 2015
View reviewed changes

clean code and added more test cases

9bfb410

daxtheduck deployed to beta.duckduckgo.com October 15, 2015 15:20 Active

moollaza added the Needs Design Review label Oct 15, 2015

moollaza assigned abeyang and unassigned moollaza Oct 15, 2015

removed author line for cocoapods spice

d05cbc0

moollaza assigned moollaza and unassigned abeyang Oct 19, 2015

moollaza removed the Needs Design Review label Oct 19, 2015

moollaza removed the Status: Needs Community Review label Oct 21, 2015

moollaza added a commit that referenced this pull request Oct 21, 2015

Merge pull request #2224 from soleo/cocoapods

319c250

New Cocoapods Spice

moollaza merged commit 319c250 into duckduckgo:master Oct 21, 2015

jagtalon mentioned this pull request Jul 27, 2016

Cocoapods Instant Answer #2927

Closed

tagawa added the Programming Mission label Aug 24, 2016

		@@ -0,0 +1,38 @@
		package DDG::Spice::Cocoapods;
		# ABSTRACT: Write an abstract here

New Cocoapods Spice #2224

New Cocoapods Spice #2224

Conversation

soleo commented Oct 11, 2015

Checklist

moollaza Oct 13, 2015

Choose a reason for hiding this comment

soleo Oct 14, 2015

Choose a reason for hiding this comment

soleo commented Oct 14, 2015

moollaza commented Oct 15, 2015

moollaza commented Oct 15, 2015

soleo commented Oct 15, 2015

moollaza commented Oct 16, 2015

soleo commented Oct 19, 2015

soleo commented Oct 19, 2015

moollaza commented Oct 19, 2015

moollaza commented Oct 21, 2015

soleo commented Oct 21, 2015

moollaza commented Oct 21, 2015

moollaza commented Apr 22, 2016

soleo commented Apr 22, 2016

moollaza commented Apr 22, 2016

soleo commented May 19, 2016 • edited

moollaza commented May 20, 2016

soleo commented May 20, 2016

floere commented May 21, 2016

soleo commented May 21, 2016

moollaza commented May 24, 2016 • edited

jagtalon commented Jul 7, 2016

floere commented Jul 8, 2016

moollaza commented Jul 13, 2016

moollaza commented Jul 13, 2016 • edited

edgesince84 commented Jul 13, 2016

floere commented Jul 15, 2016

soleo commented Jul 18, 2016

jagtalon commented Jul 27, 2016

jagtalon commented Jul 27, 2016

moollaza commented Jul 28, 2016

jagtalon commented Jul 28, 2016

moollaza commented Dec 15, 2016

jagtalon commented Dec 15, 2016

soleo commented May 19, 2016 •

edited

moollaza commented May 24, 2016 •

edited

moollaza commented Jul 13, 2016 •

edited