Rewrite of Indeed Jobs IA #1587

tagawa · 2015-03-03T09:16:38Z

What does your Instant Answer do?
Gets 10 jobs posts from indeed.com

What problem does your Instant Answer solve (Why is it better than organic links)?
Shows location, company and how recent the job posting is.

What is the data source for your Instant Answer? (Provide a link if possible)
https://ads.indeed.com/jobroll/xmlfeed

Why did you choose this data source?
Suggested by @moollaza: https://duck.co/ideas/idea/604/jobs-listings-from-indeed-com

Are there any other alternative (better) data sources?
GitHub jobs and other job sites, but I believe indeed.com aggregates a lot of these.

What are some example queries that trigger this Instant Answer?
"web developer jobs"
"jobs in boston"
"tax accountant job in san francisco"

Which communities will this Instant Answer be especially useful for? (gamers, book lovers, etc)
Adults looking for a job

Is this Instant Answer connected to a DuckDuckHack Instant Answer idea?
Yes: https://duck.co/ideas/idea/604/jobs-listings-from-indeed-com
and fixes #1584

Which existing Instant Answers will this one supersede/overlap with?
Probably supersedes github_jobs

Are you having any problems? Do you need our help with anything?

An Indeed Jobs publisher ID is required (DDG_SPICE_INDEED_JOBS_PUBLISHER). Does DDG have one?
In Firefox I'm unable to click the tiles to go to indeed.com. Is it just me? Is this a DDG template problem? I've not got any custom code so this is really confusing me.

What are the terms of use for the API? Will DuckDuckGo need specific authorization (e.g. an API key)? Are there any costs associated with API usage?
The API is free but an API key (publisher ID) is required, as well as attribution.

Where did you hear about DuckDuckHack? (For first time contributors)
Contributed IAs a couple of years ago.

What does the Instant Answer look like? (Provide a screenshot for new or updated Instant Answers)

Checklist

Please place an 'X' where appropriate.

[X] Added metadata and attribution information
[X] Wrote test file and added to t/ directory
[X] Verified that instant answer adheres to design guidelines (https://duck.co/duckduckhack/design_styleguide)
[X] Verified that instant answer adheres to code styleguide (https://duck.co/duckduckhack/code_styleguide)
[X] Tested cross-browser compatibility

    Please let us know which browsers/devices you've tested on:
    - Ubuntu 14
        [X] Chromium
        [X] Firefox *
        [X] Opera
        [X] Midori (WebKit)

    - Android
        [X] Firefox *
        [X] Native Browser
        [] Google Chrome
        [X] Opera
* Note: On Firefox the display is OK but unable to click the tiles. See comment above.

IA Page: https://duck.co/ia/view/indeed_jobs

pfirsichbluete · 2015-03-03T20:01:24Z

Travis failed the rainfall tests (due to an unrelated change). A commit to fix those failures is at #1589

zekiel · 2015-03-03T23:40:14Z

thanks a million @pfirsichbluete

moollaza · 2015-03-24T22:04:23Z

@tagawa apologies for the long delay! This is looking great. I have a couple suggestions though:

Let's set the item.snippet as the description and pass the location to the footer template

Let's use the new variants to space things out accordingly:

normalize: function(item) {
    return {
        url: item.url,
        title: item.jobtitle,
        subtitle: item.company,
        description: DDG.strip_html(item.snippet)
    };
},
templates: {
    group: 'text',
    detail: false,
    item_detail: false,
    options: {
        footer: Spice.indeed_jobs.footer
    },
    variants: {
        tileTitle: '2line',
        tileSnippet: 'large',
        tileFooter: '2line'
    }
}

<div class="one-line">
    {{formattedLocationFull}}
</div>
<span class="tile__time one-line text--secondary">
    {{formattedRelativeTime}}
</span>

The result looks like this:

/cc @abeyang @chrismorast any thoughts?

tagawa · 2015-03-25T14:15:06Z

Thanks for the suggestion @moollaza. I've made this change:

Full description show in view
Location shown in footer
Uses new variants

I also changed the IA display name to "Indeed Jobs" rather than just "Jobs" to avoid duplication with GitHub Jobs (which will also need to be changed).

moollaza · 2015-03-25T15:34:49Z

@tagawa awesome, thanks for adding those changes!

I also changed the IA display name to "Indeed Jobs" rather than just "Jobs" to avoid duplication with GitHub Jobs (which will also need to be changed).

Actually both should just have "Jobs" as their name. We're trying to avoid using any brand names for our tabs (i.e. Videos vs YouTube, Software vs GitHub).

moollaza · 2015-03-25T15:35:46Z

lib/DDG/Spice/IndeedJobs.pm

+
+triggers any => "job", "jobs";
+
+spice to => 'http://api.indeed.com/ads/apisearch?publisher={{ENV{DDG_SPICE_INDEED_JOBS_PUBLISHER}}}&v=2&useragent=DuckDuckGo&userip=1.2.3.4&q=$1&l=$2&co=$3&format=json&callback={{callback}}';


One more small fix, let's change the APIKEY placeholder to our canonical wording: DDG_SPICE_INDEED_APIKEY

tagawa · 2015-03-25T23:30:46Z

Hi again @moollaza. Appreciate the clarification - I've made those changes.
The name policy means there would be two tabs with the name "Jobs" so maybe the GitHub one could be changed to "IT Jobs" (or similar).

moollaza · 2015-03-26T00:39:34Z

The name policy means there would be two tabs with the name "Jobs" so maybe the GitHub one could be changed to "IT Jobs" (or similar).

You're correct, our policy is to prevent duplicate Instant Answers like this scenario so we're going to reduce the trigger space of the GitHub Jobs IA by changing the trigger to "github jobs"

moollaza · 2015-04-06T20:56:15Z

lib/DDG/Spice/IndeedJobs.pm

+triggers any => "job", "jobs";
+
+spice to => 'http://api.indeed.com/ads/apisearch?publisher={{ENV{DDG_SPICE_INDEED_APIKEY}}}&v=2&useragent=DuckDuckGo&userip=1.2.3.4&q=$1&l=$2&co=$3&format=json&callback={{callback}}';
+spice from => '([^/]+)/(.*?)/([^/]*)';


@tagawa it's best to put optional checks at the end and only return them if they're defined otherwise you'll end up with a double slash // which gets automatically changed to / by NGINX.

spice from => '([^/]+)/([^/]+)/?([^/]+)?';

Sorry this doesn't actually apply here -- you're returning ' ' not '' so it's OK

abeyang · 2015-04-06T21:58:51Z

Hi @tagawa . This IA looks good! There are some things we'll need to modify on our end (the template itself), but as far as yours stand, it's good! 👍

moollaza · 2015-04-06T21:59:41Z

lib/DDG/Spice/IndeedJobs.pm

+        if ($1 || $2) {
+            my $query = $1 || ' ';
+            my $location = $2 || $spice_location || ' ';
+            return $query, $location, $country;


I think we either need to handle the country a little bitter, or ignore it for now. if I try to search for job in a city outside my country, I'm going to get really weird results or none. e.g. "jobs in philadelphia" sets the country to CA because I'm in Canada, which sends some mixed signals to Indeed.

We should either check for the presence of a country and get the country code using Locale::Country (a few other Spice do this) or we should try and pass the whole location along to Indeed (I'm not sure how well they handle that?)

Looks like the Indeed API is only for the US, e.g. for "Paris" it defaults to Paris, Texas and there are no results for Paris, France. I agree we should ignore $country completely but should we also restrict the IA to searches from the US only?

I believe it supports all the countries they have websites for. It definitely had results for Canada when the country was set to CA, and I just tried Brazil and India and they have results there too. I'd like to keep it open to all users because I can imagine lots of people outside the US look for jobs there.

I think the easiest route is to mimic the Time Spice which uses a large list of the biggest cities for most countries. https://github.com/duckduckgo/zeroclickinfo-spice/blob/master/lib/DDG/Spice/Time.pm#L23

That should allow us to get great coverage for the majority of queries. If needed, we can add internal "deep" triggering.

Still wondering about the best way to handle countries. You're right that they support more than just the US but they can't handle the whole location without country being a separate parameter.

I think most practical would be a hash lookup of capitals/countries, and perhaps a separate check for the country name in the query. This still wouldn't catch your Philadelphia (Canada) example but it would be better than nothing.

I think most practical would be a hash lookup of capitals/countries, and perhaps a separate check for the country name in the query.

@tagawa I just realized we didn't implement this yet -- that approach sounds good to me!

moollaza · 2015-04-06T22:02:15Z

Up on DDH8, https://ddh8.duckduckgo.com/?q=computer+programmer+jobs+in+philadelphia

/cc @abeyang

tagawa · 2015-04-07T05:49:59Z

Thanks @abeyang and @moollaza — pleased to hear it's looking good with just a couple of things to tweak.

abeyang · 2015-04-08T22:38:43Z

Just curious @tagawa --

Are we able to show the most recent job listings first?
Does Indeed return different job types (eg, full-time, part-time, contract, etc)?

tagawa · 2015-04-12T13:35:25Z

@abeyang:

Yes, the IA now shows most recent jobs first.
Yes, it's possible to filter by full-time, part-time, contract, temporary and internship. However, I'm having trouble adding more placeholders to the Spice To URL. Is there a max limit of 3?

moollaza · 2015-04-13T01:09:50Z

However, I'm having trouble adding more placeholders to the Spice To URL. Is there a max limit of 3?

No, there's no limit. But any missing/empty params will be collapsed (i.e. spice/my_spice/$1//$3 -> spice/my_spice/$1/$3) which will throw off your regex. You'll need to return a for empty params or join params into a single string (return $foo, "$bar-$baz"->/foo/bar-baz/) and then your regex will need to capture what exists.

FYI: We're bringing front-end filtering to the UI very soon, which will allow users to filter the original returned results, and also re-query the API with added filters

tagawa · 2015-04-13T01:17:36Z

Thanks @moollaza. The value of $4 is not being recognized so I'll try out your suggestions.
Front-end filtering sounds very cool — looking forward to it.

moollaza · 2015-04-13T18:34:12Z

Everything LGTM 👍

abeyang · 2015-04-13T18:39:10Z

I agree with @moollaza. Just a head's up, we are in the process of updating the look+feel for these kind of jobs IAs, so this might get yet another update in the near future. Thanks, @tagawa !

Rewrite of Indeed Jobs IA. Fixes #1584

tagawa · 2015-04-14T06:23:35Z

Thank you both. Job-related templates sounds good so I'll look out for those.

jagtalon · 2015-05-12T16:37:02Z

@tagawa @moollaza now live! https://duckduckgo.com/?q=javascript+jobs+nyc&ia=jobs

tagawa · 2015-05-12T22:52:55Z

Looks great! Thanks for the heads-up @jagtalon.

tagawa and others added 2 commits March 3, 2015 12:19

Added original spice files from PR duckduckgo#333

df7def8

Rewrite of Indeed Jobs API

cba2c3b

moollaza added New Instant Answer Status: Needs Community Review labels Mar 3, 2015

moollaza added the Needs Design Review label Mar 24, 2015

Updated view to show full description

cdeb61c

moollaza reviewed Mar 25, 2015
View reviewed changes

Reverted IA name and updated API key placeholder

b24bb59

moollaza assigned abeyang Apr 2, 2015

moollaza removed the Status: Needs Community Review label Apr 6, 2015

moollaza reviewed Apr 6, 2015
View reviewed changes

tagawa added 3 commits April 12, 2015 05:11

IndeedJobs: Updated regex to use named captures

ff462fa

IndeedJobs: Results now sorted by date

ff1f433

IndeedJobs: Results now sorted by date

7a1f579

moollaza added a commit that referenced this pull request Apr 13, 2015

Merge pull request #1587 from tagawa/indeed

19cca22

Rewrite of Indeed Jobs IA. Fixes #1584

moollaza merged commit 19cca22 into duckduckgo:master Apr 13, 2015

moollaza removed the Needs Design Review label Apr 14, 2015

moollaza changed the title ~~Rewrite of Indeed Jobs IA. Fixes #1584~~ Rewrite of Indeed Jobs IA May 26, 2015

moollaza removed the Status: Work In Progress label Aug 20, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rewrite of Indeed Jobs IA #1587

Rewrite of Indeed Jobs IA #1587

tagawa commented Mar 3, 2015

pfirsichbluete commented Mar 3, 2015

zekiel commented Mar 3, 2015

moollaza commented Mar 24, 2015

tagawa commented Mar 25, 2015

moollaza commented Mar 25, 2015

moollaza Mar 25, 2015

tagawa commented Mar 25, 2015

moollaza commented Mar 26, 2015

moollaza Apr 6, 2015

moollaza Apr 6, 2015

abeyang commented Apr 6, 2015

moollaza Apr 6, 2015

tagawa Apr 7, 2015

moollaza Apr 7, 2015

tagawa Apr 12, 2015

moollaza Apr 14, 2015

moollaza commented Apr 6, 2015

tagawa commented Apr 7, 2015

abeyang commented Apr 8, 2015

tagawa commented Apr 12, 2015

moollaza commented Apr 13, 2015

tagawa commented Apr 13, 2015

moollaza commented Apr 13, 2015

abeyang commented Apr 13, 2015

tagawa commented Apr 14, 2015

jagtalon commented May 12, 2015

tagawa commented May 12, 2015


		triggers any => "job", "jobs";

		spice to => 'http://api.indeed.com/ads/apisearch?publisher={{ENV{DDG_SPICE_INDEED_JOBS_PUBLISHER}}}&v=2&useragent=DuckDuckGo&userip=1.2.3.4&q=$1&l=$2&co=$3&format=json&callback={{callback}}';

Rewrite of Indeed Jobs IA #1587

Rewrite of Indeed Jobs IA #1587

Conversation

tagawa commented Mar 3, 2015

Checklist

pfirsichbluete commented Mar 3, 2015

zekiel commented Mar 3, 2015

moollaza commented Mar 24, 2015

tagawa commented Mar 25, 2015

moollaza commented Mar 25, 2015

Choose a reason for hiding this comment

tagawa commented Mar 25, 2015

moollaza commented Mar 26, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abeyang commented Apr 6, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

moollaza commented Apr 6, 2015

tagawa commented Apr 7, 2015

abeyang commented Apr 8, 2015

tagawa commented Apr 12, 2015

moollaza commented Apr 13, 2015

tagawa commented Apr 13, 2015

moollaza commented Apr 13, 2015

abeyang commented Apr 13, 2015

tagawa commented Apr 14, 2015

jagtalon commented May 12, 2015

tagawa commented May 12, 2015