Refresh oDesk skills

[ipeirotis]

Fetch periodically the oDesk dumps and return the extra field with the oDesk skill by fetching from https://www.odesk.com/api/profiles/v1/metadata/skills/ and then get the details for each skill https://www.odesk.com/api/profiles/v1/metadata/skills/adobe-acrobat

[georgegg]

This is possible if we add new ones (see if we have new skills and do that for the new ones ONLY)!!! To renew all the table (empty and repopulate again) is impossible as oDesk after a while returns no data (Too many CURLs).

[ipeirotis]

No need to empty and repopulate. Just go over the oDesk skill list, and fetch one after the other, and inserting/updating the entry at the table.

Just run this with a big sleep in between. You can keep running the fetch every few minutes. For example, if you fetch one every 30 seconds, you will be done with the refresh within a day.

You may have another process to check for deleted skills as well.

[georgegg]

It wold be great to know odesk api strategy for connection limits etc.

This way I can devise the best way to do refresh for odesk skills.

Is it possible to have such info?

[ipeirotis]

Just check for 503 error and back off with an exponentially increasing delay when you hit a 503. Seems to be the best practice.