THOR-Bootcamp-2017

THOR Bootcamp, Helskinki, March 2017

Hands on with the ORCID API:
Getting Authenticated ORCID iDs, OAuth Permissions and updating records

Workshop Agenda (3 hrs, 30 mins)

1. PRESENTATION: ORCID - the technical perspective. (30 min)

Learn about ORCID and ORCID iDs and how they work. Understand how organizations are using the ORCID registry to collect and display ORCID iDs, and connect and sync information between ORCID records and their own system.

2. ACTIVITY: EXPLORE THE ORCID REGISTRY (20 min)

Set up an ORCID iD in our test environment, and explore adding works to your record. Understand ORCID’s provenance model and its implications. Learn about the components of an ORCID record, how they get populated, and how they get used.

3. PRESENTATION: ABOUT THE ORCID APIs (30 min)

Discover ORCID API types and features, and understand ORCID’s test environment and the technologies that ORCID uses.

4. ACTIVITY: Explore the public API (30 min)

Use swagger to request records from the public API. Try reading your test account on the sandbox, and search for identifiers in the live registry.

5. ACTIVITY: OAUTH BASICS (30 min)

ORCID’s API uses OAuth 2.0 as its protocol for a system client to obtain user permission to access the information stored in his/her ORCID record. In this section you will obtain system client credentials, and execute basic commands to request permission using a basic OAuth 2.0 3-legged flow. (Don’t know what that is? don’t worry! It will be covered in the session.)

6. PRESENTATION: API CREDENTIAL SETUP (10 min)

Set up ORCID Member API credentials to enable record updates. We will try it out, using Google OAuth playground to simulate your website.

7. ACTIVITY: POST A WORK AND EMPLOYMENT AFFILIATION TO A RECORD (60 min)

Format data about the person’s contirbutions to your platform and post it to his/her ORCID record. Update the data that you’ve already posted to simulate updating data when an metadata changes.

8. REFERENCE MATERIALS

--

1. WHAT IS ORCID? (30-45 mins)

PRESENTATION

Presented as a power point presentation - first two sections

-top-

2. EXPLORE THE ORCID REGISTRY (30 min)

ACTIVITY

2.1 Create an ORCID iD

In order to try out API calls, such as a reading a record and adding information to it, you will also need to create a test ORCID record. This can be done through the user interface, much like in the live ORCID registry.

1. Open a web browser and navigate to https://sandbox.orcid.org/signin
    
2. Click Register for an ORCID iD.
    
3. Complete the form with a name, email, and password. You will need to remember this information for future steps.

IMPORTANT! Don’t use a real email address! Instead, make up an address ending in @mailinator.com (use any prefix, e.g. sgarcia@mailinator.com).

4. Click the I consent… checkbox and click Register.
    
5. After completing the registration process, you will be taken to your new testing ORCID record. Make note of the 16-digit ORCID iD for this record – you will need this in order to make API calls later during the workshop.
   
    

-top-

2.2 Explore the ORCID record

In this section we will learn about the parts of an ORCID record, and explore ORCID's provenance model.

1. Using the interface, add a sentence or two to serve as the biography for your test record. The biography section is located at the top of the right column, and the edit button is the pencil icon in the upper right corner of this section, next to the visibility selector.
    
2. In the Employment section, click Add employment > Add manually to add employment information.
    
3. After saving your employment information, notice the information on the bottom of the "card" you just created. There is a Source field (that matches your name because you added this information) and the date that you created this addition to your record. This provenance information is recorded for all information added to an ORCID record.
    
4. In the main menu at the top of the screen, click on the Account settings link to display your account preferences. Notice each section to see what iD holders can control with their ORCID accounts.
    
5. In the main menu at the top of the screen, click on the Developer tools link. In this section a user may obtain credentials to access the public API. Note that this API is different from the member API that we will be discussing today.

-top-

2.3 Add some works to your record

Much of the power of ORCID comes from the connections it makes between Authors and the things they produce. Works, inlcuding datasets and journal articles can be in several ways:

• Added manually by the user, via the user interface.
• Imported manually from Bibtex by the user, via the user interface. Bibtex is not very consistent, but widely used, so it is supported for import and export of works.
• Added by the user, via a search and link wizard. These automate the discovery of works based on thrid party tools. These wizards include integrations from Crossref, Datacite, Scopus, Web of Science and many more. Not all of these tools work in the sandbox environment. Two that do work are but the Europe PMC and Scopus tools.
• Added by a trusted third party, with the users permission. This can be at any time after permission is granted, and is often done on publication. We will explore this use case later today.

1. Add a work to your record manually. Make sure it has an identifier of some kind so we can search for it later today.
2. Add a work with the "Europe PMC Test" search and link wizard. Not all search and link wizards work on the sandbox, but this one does.
3. Connect to scopus using the search and link wizard. In addition to adding works, the scopus search and link adds the Scopus Identifier to the record. If you are not an author on Scopus yourself, search for Brown, Josh D. and link to that. When asked for an email, use the @mailinator.com address you used before.

-top-

3. ABOUT THE ORCID APIs (30 min)

PRESENTATION

Presented as a power point presentation - third section

3.1 ORCID API types & features

ORCID offers several APIs (Application Programming Interfaces) that allow your systems to connect to the ORCID registry, including reading from and writing to ORCID records. Some API functions are freely available to anyone; others are available to organizations that financially support ORCID with an annual membership subscription.

API Version	Access	Features
Public API	Freely available to anyone	Authenticate: Obtain a user’s authenticated ORCID iD Read (Public): Search/retrieve public data Create: Create new ORCID records on demand
Member API	Available to organizations that support ORCID with an annual membership subscription Sandbox Member API freely available to all for testing	Read (Limited): Search/retrieve limited-access data Add: Post new items to a record (affiliations, works, etc.) Update: Edit or delete items you previously added

-top-

3.2 Sandbox test environment

In addition to the production Registry and APIs, ORCID also offers a testing environment, the ORCID Sandbox, which we will be using for this boot camp. The Sandbox is open to all users and provides a place to develop and test applications without affecting data in the live ORCID registry.

The Sandbox includes:

• Sandbox Registry: Simulates the ORCID Registry
   
• Sandbox Member API: Simulates the Member API
   
• Sandbox Public API: Simulates the Public API

The sandbox behaves the same way as the production ORCID Registry with a few exceptions:

• Search & Link tools do not always function.
   
• To avoid unintentional spamming, the Sandbox sends emails only to @mailinator.com addresses. Mailinator.com is an inbox testing service that is free and requires no registration. To receive emails from the Sandbox, use an @mailinator.com email address when creating Sandbox record(s).
   
• Links in the top menu bar (For Researchers, For Organizations, About, etc.) do not function.

-top-

3.3 ORCID API technologies

All of the ORCID APIs are based on the same set of technologies:

1. REST: ORCID APIs are “RESTful”, which means that they use HTTP (hyper-text transfer) calls to transfer information.
    
2. OAuth: ORCID APIs use the OAuth 2.0 authentication protocol in order to grant client applications access to users’ ORCID records.
    
3. XML/JSON: ORCID APIs support data exchange in either XML or JSON format.

-top-

3.4 ORCID API Tools

In order to use the ORCID APIs you will need the following software tools:

1. Web browser: Firefox (33+), Chrome (38+), Internet Explorer (10+), Safari (6+)
    
2. Internet connection
    
3. Plain text editor: Sublime, TextEdit (Mac), Notepad++ (Win), or your preferred plain text editor
    
4. Software capable of making HTTP requests:
• cURL: free, command-line application available for Mac or Windows at http://curl.haxx.se/download.html (pre-installed on most Mac OS versions; accessible within Terminal application)
   
• Online tools, e.g. hurl.it or Google OAuth Playground
   
• The ORCID swagger interface, available on the sandbox and live registries. Separate endpoints are available for the public and member APIs.
• Your own web application, in a language such as Java, Ruby, Python, PHP, etc.

For this workshop, we will be using the online tool Google OAuth Playground and the ORCID swagger interface.

-top-

4. Explore the public API with swagger(20 min)

ACTIVITY

Load up the public API swagger interface and use it to view the metadata for your new test account. This is a good way of seeing an overview of the API endpoints and the values they return.

1. Use the "Public API v2.0" to list the works. Explore the difference between a work summary and a work.
2. The "activities" endpoint lists all ORCID activities for an ID, including works, education, employment and funding. Request it for your account and see how it behaves.
3. If a record has more than one work with the same identifier, the API 'groups' them together. Add a duplicate work to your test account and see how it looks in the API

If you are planning on using JSON instead of XML, the swagger interface is the best place to discover how to structure API JSON objects

Swagger can also be used to generate client libraries in a huge number of languages. This can be done with http://editor.swagger.io/. Simply import the swagger JSON url (e.g. https://pub.sandbox.orcid.org/resources/swagger.json) into the tool, then select the client library to generate. Depending on your use case, this may be a quick way of getting your integration up and running.

An example of this in action is our bibtex processing library which uses a swagger generated client library to access the API.

5. OAUTH BASICS (30 min)

ACTIVITY

As discussed in section 3.1, the Public API can only be used to read and search ORCID records, and to get authenticated ORCID iDs. The Member API, however, can be used to add new information to ORCID records, as well as to update information previously added. To do these actions, one must obtain permission from the user/data subject. This section describes the standard OAuth process for requesting this permission.

5.1 Accessing the Sandbox Member API

Client credentials consisting of a client ID and a client secret are needed in order to access the Member API. Client Credentials for the Member APIs are issued by ORCID. For this workshop, you can use the sample Sandbox Client Credentials, but we recommend that you obtain your own Member API Sandbox Client Credentials. How to do this is discussed in section 6.

-top-

5.2 Setting up the OAuth Playground

We’ll use the Google Developers’ OAuth Playground for the next exercises. To get started, we will need to configure the environment to work with the ORCID Member API.

1. Visit https://developers.google.com/oauthplayground
    
2. Click the gear icon in the upper right corner to open the configuration form.
    



Note: For those with institutional accounts, try to use the credentials for your institution.

3. Enter the following:

   OAuth flow	Server-side
   OAuth endpoints	Custom
   Authorization endpoint	https://sandbox.orcid.org/oauth/authorize
   Token endpoint	https://sandbox.orcid.org/oauth/token
   Access token location	Authorization header w/Bearer prefix
   OAuth Client ID	Your Member API client ID (ex: APP-E422WM33OPZWKKMQ)
   OAuth Client Secret	Your Member API client secret (ex: ae857cfb-446b-4c3f-8a09-55436bf602dc)

4. The configuration screen should look similar to the image at right. After you’ve entered the settings, click Close in the lower left corner of the configuration screen.

-top-

5.3 Getting permission (an Access Token) to access ORCID records

To access an ORCID record via the Member API, you first need to get permission from the owner of the record in the form of an Access Token. ORCID uses the standard protocol, OAuth 2.0, to obtain this permission. Generally there are two steps:

1. Decide which scopes you are interested in. Different scopes enable different actions, like read or update. Have a look at the list of scopes on the ORCID website. For this excercise we will use "/activities/update"
2. Get an Authorization Code.
    
3. Exchange the Authorization Code for an Access Token.
    

4.3.1 Get an Authorization Code

To get an Authorization Code, you’ll need to prompt the user to sign into his/her ORCID account and grant permission to your application. In a real-world situation, this is done using an authorization URL that you construct. This URL looks something like

https://sandbox.orcid.org/oauth/authorize?client_id=[your client ID]&response_type=code&scope=/activities/update&redirect_uri=https://developers.google.com/oauthplayground

With the OAuth Playground, however, this step is done by configuring some additional settings and clicking a button.

1. Beneath Step 1: Select & authorize APIs on the left side of the Google OAuth Playground screen, type /activities/update in the text box (do not select any of the options presented in the box above).
    
2. Click the Authorize APIs button.
    
    
3. An ORCID OAuth permission screen will appear. If you are already signed into your test Sandbox account, click the Authorize button to grant permission. If you are not yet signed in, type in your Sandbox test account sign in credentials, and click Authorize to grant the permissions.
    
4. Clicking Authorize will send the user back to the OAuth Playground. A 6-character code will appear in the Authorization Code field.
    

5.4 Exchange the Authorization Code for an Access Token

Once you have an Authorization Code, you can exchange it for an Access Token, which allows you to read from/write to a user’s ORCID record. In a real-world situation, this exchange would be done by your system, using a programming language such as PHP, Java, or Ruby on Rails. This call is a RESTful call with information similar to the following

``` HTTP method: GET Header: Accept: application/json Data: client_id=[your client ID]& client_secret=[your client secret]& grant_type=authorization_code& code=[the authorization code]& redirect_uri=https://developers.google.com/oauthplayground Endpoint: https://sandbox.orcid.org/oauth/token" ```

With the OAuth Playground, however, this step is done by clicking a button.

1. Beneath the Authorization Code field, click Exchange authorization code for tokens.
    
2. The token will appear in the Access Token field.
    
3. Note that you are provided with additional information in the Request/Response section on the right side of the screen, such as the name and ORCID iD of the user who granted permission, the lifespan of the token (20 years), and the scope for which the token is valid.
    
    

5.5 Congratulations. You now have an authenticated ORCID

Well done. For integrations that just require ORCID ids for their own systems and don't intend to update the registry, this is all that is needed. Implementing this on your own system should be easy!

A preconfigured playground if you're having trouble

-top-

6. API CREDENTIAL SETUP (30 min)

PRESENTATION

6.1 How to get ORCID Member API Credentials

To use the ORCID Member API, you'll need credentials consisting of a Client ID (consumer KEY) and Client Secret (consumer SECRET). These work like a username and password that allow your application to access the API.

We require that all new Member API applications be built and tested using our Developer Sandbox, which mirrors production environment behavior, but does not contain production data. Learn more about the Developer Sandbox

To get Sandbox API credentials, use the form at:

Request ORCID API credentials

Note: The process is not automated - ORCID staff will usually issue your credentials within 24 hours

When requesting credentials, you'll be asked for the following information:

• Notes for ORCID Staff: let us know if you're using a vendor system, need additional redirect URIs, or have any other questions or notes.
• Name of your organization
• Contact email address Use a valid email - we'll send your credentials to this address!
• Name of your client application: Name displayed on authorization screen and list of trusted organizations in users' records
• Short description of your client application: Displayed on authorization screen when users click the question mark icon
• URL of the home page of your application: Displayed on the list of trusted organizations in users' records
• Redirect URIs: URL(s) in your web application where users should be returned to after they authorize access to their ORCID record data.
• Type of credentials: Basic allows you to read from/write to records, while premium allows read/write access and also lets you register webhooks

For the purposes of today, we have set up a number of test accounts for you to use.

7. POST A WORK TO THE REGISTRY (50 min)

ACTIVITY

In this section we will try to add and update a work to your Sandbox test ORCID record using the permission that you have already received from earlier exercises.

7.1 Anatomy of a work

Take a look at a sample work that was retrieved from the ORCID API. This contains information provided by the source of the information, and information added by ORCID, such as the creation date. It also includes a "putCode" which is the unique identifier for the work within ORCID. This is needed to update the work or request the full work details instead of a work summary.

Take a look at this partially populated 'blank' work. This has placeholders for the XML you need to post to create a work. Note that it does not include the fields added by ORCID seen above.

7.2 Post a work to your ORCID record

1. Beneath Step 3: Configure request to API, set HTTP Method to POST.
    
2. Click Add headers and enter the following values:
• Header name: accept
• Header value: application/vnd.orcid+xml
  



3. Click Add to add another header and enter the following values:
• Header name: Content-type
• Header value: application/vnd.orcid+xml
   
4. Click Add again, then click Close.
    
5. In the Request URI field, enter https://api.sandbox.orcid.org/v2.0/[orcid-id]/work, replacing [orcid-id] with the ORCID iD of the Sandbox record that you created earlier (ex: https://api.sandbox.orcid.org/v2.0/0000-0002-3791-8427work)
   
   
    
6. Click Enter request body. Here is where you’ll enter the XML for the works you wish to add.
    
7. Copy the sample work XML in the Sample work section.
    
8. Paste the copied content into the Request Body text box
9. Edit the text to reflect your institution.
10. Click Close.
    
    

 
11. Click Send the request.
     
12. The results will appear in the Request/Response section on the right side of the screen. Scroll to the bottom – if you see HTTP/1.1 201 Created, the work was successfully posted!
     
     
13. Visit the public view of your Sandbox record at http://sandbox.orcid.org/[Your sandbox iD] to see the work that you added in the user interface.

-top-

7.3 Updating a work

In a real-world situation, you may need to update a researcher's work. Updating uses PUT instead of POST and requires the activity put code.

1. Beneath Step 3: Configure request to API, set HTTP Method to PUT -- which you need to update the item.
    
2. Change the endpoint: https://api.sandbox.orcid.org/v2.0/[orcid-id]/work/[put-code]. The put-code was returned as a Location header in the response to the create POST. e.g. Location: https://api.sandbox.orcid.org/v2.0/0000-0001-6363-5714/work/783113 has the put code '783113'. If you no longer have this number on your screen, you can find it by GETing the users work summary and locating the work you added. The put-code is an attribute of the work element.
3. Click Enter request body. This is where you’ll enter the XML for the work that you wish to edit.
    
4. Modify the sample work XMLMake sure you add the put-code="" attribute to the <work:work> element.
    
5. Paste the copied content into the Request Body field and amend to reflect your institution.
6. Click Close.
    
7. Click Send the Request.
    
8. The results will appear in the Request/Response section on the right side of the screen. If the full XML of the user’s record appears, the work was successfully updated!
   If you receive an error, be sure to check that the headers value under Add headers have not been changed to garbled text, e.g. "application%2Fvnd.orcid%2Bxml".
    
9. Visit the public view of your Sandbox record at http://sandbox.orcid.org/[Your sandbox iD] to see the changes to the work in the user interface.

7.4 Post an employment affiliation to your ORCID record

In this section we will add and update an employment affiliation to your Sandbox test ORCID record using the permission that you have already received from the earlier exercise. Note: It is also possible to post an education affiliation using [different slightly different XML and endpoints](sample-education-affiliation.xml).

1. Beneath Step 3: Configure request to API, set HTTP Method to POST.
    
2. Click Add headers and enter the following values:
• Header name: accept
• Header value: application/vnd.orcid+xml
  



3. Click Add to add another header and enter the following values:
• Header name: Content-type
• Header value: application/vnd.orcid+xml
   
4. Click Add again, then click Close.
    
5. In the Request URI field, enter https://api.sandbox.orcid.org/v2.0/[orcid-id]/employment, replacing [orcid-id] with the ORCID iD of the Sandbox record that you created earlier (e.g. https://api.sandbox.orcid.org/v2.0/0000-0002-3791-8427/employment)
   
   
    
6. Click Enter request body. Here is where you’ll enter the XML for the works you wish to add.
    
7. Copy the XML.
    

<?xml version="1.0" encoding="UTF-8"?>
<education:education
	xmlns:common="http://www.orcid.org/ns/common" xmlns:education="http://www.orcid.org/ns/education"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.orcid.org/ns/education ../education-2.0.xsd ">	
	<education:department-name>Department</education:department-name>
	<education:role-title>Degree title</education:role-title>
	<common:start-date>
		<common:year>2012</common:year>
		<common:month>01</common:month>
		<common:day>01</common:day>
	</common:start-date>
	<education:organization>
		<common:name>My University</common:name>
		<common:address>
			<common:city>Some City</common:city>
			<common:region>Region</common:region>
			<common:country>US</common:country>
		</common:address>
		<common:disambiguated-organization>
			<common:disambiguated-organization-identifier>XXXXXX</common:disambiguated-organization-identifier>
			<common:disambiguation-source>RINGGOLD</common:disambiguation-source>
		</common:disambiguated-organization>
	</education:organization>	   	
</education:education>

8. Paste the copied content into the Request Body text box.
    
9. Customize the text to reflect your institution. For the disambiguated-organization-identifier, replace XXXXXX with the Ringgold identifier for your institution.
    
10. Click Close.
    
    

 
11. Click Send the request.
     
12. The results will appear in the Request/Response section on the right side of the screen. Scroll to the bottom – if you see HTTP/1.1 201 Created, the affiliation was successfully posted!
     
     
13. Each item added to the ORCID record has a unique put-code that can be used to read the full metadata of the item, or to edit or delete the item using the client that has added it to the record. The put-code is displayed in the response following Location: https://api.sandbox.orcid.org/v2.0/[orcid-id]/employment/[put-code]. In a real-world situation, your system would store this put-code along with the affiliation data in your database.
    
14. Visit the public view of your Sandbox record at https://sandbox.orcid.org/[Your sandbox iD] to see the work that you added in the user interface.

-top-

7.5 Updating an affiliation

In a real-world situation, you may need to update a researcher's affiliation when they finish a degree, change departments, or finish a contract. To update an affiliation, you will require the unique put-code for the affiliation.

1. Beneath Step 3: Configure request to API, set HTTP Method to PUT, which is required to update the item.
    
2. At the end of the request URI, add the put-code of the affiliation you just created, e.g. https://api.sandbox.orcid.org/v2.0/0000-0002-3791-8427]/employment/employment/26651. Without the put-code, you cannot update the affiliation.
    
3. Click Enter request body. This is where you’ll enter the XML for the work that you wish to edit.
    
4. Go to [sample-employment-affiliation-edit.xml](sample-employment-affiliation-edit.xml) and copy the XML.
    
5. Paste the XML in the Request Body field and amend to reflect your institution.
   Tip: The amended area are lines 1 (edit) and lines 13-11 (new data). You can copy line one and use it to replace the first line of your data; enter the unique put-code in place of "put-code="XXXXX"". The new data begins begins with <common:end-date>, and you can can paste it after the </common:start-date> in the active Request Body.)
   
    
6. Click Close.
    
7. Click Send the Request.
    
8. The results will appear in the Request/Response section on the right side of the screen. Scroll down – if you see HTTP/1.1 200 OK, the affiliation was successfully posted!
   If you receive an error, be sure to check that the headers value under Add headers have not been changed to garbled text, e.g. "application%2Fvnd.orcid%2Bxml".
   
    
9. Visit the public view of your Sandbox record at https://sandbox.orcid.org/[Your sandbox iD] to see the changes to the work in the user interface.

-top-

7.6 Searching the registry

If you run a repository of some kind, you may be interested in seeing who has claimed things from your repository. Depending on the identifiers you use (for example, DOIs), this may be possible via the Search API.

A list of supported identifier types can be found through the API. New identifier types are added to the requistry when requested by ORCID members. If you have a DOI prefix or other means of distinguising your identifiers from others, it may even be possible to find all occurances at once.

There are some detailed instructions on how to search the registry, complete with examples. The will work in the browser, or via the google playground and can be done against the live or sandbox environment.

• Try searching the live environment for a DOI or DOI prefix. To search for a doi you need to use something likehttps://pub.orcid.org/v2.0/search/?q=doi-self:10.6084%2FM9.FIGSHARE.4134027.V1 (note the / in the DOI is URL encoded to %2F). To search for a prefix, use something like https://pub.orcid.org/v2.0/search/?doi-self:10.6084*
• Try searching for the work you just added to your test account
• Experiment with other fields

7.4 More!

If you have time, there is an v2.0 API guide with XML samples and curl requests that you can use to add employment activities, external identifiers, biography elements and more.

-top-

8. REFERENCE MATERIALS

• See example implementations and workflow guides https://members.orcid.org
   
• Read technical documentation https://members.orcid.org/api
   
• v2.0 API guide with XML samples and curl requests
• Join the ORCID API Users Group https://groups.google.com/group/orcid-api-users
   
• Sign up for a Technical Webinar https://members.orcid.org/event-list
   
• Email the ORCID Community Engagement & Support support@orcid.org