Use Cases
1 Introduction
2 Authentication
2.1 Client Authentication
2.2 Back-end Authentication
3 Use cases
3.1 Content-Based Analysis
3.2 Content-Based Search
3.3 Facebook Profile Summarization
3.4 Facebook Profile Summarization Feedback
3.5 File Upload and Analysis
3.6 Id-Based Search
3.7 Keyword-Based Search
3.8 Similarity Feedback
3.9 Twitter Profile Summarization
3.10 User Feedback
3.11 User Feedback - Face Recognition
This page is contains use cases for the Content Analysis (CA) API. The purpose of the documentation is to illustrate the use of APIs needed to implement basic use cases required for content analysis ecosystem: uploading content; executing content analysis on the service back-ends; and retrieving content – or content metadata – based on search terms provided by a client (or user). In the scope of this documentation content is limited to photographs (image-based content).
In the current implementation, the client authentication process uses the default front-end built-in authentication scheme (username + password) when needed. The supported authentication methods are:
- HTTP Basic Authentication.
- OAuth/OAuth2 using one of the supported 3rd party services (Google, Facebook, Twitter).
- Use of TLS/HTTPS if so desired.
All 3rd party storage solutions (e.g. Picasa, Facebook) are authenticated by the front-end for back-ends by using URL redirection.
In the current implementation the back-ends are assumed to be on trusted zone, and thus, are not authenticated in any other way than using a simple id check or IP-based authentication. The back-ends can be authenticated by utilizing one the methods available to clients if required.
The use cases are illustrated by sequence diagrams, and further explained by REST method calls and example responses. The uses cases omit user authentication, but assume that authentication has been performed when required by the methods used in the use cases.
This use case is for scheduling analysis tasks on the analysis back-ends. The tasks are generated automatically when user connects an external photo account (e.g. Picasa), or when content of a previously connected account are synchronized with the front-end.
This example assumes that connecting an account has already been made by using the front-end. The connecting process is not explained as it is performed by the front-end’s internal functionality, and thus, it is out of scope of this documentation.
addTask (Front-end -> Analysis Back-end):
Example Query:
POST /rest/ca/addTask
Content-Type: text/xml; charset=UTF-8
[HTTP BODY STARTS]
<?xml version="1.0" encoding="UTF-8"?>
<taskDetails>
<taskId>1</taskId>
<taskType>ANALYSIS</taskType>
<backendId>0</backendId>
<userId>1</userId>
<userConfidence>1</userConfidence>
<callbackUri>http://fe.example.org/rest/ca/taskFinished</callbackUri>
<mediaList>
<media>
<UID>1</UID>
<serviceId>0</serviceId>
<userId>1</userId>
<url>http://fe.example.org/rest/ca/r?uid=1&service_id=0</url>
<credits>Creative Commons</credits>
<name>Photo</name>
<description>cute photo</description>
<visibility>PRIVATE</visibility>
<mediaType>PHOTO</mediaType>
<objectList>
<object>
<objectId>123</objectId>
<objectType>KEYWORD</objectType>
<status>CANDIDATE</status>
<backendId>0</backendId>
<value>cute</value>
<confidence>0.5</confidence>
<rank>0</rank>
<visualObjectId>1</visualObjectId>
<mediaType>PHOTO</mediaType>
</object>
</objectList>
</media>
</mediaList>
</taskDetails>
[HTTP BODY ENDS]
Example Result:
<?xml version="1.0" encoding="UTF-8"?>
<response method="addTask" service="ca">
<status>ok</status>
</response>
Retrieve photo (redirect) (Analysis Back-end -> Front-end):
Example Query:
GET /rest/ca/r?uid=1&service_id=0
Example Result:
REDIRECT/PHOTO CONTENT
taskFinished (Analysis Back-end -> Front-end):
Example Query:
POST /rest/ca/taskFinished
Content-Type: text/xml; charset=UTF-8
[HTTP BODY STARTS]
<?xml version="1.0" encoding="UTF-8"?>
<taskResults>
<taskId>1</taskId>
<taskType>ANALYSIS</taskType>
<backendId>0</backendId>
<status>COMPLETED</status>
<message>Analysis completed successfully.</message>
<mediaList>
<media>
<UID>1</UID>
<backendStatusList>
<backendStatus>
<backendId>0</backendId>
<status>COMPLETED</status>
<message>Completed successfully.</message>
</backendStatus>
</backendStatusList>
<objectList>
<object>
<objectId>123</objectId>
<objectType>KEYWORD</objectType>
<status>CANDIDATE</status>
<backendId>0</backendId>
<value>cute</value>
<confidence>0.5</confidence>
<rank>0</rank>
<mediaType>PHOTO</mediaType>
</object>
</objectList>
</media>
</mediaList>
</taskResults>
[HTTP BODY ENDS]
Example Result:
<?xml version="1.0" encoding="UTF-8"?>
<response method="taskFinished" service="ca">
<status>ok</status>
</response>
By using this method, the user can use a photo from external source to search similar photos from his/her photo collection, or from photos that other users have defined public.
similarPhotosByContent (Client -> Front-end):
Example Query:
POST /rest/ca/similarPhotosByContent?url=http%3A%2F%2Fexample.org%2Fimage.jpg&data_groups=all
similarPhotosByContent (Request; Front-end -> Analysis Back-end):
Example Query:
POST /rest/ca/similarPhotosByContent?url=http%3A%2F%2Fexample.org%2Fphoto.jpg&data_groups=all
Retrieve photo (direct access from the given URL):
Example Query:
GET /photo.jpg
Example Result:
PHOTO CONTENT
Final Result (Response; Analysis Back-end -> Front-end):
Final Search Result:
<?xml version="1.0" encoding="UTF-8"?>
<response method="similarPhotosByContent" service="ca">
<status>ok</status>
<mediaList>
<media>
<UID>1</UID>
<objectList>
<object>
<objectId>123</objectId>
<objectType>KEYWORD</objectType>
<status>CANDIDATE</status>
<backendId>0</backendId>
<value>cute</value>
<confidence>0.5</confidence>
<rank>0</rank>
<visualObjectId>1</visualObjectId>
<mediaType>PHOTO</mediaType>
</object>
</objectList>
</media>
Final Result (Front-end -> Client):
The results may include a combined result set received from multiple analysis back-ends.
In this example case only the results provided by a single back-end is shown.
Final Search Result to Client:
<?xml version="1.0" encoding="UTF-8"?>
<response method="similarPhotosByContent" service="ca">
<status>ok</status>
<mediaList>
<media>
<UID>1</UID>
<objectList>
<object>
<objectId>123</objectId>
<objectType>KEYWORD</objectType>
<status>CANDIDATE</status>
<backendId>0</backendId>
<value>cute</value>
<confidence>0.5</confidence>
<rank>0</rank>
<visualObjectId>1</visualObjectId>
<mediaType>PHOTO</mediaType>
</object>
</objectList>
</media>
</mediaList>
</response>
This use case is for scheduling summarization tasks on the analysis back-ends. The tasks are generated automatically when user connects an external account with profile content, or when content of a previously connected account are synchronized with the Front-end.
This example assumes that account connection has already been made using the Front-end. The connecting process is not explained as it is performed by The Front-end’s internal functionality, and thus, is out of scope of this documentation.
addTask (Front-end -> Summarizer):
Example Query:
POST /rest/fbj/addTask
Content-Type: text/xml; charset=UTF-8
[HTTP BODY STARTS]
<?xml version="1.0" encoding="UTF-8"?>
<taskDetails>
<taskId>1</taskId>
<taskType>FACEBOOK_PROFILE_SUMMARIZATION</taskType>
<backendId>0</backendId>
<userId>1</userId>
<callbackUri>http://fe.example.org/rest/ca/taskFinished</callbackUri>
<profile>
<groupList>
<group>
<name weight=”5”>Super Secret Society of Super Secrecy</name>
<description weight=”3”>Waaaaay too secret for public description</description>
<owner>Pohtii Block</owner>
<updatedTimestamp>2013-04-24T15:20:13+0300</updatedTimestamp>
<groupPrivacy>SECRET</groupPrivacy>
<location>
<name>San Francisco, California</name>
<country>USA</country>
<state>California</state>
<city>San Francisco</city>
</location>
</group>
</groupList>
<likeList>
<like>
<category>Product/service</category>
<name>Facebook Developers</name>
</like>
</likeList>
<statusMessageList>
<statusMessage>
<commentList>
<comment>
<createdTimestamp>2013-04-11T16:02:21+0300</createdTimestamp>
<from>Pohtii Block</from>
<message weight=”1”>ooo, a real facebook developer :O</message>
<likeCount>0</likeCount>
</comment>
</commentList>
<from>Mary Peettersson</from>
<message weight=”2”>uuu... I am a real developer now!</message>
<updatedTimestamp>2013-04-11T12:02:21+0300</updatedTimestamp>
<likeCount>0</likeCount>
</statusMessage>
</statusMessageList>
<photoDescriptionList>
<photoDescription>
<photoUID>0</photoUID>
<commentList>
<comment>
<createdTimestamp>2013-05-07T17:36:12+0300</createdTimestamp>
<from>Mary Peettersson</from>
<likeCount>1</likeCount>
<message weight=1”>testing facebook comments</message>
</comment>
</commentList>
<createdTimestamp>2013-04-11T16:07:28+0300</createdTimestamp>
<description weight=2”>Except for this picture, which also has Mona Lisa.</description>
<from>Mary Peettersson</from>
<likeCount>0</likeCount>
<location>
<city>Paris</city>
<country>France</country>
<name>Musée du Louvre</name>
</location>
<updatedTimestamp>2013-05-07T17:04:21+0300</updatedTimestamp>
<photoTagList>
<photoTag>
<value>Museum</value>
</photoTag>
</photoTagList>
</photoDescription>
</photoDescriptionList>
<videoDescriptionList>
<videoDescription>
<createdTimestamp>2013-04-11T16:07:28+0300</createdTimestamp>
<description weight=”2”>An example of a video description.</description>
<from>Mary Peettersson</from>
<updatedTimestamp>2013-05-07T17:04:21+0300</updatedTimestamp>
</videoDescription>
</videoDescriptionList>
<eventList>
<event>
<RSVPStatus>ATTENDING</RSVPStatus>
<location>
<name>The Moon</name>
</location>
<owner>Pohtii Block</owner> <!-- owner of this event -->
<description weight=”3”>The coolest party on this wo... outside this world</description>
<name weight=”5”>Väärä's BDay Party</name>
<startTimestamp>2013-06-08T00:00:00+0300</startTimestamp>
<endTimestamp>2013-06-09T00:05:00+0300</endTimestamp>
<updatedTimestamp>2013-05-07T00:05:00+0300</updatedTimestamp>
<groupPrivacy>SECRET</groupPrivacy>
</event>
</eventList>
<userDetails>
<bio>I'm a fun loving girl. I like sports, camping and kids. Here ends the
first chapter of my bio. Here begins the next chapter of my bio. Just typing
here some stuff about me. Pretty cool, eh?</bio>
<birthday>1992-02-07T00:00:00+0200</birthday>
<gender>FEMALE</gender>
<hometown>Jakarta, Indonesia</hometown>
<location>
<name>London, United Kingdom</name>
</location>
<name>Peettersson, Mary</name>
<political>Republican Party</political>
<relationshipList>
<relationship>
<relationshipType>Married</relationshipType>
<with>Peter Peettersson</with>
</relationship>
</relationshipList>
<religion>Catholic</religion>
<updatedTimestamp>2013-05-07T17:04:21+0300</updatedTimestamp>
<userId>3</userId> <!-- Front-end UserId for this FB User -->
<facebookId>0</facebookId>
</userDetails>
</profile>
</taskDetails>
[HTTP BODY ENDS]
Example Result:
<?xml version="1.0" encoding="UTF-8"?>
<response method="addTask" service="fbj">
<status>ok</status>
</response>
taskFinished (Summarizer -> Front-end):
Example Query:
POST /rest/fbj/taskFinished
Content-Type: text/xml; charset=UTF-8
[HTTP BODY STARTS]
<?xml version="1.0" encoding="UTF-8"?>
<taskResults>
<taskId>1</taskId>
<taskType>FACEBOOK_PROFILE_SUMMARIZATION</taskType>
<backendId>0</backendId>
<status>COMPLETED</status>
<message>Summarization completed successfully.</message>
<objectList>
<object>
<objectId>123</objectId>
<objectType>KEYWORD</objectType>
<status>CANDIDATE</status>
<backendId>0</backendId>
<value>cute</value>
<confidence>9.8058747237</confidence>
<rank>0</rank>
</object>
</objectList>
<mediaList>
<media>
<UID>0</UID>
<objectList>
<object>
<objectId>124</objectId>
<objectType>KEYWORD</objectType>
<status>CANDIDATE</status>
<backendId>0</backendId>
<value>biking</value>
<confidence>4.58747237</confidence>
<rank>0</rank>
<mediaType>PHOTO</mediaType>
</object>
</objectList>
</media>
</mediaList>
</taskResults>
[HTTP BODY ENDS]
Example Result:
<?xml version="1.0" encoding="UTF-8"?>
<response method="taskFinished" service="fbj">
<status>ok</status>
</response>
This use case is for scheduling summarization feedback tasks on the analysis back- ends. The tasks are generated automatically when gives feedback (i.e. ranks the tags) on the previously generated tags.
setRank (Client -> Front-end):
Example Query:
POST /rest/fbj/setRank?rank=1:100,2:-10
Example Result:
<?xml version="1.0" encoding="UTF-8"?>
<response service="fbj" method="setRank">
<status>ok</status>
</response>
addTask (Front-End -> Summarizer):
Example Query:
POST /rest/fbj/addTask
Content-Type: text/xml; charset=UTF-8
[HTTP BODY STARTS]
<?xml version="1.0" encoding="UTF-8"?>
<taskDetails>
<taskId>1</taskId>
<taskType>FACEBOOK_PROFILE_SUMMARIZATION</taskType>
<backendId>0</backendId>
<userId>1</userId>
<callbackUri>http://fe.example.org/rest/ca/taskFinished</callbackUri>
<objectList>
<object>
<objectId>123</objectId>
<objectType>KEYWORD</objectType>
<status>CANDIDATE</status>
<backendId>0</backendId>
<value>cute</value>
<confidence>9.8058747237</confidence>
<rank>0</rank>
<visualObjectId>1</visualObjectId>
</object>
</objectList>
</taskDetails>
[HTTP BODY ENDS]
Example Result:
<?xml version="1.0" encoding="UTF-8"?>
<response method="addTask" service="fbj">
<status>ok</status>
</response>
Note that the Facebook Summarization Feedback tasks may not produce any results, and thus, no further communication is needed. Responding with taskFinished is not required. Responding with a proper status code without results is accepted and will simply update the analysis status on the front-end.
This use case is for uploading a file to the service and starting a photo analysis task for the file. The example assumes that the uploaded file is an image file of a supported format. This example use case utilizes the default back-ends provided by the system, and omits the parts of the process involving back-ends for simplification.
This example omits the analysis task creation and delivery to the analysis back-ends.
addFile (Client -> Front-end):
Example Query:
POST /rest/fs/addFile
[HTTP BODY STARTS]
[File Contents]
[HTTP BODY ENDS]
Example Result:
<?xml version="1.0" encoding="UTF-8"?>
<response method="addFile" service="fs">
<fileList>
<file>
<fileId>0</fileId>
<url>http://fe.example.org/00000000-abcd-efgh-ijkl-000000000000.jpg</url>
</file>
</fileList>
<status>ok</status>
</response>
Add received URL to analysis (Client -> Front-end):
Example Query:
POST /rest/cos/addUrl?url=http%3A%2F%2Ffe.example.org%2F00000000-abcd-efgh-ijkl-000000000000.jpg
Example Result:
<?xml version="1.0" encoding="UTF-8"?>
<response method="addUrl" service="cos">
<status>ok</status>
</response>
This method requires the photo UID, which is automatically assigned to each uploaded photo by the front-end. Thus, the method can only use previously uploaded photos as reference. If the user is not authenticated, only UIDs of public photos can be given as reference and the search will only return photos that are defined public by the users.
similarPhotosById (Client -> Front-end):
Example Query:
POST /rest/ca/similarPhotosById?uid=1&data_groups=all
similarPhotosById (Request; Front-end -> Analysis Back-end):
Example Query:
POST /rest/ca/similarPhotosById?uid=1&data_groups=all
Final Result (Response; Analysis Back-end -> Front-end):
Final Search Result:
<?xml version="1.0" encoding="UTF-8"?>
<response method="similarPhotosById" service="ca">
<status>ok</status>
<mediaList>
<media>
<UID>1</UID>
<objectList>
<object>
<objectId>123</objectId>
<objectType>KEYWORD</objectType>
<status>CANDIDATE</status>
<backendId>0</backendId>
<value>cute</value>
<confidence>0.5</confidence>
<rank>0</rank>
<visualObjectId>1</visualObjectId>
<mediaType>PHOTO</mediaType>
</object>
</objectList>
</media>
</mediaList>
</response>
Final Result (Front-end -> Client):
The results may include a combined result set received from multiple analysis
back-ends. In this example case only the results provided by a single back-end
is shown.
Final Search Result to Client:
<?xml version="1.0" encoding="UTF-8"?>
<response method="similarPhotosById" service="ca">
<status>ok</status>
<mediaList>
<media>
<UID>1</UID>
<objectList>
<object>
<objectId>123</objectId>
<objectType>KEYWORD</objectType>
<status>CANDIDATE</status>
<backendId>0</backendId>
<value>cute</value>
<confidence>0.5</confidence>
<rank>0</rank>
<visualObjectId>1</visualObjectId>
<mediaType>PHOTO</mediaType>
</object>
</objectList> Tampere University of Technology 24 (45)
</media>
</mediaList>
</response>
By using the keyword based search, the user can search photos which are related to each other by keywords.
In this example, the optional result_info data group is used to request
additional information about the search results.
Example Query:
GET /rest/ca/similarPhotosByKeyword?keywords=cute&data_groups=all,result_info
Example Result:
<?xml version="1.0" encoding="UTF-8"?>
<response method="similarPhotosByKeyword" service="ca">
<status>ok</status>
<mediaList>
<resultInfo>
<resultCount>1</resultCount>
</resultInfo>
<media>
<UID>1</UID>
<serviceId>0</serviceId>
<userId>1</userId>
<url>http://fe.example.org/rest/ca/r?uid=1&service_id=0</url>
<credits>Creative Commons</credits>
<name>Photo</name>
<description>cute photo</description>
<visibility>PRIVATE</visibility>
<mediaType>PHOTO</mediaType>
<objectList>
<object>
<objectId>123</objectId>
<objectType>KEYWORD</objectType> Tampere University of Technology 26 (45)
<status>CANDIDATE</status>
<backendId>0</backendId>
<value>cute</value>
<confidence>0.5</confidence>
<rank>0</rank>
<visualObjectId>1</visualObjectId>
<mediaType>PHOTO</mediaType>
</object>
</objectList>
</media>
</mediaList>
</response>
User can submit feedback on the similarities of photos. The feedback does not have
to relate to an existing search query or task, though in general the feedback is
provided for a search query. This use case uses similarity search (ID-Based Search) as an example, but omits the front-end <-> back-end phase (with
responses) of the similarity search.
The feedback is sent using the similarityFeedback method, which in turn, will trigger
the generation of feedback task, which is sent to analysis back-ends. As with the
regular Feedback Task ((User Feedback](#310-user-feedback)) the back-end could reply with additional
taskFinished call to update metadata on the front-end.
similarPhotosById (Client -> Front-end):
Example Query:
POST /rest/ca/similarPhotosById?uid=1&data_groups=all
Example Result:
<?xml version="1.0" encoding="UTF-8"?>
<response method="similarPhotosById" service="ca">
<status>ok</status>
<mediaList>
<media>
<UID>1</UID>
<serviceId>0</serviceId>
<userId>1</userId>
<url>http://fe.example.org/rest/ca/r?uid=1&service_id=0</url>
<credits>Creative Commons</credits>
<name>Photo</name>
<description>cute photo</description>
<visibility>PRIVATE</visibility>
<mediaType>PHOTO</mediaType>
<objectList>
<object>
<objectId>123</objectId>
<objectType>KEYWORD</objectType>
<status>CANDIDATE</status>
<backendId>0</backendId>
<value>cute</value>
<confidence>0.5</confidence>
<rank>0</rank>
<visualObjectId>1</visualObjectId>
<mediaType>PHOTO</mediaType>
</object>
</objectList>
</media>
<media>
<UID>2</UID>
<serviceId>0</serviceId>
<userId>1</userId>
<url>http://fe.example.org/rest/ca/r?uid=2&service_id=0</url>
<credits>Creative Commons</credits>
<name>Photo</name>
<description>cute photo</description>
<visibility>PRIVATE</visibility>
<mediaType>PHOTO</mediaType>
<objectList>
<object>
<objectId>124</objectId>
<objectType>KEYWORD</objectType>
<status>USER_CONFIRMED</status>
<backendId>0</backendId>
<value>cute</value>
<confidence>0.7</confidence>
<rank>0</rank>
<visualObjectId>1</visualObjectId>
<mediaType>PHOTO</mediaType>
</object>
</objectList>
</media>
</mediaList>
</response>
In practice, the query is first send to applicable back-ends, and based on the
back-end responses the client response is generated. This phase is omitted for
simplicity. See the example in id-based search for full listing (section 3.6).
similarityFeedback (Client -> Front-end):
Example Query:
POST /rest/ca/similarityFeedback
Content-Type: text/xml; charset=UTF-8
[HTTP BODY STARTS]
<?xml version="1.0" encoding="UTF-8"?>
<feedbackList>
<mediaList>
<media>
<UID>1</UID>
<serviceId>0</serviceId>
<userId>1</userId>
<url>http://fe.example.org/rest/ca/r?uid=1&service_id=0</url>
<visibility>PRIVATE</visibility>
<mediaType>PHOTO</mediaType>
</media>
</mediaList>
<similarMediaList>
<media>
<UID>2</UID>
<serviceId>0</serviceId>
<userId>1</userId>
<url>http://fe.example.org/rest/ca/r?uid=2&service_id=0</url>
<visibility>PRIVATE</visibility>
<mediaType>PHOTO</mediaType>
</media>
<UID>3</UID>
<serviceId>0</serviceId>
<userId>1</userId>
<url>http://fe.example.org/rest/ca/r?uid=3&service_id=0</url>
<visibility>PRIVATE</visibility>
<mediaType>PHOTO</mediaType>
</media>
</similarMediaList>
<dissimilarMediaList>
<media>
<UID>4</UID>
<serviceId>0</serviceId>
<userId>2</userId>
<url>http://fe.example.org/rest/ca/r?uid=4&service_id=0</url>
<visibility>PRIVATE</visibility>
<mediaType>PHOTO</mediaType>
</media>
</dissimilarMediaList>
</feedbackList>
[HTTP BODY ENDS]
Note that the feedback items are not limited to the content of the original query. In this example the client provides similarity feedback containing photos with UID 3 and 4 even though those photos are not part of the original search results. The only limitation is that the client (or user) has the appropriate permissions to access the content provided in the feedback.
Example Result:
<?xml version="1.0" encoding="UTF-8"?>
<response method="similarityFeedback" service="ca">
<status>ok</status>
</response>
addTask (Front-end -> Analysis Back-end):
Example Query:
POST /rest/ca/addTask
Content-Type: text/xml; charset=UTF-8
[HTTP BODY STARTS]
<?xml version="1.0" encoding="UTF-8"?>
<taskDetails>
<taskId>1</taskId>
<taskType>FEEDBACK</taskType>
<backendId>0</backendId>
<userId>1</userId>
<userConfidence>1</userConfidence>
<callbackUri>http://fe.example.org/rest/ca/taskFinished</callbackUri>
<mediaList>
<media>
<UID>1</UID>
<serviceId>0</serviceId>
<userId>1</userId>
<url>http://fe.example.org/rest/ca/r?uid=1&service_id=0</url>
<visibility>PRIVATE</visibility>
<mediaType>PHOTO</mediaType>
</media>
</mediaList>
<similarMediaList>
<media>
<UID>2</UID>
<serviceId>0</serviceId>
<userId>1</userId>
<url>http://fe.example.org/rest/ca/r?uid=2&service_id=0</url>
<visibility>PRIVATE</visibility>
<mediaType>PHOTO</mediaType>
</media>
<UID>3</UID>
<serviceId>0</serviceId>
<userId>1</userId>
<url>http://fe.example.org/rest/ca/r?uid=3&service_id=0</url>
<visibility>PRIVATE</visibility>
<mediaType>PHOTO</mediaType>
</media>
</similarMediaList>
<dissimilarMediaList>
<media>
<UID>4</UID>
<serviceId>0</serviceId>
<userId>2</userId>
<url>http://fe.example.org/rest/ca/r?uid=4&service_id=0</url>
<visibility>PRIVATE</visibility>
<mediaType>PHOTO</mediaType>
</media>
</dissimilarMediaList>
</taskDetails>
[HTTP BODY ENDS]
Example Result:
<?xml version="1.0" encoding="UTF-8"?>
<response method="addTask" service="ca">
<status>ok</status>
</response>
This use case is for scheduling summarization tasks on the analysis back-ends. The tasks are generated automatically when user connects an external account with profile content, or when content of a previously connected account are synchronized with the front-end.
This example assumes that account connection has already been made using the Front-end. The connecting process is not explained as it is performed by The Front-end’s internal functionality, and thus, is out of scope of this documentation.
addTask (Front-end -> Summarizer):
Example Query:
POST /rest/ca/addTask
Content-Type: text/xml; charset=UTF-8
[HTTP BODY STARTS]
<?xml version="1.0" encoding="UTF-8"?>
<taskDetails>
<taskId>1</taskId>
<taskType>TWITTER_PROFILE_SUMMARIZATION</taskType>
<backendId>0</backendId>
<userId>1</userId>
<callbackUri>http://fe.example.org/rest/ca/taskFinished</callbackUri>
<profile>
<userDetails>
<bio>czhXX</bio>
<favoritesCount>1992245036</favoritesCount>
<followersCount>993252504</followersCount>
<friendsCount>296699425</friendsCount>
<twitterId>1</twitterId>
<name>kfZmBq</name>
<screenName>bBsc13UxU</screenName>
<location>
<name>wlRMa</name>
</location>
<userId>8367131745868141232</userId>
</userDetails>
<statusMessageList>
<statusMessage>
<from>S9QNEwFnR</from>
<message>#BQtQp #KqT #rFA22 cg55 #su99 LIA 7PtLs #MAZv #4t9qBc6 zCMWL #AC1lyE #L29 XocJ9 9b6UzWWd0 #vUmUdItP aSZp5NuK KKnIfo soR MJVBXJC5 CRFvw0S</message>
<updatedTimestamp>1991-04-06T09:58:19+0300</updatedTimestamp>
</statusMessage>
</statusMessageList>
<photoDescriptionList>
<photoDescription>
<photoUID>55ba2f66-4f20-4843-9fba-0382c85f9ab1</photoUID>
<createdTimestamp>2014-03-30T09:53:13+0300</createdTimestamp>
<description>#83tGRK #AFT0 sI1Xi hWRNsR #PHT qWxF 3LB ZVP1n81P TIIM HUpp4Tp Mfxpw86eh WittXhz #WQ5MTP #bxabX #Uykk2RP #2iDgkKP xfyMR #uYfLyvP qJhZugRb3</description>
<from>gf6UaPve</from>
<source>vrYi</source>
<location>
<latitude>-22.0</latitude>
<longitude>132.0</longitude>
</location>
</photoDescription>
</photoDescriptionList>
<videoDescriptionList>
<videoDescription>
<createdTimestamp>1980-03-06T23:28:45.293+02:00</createdTimestamp>
<from>LLNb2Yq</from>
<description>oCKh7Gz #tgJN0M7qm ez4s2 h4Zeo GiY #olfT8l pk9IuioPd #k3H7A gA6rPka 74pZSVy1t #DDefKiP0a #nz0 #eyxNl4I GVJgBmRL P53oo Y9Jz3L BNqEpkCwi</description>
<location>
<latitude>50.0</latitude>
<longitude>88.0</longitude>
</location>
</videoDescription>
</videoDescriptionList>
</profile>
</taskDetails>
[HTTP BODY ENDS]
Example Result:
<?xml version="1.0" encoding="UTF-8"?>
<response method="addTask" service="ca">
<status>ok</status>
</response>
taskFinished (Summarizer -> Front-end):
Example Query:
POST /rest/ca/taskFinished
Content-Type: text/xml; charset=UTF-8
[HTTP BODY STARTS]
<?xml version="1.0" encoding="UTF-8"?>
<taskResults>
<taskId>1</taskId>
<taskType>TWITTER_PROFILE_SUMMARIZATION</taskType>
<backendId>0</backendId>
<status>COMPLETED</status>
<message>Summarization completed successfully.</message>
<objectList>
<object>
<objectId>123</objectId>
<objectType>KEYWORD</objectType>
<status>CANDIDATE</status>
<backendId>0</backendId>
<value>cute</value>
<confidence>9.8058747237</confidence>
<rank>0</rank>
</object>
</objectList>
<mediaList>
<media>
<UID>0</UID>
<objectList>
<object>
<objectId>124</objectId>
<objectType>KEYWORD</objectType>
<status>CANDIDATE</status>
<backendId>0</backendId>
<value>biking</value>
<confidence>4.58747237</confidence>
<rank>0</rank>
<mediaType>PHOTO</mediaType>
</object>
</objectList>
</media>
</mediaList>
</taskResults>
[HTTP BODY ENDS]
Example Result:
<?xml version="1.0" encoding="UTF-8"?>
<response method="taskFinished" service="ca">
<status>ok</status>
</response>
When user updates the details of his or her photos, a feedback task may be generated by the front-end. The analysis back-ends may or may not respond to the given feedback. If the feedback affected photos that were not included in the original task, these can be added by the analysis back-end to the results (taskFinished method). If the analysis back-end does not have anything to respond, it does not need to call taskFinished method. Responding with a proper status code without results is accepted and will simply update the task status on the front-end.
updatePhotos (Client -> Front-end):
The feedback process is initialized by the user (or client) by calling updatePhotos on the front-end, which will trigger the generation of applicable Feedback Tasks, if needed. In this example the user has both rejected and confirmed a keyword. Note that the second object has name element, which can be used to declare clear text description of a keyword. This description can be added by the user, or by the system.
Example Query:
POST /rest/ca/updatePhotos
Content-Type: text/xml; charset=UTF-8
[HTTP BODY STARTS]
<?xml version="1.0" encoding="UTF-8"?>
<mediaList>
<media>
<UID>1</UID>
<serviceId>0</serviceId>
<userId>1</userId>
<url>http://example.org/photo.jpg</url>
<credits>Creative Commons</credits>
<name>Photo</name>
<description>ugly photo</description>
<visibility>PRIVATE</visibility>
<mediaType>PHOTO</mediaType>
<objectList>
<object>
<objectId>123</objectId>
<objectType>KEYWORD</objectType>
<status>USER_REJECTED</status>
<backendId>0</backendId>
<value>cute</value>
<confidence>1</confidence>
<rank>0</rank>
<visualObjectId>1</visualObjectId>
<mediaType>PHOTO</mediaType>
</object>
<object>
<objectId>124</objectId>
<objectType>KEYWORD</objectType>
<status>USER_CONFIRMED</status>
<backendId>0</backendId>
<value>ugly</value>
<name>hideous</name>
<confidence>1</confidence>
<rank>0</rank>
<visualObjectId>1</visualObjectId>
<mediaType>PHOTO</mediaType>
</object>
</objectList>
</media>
</mediaList>
[HTTP BODY ENDS]
Example Result:
<?xml version="1.0" encoding="UTF-8"?>
<response method="updatePhotos" service="ca">
<status>ok</status>
</response>
addTask (Front-end -> Analysis Back-end):
Example Query:
POST /rest/ca/addTask
Content-Type: text/xml; charset=UTF-8
[HTTP BODY STARTS]
<?xml version="1.0" encoding="UTF-8"?>
<taskDetails>
<taskId>1</taskId>
<taskType>FEEDBACK</taskType>
<backendId>0</backendId>
<userId>1</userId>
<userConfidence>1</userConfidence>
<callbackUri>http://fe.example.org/rest/ca/taskFinished</callbackUri>
<mediaList>
<media>
<UID>1</UID>
<serviceId>0</serviceId>
<userId>1</userId>
<url>http://fe.example.org/rest/ca/r?uid=1&service_id=0</url>
<credits>Creative Commons</credits>
<name>Photo</name>
<description>ugly photo</description>
<visibility>PRIVATE</visibility>
<mediaType>PHOTO</mediaType>
<objectList>
<object>
<objectId>123</objectId>
<objectType>KEYWORD</objectType>
<status>USER_REJECTED</status>
<backendId>0</backendId>
<value>cute</value>
<confidence>1</confidence>
<rank>0</rank>
<visualObjectId>1</visualObjectId>
<mediaType>PHOTO</mediaType>
</object>
<object>
<objectId>124</objectId>
<objectType>KEYWORD</objectType>
<status>USER_CONFIRMED</status>
<backendId>0</backendId>
<value>ugly</value>
<name>hideous</name>
<confidence>1</confidence>
<rank>0</rank>
<visualObjectId>1</visualObjectId>
<mediaType>PHOTO</mediaType>
</object>
</objectList>
</media>
</mediaList>
</taskDetails>
[HTTP BODY ENDS]
Example Result:
<?xml version="1.0" encoding="UTF-8"?>
<response method="addTask" service="ca">
<status>ok</status>
</response>
taskFinished (Analysis Back-end -> Front-end):
Example Query:
POST /rest/ca/taskFinished
Content-Type: text/xml; charset=UTF-8
[HTTP BODY STARTS]
<?xml version="1.0" encoding="UTF-8"?>
<taskResults>
<taskId>1</taskId>
<taskType>FEEDBACK</taskType>
<backendId>0</backendId>
<status>COMPLETED</status>
<message>Feedback affected other photos.</message>
<mediaList>
<media>
<UID>2</UID>
<serviceId>0</serviceId>
<userId>1</userId>
<url>http://fe.example.org/rest/ca/r?uid=2&service_id=0</url>
<backendStatusList>
<backendStatus>
<backendId>0</backendId>
<status>COMPLETED</status>
<message>Keyword updated based on user feedback.</message>
</backendStatus>
</backendStatusList>
<objectList>
<object>
<objectId>125</objectId>
<objectType>KEYWORD</objectType>
<status>CANDIDATE</status>
<backendId>0</backendId>
<value>ugly</value>
<confidence>0.75</confidence>
<rank>0</rank>
<visualObjectId>1</visualObjectId>
<mediaType>PHOTO</mediaType>
</object>
</objectList>
</media>
</mediaList>
</taskResults>
[HTTP BODY ENDS]
Example Result:
<?xml version="1.0" encoding="UTF-8"?>
<response method="taskFinished" service="ca">
<status>ok</status>
</response>
Content-Based Analysis (Content-Based Analysis) combined with User Feedback (User Feedback). The only difference is in the transferred visual object metadata. As with the regular feedback, the back-end may (if it wants to update previously submitted metadata, or submit new metadata) or may not respond to the feedback provided by the user by calling taskFinished on the front-end. Responding with a proper status code without results is accepted and will simply update the task status on the front-end.
taskFinished (Analysis Back-end -> Front-end):
Note that the user feedback is not initiated by the taskFinished call from the analysis back-end. In the example case the first taskFinished call simply illustrates the fact that the original metadata have been (in this use case) been provided by a finished analysis task. The feedback task is generated based on the updated metadata provided by the user in the updatePhotos call. In this case, the initial metadata provided by the back-end is an unknown face, as shown in the example code below (the object element could also include an empty value element).
Example Query:
POST /rest/ca/taskFinished
Content-Type: text/xml; charset=UTF-8
[HTTP BODY STARTS]
<?xml version="1.0" encoding="UTF-8"?>
<taskResults>
<taskId>0</taskId>
<taskType>ANALYSIS</taskType>
<backendId>0</backendId>
<status>COMPLETED</status>
<message>Analysis completed successfully.</message>
<mediaList>
<media>
<UID>2</UID>
<backendStatusList>
<backendStatus>
<backendId>0</backendId>
<status>COMPLETED</status>
<message>Recognized an unknown face.</message>
</backendStatus>
</backendStatusList>
<objectList>
<object>
<objectId>124</objectId>
<objectType>FACE</objectType>
<status>CANDIDATE</status>
<backendId>0</backendId>
<confidence>0.75</confidence>
<rank>0</rank>
<mediaType>PHOTO</mediaType>
</object>
</objectList>
</media>
</mediaList>
</taskResults>
[HTTP BODY ENDS]
Example Result:
<?xml version="1.0" encoding="UTF-8"?>
<response method="taskFinished" service="ca">
<status>ok</status>
</response>
getPhotos (Client -> Front-end):
Note that the client could have received the metadata of the photo in question by any number of ways (such as a result of a search query), and getPhotos is used here only for an example.
Example Query:
GET /rest/ca/getPhotos?uid=2&datagroups=all
Example Result:
<?xml version="1.0" encoding="UTF-8"?>
<response method="getPhotos" service="ca">
<status>ok</status>
<mediaList>
<media>
<UID>2</UID>
<serviceId>0</serviceId>
<userId>1</userId>
<url>http://fe.example.org/rest/ca/r?uid=2&service_id=0</url>
<visibility>PRIVATE</visibility>
<mediaType>PHOTO</mediaType>
<objectList>
<object>
<objectId>124</objectId>
<objectType>FACE</objectType>
<status>CANDIDATE</status>
<backendId>0</backendId>
<confidence>0.75</confidence>
<rank>0</rank>
<visualObjectId>1</visualObjectId>
<mediaType>PHOTO</mediaType>
</object>
</objectList>
</media>
</mediaList>
</response
updatePhotos (Client -> Front-end):
If the user wants to update the recognized face with a proper value (name, “Jack”, in the example), he can do so by using the updatePhotos method, as seen in the example below. User can also return other status information, such as “USER_REJECTED”, if he/she deems that the information provided by the back-end was incorrect.
Example Query:
POST /rest/ca/updatePhotos
Content-Type: text/xml; charset=UTF-8
[HTTP BODY STARTS]
<?xml version="1.0" encoding="UTF-8"?>
<mediaList>
<media>
<UID>2</UID>
<serviceId>0</serviceId>
<userId>1</userId>
<visibility>PRIVATE</visibility>
<mediaType>PHOTO</mediaType>
<objectList>
<object>
<objectId>124</objectId>
<objectType>FACE</objectType>
<status>USER_CONFIMED</status>
<backendId>0</backendId>
<value>Jack</value>
<confidence>1</confidence>
<rank>0</rank>
<visualObjectId>1</visualObjectId>
<mediaType>PHOTO</mediaType>
</object>
</objectList>
</media>
</mediaList>
[HTTP BODY ENDS]
Example Result:
<?xml version="1.0" encoding="UTF-8"?>
<response method="updatePhotos" service="ca">
<status>ok</status>
</response>
addTask (Front-end -> Analysis Back-end):
Example Query:
POST /rest/ca/addTask
Content-Type: text/xml; charset=UTF-8
[HTTP BODY STARTS]
<?xml version="1.0" encoding="UTF-8"?>
<taskDetails>
<taskId>1</taskId>
<taskType>FEEDBACK</taskType>
<backendId>0</backendId>
<userId>1</userId>
<userConfidence>1</userConfidence>
<callbackUri>http://fe.example.org/rest/ca/taskFinished</callbackUri>
<mediaList>
<media>
<UID>2</UID>
<serviceId>0</serviceId>
<userId>1</userId>
<url>http://fe.example.org/rest/ca/r?uid=2&service_id=0</url>
<visibility>PRIVATE</visibility>
<mediaType>PHOTO</mediaType>
<objectList>
<object>
<objectId>124</objectId>
<objectType>FACE</objectType>
<status>USER_CONFIMED</status>
<backendId>0</backendId>
<value>Jack</value>
<confidence>1</confidence>
<rank>0</rank>
<visualObjectId>1</visualObjectId>
<mediaType>PHOTO</mediaType>
</object>
</objectList>
</media>
</mediaList>
</taskDetails>
[HTTP BODY ENDS]
Example Result:
<?xml version="1.0" encoding="UTF-8"?>
<response method="addTask" service="ca">
<status>ok</status>
</response>
taskFinished (Analysis Back-end -> Front-end):
Based on the feedback, the analysis back-end may update the visual objects it has previously submitted by calling taskFinished. In the example, if visual object with objectId 125 already exists, it is updated, if not, new object is associated with the photo (with objectId=125). In the example, another face with value “Jack” has been found in another photo.
Example Query:
POST /rest/ca/taskFinished
Content-Type: text/xml; charset=UTF-8
[HTTP BODY STARTS]
<?xml version="1.0" encoding="UTF-8"?>
<taskResults>
<taskId>1</taskId>
<taskType>FEEDBACK</taskType>
<backendId>0</backendId>
<status>COMPLETED</status>
<message>Feedback affected other photos.</message>
<mediaList>
<media>
<UID>3</UID>
<serviceId>0</serviceId>
<userId>1</userId>
<url>http://fe.example.org/rest/ca/r?uid=3&service_id=0</url>
<status>COMPLETED</status>
<message>Face value updated based on user feedback.</message>
<objectList>
<object>
<objectId>125</objectId>
<objectType>KEYWORD</objectType>
<status>CANDIDATE</status>
<backendId>0</backendId>
<value>Jack</value>
<confidence>0.75</confidence>
<rank>0</rank>
<visualObjectId>1</visualObjectId>
</object>
</objectList>
</media>
</mediaList>
</taskResults>
[HTTP BODY ENDS]
Example Result:
<?xml version="1.0" encoding="UTF-8"?>
<response method="taskFinished" service="ca">
<status>ok</status>
</response>