~GSoC2010: nmudgal's Data Source Input Plugin

anildash edited this page Mar 26, 2011 · 1 revision

Nitin Mudgal
irc: nmudgal

Goal :
To create ThinkTank plugins that can pull data from social networks like LinkedIn & YouTube.
Since ThinkTank is right now used to fetch twitter status updates so it would be a good deal to make such that it can fetch data from other social networks too.
Substantial work has already been done on the support of social network since facebook has been implemented & bit.ly is used for shortening URL.

The goal is to enhance the present scalability & capability of the ThinkTank . Due to lack of various social support , the present model do not serve the community to the horizon. The proposed system would elaborate the it’s scalability so that it would be easier for everyone to see their updates of various social network account in one place .
The popularity and ease of APIs provides various API for the project. Once done it can be can be implemented for other networks too.

What exactly is it ?

It is pack of plugin which includes LinkedIn & YouTube plugin which are written using APIs provided by these organisations that aggregates data from & show up to make it easier for user & increases scalability of ThinkTank.

How ?
Application programming interface (API) is an interface implemented by a software program to enable interaction with other software, similar to the way a user interface facilitates interaction between humans and computers. It has allowed web communities to create an architecture for sharing content and data between communities and applications. Actually , the content that is created in one place can be dynamically posted and/or in multiple locations :
Like Content can be shared from sites like LinkedIn to social networking sites like Facebook etc.
So this project will includes various APIs in PHP for pulling data. My highest priority list includes :

  • Youtube Integration : The API provided by Youtube Data API allows client applications to retrieve and update YouTube content in the form of Google Data API feeds.Client application can use the YouTube Data API to fetch video feeds, comments, responses, and playlists, as well as query for videos that match particular criteria. The API can also be used to make authenticated requests to modify this information and to upload new video content to the site.

How to : Since this plugin requires need to use Zend_Gdata >= 1.7.7, which is distributed as part of the Zend Framework.

  1. Authentication : All API operations that do not involve retrieving public content will require authentication only. A crawler plugin,in which PHP client library will be used or can be used to retrieve public feeds or to execute authenticated operations. Since public feeds are read only, whereas after authentication , this includes the retrieval of privated feeds which will include write, upload update and delete operations.This crawler will include AuthSub proxy authentication method with will not require users to access their YouTube username & password actually. this will include a method getAuthSubRequestUrl which will point the user to a Google Login page & after authorization redirect back to the application. This will retrieve a nonsecure token that will be exchanged for a session token , which will not expire & can the be used multiple times. A fully authenticated service can be created by passing the $httpClient object to the Zend_Gdata_YouTube service object’s constructor & it also need to pass an ID for the application, dev key and client ID to the constructor. The developer key(dev key)is only necessary if private feed retrieval is done.
  2. UI : Extending the front administration page template , a UI through which administrator can make configuration respective to accounts, this will include a template directory under which application.account.index.tpl , schema design for storing this information .
  3. Retrieval : lib/ directory will include files to display a feed of videos after determining if a user can edit a video entry.This will include a file which will have a method getAndPrintVideoFeed will will prints information about each entry in the feed & stores it in the database. A separate file which will will check for user authentication. By default the getVideoEntry method will return a read only video entry. By passing true , this method can retrieve an editable video entry. A class here will also include method getAndPrintStandardFeeds for retrieval for standard categorized feeds which will erase the need of calling of crawl function to retrieve data since this will be done simultaneously after authentication as feeds will be modeled most simply as Zend_Gdata_YouTube_VideoFeed (VideoFeed) objects, each of which contains a number of Zend_Gdata_YouTube_VideoEntry (VideoEntry) objects. Each video entry corresponds to exactly one YouTube video and contains information about that video or class that included method for categorized feed retrieval will reflect YouTube user behavior, such as top-rated and most viewed video feeds, or were selected by YouTube staff, such as recently featured and mobile video feeds rather than user-specific. Crawler will be only used to authenticate to services & each time if user like to get update he/she has to auth to services & updates will be retrieved using various methods after it .
    Since there is good & huge documentation to work with so many other methods for searching , uploading & deleting videos , user profile integration can be done.
  • LinkedIn Integration : Using REST/HTTP API provided by this network professional network can be directly added to the app.That will provide
    • Provide a better experience
    • Increase productive engagement
    • Increase your reach
  1. Authentication: Same as others firstly user will request for an api key on linkedIn network, which will be passed into the crawler file to auth to services. This crawler is used to build a feature in the app that leverages the user’s linkedin network.User click on to make a request & then in turn a reques token will be passed. & LInkIn replies with an OAuth Token indicating that user can use the authentication system.Further the user grants access to app by signing into the page & upon successful sigon, LinkedIn will return the user . Then this will make a call to LInkedIn to get an Access Token. Then an Access Token for any API call so LinkedIn for identification . Nutshell, An API key, for identifying You & Your application & Access Token, Identifying the user’s account you are requesting the information from.
    Authentication system is as same as twitter OAuth Flow.
  2. UI : Extending the front administration page template , a UI through which administrator can make configuration respective to accounts, this will include a template directory under which application.account.index.tpl , schema design for storing this information .
  3. Retrieval : After the successful authentication, various APIs provided by network can be used such as for accessing & showing profile & connection & for network updates etc.
    Status Update API includes one is Profile API & another is Network API.
    Get Network API can be used to get updates about the network.This will be done using a crawler class which upon calling can show updates on the page .
    All returns follow a standard pattern with variation in the update details section:
    • Network opening tag
    • Network stats: the count of members in the first and second degrees
    • Updates: number of
      Post network API are used to post status on network .
      While Profile API are used to change status of profiles of the user who has grant the access.

The crawler class will update the authentication & take parameters from the LInkedIn to update the status using various string parameters passed into the URI that can shown into the class file that on refresh show the content of that specific .A successful call will generate the xml document & return it which will be passed onto the method used to show the content in the class.
The XML returns two URLs that let easily display a standard profile

  • api-standard-profile-request/url
  • site-standard-profile-request/url
    All the data that has been fetched using the class will be saved in the db & whenever this class is called using the link provided in the UI all the data is fetched again .

+Point :
As I will work on Google Map API integration which will help to show updates on the world map & will help to create a mash up around the ThinkTank, as Google maps getting very much popularity due to it’s OPEN STREET MAP Project so has a very good documentation to work with. This will be useful as it will tell about the widening database & help show up the data on world map so as to locate easily.

When ?
A rough timeline for project :

  • April 26 th – May 1 st : Deciding a code name for the project :) getting familiar with mentor & community.
  • May 1 st – May 24 th : Getting familiar with ThinkTank code, coding convention, reading up the specs, learning how to write ThinkTank plugin, setting up the development environment, setting up the project version control system like an git account.

Checkpoint 1 st :

  • May 24 th – June 7 th : Start Coding !Write REST API/crawler for pulling data.
  • June 7 th – June 30 th :Modifying & writing lib files for the crawler & authentication support .

Checkpoint 2 nd:

  • June 2 nd -July 12 th : Designing UI & integration with others.
  • July 12 th – July 16 th : Combine everything so far, release first testing version. Testing debugging.

CheckPoint 3 rd:

  • July 16 th – August 9 th : Working on Google Map API Integration . & integrating with others.
  • August 9 th – August 16 th :Final release, testing, testing…..& Debugging !

Yeppi !! Done !

Me why ?
I m an undergraduate student at College of Engineering & Technology, Bikaner(India). With Computer Science as my major, I have been associated with FOSS from more than 1 year now. I recently joined ThinkTank, worked with it & now making efforts in contributing. I have worked with various frameworks & content management system includes Joomla, Wordpress, Drupal, Tikiwiki . I had made a plugin for wordpress can be seen here(1). Besides these I am the administrator of Linux User Group Bikaner(2) & one of the founder. I have a keen interest in java language too & has made a download manager client in this language can be seen here(3). Besides these I m working on firefox extension Jetpack & has made a patch using Twitter API for it can be seen here(4). I have hands on experience on working with DIGG API, GOOGLE APIs viz Google maps api, google blogger api etc, Delicious API. Except it I am an amateur white hat hacker love playing with backtrack-linux & has experience in maintaining access using tunneling & graphis designer, love to play with gimp. All in all, I m quite confident that I am the
right guy to take this project out of this document. I am already well versed with community interactions/dynamics of OSS projects, I
use irc, mailing lists, version control systems, such as git for various projects hence I can start early with coding. I take Summer of Code as an
opportunity to get engaged & giving back to the community what it is to me & continue this association even after Gsoc.
You can know more about me on my personal blog(5), and I would love to be contacted if any part of this proposal is not clear to you. Thank you for considering this proposal and for your time!

Except it additional link to proposal

Related Links:
(3)http://code.google.com/p/dmanag/ .

Clone this wiki locally
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.