Skip to content

English plugin dev 5 3

Jump to bottom
semuel edited this page Jun 8, 2012 · 9 revisions

Cooperation with external Web API

Introduction

Until now everything that we talked about was about MT and modifying MT itself.

But the real fun is when you connect things together. We will see who to connect MT and external Web APIs

What are Web APIs?

Basically, A Web API is a pre-defined URL, to which a program can access and a certain operation will happen. For example, MT itself have a few Web APIs: Trackback, XML-RPC and Atom.

And there are APIs all over the internet. A lot of sites expose API, which you can use for fun and profit.

Developing Plugin connecting MT and Web API

Specification

  • Can setup an Yahoo! Japan Application ID in the configuration screen
  • When an entry is saved (POST SAVE) the following should happen:
    • If api_id is not set, abort this callback
    • Summarize the Title, body and “more text” fields of the entry, and submit them (with the api_id) to Yahoo! Japan’s text scoring API
    • Extract keywords and scores from the returned XML
    • If the score for any keyword is above 50, add this keyword as a tag to the entry

Yahoo! Japan key-phrase extraction API

For using Yahoo! Japan Web APIs, we need an application ID. This can be obtained from their Developer Network Website

Yahoo! have many web APIs open to the public, and will will use their keyword-phrase extraction API

The API analize text, returns the most important keywords and phrases in the text, and also score each keyword or phrase to how important it is. The returned output can be (configurable in the request) either XML, JSON or PHP Serialize. We will use the default, XML.

config.yaml

id: MyPlugin17
key: MyPlugin17
name: <__trans phrase="Sample Plugin API">
version: 1.0
description: <__trans phrase="_PLUGIN_DESCRIPTION">
author_name: <__trans phrase="_PLUGIN_AUTHOR">
author_link: http://www.example.com/about/
doc_link: http://www.example.com/docs/
l10n_class: MyPlugin17::L10N

system_config_template: yahoo_japan_api_id_setting.tmpl
settings:
    yahoo_japan_api_id:
        default:
        scope: system

callbacks:
    MT::App::CMS::cms_post_save.entry: $MyPlugin17::MyPlugin17::Callbacks::post_save_entry

Commentary

  • The plugin system-level setting template: yahoo_japan_api_id_setting.tmpl
  • The settings themselves under settings
    • Setting key: yahoo_japan_api_id
      • Default value: empty
      • Scope: System
  • Callback to be called right after an entry is saved in the CMS: MT::App::CMS::cms_post_save.entry
    • Callback handler: $MyPlugin17::MyPlugin17::Callbacks::post_save_entry

Callbacks.pm

package MyPlugin17::Callbacks;
use strict;

use constant YAHOO_API_URI => 'http://jlp.yahooapis.jp/KeyphraseService/V1/extract';
use constant TAG_SCORE => 50;

sub post_save_entry {
    my ($cb, $app, $obj, $org_obj) = @_;
    my $plugin = MT->component('MyPlugin17');

    my $api_id = $plugin->get_config_value('yahoo_japan_api_id', 'system');
    return 1 unless $api_id;

    require LWP::UserAgent;
    require HTTP::Request::Common;

    my $text = $obj->title . ' ' . $obj->text . ' ' . $obj->text_more;
    my %data = ( 'appid' => $api_id,
                 'sentence' => $text,
               );
    my $req = HTTP::Request::Common::POST(YAHOO_API_URI, [%data]);
    my $ua = LWP::UserAgent->new;
    my $res = $ua->request($req);

    require XML::Simple;
    my $results = XML::Simple::XMLin($res->content)->{'Result'};

    my @tags = ();
    foreach my $i (0..$#$results) {
        my $result = $results->[$i];
        my $score = $result->{'Score'};
        my $keyphrase = $result->{'Keyphrase'};
        push(@tags, $keyphrase) if $score >= TAG_SCORE;
    }
    return 1 unless @tags;

    $obj->add_tags(@tags);
    $obj->save
        or die $obj->errstr;
}

1;

Commentary

package MyPlugin17::Callbacks;
use strict;
  • package declaration and use strict; 
use constant YAHOO_API_URI => 'http://jlp.yahooapis.jp/KeyphraseService/V1/extract';
use constant TAG_SCORE => 50;
  • YAHOO_API_URI: Key phrase extraction API URL
  • TAG_SCORE: phrase with score about/equal to this, should be added as tag
sub post_save_entry {
    my ($cb, $app, $obj, $org_obj) = @_;
    my $plugin = MT->component('MyPlugin17');
  • Callback handler function declaration
  • Parameters: callback, MT app, the saved object, the original object (before the changes of this save were applied)
  • Getting the plugin object (we need it for the plugin configuration)
    my $api_id = $plugin->get_config_value('yahoo_japan_api_id', 'system');
    return 1 unless $api_id;
  • get the yahoo_japan_api_id config into $api_id
  • If yahoo_japan_api_id is not set, stop the callback
    require LWP::UserAgent;
    require HTTP::Request::Common;

    my $text = $obj->title . ' ' . $obj->text . ' ' . $obj->text_more;
    my %data = ( 'appid' => $api_id,
                 'sentence' => $text,
               );
    my $req = HTTP::Request::Common::POST(YAHOO_API_URI, [%data]);
    my $ua = LWP::UserAgent->new;
    my $res = $ua->request($req);
  • We need the LWP::UserAgent and HTTP::Request::Common Perl modules for communicating with Yahoo!’s API
  • Combine the title, text and text-more fields to one variable $text
  • Preparing the data for sending: combining $api_id and $text into %data
  • Preparing the POST request: using HTTP::Request::Common::POST
  • And the use LWP::UserAgent to send the request and receive the response $res 
    require XML::Simple;
    my $results = XML::Simple::XMLin($res->content)->{'Result'};
  • We will use XML::Simple to parse the response (not the best XML module out there, but it is sufficient for us here)
  • XML::Simple supply a parsing function XMLin
  • The XML itself is in the response in the content function. (there are other things in the response, such as error code that we should really check that is OK, but we didn’t. my bad.)
  • After the XML was parsed and XMLin returns a hashef, we are interested in the ‘Result’ sub-structure
    my @tags = ();
    foreach my $i (0..$#$results) {
        my $result = $results->[$i];
        my $score = $result->{'Score'};
        my $keyphrase = $result->{'Keyphrase'};
        push(@tags, $keyphrase) if $score >= TAG_SCORE;
    }
    return 1 unless @tags;
  • @tags – an array for the tag that we need to add
  • foreach will loop over the $results
  • take individual $result from the array
  • Extract $score and $keyphrase from the $result
  • if $score is more then TAG_SCORE (50) add the $keyphrase to @tags
  • If @tags is empty, exit from this callback. there are no phrases important enough
    $obj->add_tags(@tags);
    $obj->save
        or die $obj->errstr;
  • Use the Entry object function, add_tags, to add these tags to $obj
  • Then, as we changed the entry object, we will save it

Directory Structure

$MT_DIR/
|__ plugins/
   |__ MyPlugin17/
      |__ config.yaml
      |__ lib/
      |  |＿ MyPlugin17/
      |     |__ Callbacks.pm
      |     |__ L10N.pm
      |     |＿ L10N/
      |     |  |＿ en_us.pm
      |     |  |＿ ja.pm
      |__ tmpl/
         |＿ yahoo_japan_api_id_setting.tmpl

Plugin Download

MyPlugin17.zip(3.21KB)

Summary

This is quite a simple example, yet the principle is the same when connecting to any other Web API.

But please don’t forget to check for errors. The API can one day stop responding. or return the answer in different format. Or return error. You should alway check that you are getting the right thing, and if not alert the user.
Also, there are problem that are inherent in the plugin: once a tag was added, we don’t remove it, even if the user changed the article and it no longer appear it in. When developing a plugin, you should consider this too.

But still, Web API give you the power of the internet. use it wisely!

Navigation

Prev:Add and display a modal window << Index >> Next:Permissions and Roles

Clone this wiki locally