Creating a Xideco Twitter Bridge

Alan Yorinks edited this page Jan 16, 2016 · 26 revisions

To demonstrate just how easy it is to add functionality to an existing Xideco system, let's add a "Twitter" block to Scratch and a Xideco Twitter Bridge to send out live tweets.

Here is what we need to do:

  1. Add a custom Twitter block to Scratch.
  2. Update the HTTP Bridge to accept data from the new Scratch block
  3. Create a unique Xideco Twitter protocol message that will be sent to the Xideco Router
  4. Test that a Scratch to HTTP Bridge message is received by the bridge and a message is prepared to be sent to router.
  5. Create a new bridge, called the Twitter Bridge that subscribes to the new protocol message.
  6. Test end to end tweet creation without actually sending out a tweet.
  7. Create a Tweet in Scratch and verify that it is actually sent.

1. Add a custom Twitter block to Scratch

Scratch extension blocks are described in a special JSON file. The details of creating such a file can be found here.

To add the custom block shown above, the following lines of code were added to [m_s2aio.s2e] (https://github.com/MrYsLab/xideco/blob/master/xideco/data_files/scratch_files/extensions/m_s2aio.s2e):

      [   "",
            "Enter Your Tweet: %s",
            "tweet",
            "Hello World"
        ],

A block is described as follows:

The first line describes the block type. " " = A command block, rectangular in shape.

The next line describes the static text of the block ("Enter your Tweet:") and an input field specifier, "%s". The %s will create a text input field within the block.

The third line containing "tweet", specifies the command value that Scratch will attach to the HTTP GET message it generates when the block executes.

The next line, "Hello World" is the default text contained in the block when it initially appears on the screen.

Using the updated extension description file, a project file was created and can be found here.

If you save the project file and load it into Scratch using File/Load in the Scratch editor, you should see the new Tweet block at the bottom of the block stack.

2. Update the HTTP Bridge to accept data from the new Scratch block

The HTTP Bridge contains an instance of an aiohttp HTTP server. It receives and parses the HTTP GET requests from Scratch, so we need to update the HTTP Bridge to process the "tweet" command.

To do so we add a dispatch entry into the HTTP Bridge:

app.router.add_route('GET', '/tweet/{message}', self.tweet)

3. Create a unique Xideco Twitter protocol message that will be sent to the Xideco Router

To process a "tweet" command we need to add a new method that we specified in the line above.

    async def tweet(self, request):
        """
        Create a tweet command message
        :param request: command and data from GET request
        :return: HTTP response
        """
        # retrieve the users tweet text  from the GET request
        message = request.match_info.get('message')

        # print the message for debug purposes
        print(message)

        # create a msgpack message for the text
        command_msg = umsgpack.packb({u"message":message})

        # Use the pseudo board 100 for message envelope and send the message to the router
        await self.send_command_to_router("100", command_msg)
        return web.Response(body="ok".encode('utf-8'))

You can find an updated HTTP Bridge file, called xihbt.py, here.

4. Test that a Scratch to HTTP Bridge message is received by the bridge and a message is prepared to be sent to router.

Start up the modified HTTP Bridge called xihbt.py and load the modified Scratch project into Scratch.

Click on the Tweet block. You should see "Hello World" appear the xibht.py console window.

5. Create a new bridge, called the Twitter Bridge that subscribes to the new protocol message.

A new file called xitwitter_bridge.py is created. It receives the twitter generation message from the Xideco Router and sends communicates with twitter.com. Here is the salient code for this new bridge, and you can find full copy of the file here.

This file contains a class called TwitterBridge. The init method connects to the router publisher socket, and subscribes to messages from pseudo board 100.

class TwitterBridge:
    """
    The Twitter Bridge provides the protocol bridge between Xideco and a Twitter generation request from Scratch
    """
    def __init__(self):
        """
        :param pymata_board: Pymata-aio instance
        :param board_num: Arduino Board Number (1-10)
        :return:
        """
        # establish the zeriomq sub and pub sockets

        self.context = zmq.Context()
        self.subscriber = self.context.socket(zmq.SUB)
        connect_string = "tcp://" + port_map.port_map['router_ip_address'] + ':' + port_map.port_map[
            'command_publisher_port']
        self.subscriber.connect(connect_string)

        # subscribe to the "twitter topic"
        env_string = "A100"
        envelope = env_string.encode()
        self.subscriber.setsockopt(zmq.SUBSCRIBE, envelope)

The second method, run_twitter_bridge, is a forever loop that receives the twitter commands destined for pseudo board 100.

    def run_twitter_bridge(self):
        try:
            msg = self.subscriber.recv_multipart(zmq.NOBLOCK)
            self.payload = umsgpack.unpackb(msg[1])
            print("[%s] %s" % (msg[0], self.payload))
            message = self.payload["message"]
            print(message)

            command = 'twitter -eYOU@email.com set %s' % message
            # subprocess.call(command, shell=True)
            time.sleep(.001)
        except zmq.error.Again:
            time.sleep(.001)

Change "YOU@email.com" to your email address

The message receive uses a NOBLOCK command for a non-blocking receive. When self.subscriber.recv_multipart(zmq.NOBLOCK) is called and a message is not available, a zmq.error.Again exception is thrown. We just need to catch it, and try again.

Notice the commented out "subprocess.call(command, shell=True)". This is to allow us to test end to end. Note: there are more elegant ways to interface with Twitter - we took the quick and easy way.

6. Test end to end tweet creation without actually sending out a tweet.

With subprocess.call(command,shell=True) commented out, start up xirt, xihbt and xitwitter_bridge. Load the twitter Scratch project into Scratch and execute the Twitter block.

You should see something like the following in the xitwitter_bridge console:

xitwitter_bridge.py
[b'A100'] {'message': 'Hello World'}
Hello World
[b'A100'] {'message': 'Hello World'}
Hello World

7. Create a Tweet in Scratch and verify that it is actually sent.

Uncomment out subprocess.call(command, shell=True), enter text you would like to Tweet in the Scratch block and execute the block. You should see the tweet appear in Twitter.

To Summarize

To add Twitter functionality we:

  1. Added five lines to a Scratch extension description file to create the Tweet block.
  2. Added 1 line to a new HTTP_Bridge file so that the HTTP server would recognize the new Scratch Command.
  3. Added about 7 lines of new code to the modified HTTP_Bridge to process the receipt of the Tweet command.
  4. We created a new xitwitter_bridge file consisting of 2 methods to receive the "generate tweet" message from the modified HTTP_Bridge and to send the tweet.

So with very little code, we were able to add some substantial functionality. Neither the Arduino Bridge nor the Xideco Router needed any modification and we did not have reconfigure the network in any way.

Another thing to note, is that if you had had a Scratch script running, let's say blinking an LED and then started the xitwitter_bridge, the bridge would have started seamlessly without disturbing the running script and enabling the twitter functionality.