2007-03-24-yahoo-pipes-and-the-mashed-up-world-of-aggregated-filtered-blended-information.html

---
layout: post
title: Yahoo Pipes and the Mashed Up World of Aggregated, Filtered, Blended Information
date: 2007-03-24 01:11:47.000000000 -07:00
categories:
- technical-writing
- web-design
- web-design
tags:
- aggregation
- mashups
- Yahoo Pipes
status: publish

published: true
---

{% include toc.html %}

<p>Yahoo Pipes is a new online tool that allows you to blend, manipulate, and combine feeds from various data sources to create a streamlined, single feed of information. Essentially Yahoo Pipes allows you to create feed mashups of different data sources without having a knowledge of programming.<a href="https://s3.us-west-1.wasabisys.com/idbwmedia.com/images/2007/03/yahoopipes.gif" title="Yahoo Pipes"><img src="https://s3.us-west-1.wasabisys.com/idbwmedia.com/images/yahoopipes1.gif" alt="Yahoo Pipes" align="left" height="345" hspace="5" vspace="5" width="480" /></a><br />
<br />&nbsp;</p>
<p><a href="http://pipes.yahoo.com" target="_blank">Yahoo Pipes has received a lot of praise. In its debut, </a><a href="http://radar.oreilly.com/archives/2007/02/pipes_and_filte.html" target="_blank">Tim O'Reilly said</a>:</p>
<blockquote><p>Yahoo!'s new Pipes service is a milestone in the history of the internet. It's a service that generalizes the idea of the mashup, providing a drag and drop editor that allows you to connect internet data sources, process them, and redirect the output. Yahoo! describes it as "an interactive feed aggregator and manipulator" that allows you to "create feeds that are more powerful, useful and relevant." While it's still a bit rough around the edges, it has enormous promise in turning the web into a programmable environment for everyone.</p></blockquote>
<p>A few examples will bring this tool's capability into focus. Suppose you want to combine your favorite feeds into one feed. Yahoo Pipes allows you to aggregate the feeds, choose a sorting order, define filters, truncations, keywords, and so on. The pipe then generates a single RSS feed containing this information.</p>
<p><a href="http://pipes.yahoo.com/pipes/pipe.info?_id=0g8N7Hu82xGydYNOJjBjOg" target="_blank" title="Yahoo Pipes"><img src="https://s3.us-west-1.wasabisys.com/idbwmedia.com/images/yahoopipes21.gif" alt="Yahoo Pipes" title="Yahoo Pipes" align="left" hspace="5" vspace="5" /></a>In another example, let's say you want to monitor the web for keyword searches on "technical writing." Traditionally, you could perform individual searches on various search engines, even subscribing to RSS feeds for each engine's search results. But Yahoo Pipes allows you to define different data sources and then run your keyword search across all the data sources at once.</p>
<p><a href="http://pipes.yahoo.com/pipes/pipe.info?_id=0g8N7Hu82xGydYNOJjBjOg" target="_blank">The Aggregated News Alerts Pipe</a> (shown visually by the image on the left) takes the keywords you input and searches for the across the following search engines: www.blogpulse.com, news.search.yahoo.com, technorati.com, blogsearch.google.com, icerocket.com, findory.com, yahoo.com, www.bloglines.com, search.yahoo.com, api.technorati.com, www.icerocket.com, bloglines.com, search.live.com, news.google.com, rss.findory.com, blogpulse.com, google.com</p>
<p>After you type a keyword and click "Run this pipe," the pipe retrieves the data and allows you to subscribe to the results as an RSS feed. This is an excellent way to keep your eye on the blogosphere for particular topics.</p>
<p>The <a href="http://pipes.yahoo.com/pipes/pipe.info?_id=HDWYJBq52xGK4P_rZoQMOQ" target="_blank">BI Jobs Pipe</a>, another piping application, allows you to search various sources for jobs, after inputting a location and job title.</p>
<p>Yahoo Pipes is not immediately intuitive (but here's a <a href="http://content.zdnet.com/2346-11406_22-54748.html" target="_blank">good tutorial</a> for doing the simplest mashup). Essentially, the pipe-building is a bit like creating an advanced search with different criteria. You can define operators, filters, number of returns, dates, locations, and other parameters for the search returns. You also define the data sources. The pipes can be robust and intricate, or they can be simple. What's totally new is the visual drag-and-drop pipe interface.</p>
<p>Yahoo Pipes also allows you to clone well-built pipes and tweak them to suit your own purposes. Because of this, Yahoo Pipes is very web 2.0. You can browse the <a href="http://pipes.yahoo.com/pipes/pipes.popular" target="_blank">most commonly run and cloned pipes</a> by clicking "Browse Pipes" from the Pipes home page.</p>
{% include ads.html %}
<p>Although I think the product is unique, innovative, and powerful, I was little disappointed in Yahoo's documentation. I may be mistaken, but it appears Yahoo launched its innovative product with little instruction. Their home page says,</p>
<blockquote><p><strong>How do I use pipes?</strong></p>
<p>Please check back, in a few days we'll have online tutorials demoing how to get the most out of Pipes.</p></blockquote>
<p>The product was launched more than a month ago. I guess if the product is cool enough, users <em>will </em>write the help.</p>
<p>The drag-and-drop interface, with connecting pipes all over the place, make it visually appealing and fun. In the near future, I'd like to create technical writing pipes that would be useful to our community. I created a <a href="http://pipes.yahoo.com/pipes/pipe.info?_id=8soeGL7Z2xGiLaMMo_NLYQ" target="_blank">simple pipe</a> that aggregates 10 different tech writing feeds, but for some reason when I subscribe to the pipe's RSS feed in IE or FeedDemon, it strips the feed of its formatting. Another limitation is that you can't see the source each feed is coming from (although Lifehacker apparently has a workaround).</p>
<p>In using Yahoo Pipes, it's probably easiest to find successful and interesting pipes, clone them, and then tweak them to suit your purpose.</p>
<p><strong>Update:</strong> I found some <a href="http://usefulvideo.blogspot.com/2007/02/yahoo-pipes-tutorials.html" target="_blank">excellent video tutorials</a> for using Yahoo Pipes at <a href="http://usefulvideo.blogspot.com/">http://usefulvideo.blogspot.com</a>. There's also a <a href="http://mrspeaker.webeisteddfod.com/2007/02/10/yahoo-pipes/" target="_blank">written tutorial here</a>.</p>