According to former United States Senator Ted Stevens, the internet is a Series of tubes

[…] They want to deliver vast amounts of information over the Internet. And again, the Internet is not something that you just dump something on. It’s not a big truck. It’s a series of tubes. And if you don’t understand, those tubes can be filled and if they are filled, when you put your message in, it gets in line and it’s going to be delayed by anyone that puts into that tube enormous amounts of material, enormous amounts of material

Well, I was stunned to find out that this is true…well sort of. Yahoo has a service out called Pipes

Pipes is a powerful composition tool to aggregate, manipulate, and mashup content from around the web.

The short version is you can create your own pipe or use/clone one of hundreds out there. I was drawn to this one: Social Media Firehose So you enter in a few terms to search on and in one rss/xml/json feed you get a result set from sources all over the web: Twitter, blogs, etc.

Why should you care? Well, as soon as I saw this I thought of the project I’ve been working on to integrate Twitter into Omniture. I figured that I could take that same approach and use Pipes to further extend the level to which we would be able to monitor our brand across social media. And I was right!

After pulling the JSON feed I wanted I was able to parse through it to create the XML object I would need to take advantage of the Data Insertion API.

I then changed the settings in SiteCatalyst to combine this with the data I was already gathering from twitter. I created a new event called Social Media and changed both the twitter data and the data from Pipes to feed into that. Then I added an evar for the source ( Twitter, or the source that Pipe pulled from). Now when I go into the Social Media report suite, I see a report with a list of the terms I monitor, which can be broken down by source of the buzz, author, link of the blog post, link to the twitter user and so on.

I will be modifying the code I have on RiaForge for just twitter to show the combined files for both pipes and twitter, and posting that soon. Now I will keep my eyes out for other API’s I can tap to continue to extend the monitoring reach.

-Rudi