Our Architectural Overview of the PostRank Engagement System intends to define what we are doing and why.  While this paper will not go into specific technical details of how we build our stuff and scale it out across Amazon’s EC2 cloud, we are setting the context for our system, defining what we calculate and how. (We’ll geek out about some of the challenges and lessons of building and scaling our stuff a bit later, you can be sure.)  For those that are interested in how Social Engagement is calculated, this is the document for you.

Social Engagement is an evolving concept. We’ve been talking to advertising agencies and networks, marketers, researchers, public relations, news sites, media companies and analytics platforms. Across the board they all have highly variable and interesting points of view on what engagement is, how useful it is and what to do with it.

We’d love to get your take on our approach and especially hear what you would do with content and engagement data since the real-time content stream, social engagement monitoring, and two years of archival data are available as part of our Data Services products. We want to hear your ideas. Let’s Get Started.

“Now, besides blogs, there is Twitter, social networking, and the real-time web.” – Richard Jalichandra

Technorati’s announcement this morning that they were re-inventing themselves to become relevant to today’s web was cheered by all of us at PostRank. It’s complete validation of what we’re all about, and it proves that the hard work we’ve put into collecting and making sense of social media metrics for the last two years has been time well spent. Yes, there are dozens of social hubs now — Digg, Delicious, Reddit and Friendfeed among them — which in the course of the past two years have begun to dramatically transform how users interact with and distribute content. These social sites can now be consistently found in the top three traffic drivers (competing with search and direct visits) and more and more publishers are focusing on their communities as a way to build more engaging content and conversations, drive traffic, and expand their influence.

Welcome to the distributed, real-time, social web!

Earlier this year we did a data mining experiment, and discovered two really important trends. First, the half-life of a story on the Internet today is roughly 50 minutes. In other words, more than half of any engagement a story gets occurs in the first hour after it’s published. Without a doubt, this trend is amplified by the numerous social networks, all of which value the timeliness of a story above many other factors.

Secondly, because the web has become social, distributed, and real-time, users are free to share, critique, and comment on stories away from where they were published. In fact, over 80% of all the engagement for an average story on the Internet happens off the publisher’s property; there are conversations on Twitter, votes on Digg, bookmarks on Delicious, and the list goes on. Traditional analytics don’t capture this; they’re still just measuring pageviews, links and unique visitors. Clearly, better tools are needed to measure engagement. PostRank Analytics is the first step in this direction.

Leveraging these emerging trends, our focus at PostRank has been on building a real-time ranking and monitoring platform that is relevant to the new social web. Links matter — they are a vote of confidence — but sharing, bookmarking, voting and dozens of other actions that millions of users take every day are all indicators of interest and engagement. Collecting these metrics gives a publisher a much more complete picture of audience engagement. PostRank tracks all of these metrics in real-time. It is a measure of total story engagement, on and off the publisher’s site.

When you type in a topic (E.g. Wine) on postrank.com you see a ranked list of blogs. That ranking is done in real-time based on all the engagement that each of those blogs has received over the last two weeks. It is relevant, dynamic, real-time, and it takes into account the social web.

The same technology that powers PostRank is now also powering the AdAge Power 150 list. AdAge migrated to using PostRank as their most weighted scoring factor in order to capture the feedback from the social web. The Atlantic Wire also just launched the Atlantic 50, tracking the most influential political commentators in the US, building in PostRank metrics.

New Real-time Ranking APIs: Feed Engagement & Metrics

So what’s the point? When we talk about engagement, we cannot ignore the social web. That’s the thesis of PostRank, and today we’re happy to announce that the same real-time APIs that power PostRank and our partners’ sites are becoming public.

Feed Engagement API (docs): real-time ranking of blogs based on all the social engagement data collected by PostRank (comments, discussions, votes, etc.) The API allows you to define a time interval to consider (from a day to a year), and you can even look at the daily engagement data for each blog.

Metrics API (docs): real-time metrics for any blog post. Need to know how many times a story has been shared, retweeted, or bookmarked? Metrics API can answer that.

So what does this mean? Now anyone can use the PostRank APIs to dynamically rank stories and entire feeds in real-time. As an industry, we need to move beyond page views and acknowledge the social web. Feed Engagement, Metrics, and the PostRank APIs are our contributions to this conversation.

Press Release
ReadWrite Web: As Technorati Falters, PostRank Launches New Blog Ranking Tools

The PostRank Real-Time Engagement APIs enable you to keep on top of when and where conversations about news and blog stories are taking place online. Over 50% of a story’s engagement happens within the first hour after it’s published, so keeping on top of where and how your audience is engaging with your content is critical. Answer questions, catch breaking news, and join conversations as they’re starting.

Over 80% of engagement with online content doesn’t take place on the publisher’s site. It’s scattered around the social hubs, like Twitter, FriendFeed, and digg. Keeping up with these activities manually can be extremely time- and resource-intensive. But PostRank tracks that activity on the most popular social sites and brings it all together, enabling stakeholders to see the full story around their content.

Each engagement notification includes metadata that helps identify velocity, the latest source of engagement activity (e.g. the network on which the content is popular), the feeds, and their respective engagement where the article appeared. Being notified when new interactions with your content take place is a powerful tool for people managing publishing, communications, community building, and marketing.

PostRank has also added Sentiment and Emotional Analysis to its Real-time Content stream APIs. We analyze each story with machine learning algorithms to provide general sentiment (positive, neutral or negative), an overall emotional score for the story, and a detailed emotional score for the story. (The detailed scores are based on Paul Ekman’s emotional classifications of anger, disgust, fear, happiness, sadness and surprise.)

Since we gather up what folks are saying about stories, adding Sentiment and Emotional Analysis to those conversations was the natural next step. Publishers can now learn not only when and where people are talking about their content, but what the tone of those conversations is. This is an invaluable tool for brand managers, assisting them in determining when and where their engagement is best invested. For example, a negative article that gets no engagement may not require a response, but a negative article accompanied by a vigorous ongoing debate very well might.

It’s no longer enough for publishers, corporate entities, marketing professionals, and agencies to be online, they need to know when and where people are engaging with their content and how those people (their customers and potential customers) are reacting. Intelligent and agile responsiveness is becoming a key skill, and PostRank’s Data Services — including our new Real-Time Engagement Data APIs and Sentiment Analysis — are a key competitive advantage.

For the formal types: the press release.

So how do you separate the wheat from the chaff? Where do you access and make sense of the content people are engaging with? PostRank to the rescue!

Today we’re announcing Data Services — a group of APIs that enable customers to access PostRank data for their own information and application needs.

The Real-Time API will deliver up-to-the-minute, socially relevant, top quality news and blog content. There are several engagement thresholds available, which access the top 1K, 10K, 50K or 100K feeds that have generated audience interaction in the past 30 days. And for the real info junkies, there’s the BuzzFeed (the full pipe of all engagement-generating feeds).

Metrics are gathered from nearly two dozen social networks and apps. Immediate content indexing impacts feed rankings, automatically surfacing the content garnering the most audience interaction. This enables clients to easily highlight the latest, most socially relevant content to suit their needs.

The new Data Mining APIs provide access to the rich archive of metadata PostRank collects. These APIs enable users to retrieve, analyze, and display a vast range of feed and story information, be it feed-level engagement data, topic or theme-specific feeds or content, or top post identification.

The Data Mining APIs are like tasty pizza — you not only choose your unique topics, you also choose how it gets sliced up to best meet your needs.

All of the PostRank Data Services APIs are licensed on a subscription basis.

To round off the Data Services offerings, PostRank also now offers custom reporting, with data presented in a variety of formats, depending on the user’s needs, and priced per report.

Intrigued? Got a killer idea that’s itching to be developed and think our data is just what you need? Give us a holler and let’s get started!

For the formal types, here’s our press release.

There are already some noticeable changes in the listings, and we’re hoping our real-time analysis will keep the rankings current and interesting for a long time to come. Our congratulations to the bloggers on the list! (We’re not even playing favourites with the Canadians.)

The AdAge blog post has a great explanation of how and why they integrated PostRank into their analysis, and what it brings to the list and its rankings:

Why’s PostRank so great? For one, unlike other metrics, they provide real-time data. We’ll only be grabbing it once a day like any other metric, but you can be reasonably sure that your recent hot post will be accounted for. But here’s an even bigger reason: we have a specific time window. For each blog, we’re grabbing PostRank scores for only the previous 30 days, an intentionally small window of time. We wanted to make the Power 150 more dynamic and current, and, by basing 1/3 of your overall score on recent activity, we’re hoping this will affect rankings in three ways:

Level the playing field a bit. If you’re a king now, you could be demoted within 30 days. If you’re at the bottom, you could enter the upper reaches yourself within a month, if you do well enough. Of course, the latter still won’t be easy, but hopefully not the insurmountable task it once was, because recent successes will count for more and long-accrued success a little less.

Encourage people to keep up with their blogs. If you stop blogging for a month (as some of you have — tsk, tsk), your score will suffer. We don’t necessarily want to punish you for taking a break, but we think those who are still active and working should have an advantage.

Give a better picture of what’s happening now. We want the Power 150 to be a more dynamic place, a better reflection of the fluidity of conversations on the web rather than a calcified “Who’s Who.” It’s a pretty lofty goal, but this our best shot so far.

Additionally, there’s more information here, as well as a chance to win Power150 prizes!

We’re thrilled to be a key part of the Power150 analysis and list, and we hope we can help keep the list timely and relevant and help people even more to Read What Matters.

This means immediate feedback for PostRank scores as users are engaging with publishers’ content (faster and better discovery) and an immediate feedback loop for the publishers (to help them engage in new and ongoing conversations).

As part of the upgrade, we have also added a number of new sources to our platform:

Among the new additions are: Diigo, Feecle, Furl, Identica, Jaiku, BrightKite and MéxicoDiario. And of course, we are always working on adding new sites and improving our global coverage.

Real-time Authoritative Search

Over the course of past five years, online social networks, and a proliferation of socially-oriented tools such as wikis, blogs, microblogging, bookmarking and sharing sites, have transformed the Web from a technical infrastructure to a social platform, which is now often referred to as the Social Web.

Vik Singh, architect of the Yahoo! Search BOSS team, recently highlighted some of the challenges and opportunities that this change has brought forward:

One emerging area of search that I think no one has really solved is real-time authoritative search. When breaking news happens (like the Mumbai bombing, Hudson River plane crash, or wildfires), it’s difficult for traditional news sources to discover and prioritize all the information in a timely fashion. It can take several minutes or hours for traditional media to converge on the important stories. However, new social media outlets like Twitter are breaking these important stories faster than traditional media. By looking at the number of users chatting about these topics, one can measure the future newsworthiness of a very fresh story despite its potentially minimal traditional news coverage at that moment.

Later in his Q&A, Vik also highlights PostRank as a potential and promising solution to the problem he outlines. The combination of real-time social media monitoring and PostRank analysis provides a great a signal for the new and upcoming stories as they develop. Want to give it a try? Try our BOSS+PostRank mashup:

BOSS + PostRank

To get started simply type in a query (e.g. yahoo), select news, and click search. To power the search, we use the BOSS Mashup framework to retrieve the latest stories from around the web, and then apply a PostRank filter on top – stories with the highest engagement have the highest scores and are displayed first in the list. You can preview the story, click on the PostRank score to view some of the metrics behind it, and even see the filtered out stories at the bottom of the page:

The combination of BOSS and PostRank APIs is a great demonstration of leveraging the data generated in the Social Web. Give it a try (boss.postrank.com), and let us know on your thoughts and suggestions.

The Ubiquity team at Mozilla recently launched a new homepage for their project which features a really interesting integration of Yahoo Pipes and PostRank API to identify the most influential posts. Instead of using a numeric score, the engagement of the story is visualized as an underline with the help of color and length – neat!

For those of you who are not familiar with Ubiquity, make sure to check out the video summary, it’s an amazing Firefox extension which can streamline your day to day browsing experience.

On that note, we’re also big fans of the project and have built a couple of custom commands, one of which you can now install in your browser: preview top posts of any page with a single command! To get started, click on the link, and follow the instructions to install the command. Once the script is installed, simply navigate to any website with an RSS feed, open your Ubiquity prompt, and type in “top” to preview the most influential stories from that site! Easy as that.

Let us know how it works, and if you have any suggestions for other commands you’d like to see.

AideRSS provides a service that ranks online content such as RSS feeds, blog posts and so on. I have always been impressed with AideRSS and wanted to educate myself with writing REST interface logic, ASP.Net and Web 2.0 in general. AideRSS also provides a PostRank API to their service so I decided to write a .NET client wrapper for this API to make it easier to use.

A great addition to our API library. Thanks, Declan!

I’ve created a Ruby wrapper around the API that people will hopefully find useful. I’ve currently tagged it as a 0.5 beta release until I can get some feedback on if the API is right for other people as well. (Plus, I know I want to change the Exception handling at some point so there will be changes before a 1.0.)

You can grab the postrank-0.5.gem or install it through the GitHub gem server by entering:

gem sources -a http://gems.github.com
sudo gem install dj2-postrank

Or, if you want to can grab the code from GitHub and build it yourself.

If you find any problems, have any suggestions or patches, you can file them on the LightHouse project page.

Another well done library for PostRank API – w00t!