The Short Version

Appsfire is introducing today a powerful and innovative signal to assess whether an app is worth downloading. App Score is a dynamic score which processes dozens of parameters several times a day across all the apps in the iOS App Store. The App Score for a particular app goes beyond the current ratings, rankings and reviews for that app; it also analyzes the rating and ranking performance over time, incorporates additional metadata and reviews for that app across the web, identifies apps engaging in suspicious rating/review behavior in the App Store, and synthesizes the publisher’s record across all of his or her apps.

The Long Version

The biggest pain point affecting mobile app consumers is discovery. To be more precise, it’s a pain to find GOOD apps. Billions of apps get downloaded, but only a small fraction are truly appreciated and used.

To discover great apps, you can rely on certain curated sources, from the App Store itself (whose Featured Apps section drives massive downloads) to myriad blogs and app review sites. But often that is not enough, as editorial curation is only one point of view, and typically fraught with bias.

Moreover, acquiring an app is more tedious than browsing a new website. Even when you’ve identified an app that interests you, you’ll still need to press the download button for an app, wait for the installation, open the app, and spend a few minutes figuring out if this app is right for you. Let alone the fact that few, if any, websites require payment to merely browse; in contrast, there are no trial periods for paid apps in the App Store. We have all experienced buyer’s remorse after paying for an app that quickly revealed itself to be a waste of time and money.

What we are introducing today is a powerful yet simple way to decide whether an app is worth downloading. App Score is the fruit of the intense labor we have put into building AppGenome over the past two years. AppGenome, an app metadata engine, aggregates hundreds of millions of data points on apps per week. As PageRank was the foremost signal that identified quality and relevance across the web, App Score is the foremost signal that emerges from our AppGenome.

 

THE PROBLEM: THE EXPERIENCE OF DISCOVERING QUALITY APPS IS BROKEN

Discovery in the App Store is broken because it is primarily based on two factors that are only loosely correlated to the quality of the app: App Store rankings and ratings/reviews. While intuitively ranking and ratings should be a measure of quality, a number of factors have undermined this.

• Rankings identify the top 200 apps, how about the other 630,000? Too many great apps never make it to the rankings. Why should they be marginalized? Every great app deserves its chance to shine.

• Rankings is easily gameable by marketers: Rankings have become the playground of marketing experts. The objective of most marketing campaigns is not to acquire new users, but to vault the app to top of the rankings. While presence in the rankings is in itself a way to acquire users, many consumers are under the illusion that rankings are a measure of merit. In fact it’s often a measure of marketing dollars.

• Punctual rankings are misleading: Although refreshed regularly, App Store rankings are merely a snapshot and not a measure of quality (as discussed above). An end-user has no idea which apps have shot up the rankings merely via marketing channels. Apps which remain atop the rankings over a sustained period of time - because they’re good apps - should be rewarded.

• Ratings are irrelevant for new apps: It takes some time until you gather enough traction and ratings/reviews from users to find out to build a base of reviews that accurately informs potential new consumers (for better or wrose). A user exposed to a new app has scarce information in deciding whether or not a new app is high-quality.

• Ratings and reviews are also gameable: Many apps use third-party services to rate their app. That’s disingenuous and overtly misleading.

• Ratings are local, not gobal: Ratings are attributed per geographic store (there are 123 stores). If a user gives rates an app in the US App Store, it will not be exposed to users in the French App Store. A great app could have a wonderful rating pattern in one country (say the USA) but none in another.

Does this mean that rankings, ratings and reviews are irrelevant? Not at all. But this is how the App Store has educated users to evaluate apps. And it just doesn’t work.

Ratings and reviews are relevant, but to derive value from them you need to see the forest through the trees - weeding out apps which have gamed the system while rewarding apps which have generated genuine, positive ratings and reviews around the world and around the web. Not even the most discerning consumer can collect or process this information, and most average consumers don’t have the time to browse reviews in making their purchasing decision.

Simply put, users need a clear signal: something that inspires the thought, “I should download”, or the thought, “I’ll pass”. The App Store is not built to help users make an easy purchasing decision.

 

THE APPSFIRE APP SCORE: THE QUALITY SCORE FOR APPS

When we created Appsfire, our mission was to help users find the great apps because they’re great apps. Not because they’re highly rated, highly ranked or highly marketed. Since Appsfire’s inception, we’ve been building AppGenome, our app metadata engine, as the quantitative foundation for this discovery experience. In that time, we’ve also served users over 1 billion app recommendations.

:: Introducing App Score :: App Score is a dynamically computed and frequently refreshed score that synthesizes numerous sources into a single quality score, emerging from three primary components:

• Rankings Score: App Score evaluates the ability of an app to sustain its rankings in the App Store over time. We call that “stay power”. Apps which market their way to the top of rankings typically exhibit poor stay power. Apps which top the rankings based on merit - Angry Birds, Instagram - typically sustain their rankings through massive consumer interest.
• Ratings Score: App Score evaluates the consistency, frequency and velocity of good and bad ratings that an app generates across stores and versions.
• Developer Score: App Score evaluates the reputation and success of a developer across the apps he has created over time (a Klout for developers, if you will). This way we can spot from the very first moment after an app goes live whether the app is promising or not. In exceptional cases, both good and bad, we expose some aspects of the Developer Score to inform users:
• Top Developer: Developers who have built at least one highly successful, high quality app [Android identifies this information as well, but this is most likely editorial]. A “Top Developer”, by our measure, represents the top 1% of developers in the App Store.
• Dodgy Developer: Developers with a track record of building at least one app exhibiting suspicious rating/review behavior.

App Score also incorporates data from sources beyond the App Store: Mentions on Twitter and Facebook (for reference, Appsfire invented AppTrends), mentions on key publications and review sites (like Macworld, TechCrunch, The New York Times) and many more (we’re not going to reveal everything!) in search of quality signals (whether positive or negative).

That’s it! No more need to read all the reviews, check ratings and rankings accross stores (who has THE time for that?). One all-encompassing score will tell you if this app is worth your time.

Bottom line: App Score looks at the big picture and identifies the patterns that reveal an app’s quality, not just the basic rating and review parameters. Ratings and reviews exposes just the tip of the iceberg; App Score exposes the iceberg.

 

SO HOW DOES IT WORK FOR USERS?

 

It’s super, duper simple. Mobile users require simple stuff.

Film buffs have Metacritic and Rotten Tomatoes. We wanted something as simple for apps from a user’s perspective (especially considering that there are many more apps and parameters for apps than for movies). We also wanted something that paralleled PageRank, Google’s initial method to rank search results based on quality and relevance.

App Score is a single number between 1 (Cr-app) to 100 (Masterpiece), which defines the quality of an App. Users don’t have to know all the details behind it - the same way they don’t have to know why a search result appear #1 in Google’s search engine. They will know instantly if an app is good. We crafted beautiful badges to display this information clearly and cleanly..

[91 >100] : “Red hot”

[80 > 91]: ”Super”

[70 > 80]: ”Good stuff”

[60 > 70]: “Promising”

[30 > 60]: “Hmmmmm”

[10 > 30]: “Cr-app-y”

SO HOW USEFUL IS THIS, REALLY? CASES AND MORE CASES

Well let’s take some examples. All of them were computed automatically by App Score.

Case 1: An app with no rating in a given store

Here is the case of an app that has no rating in the App Store because it doesn’t have enough traction yet. If you were to view MoveTheEggs from here Israel, which is not a major geography in the App Store, you see this: No ratings. There’s no way to know if the app is good or not. And this is what App Score says (83 - Super). So you go from “no clue - how good or bad is the app?” to a clear signal saying you should not be disappointed by this app and that it therefore worth it.

Case 2: A bestseller, but low in quality

Let’s take a super famous app: Facebook. Yes, it is a really great service. But how good is the app? App Score says it’s bad (32/100). And apparently there are many good reasons for that. On the other side Facebook Messenger (which is less buggy) has a very high score (92/100).

Case 3: An unpopular app that is actually worth your attention

This app - Mr Chiizu Plus - has never been highly ranked in the US. It is a very nice Photo Collage app and has a very high app score because the high ratings are consistent, as is its position in the App Store rankings, and it has receive very favorable reviews across many sites. By our measure, it has an App score of 83 - Good Stuff.

Case 4: A great and very popular app with high App Score

Apps like Angry Birds [99 - Red Hot] and Infinity Blade [95 - Red hot] are great examples of great apps with a very high-quality score.

Case 5: How the App Score considers developer reputation and automatically rewards that developer’s apps

In the hours after Rovio released Angry Birds Space, the app naturally had fews ratings, little history, and scarce reviews. App Score knew it was from Rovio - the developer behind the successful Angry Birds - and it instantly attributed an App Score of (67 - Promising). Later in the afternoon it was already boosted to an appscore of 94 (due to massive popularity and good reviews). The day after it stabilized to 95.

Case 7: How great apps are spotted quickly and gain a higher rank

Snapguide - a great how-to guide for iPhone - got a 80 (Good app) score 1 day only after its release. Today after several more reviews Snapguide is at 87. The best part? Snapguide is a wonderful app and is not at all in the rankings - not even in the top 50 of its own category (Lifestyle, in the US).

Paper by fiftyThree (iPad only), a wonderful note taking app, less than 24 hours after its release had a very high App Score of 82.

Case 8 : An app can be highly rated, but of very low quality

Take this app showing over 4 stars review. It’s tempting to buy it. Well don’t! Its App Score suggests that it’s low quality - yep, another fingerprinting app with limited value (App Score = 13).

Case 9 : Some apps have no ratings over time.

How do you make a decision in this case? On the iPhone, the reviews are 2 clicks away - most people don’t even get there - and the ratings are non-existent. App Score would tell you it’s not worth wasting your time

Case 10: Being a good developer does not guarantee you a good App Score for your next app

Chillingo, who is famous for publishing many bestseller games, released this Volley Ball game. It could not escape a low App Score (19), as the game is not so great and their Developer Score did not help them on that app.

Want to try for yourself ? open Appsfire Deals (soon in Appsfire, our original app discovery app, too) or run a search on our web site to find out the App Score on any app.

 

FAQS

CAN THE APP SCORE BE GAMED?

Nothing is impossible. But it would be really, really, really hard. A developer, especially if he has several apps, would have to game consistently the rankings, ratings, reviews, Twitter mentions, Facebook likes, review sites of his or all his apps on a consistent manner over time. You can game a ranking for a day or two. It is really difficult or near impossible to game all those paramaters at once - all the time. App Score is built to withstand this..

In addition, we created special safeguards/detectors that highlight suspicious activity around an app (e.g.,: a 1-day-old app suddenly receiving thousands of ratings). We also identify apps that have a limited value or that are clones (Temple what?).

Example 1: Copycats can be popular

If we re dealing with the number 10th clone version of Temple Run or Angry Birds, users will be able to spot it with a special label. Like here.

Example 2: Apps with low utility can be popular

This app - A Lie Detector - has been quite popular but App Score reveals it has a low utility. Save you a few clicks.

This other app is also consistently popular but has really low value for users - it catches the eye but not the fingers. App Score will attribute to it a “Low utility” label

In both, the apps were detected automatically but were validated manually.

Soon more safeguards will be added and make app score even more reliable: For example, a special badge will highlight apps with an unstable pricing policy (meaning they change their pricing up and down too frequently).

 

WHAT AND WHO IS APP SCORE GOOD FOR?

• Good for Discovery and mobile users: We use it first to provide a greater discovery experience: in Appsfire we won’t show you the crapps and showcase only the best. Unless you really want to see everything (there is a search for that). This is why our users feel compelled by appsfire and its discovery experience. The day we introduced quietly the app score as an automatic filter in our apps the stickyness and satisfaction have nearly doubled. Users “felt” the difference. Specially comparing to other discovery services which use only the ratings as a filter or have no quality filter at all…
• Good for Developers: you can use it to benchmark their apps vs their competitors or the apps or their own portfolio.
• Good for Journalists and Bloggers who can use to find out whether this is worth their attention
• Good for Investors can use for due diligence or tracking the performance of their investment (instead of a complex method...)
• Good for Ad Networks or Advertisers who can use to find whether this is worth promoting an app or not or whether you buy media on a good quality app

CAN APP SCORE BE USED BY OTHER SITES/APPS?

Yes but not yet. Eventually we’ll allow third party services to use and display the App Score via a private API.

 

WHAT ARE THE LIMITS OF THE APP SCORE?

The limit is simple: The less data, the less the score is relevant… (makes sense no? ) If, for example, an app is live without any track record, rating record, or developer record, there is no way to evaluate the quality of the app. The only way is to try it. But very very few apps stay without track record for long. Within hours, apps usually start to collect some interesting data points which we capture.

The other limit of App Score is that it is algorithmic, just like the Google PageRank. A few apps could be trapped by mistake within our Crapp filter as we have limted information on them, but that will be soon completed by a crowdsourced system in addition to more safeguards detectors that will be added.

For most apps the App Score is rather static and won’t change from one day to another: it is unlikely that an app goes from RED HOT to “Hmmm” or that a crappy app becomes RED HOT. But certain events like a major update can affect the App Score of an app. We track those events and obviously the App Score gets updated.

 

TWO MORE THINGS

We’re not done yet with the App Score.

1. We will unveil many innovations over the next few weeks and we’ll extend it to Android as well.
2. We’ll include many external new parameters and will open it to crowdsource for improvements and fine tuning.

You can find the App Score of any app either by searching on our site Appsfire.com or by browsing and searching apps in Appsfire Deals (soon also in Appsfire)

Hope you will like it!

 

UPDATE

Check out the excellent coverage on GigaOm.