23rd
Results from the iPhone’s new rate-on-delete dialog
Marco is one of the founders of Tumblr (the blog platform that this blog is on). He also has an iPhone app called instapaper. The point he makes below is very interesting in how rating systems do not always reflect the real sentiment of the overall user base.
Marco is pointing out a real problem that Apple is creating, if you create a system that over punishes app publishers with negative reviews then you take out the incentive for them to work on your platform.
If you are running a product/platform balancing user feedback to establish the true satisfaction can be very hard…
The new rate-on-delete popup in iPhone OS 2.2 has already destroyed the average for Instapaper Free.
Before 2.2, it was a solid 4.5-star average (the same as Instapaper Pro). Since then, very few written reviews in the App Store have been added (about 10 of the 180 total in the U.S. store). And if you look at the written reviews (sort by Most Recent), they’re overwhelmingly positive, with very few lower than 5 stars.
Now, the average for Instapaper Free is 3 stars, and since it was pulled down so significantly and so quickly after this change, I expect it to go lower.
I wouldn’t be complaining if this were an accurate reflection of people’s usage and opinion. But it’s not. I estimate that the number of delete-ratings to affect it so far has been in the low hundreds. But thousands of people use Instapaper Free every day, as measured by the server-side update API. If they’re using it every day, they’d probably rate it higher than 3 stars. If even a small portion of them rated the app, they’d dramatically overpower the low average set by the deleters.
We really need a corresponding rating dialog on positive experiences. This is going to destroy too many good apps’ averages.
