Quantifying research quality and impact is a tricky issue. "Altmetrics are new metrics proposed as an alternative to the widely used journal impact factor and personal citation indices like the h-index." Instead of waiting for your peers to read and cite your work, you can now get instant kudos from retweets and Facebook likes.
Of course this can be a useful tool for tracking engagement (or massaging your ego), but not as a serious metric for quality or impact as many are now proposing (and some institutions implementing). I was disappointed to receive an email this week from the journal Energy & Environmental Science ranking recent publications by the arbitrary numbers given above.
Some potential flaws (for better worded opinions: [1],[2]):
- Open to manipulation. If science was a popularity contest, then virtually every major breakthrough would have been forgotten. Most Nobel laureates have a story of how their pioneering work was rejected for publication in a top-tier journal or poorly received by their peers. The danger here is aggressive marketing, e.g. a legion of university minions spamming all forms of social media or coercing students and alumni to "support" their academics. Similar to the music industry, the crowd pleasers from big institutions would come out on top, while real science could be driven underground (although I would be interested to experience no wave chemistry).
- Encouraging exaggeration. As press offices fight for attention, press releases are becoming increasingly misleading. In the field of solar cell research, paradigms are apparently being shifted multiple times per week, while there is rarely an actual increase in efficiency or a change in architecture. Real progress takes time, while inflated claims gather clicks. Maybe I should just stop looking at Science Daily.
- Arbitrary weightings. All citations are created equal, but does a one-click "retweet" or "like", imply that someone has read, appreciated or even opened the paper? A tweet from my mum should count double of course.
It is important to see research metrics evolve, but that fact that a company can produce a pretty graph doesn't make it any more relevant. Never Mind What's Been Selling, It's What You're Buying.