On detectives

It’s the New Year, and both ITV and BBC have come out with their headline crime dramas to keep us cosy during the long miserable nights in that make up Dry January. For the BBC it’s Silent Witness and Death In Paradise, while for ITV it’s Endeavour and Midsomer Murders.

I prefer the longer format of the ITV series. I also like the fact that they tend to take their structure and references more seriously, too. Which was why the first episodes of both new series were notable.

Firstly Endeavour broke one of the Decalogue (the golden rules of crime fiction defined during crime fiction’s heyday). Fairly and squarely, no apology, 85 minutes in we get to the big reveal and it’s a rule breaker. There’s no way the writers didn’t know what they were doing. Interesting, if nothing else.

And then Midsomer Murders, only 3 days later, was an episode without a single murder. Let’s be clear, Midsomer is second only to Cabot Cove in terms of body count, so an entire episode without even one measly murder? Unthinkable.

Crime stories are predictable. They follow rules. That’s why we love them. It appears ITV has other ideas…

Using Gmail and Thunderbird: Unsubscribing from the automatic “All Mail” folder

If, like me, you can’t get to grips with the Gmail web interface (however good your colleagues tell you that it is) then you’re going to end up with a desktop IMAP client. And despite it’s many and varied flaws if you’re on Linux it’s been Thunderbird for a long time now.

Because Gmail operates on the concept of a single, searchable, tagged, bucket of mail under the hood there is an “All Mail” folder, in which all of your mail sits. This is exposed via the Gmail IMAP server, and your email client is automatically subscribed to it. If, like me, you have tens of thousands of pieces of mail this will mean that Thunderbird tries to download every piece of mail from the server into the “All Mail” folder, as well as syncronising the same pieces of mail in whatever folder structure you’ve created for yourself. Even if you operate on Gmail’s model of leaving everything in your Inbox you’ll download everything twice – once into the Inbox and once into “All Mail”.

Thunderbird allows you to unsubscribe from folders, but the current interface is a bit weird.

Right click over the “All Mail” folder, and choose the Subscribe… option. Then untick every folder except “All Mail”, and hit the Unsubscribe button on the right. Then hit OK, and then restart Thunderbird.

Now all your Gmail folders have disappeared! Don’t panic. Right click over the empty “Gmail” folder, select Subscribe… again. Then check “Bin”, “Drafts”, and “Sent Mail” and hit the Subscribe button, followed by OK. Restart Thunderbird again.

Finally, you are in the right place.

Good luck.

Update on care.data

Unsurprisingly it all kicked off about the care.data scheme that I last posted about, and it turns out that there are a lot better informed and better resourced people than I with an eye for this sort of thing. Reading, for example, the Guardian’s coverage of the topic will be far more informative than anything I could post.

I will however, update on their response to my letter.  I had hoped that by asking for them to clarify how and who the information was shared with, and thereby allowing me to take an informed decision I would get a nice, hopefully templated, response.

Instead I got the following:

Dear Mr Theyers

Thank you for your question about care.data. If you wish to opt out you must inform your GP. If you have any questions about how your data will be used please contact your GP.

Best regards

Not exactly the most positive response. Nor the more informative.

care.data and the HSCIC

You’ll probably have seen that Twitter is full of people encouraging us to opt out of the government’s care.data scheme (whereby non-anonymised health records are made available to non-NHS organisations).  The scheme is run by the HSCIC and, unless you specifically opt out with your GP, your data will be taking part.

What’s painfully unclear is exactly who gets access to this data. Universities and research organisations? Yay! Insurance companies and law enforcement? Boo! (you get my drift)

The magic key is “section 251 support“. Organisations with section 251 support have access to this information. So I’ve written to HSCIC asking them about it:

Dear Sir/Madam

As you may have seen Twitter is all a-flutter today about opting out of care.data.

I wonder if you could tell me a couple of things so that I can inform my decision as to whether to opt out or not?

  •  is there a public and up to date (and regularly updated) list of the organisations that have section 251 support?
  •  is the process by which an organisation applies and is given section 251 support public?
  •  is there a public and up to date (and regularly updated) list of current applications for section 251 support?

While I understand the need for the NHS to share information more freely to improve care outcomes I am extremely concerned about third parties (such as insurance companies) getting hold of this information and as such hope that you will be transparent about the organisations with access to this information.

Yours faithfully

Andy Theyers

I’ll let you know if I hear back.

A year in music 2013

2013. In music. Spotify has taken over. Whether I like it or not, and despite the fact that I continue to purchase CDs like some weird old dinosaur, almost all of my music consumption is done via Spotify. I promised myself this wouldn’t happen but it’s just too damn convenient. To try and make up for this I’ve made a concerted effort to buy from BandCamp if at all possible, but too few bands use it to make it a real viable way of guaranteeing that the band gets the money for what I’m listening to.

Albums of the year:

  • Volcano Choir: Repave (spotify) (amazon). Justin Vernon’s 4th album in the various versions of Bon Iver. The best yet.
  • Matthew E White: Big Inner (spotify) (domino). Gospel music like you’ve never heard.
  • William Onyeabor: Who Is William Onyeabor (spotify) (amazon). Fantastic, heavily disco influenced, Afro-beat from the eighties.
  • Darkside: Psychic (spotify) (amazon). Proper headphone music.
  • CHVRCHES: The Bones Of What You Believe (spotify) (amazon). 2013’s perfect pop album.

Albums I discovered and loved in 2013, despite earlier release dates:

  • The Asphodells: Ruled By Passion, Destroyed By Lust (spotify) (beatport). Andrew Weatherall. ‘Nuff said.
  • Mungolian Jetset: Schlungs (spotify) (amazon). “Too trippy for it’s own good” said the reviews (see also Tranquility Bass, below). Bollocks, says I. Moon Jocks and Prog Rocks is disco perfection.
  • Moonface with Siinai: Heartbreaking Bravery (spotify) (amazon). The Wolf Parade album I’d been waiting for.
  • Gallops: EP (spotify) (bandcamp). Um. Just try it. You might like it.

And let’s talk about albums released in 2013 that I desperately wanted to like…

New Arcade Fire albums have been, for the last 6 years, genuine highlights. I just couldn’t get behind Reflektor. Or rather, I thought the first 30 minutes were fantastic, and the rest was shite.

John Grant’s Pale Green Ghosts. I loved Queen of Denmark. Pale Green Ghosts is lyrically just a bit too close to the bone to listen to too often. Which is a shame, because he’s clearly brilliant.

And Midlake’s first album without Tim Smith. To be honest I’ve not listened to this enough to know outright. I think it’s definitely better than Courage of Others, but probably isn’t quite Van Occupanther. Time will tell – I’m still trying.

I managed to get a lot of gigs in this year too. I gig for the dancing so there’ll be no surprise to find that various versions of Hawkind (both Psychedelic Warlords and Hawklords), Wooden Shjips and LoOp provided the best gigs of the year by a country mile. The surprise package was Parquet Courts (who were fantastic, against expectations) and the disappointment was Junip (just too much head nodding and beard stroking).

I continue to be confused why no one dances at gigs any more. I’m clearly getting old. Perhaps the gig highlight was being shushed by some earnest beard toting chap during the opening chord of the Wooden Shjips set. We are not hear to listen, people, we are here to dance. He sidled away from us quite quickly once our limbs started flailing…

As previous years I’ve compiled a list of my most played albums per month, and linked to a Spotify playlist for every album played in the given month.

  • January: Tranquility Bass new album. Back in 1997 Tranquility Bass released what fast became my favourite album of all time (Let The Freak Flag Fly). Favourite to the point that I don’t listen to it too often, in case I accidentally ruin it through over-playing. Yeah. I know. Anyway. He disappeared. And then in late 2012 he reappeared. With the rather excellent Heartbreaks & Hallelujahs. You should try it.
  • February: Matthew E White’s Big Inner. Oh, as they say, My God. One or other remix of the main track Big Love was on pretty much every mixtape of the year, but the whole album (and unremixed Big Love) is absolutely glorious southern swampy soul music to die for.
  • March: I’ve never really listened to hip hop. Through not having a route in, rather than any disliking it. In March someone spent the time working out what I might like. Hence A Tribe Called Quest’s The Low End Theory being my most played album in March (along with Black Star and Madvillainy).
  • April: All excursions into hip hop aside, I’m still a fuzzy guitar space rock kind of guy… So Life Coach’s rather excellent Alphawaves & the new Besnard Lakes happily filled April.
  • May: Thisismyjam introduced me to Matthew E White in January. It also introduced me to !!! in May. Thr!!!er became one of my most played albums of the year, and led me to a more detailed investigation of the DFA/dance punk thing: The Juan Mclean, Outhud and Maserati all ended up on rotation too.
  • June and July: God I miss Wolf Parade. Seriously miss them. So Moonface’s Heartbreaking Bravery was a fantastic discovery. Spencer Krug’s half of Wolf Parade was the half I really liked – this is the natural successor to Wolf Parade for me, in a way that Handsome Furs and Divine Fits never were.
  • August: August (like December) is always musically a bit light through holidays and so on. Junip made a perfect soundtrack to those few evenings I was able to spend in the garden.
  • September: There is no question that I love Justin Vernon. Not only that, I prefer each new album to the last. Volcano Choir’s second album (Repave) is the best yet.
  • October and November: Like 2012’s Alt-J, CHVRCHES are all a bit calculated, industry insider, type stuff. But it’s absolutely perfect pop music, so fuck it.
  • December: Dunno why I bother recording what I listen to in December. Sufjan Stevens. Every year. (Although it’s worth noting that CeeLo made a Christmas album!)

Amazon free Christmas

It will come as no surprise that, whilst truly disliking Amazon, I continue to purchase more and more from them. From all my CDs and books through to shoelaces, toasters, kettles, headphones, humane mouse traps, flea spray, you name it…

Following on from seeing Brad Jones’ keynote at FUTUReBOOK last week, from recent BBC documentaries (“Robert Peston Goes Shopping” and “Amazon: The Truth Behind The Click“), and from the very public way they mess with the UK tax system and our high streets I thought that this Christmas I really would do what I’ve been saying for years and actually try and have an Amazon-free Christmas.

I’m going to record my experiences and publish them here in the New Year.  So far, two purchases in, I’ve ended up paying a premium of £1.94 and have made both purchases from play.com having tried independent online retailers first for both but failed in some way both times.

In defence of sluts

As fun as it is to take the piss out of Godfrey Bloom’s ludicrousness, it’s worth remembering that just as we thirty/forty something bleeding heart liberal Twitter users aren’t down with the kids we also aren’t actually down with the older generation either.

In 1963 Katharine Whitehorn wrote an article for the Observer called “In Defence of Sluts”. In it she meant slut like Godfrey Bloom meant it. It was considered a watershed in women’s journalism. For a brief period every one of the newly liberated single girls with jobs, living alone in bedsits or in groups in shared houses, was a slut; she identified with the article, she bought books like Whitehorn’s own “Cooking in a Bedsitter”, and she took dirty clothes from the clothes basket because they were the cleanest thing in the house.

In 1963 my mum was 18 and living in the YWCA. My dad was 20 and living “in digs”. To them the word slut has always meant slovenly. They’ve always known that it could also be an insult — sexually derogatory — but like so many words they’ve never really accepted the change in meaning.

And that’s the generation of the UKIP heartland.

So when we all get up in arms about the out of touch dinosaur that is Godfrey Bloom and his ludicrous choice of words pretty much the entire UKIP core vote sighs dramatically and slams the door to its bedroom shouting “you just don’t understand” over its shoulder as it goes.

For all of Nigel Farage’s complaining he knows that what it really did was to highlight the difference, highlight how even David Cameron doesn’t understand them and build the barricades just a few inches higher.

VAT on eBooks, and yet more reasons to dislike Amazon

We all know that Amazon doesn’t pay tax. It’s a matter of public record and the subject of many a vainly attempted Christmas boycott. One of the many wheezes they use is to sell all of their digital books from a Luxembourg subsidiary. They do this because Luxembourg only charges 3% VAT on digital goods, whereas the UK government levies 20% VAT on ebooks.

This means that the largest retailer of digital books in the UK pays only 3% VAT on its ebook sales, but any UK based retailer is forced to pay 20% VAT. This dramatically skews the market in Amazon’s favour, and is hugely detrimental to any attempt by another retailer to break Amazon’s grip on digital reading.

Right now, for example, a book sold at £4.99 by Amazon nets Amazon £4.84 after tax, but only nets a UK based retailer £4.16. In a retail sector with such aggressive competition that 68p is vital and at the moment UK.gov is heavily favouring Amazon’s position.

Amazon are also playing fast and loose with the VAT rules in other ways. Right now a self published author through KDP only pays the 3% rate that Amazon pays. This means that the author lists their book at £1.93 and it appears for sale at £1.99 on the Amazon site. However, Amazon is charging traditional publishers the full 20% rate, despite only paying 3% themselves. In other words, they are screwing additional margin out of the traditional publishers while further tipping the balance in favour of the self published authors (and, one assumes, those published by Amazon themselves).

Currently there is a ruckus about VAT on ebooks in the UK, with a legal challenge to the validity of the tax underway. Physical books are VAT exempt in the UK, and there are two strong legal arguments that ebooks should be too. Firstly the book VAT exemption simply uses the word “book”; a digital book is still a book and any attempt to levy VAT on an ebook could be ruled a misinterpretation. Secondly, previous case law has ruled that where an item meets a consumers needs “in the same way” it should be classified similarly for VAT purposes; clearly an ebook satisifies this criteria too.

I’m hoping HMRC rules that digital books are VAT exempt:

  • Physical books are already VAT exempt for a good reason; as we move to digital by default we don’t want a stealth tax on reading
  • The UK government doesn’t get any revenue from ebook VAT anyway, as Amazon pays its VAT in Luxembourg
  • It’s hugely skewing the UK market in Amazon’s favour, making competition even harder
  • If the EU does force France and Luxembourg to increase their VAT rates on digital goods while we are VAT exempt it will create an advantage for UK online book retail throughout Europe

Back in 2001 Gordon Brown scrapped Betting Duty (where the punter paid 9% on either their stake or their winnings) because offshore gambling sites that paid no Betting Duty were skewing the market and killing UK bookmakers. Now is the time for a similarly enlightened view on ebook VAT.

[1] Amazon force publishers to pay full VAT

[2] VAT may be dropped on eBooks

[3] Gordon Brown scraps Betting Duty

A year in music 2012

My albums of the year:

  1. Goat’s World Music (spotify). Daft costumes and songs with Goat in the title. I don’t think they intend themselves to be taken seriously, but it’s hard not to. Think acid soaked math rock; proper WTF, proper awesome
  2. Peter Broderick’s These Walls of Mine (spotify). Apparently this is a collection of “experimentations” but it sounds pretty darn complete to me… Glitchy, sparse, laid back and laid bare; I just love this album
  3. Golden Void’s Golden Void (grooveshark). Almost a Black Sabbath tribute album, but original enough to be much more than that. Gloriously heavy heavy psych
  4. Breton’s Other People’s Problems (spotify). I couldn’t stop listening to this for months. Like a South London LCD Soundsystem or Alt-J’s ASBO-toting cousin. Disaffected and dark, but pop music all the same
  5. Heavy Electrics (spotify). I might be on my own with this one. Driving, pulsing, krautrock designed to be played far too loud in seedy basement bars in Bladerunner’s Los Angeles.

Yet again I stumbled across a bunch of albums from previous years that I’d managed to miss, so special mention has to go to Citay, Yellow Moon Band and Tweak Bird (you might sense a theme among them…)

Like the last couple of years I’ve also done a month by month breakdown of what I listened to the most (as compared to what I liked the most). In a “measured life” kind of way I also started creating spotify playlists of every album that I enjoyed in each month. These are intended for my own (imagined) enjoyment in my (imagined) dotage, but they’re linked to too, in case you’re interested:

For those who wish to run the same analysis of their own habits you can download the script from my github. As long as you have python installed it will work like so:

python fetcher.py username year


python fetcher.py offmessage 2012

A dirty race to the bottom

tl;dr: When there’s only one book retailer (and increasingly only one publisher) reviews are the only differentiator between books, yet they are being widely abused. At the same time there is a rapid race to the bottom of both content quality and marketing tactics. The utopian vision that Amazon’s self publishing tools presents is tainted and won’t get better until Amazon accept some responsibility, or until an alternative retailer can make a big enough dent in the market to matter.

A story for you.  My wife has written a diet book.  Claire is doing the diet herself, it’s working for her, and the text and recipes within the book are her own original work.  And people are (or were) buying it.

Let me also state right now that, despite how it will doubtless sound later on, this is not intended to be a whinge about competition.  As dispassionately as possible I’d like to discuss how Amazon, as sole retailer and publisher, also owns the only way consumers can discriminate between products, and how this is fundamentally broken.  I’ve deliberately not linked to the books in this article to try and avoid making this anything other than a story about some stuff that happened.  That this is based on Claire’s experience is a necessary fact, but not the point.

Claire’s book is published via Amazon’s KDP and Createspace services.  This is an increasingly common way for authors to bring their books to market; between them they provide both digital and physical publishing tools to allow authors to self publish their work on the Amazon network of sites.

Private pressings (or “vanity publishing” as it’s sometimes disparagingly known in the trade) have always existed.  Having invested long years of your life in preparing your magnum opus it’s obvious (if you have the resources, at least) that the shortsightedness of a closed, cliquey, distant few shouldn’t affect whether or not people have access to your book.  After all, we all have a book inside us; self publishing allows those with a little cash to taste the pleasures of seeing their name on a dust jacket.

It used to be that when an author approached a specialist book binder they were gently talked down from a print run of thousands, while at the same time talked up into taking additional services such as proof reading, editing and jacket design; all the things that help turn a manuscript into a book, in fact.  And so what might have been two thousand error strewn paperbacks stored in the spare bedroom for years to come becomes 100 or so beautifully bound and grammatically correct hardbacks for distribution to exactly the right bookshelves.  The author’s goals are met, the printer’s margins have been increased and the practical realities of the dead tree publishing industry explained and (in most cases) understood.

Like with so many old school media industries a few years ago the Internet began to change all that.  First there were print on demand services like Lulu, and then five years ago Amazon waded in with KDP (Kindle Direct Publishing) that allowed authors to sell digital versions of their books directly on the Amazon site.  And then two years ago Amazon bought Createspace (a print on demand rival to Lulu).  At this point the circle was complete; with a few simple tools an author can now get both paper and digital versions of their books for sale in the world’s largest bookshop, linked together, consistently listed and ranked.

As a long term resident of the Internet I whole heartedly approve of yet another dinosaur of an industry being disintermediated.  Seriously, an industry where the recommendation is to send a copy of your manuscript to upwards of 50 people from whom you are warned you shouldn’t expect a response within 6 months is ripe for being torn a new one.  Amazon saw that, and started tearing.  And more power to them, frankly.

In this process we have disintermediated traditional publishers who took, on average, 75% of net receipts (this is important – it means the author bears the cost of discounting to retailers) and replaced them with a retail channel that offers 65% to 70% of list price (meaning that any discounting is borne by the retailer).  A £7 paperback published by a traditional publisher will earn the author around £1 per copy.  Traditionally published ebooks are sold at the same price and under the same contractual terms, so that £7 book as an ebook also earns the author £1.  Published via KDP for £7 the book will net the author around £4.55 per copy.  Sold as a print on demand paperback via Createspace it will earn about £2.10.  And let’s not forget that due to the structure of publishing deals (net receipts again) if that £7 traditionally published book is sold in Tesco it might only earn the author 10p per copy sold.

So if you know, in your heart of hearts, that your book is not going to be one of the 10 a year that get the full marketing weight of the publishing house behind it (and that’s increasingly true for many major authors let alone new ones) then why go through the heartbreak of dozens of rejection letters when you could get straight to market via Amazon, sell your book digitally for £4 a copy and earn £2.50 a sale?

Of course, you’d be advised to buy freelance proof reading at a minimum (no one can proof their own work. No one), and maybe editing too.  Proofing costs somewhere in the region of $150 for a medium sized novel (there are literally hundreds of people offering this service in the Kindle forums), so hopefully won’t break the bank.  And jacket design isn’t quite as important as it once was, now that people only see a postage stamp sized thumbnail of the jacket before buying.

All very utopian, no?

The big question, of course, is how does the author market their work?  65% of no sales works out just the same as 25%.  In the traditional publishing view of the world there is poster and TV advertising, weekly book supplements in the newspapers, placement on the 2 for 1 table in Waterstones and a myriad of other ways where the publisher’s resources ensure a route to market.

Self-published authors can’t tap into those routes.  There are no posters, no adverts, no ready and willing newspaper columnists with a deadline and inches to fill and no two for one table upon which to place their books.  Instead, after self publicity through channels like blogs, Twitter and Facebook authors are singly and utterly reliant on Amazon reviews.

And therein lies the rub.  Amazon’s review system places very few controls on who can review what.  You can’t, for example, leave reviews on the same item from different accounts that share a credit card, but that’s about it.  That’s not much of a control though; all it takes is a quick ring round your friends to get the reviews up.  Or if you’re too embarrassed to ask your friends you can buy them on Fiverr instead.

A couple of fake five star reviews to kick start the book isn’t too bad.  Friends and family want you to succeed and are going to rate your book highly whatever.  What’s killing the Amazon review system is the lack of control of negative reviews from competing authors and publishers.  Consumers have to take reviews at face value, and a couple of scathing reviews completely destroy sales.

And here’s where we get to Claire’s experience.  The book she’s written is about the “5:2 diet”.  There’s been a bunch of publicity around the diet recently, following a BBC Horizon documentary and a couple of articles in the mainstream press (including one in The Sun by, of all people, Dom Joly).  The big publishers move too slowly to get books out quickly, and when everyone suddenly wants a book about this miracle diet it’s only the self published authors who are there to service that market (take note, publishers, by the way). I’d argue that this is one of the many beauties of the KPD/Createspace system.

Claire’s book was receiving some nice reviews and selling very well until the negative reviews started appearing.  And they were vitriolic: “Cashing in”, “Read the Times instead”, “rubbish”, “This book has been rushed and it shows”, “Nothing new”.  It’s hard not to take those sorts of comments to heart. We spent a lot of time staring at the book and wondering how some people could interpret it the way they had, and why the reviews were so polarised.  We certainly didn’t feel it was rushed, it wasn’t badly formatted, all the content was original, including calorie tables and tried and tested recipes, and the description of the product made clear that it was a short book.

It took us a while to work out what was going on, but eventually we twigged that other people had had the same idea as Claire to write a book about the diet, and saw negative reviews on her book as a way of boosting the sales of their own.  At one point, with some lovely (real!) positive reviews of the book, she was selling close to 80 copies a day.  Now that the book has been swamped with negative reviews (all completely at odds with the positive ones) she’s down to 10 copies a day and falling.  At the same time less scrupulous publishers are riding high in the charts, bouyed both by the positive reviews they’ve left on their own book and by the negative ones they’ve left on Claire’s.

We’ve complained to Amazon.  We’ve marked reviews as inappropriate. But nothing’s been done. And now we live in fear of another negative review hitting our main competitor, because each time a bad one lands on theirs, another negative one appears on Claire’s.

Of the ten 1 and 2 star reviews on Claire’s book 5 are not verified purchases.  (Of the verified purchases one is a genuine complaint about a misprinting by Createspace, one is from a nutter, and one is by the author of the “rival” book, but more of that later.)  Given that the book is published by Amazon and only available through Amazon it seems strange that such a high proportion of reviews are not verified purchases; the user leaving the review has to explicitly choose to remove the Verified Purchase message from their review.

Indeed, if we look at other KDP/Createspace books where there are lots of reviews (I used books with more than 30 reviews) and where the reviews are relatively consistent (i.e. not polarised) we can see that on average only between 10 and 15% of reviews are marked as not verified purchases.  On books where the reviews are polarised the reviews show the same characteristics as those on Claire’s – a disproportionate number of either the positive or the negative reviews are not verified purchases.

To continue this train of thought we can examine the reviews on the rival diet book.  Here we see that none of the 5 star reviews are verified purchases (and all were written in the same 48 hour period), yet both of the one star reviews do come from verified purchases.

And it’s not just the unverified reviews that can be untrustworthy.  One of the negative verified purchase reviews on Claire’s book is from a user that calls herself “Crafty Mama”.  She used to have a five star review on the rival book until it disappeared (one assumes because she tried to leave another review on the same book from an account that shared a credit card).  This user has reviewed very few books but appears to have very strong opinions about Ukelele books.  The Ukelele book she likes is published by the same publisher as the rival diet book.  The one she doesn’t is not.

There’s also an unverified 5 star review left on the rival book by “Cooking Mama” who has only reviewed one other book, published by the same publisher and being a book to which “Crafty Mama” has uploaded customer images.

With most fake reviews you can’t prove anything. If, like Cooking/Crafty Mama, you do it consistently there’s an easy to follow trail, but there’s no real way for a consumer to tell if a one off review is fake or not. And it’s not like most people would even notice those with the obvious trails – it’s just too many clicks.

I can only assume that if this is happening to Claire’s book (a pretty niche title) it’s happening to hundreds if not thousands of others as you read this.

Ideally the vision of self publishing books via Amazon is that every author gets a fair crack of the whip in an open marketplace.  One of our friends (the lovely Sophie, whose second novel is out now) had huge success with her first novel last year, selling tens of thousands of copies via KDP, and it’s that type of story that Amazon would have us buy in to.  What we aren’t hearing so much about (yet) is the vast morass of books produced by unscrupulous authors/publishers who are unafraid of violating copyright or plagiarism, or of damning competitors with fake reviews.

Where the Amazon self publishing tools should really be helping is in the ‘long tail’ of low volume books where publishers are less and less inclined to take an interest. Unfortunately it’s exactly in these low volume books where the reviews are least well policed. There’s been considerable noise and bluster recently about established authors posting both positive and negative fake reviews, but down the sales ranks and in the niche subjects no one’s paying any attention. Yet it’s there that thousands of authors are trying to eke out a living with no external support.

Because the review system is so baked-in to the Amazon experience I would argue that it has to be managed very closely – certainly more closely than it is now.  Amazon’s success with the ubiquity of the Kindle is even more powerful than Apple’s early success with the iPod.  Right now the self published book market doesn’t have an editorial voice or a recommendation system outside Amazon.  Unlike, say, Pitchfork or The Quietus for music there isn’t an established independent voice (or voices) yet. The book social networks like Goodreads are a way off providing recommendations of the quality of Last.fm. And because getting content onto a Kindle by any other means than via the Amazon store is so cumbersome there’s little to no opportunity for direct sales via your own site or via an alternative service (like Bandcamp, for music).  In every way we are more tied to Amazon and the Kindle than we ever were to iTunes and the iPod.

The utopian ideal of disintermediating the large publishers is turning into a filthy race to the bottom, shepherded by Amazon.  Their apparent position of “as long as you buy something we don’t care” allows the unscrupulous to rise to the top in all but the very biggest genres, ruining the experience for authors and readers alike.

What could Amazon do to fix it?  I think there are a couple of easy steps.

Firstly, if a book is published via KDP or Createspace they should only allow verified purchase reviews. These books are only available through Amazon, so any review that comes from an unverified purchase is either too close to the author or acting against them.  It’s not a huge barrier to fake reviews, of course, but it stops a quick ring round your mates having quite the effect it does now.  This would solve almost all of Claire’s book’s problems in one fell swoop, and I imagine would solve a lot of other people’s problems, too.

Secondly, they need to take the ‘flag this review as inappropriate’ reports seriously.  If I can identify a proportion of the fake reviews simply from the public information available on the site imagine how easy it would be with additional information like account age, IP addresses and browsing history?  Claire’s income from her book has dwindled to nothing since the negative reviews started piling on; I don’t think it’s too much to ask from Amazon as the only intermediary between author and reader to at least investigate these reports and remove the obviously fake reviews.

A couple of relatively simple algorithms would make this easy, I think.  Viewed less then 4 items on the site?  Only bought one item?  Repeatedly view only the same one or two items?  Repeatedly logging in from the same IP as other users with reviews on the same item?  Always logging in from the same browser session as another user with reviews on the same item?  Always logging in from the same IP, but with no cookies set (using private browsing)?  Funded by gift card, rather than credit card?  Reviewing items negatively/positively in the same chart as items reviewed positively/negatively by related users (shared credit card, IP, etc)?

If 4 or 5 of those 8 questions come up yes, just don’t publish the review.  For a long time I had assumed that it was simply a case of the more reviews the better, but when upwards of 50% of reviews are fake this no longer holds true.  Only 3 reviews, but all good quality, is far better than 6 good quality ones out of a total of 12.

At the time of writing Claire’s book has 16 reviews.  We’re pretty confident that 9 of the negative ones are fake (and 2 more are just insane, but you can’t really adjudicate for that).  At the same time, the rival book has 10 reviews of which we’re confident 6 of the positive ones are fakes (timing, lack of verified purchase, links to other books from the same publisher, etc).

Yet the only way readers can decide between the two titles is by this review system.  It’s all a bit messed up.

Claire has just published a second edition of her book. What do we do if that starts getting poisoned by fake reviews too? Stoop to the same level and start slating our competitors? Or take some ineffectual moral high ground and watch sales plummet again? With the Amazon marketplace structured the way it is these are currently our only two options. Neither work for me.

Technology, cricket, music, food.