On conflating Reviews and Comments

Sometimes, when you look at distribution systems with build-in user interaction, you will see that system-designers haven’t really thought about what the user interaction actually is.

More specifically, it seems that they haven’t really thought about who the interactions their system allows are for. Is the text field for readers to give feedback to authors, or to other readers? Or is it just for the website itself to know?

An example of this would be YouTube. YouTube’s user-interactions consist of a convoluted subscription system, a like/dislike vote bar and comments. The convoluted subscription system is for the watcher? But then the author of a video also gets informed in detail how much growth and whatever they have, so maybe it’s for the author? The like/dislike system is… no clue for who it is, it doesn’t show up in search results or elsewhere. But apparently people clicking on the thing is good for the recommendation algorithm??? Then there’s the comments system, which… is for the audience to talk to the author? Or is it for the audience to talk to the rest of the audience? Some certainly treat it as such?

Compare this to for example, the comment and like functionality on a blog. It’s clear that this is intended to communicate something to the blog writer, maybe to other people, but like, everyone involved knows the blog writer sees it. Similarly, on a link aggregator, it’s clear the discussion is for the other readers of the aggregator, and the likes/dislikes are about recommending the link to other readers of the aggregator.

Places like fanfiction.net have a function called ‘reviews’, but a cursory investigation reveals that people just treat them as comment sections. AO3, similarly, has a like and comment functionality which understand these to be for the reader to the author. It also has a recommendation system for readers to recommend to other readers, which may include a description, that authors are not informed about.

A lot of the big software stores are especially weird in this regard: They have a review section, with a rating, but then the author of the software can comment on the review. So, is the review for other users, or is it for the author of the software?

The thing is, the meaning of a review changes significantly whether it exists for other users or for the author. If you rate something for other users, it is clearly a way to indicate recommendation. If you rate something for the author… it’s a grade, basically, isn’t it? Like high-school?

And it gets even more confusing, some of these stores will list the top rated entries, but, like, what does top-rated mean? Is it the report card grade, or is it an indication of how popular the entry is? It’s no wonder that algorithms like Youtube’s and Reddit’s have just given up on using the values themselves, and instead use engagement. Which leads to a pretty clear meaning: People are emotioned enough to go into motion to press that like button. A top ten of stuff that makes people really upset, if you will. And I am hardly the first person to state that this isn’t very good for everyone’s sanity.

The reason I am worrying about this is because I have noticed that the KDE store, and by extension, KNewStuff, conflate comments and reviews. Which seems to be due to a hack (a review being a comment with a rating attached), but I think we might be able to conceive of solutions…

Distribute the reviews

So, the first problem is that the reason we expect reviews in content distribution systems is because readers and users should be able to find something that appeals to them. Reviews then are another user’s opinion of a thing, intended for potential users. By putting reviews into a spot where the reviewer knows the subject’s author can read it, even comment on it, a dilemma appears: Are they supposed to prioritize being polite, or are they supposed to prioritize honesty? It’s possible to be both, but that requires a lot of nuance and energy to accomplish. So in effect, this is a usability problem where we are expecting the user to be really skilled.

Similarly, by putting the reviews in a place where the reviewer knows the subject’s author can read and comment on them, the author is also put into a precarious position. If a review is really bad or insulting, should the author respond nicely (reviewer might be having a bad day?) or flag it? And what if they don’t respond at all? Why are we making all these people so potentially anxious?

Let’s distribute the reviews. The basics here would be that if someone wants to review a thing, we help them write something on their blog, or microblog, or maybe one of the 5 chat protocols KDE supports, mailinglist?

For aggregating, we could use something like webmention (with the vouch extension, because otherwise mods will go nuts), or special API to let the user link their review to the thing (if a user doesn’t want to link the review, that’s their choice).

The store doesn’t necessarily need to be the aggregator, we can also just look at the social feeds someone has, such as rss feeds or other social media feeds (I don’t quite know the logistics of this yet, let someone store a review for later, maybe?). This has the side-effect people will more likely get the recommendations from folks they know, making them more effective.

We also don’t really need to worry about authors not reading reviews, because they do that regardless, the masochists, but we should make it possible for an author to ignore a review.

I’m still not quite sure about the ratings themselves either. Like, we can disambiguate it by stating that it’s a rate of recommendation, not a rate or ‘whether or not this is the best thing to exist’. But another thing to think about is whether or not it applies to all things equally. Like, with software, a 5-star rating could imply that it did everything the reviewer needed from it, and a 2-star rating that it did not. But for a story this is much harder to determine, and I’ve been wondering if a different kind of rating system, like one based on a limited set of emoji? A horror story is going to be rated 2 by someone who hates to be afraid, and 5 by someone who loves the feeling, but both will likely rate it with ‘😱’. It’s proly not terribly useful for top-rated lists, but at the least the reader knows what they’re in for when selecting the story? But that’s all pipe-dreams for now.

Comments

One of the other issues I see with the store and stores in general is that putting content on there means there’s yet another place where an author needs to manage comments. And that can be quite a deal breaker.

I spend some time on getting OPDS 1 implemented into KNewStuff, because it’s a really simple way of representing available content, and I am thinking that for comments I am going to let people link in an rss/atom feed with comments. All the major content management systems have the ability to generate feeds for the comments of a single article, so authors can just link the comment feed for a blogpost, and then on our end we should direct the user to go to the blog if they want to comment.

I think it would be pretty valuable if people could disable comments and instead point at the feed where they keep their comments. There’s some side effects there we need to keep track of, like making sure it’s clear these comments are on that blog and not on the store, as well as some vetting of the comment feed in general, but at the least it’d be in a place where the author can actually control.

This kinda ends up making commenting on the store somewhat pointless at first glance. We could try to see if some of the distributed/federated stuff is useful for assisting people to comment on the author’s comment-feed, but that’s also something that needs investigation.

Conclusion

I’ve been thinking about this all a lot over the past few years. On Saturday there was an Akademy talk by leinir about distributed app stores, and there’s going to be a birds-of-a-feather about that on Friday morning. I might not be available then, so I just wanted to get my thoughts about reviews and comments out there.

In general, I think my adjustments tend to come from a place where I have experience sitting in the author chair, as well as consuming a whole lot of indie stuff, and when looking at those, the approach of the big stores seems really weird.

I don’t think we should copy that kind of behaviour.