User:Doc James/Travel reviews

Proposal: That we, in principle, look at creating a system that allows our readers to easily provide reviews of hotels and restaurants.

Goals

Increase our base of editors. Allowing people to provide feedback on a hotel / restaurant is an easy entry to writing travel articles.
Travelers obviously want reviews which is why TripAdvisor / Yelp is so popular. We are writing for the traveler so should consider providing this sort of content.

Implementation

Note: This is just one possibility. I am in no way attached to this particular implementation. Please suggest others.

Possibility A

Create a separate article for each hotel / restaurant already on WV (supposedly their maybe a half million). These pages could be created by a bot.
Than link the hotels / restaurants in the main articles to these page and have the feedback tool run on them. Here is the current feedback tool The "feedback" would be feedback about the hotels / restaurants themselves and would be displaced after some details about the establishment.
Once enough feedback has been provided a rating out of 5 would be generated automatically and feedback to the main article listing. How to calculate this of course will be a work in progress and require adjusting to deal with issues that arise from people trying to game the system.
Mechanisms to take into account how established the editor providing the feedback will be needed.
The hope is that this would be an adaptation of the "feedback tool" which is currently running on nearly ever page on Wikipedia. Note that it currently does not request feedback from long in user and it shows different results when requested via the talk page depending on if you are logged in or not. Also there is more than one version currently live so you will need to look at a few pages to see the different versions.

Links to examples

An overview
An example for a specific page (reached from the talk page of the article)
The central feedback page
Here you can seen version 4 but must be logged out
A blog discussing why it exists

Things that can go wrong

Organizations try to game the system
People post reviews instead of plunging forward and editing out inaccuracies - in some cases, because they aren't even aware they can.

Measures of success

Volume of useful feedback given
Total page views for this type of article
Growth in editors on other areas who began via this type of edit (on Wikipedia 3% of people who used the tool created an account and 3% of these go on to edit). So only 1:1000 become editors but with a high readership this can become significant.

Discussion

Travel guide is not a directory

I think that it is a serious change in the concept. You can not just add user reviews and ratings to a travel guide. Our present strategy is to select a reasonable number of hotels, restaurants, and other services that may be interesting to the traveler. Paris has thousands of hotels, but we choose only few of them and try to supply the list with fair descriptions. With feedback feature added, we have to list all 2000+ hotels in Paris, and we suggest that users make their choice based on ratings instead of descriptions. We basically kill the idea of fair descriptions and replace it with user's feedback.

I know it is somewhat personal, but for me a list of reviews is opposite to the very idea of a travel guide. I would strongly miss the option of looking through a list of 10-20 hotels with short, well-balanced descriptions. I would be happy to see the feedback feature as a new project or at least as an independent branch of Wikivoyage, but I am very unhappy about the idea of integrating it into existing articles. If you want to read user reviews, you can choose between TripAdvisor, booking.com, and many other websites. If you want to find a fancy place for dinner, you take a book or you go to Wikivoyage. Now if Wikivoyage switches to the feedback system, where should we go? --Alexander (talk) 12:53, 23 January 2013 (UTC)

I'm new to editing in wiki markup, so please forgive any formatting errors. I agree that reviews are not very travel guide like but, a travel guide with potentially thousands of editors, has it's own set of challenges. A group of 10 or so editors could likely discuss and come up with consensus to a very subjective subject matter. Currently, how do we decide which restaurants/hotels/bars should be included in the short list posted to a page? My list of 20 min-range restaurants to eat at in the [District] are most definitely different from the ones that are being offered up.

If nothing else, user reviews would give a more balanced representation of the places that should be in the short list. --Ceazar77 (talk) 23:04, 23 January 2013 (UTC)

Reviews and ratings are easily skewed. They are not panacea. The short list should include places where "we recommend people go" because these places are interesting and have some flavor. It is not a list of places where "most people go" or places that "most people like". --Alexander (talk) 00:06, 24 January 2013 (UTC)

Who is the "we" in the short list "we recommend people go" to? This would be easy with a small group of contributors, but if the hope is to get a large number of travelers to add content then these lists would either become large enough that they would lose their usefulness, or we would have to have some way to determine who's subjective opinion is more valuable. A review section (whether it is part of the wiki site or in a separate directory) would help address this.--Ceazar77 (talk) 17:42, 24 January 2013 (UTC)

I can get useful information out of anonymous reviews, but I don't trust them. The only thing I trust is the opinion of friends and fellow travelers. Seasoned editors of this site, who are trusted fellow travelers, take some responsibility for making a fair selection. While we may have a situation where several experienced editors do not agree on the "short list", this is not going to happen in the near future and, likely, will not happen at all.

User ratings will lead you to any decision depending on how you interpret the numbers. I think it is pretty obvious that 1-2-5 reviews do not provide a good and unbiased assessment. The numbers will only matter starting from X reviews, and in fact nobody knows how large this X should be (to make things worse, this threshold number also depends on the destination, e.g., how many locals vs. foreigners are writing the reviews). The scale will also depend on the price range and on how many budget/rich travelers visit this particular place. The review system becomes ideal when you deal with an infinite number of reviewers covering all possible origins, interests, and salaries.

Unfortunately, even TA with its long track record and a large number of contributors is far from being objective. When I check restaurants in Tallinn (which is a moderately popular European destination), I see that their #1 is a stupid French restaurant that I (as a semi-permanent resident of this city) would never recommend because it lacks any local flavor. You basically suggest that we should promote places where people like to go. I say that we should promote interesting places and, particularly, places with local cuisine or fancy environment. This is a fundamental difference. --Alexander (talk) 21:25, 24 January 2013 (UTC)

That's a very good point.

If the intent is to allow only reviews of listings that have been selected for inclusion in our travel guide, we are going to get a lot of grief from non-listed business owners and their fans, who want to know why they can't even get listed to let The People express their opinion.
If the intent is to allow rating of any and all establishments in a destination, then we would need to change one of our founding principles and become a directory, rather than just a travel guide.

If there's a middle way here that I'm missing...

-- LtPowers (talk) 13:28, 23 January 2013 (UTC)

We could have this independent of WV and contained all within the feedback tool. The hope will be that a community of editors will grow up around it. All we would have is a single link from each hotel / restaurant on WV to these review contained within the feedback tool. Just a small icon maybe.

If you look at the bottom of this article on Wikipedia you will see one example of how they add feedback

Here is another method . Than to view the feedback one needs to go to the talk page

When one clicks on viewfeedback it takes you to a different area . It shows different options depending on if you are logged in or not. So yes this is more or less an independent branch.

By the way this suggestion would not change this one bit "I would strongly miss the option of looking through a list of 10-20 hotels with short, well-balanced descriptions" I have never suggested getting rid of a list of 10-20 hotels. Travel Doc James (talk · contribs · email) 13:49, 23 January 2013 (UTC)

Well, despite the long conversation in the mailing list, this aspect remained unclear. I suggest that you rename this proposal into something like "adding directories to Wikivoyage" because hotel or restaurant directories are what you actually want to have. Feedback is one possible feature of a directory, but you also need maps, sorting by price, sorting by amenities, etc. Additionally, you will need an independent geographical hierarchy, because Wikivoyage is not supposed to have an article on every village with a hotel in it.

Overall, that's all good, but that has little to do with the existing Wikivoyage except for some links to these new directories. Personally, I don't think that this project is viable, and I see the travel quite differently, but you are welcome to try. As long as it does not touch the existing content of Wikivoyage... --Alexander (talk) 14:33, 23 January 2013 (UTC)

I like the idea of having the 'reviews' site as a separate branch using some mutation of the Feedback tool. The 10-20 hotels we have decided to list in the destination article can have the small icon linking to their reviews, and possibly at the bottom of the "Sleep" section could be a link to "See all hotels", which would take you to the reviews site which acts as a directory. Of course, then there is opportunities to integrate with Wikidata, keeping every hotel and restaurant as a piece of data with facilities info, keeping the reviews on the Wikivoyage branch. There are many exciting possibilities, without sacrificing what makes Wikivoyage unique. JamesA ^>talk 14:34, 23 January 2013 (UTC)

Yes using Wikidata is another excellent idea if we ever want to expand to an exhaustive list. The goal is not to have these completely independent as one of the primary reason for attempting this is to draw more people into editing WV. No one would be able to edit the existing content of WV through this mechanism just as no one is able to edit WP content through the version there. Travel Doc James (talk · contribs · email) 14:39, 23 January 2013 (UTC)

James, I think you've completely missed the point. If we are going to include user reviews, then we either a) have to allow any establishment to be reviewed, requiring us to maintain a directory; or b) have to restrict reviews to establishments we already list, subjecting us to accusations of bias, unfairness, and incompleteness. The fact that the reviews are "separate from" WV doesn't solve either of these problems. LtPowers (talk) 16:19, 23 January 2013 (UTC)

Copied from Wikivoyage:Travellers' pub#Article feedback tool, where it's equally relevant:

I would not like this to be a substitute for people plunging forward and eliminating inaccuracies in articles, and that's what I fear would happen. It's counter-intuitive to quite a few new users that they can edit articles, as shown by the number of complaints I've read on talk pages of problems the complainer is best able to fix, him-/herself. There could be a positive aspect, though: There's a degree of arbitrariness to which restaurants (e.g.) are listed, and perhaps a spate of bad reviews could get a mediocre restaurant like Gandhi on 6th St. de-listed (which I'd love to do but won't take individual responsibility for). Ikan Kekek (talk) 20:14, 23 January 2013 (UTC)

Further comment: I do not want us to reflexively reject all suggestions for major changes to this guide. I ultimately came out against in-line Wikipedia links, for now, and I'm expressing skepticism about this one, but my point isn't to reject change for the sake of inertia, but to think through the implications carefully. Ikan Kekek (talk) 20:20, 23 January 2013 (UTC)

I think one thing that should be cleared up is that we won't be adding this type of review content on-wiki. It should be a separate database, with an easier interface for non-wiki editors. That allows us to pull information on a much wider scale, from a much wider range of people (simply put, most people do not and will not deal with wiki markup, but almost everyone likes writing (first-person) reviews of places they've been). The two ways I see us integrating the information into the guides on-wiki are 1) linking listings to a page showing reviews (and soliciting more), and 2) review content would be CC licensed, allowing us to take good descriptions and put them into the more brief listing description.

Not all places reviewed would be put in the travel guides themselves, because of our sensible "not a directory" policy. --Peter ^Talk 00:09, 24 January 2013 (UTC)

Filtering user's feedback

The feedback feature will substantially increase the amount of janitorial work. While posting a short review is more easy than editing an existing article, it is also much easier to post spam, obscenities, and other bad stuff. Do we have a team of janitors who are willing to check this on daily (well, in fact hourly) basis? I, for one, do not plan to spend my time on this, because I believe that I can contribute to Wikivoyage in a more fruitful and intelligent way. Of course, I would be happy if others are willing to deal with the feedback, but this is something that we should check before launching the feature. --Alexander (talk) 12:53, 23 January 2013 (UTC)

Comment - I agree that in the long-term we would want to implement some form of rating/review system for all listings (even internet cafes and tourist attractions). However, the biggest issue is the implementation, and I could only support if it was clear it would be seamless and fit well in the scope of our wiki. I feel the best implementation would allow users to review/rate sites directly from the destination page using a popup, without having to have separate pages. Maybe it could function similarly to the Article Rating system on Wikipedia, but with many instances? It'd have to be discussed over many months as this is an extremely complex sort of functionality that no other wiki has attempted before. JamesA ^>talk 12:56, 23 January 2013 (UTC)

Do you mean the article feedback tool ? Travel Doc James (talk · contribs · email) 13:39, 23 January 2013 (UTC)

That's the one. Looks like there may also be some potential to integrate the tool into our articles as an actual article feedback tool, so we know how the articles can be improved. A lot of people would use our travel guides while they're away and come back knowing that many things are out of date but are too lazy to edit. Leaving feedback is much simpler. Though that may need a separate thread in the Pub. JamesA ^>talk 14:54, 23 January 2013 (UTC)

Yes we could definitely try that first. I will start a discussion at the pub. Travel Doc James (talk · contribs · email) 14:59, 23 January 2013 (UTC)

Agreed with JamesA's understanding of how this should work. Re: Alexander, I actually think we'll have more people interested in managing this type of content than we do in terms of wiki admins. It's a more popular medium for dealing with travel—albeit one that, on its own, isn't as helpful for a traveler, as the information isn't distilled into a portable, digestible guide. --Peter ^Talk 00:11, 24 January 2013 (UTC)

top 25

James, i like the idea because reviews give an impression how people perceived a hotel, its facilities and location. I think we can't beat TA and we shouldn't. Imho we should start something like a voting process to channel the top 25 hidden gems in a region (e.g. Europe, Middle East, Africa, S.America, N. America, Oceania, SEA, East Asia and Continental Asia). People would be allowed to nominate hotels and write why they like it. Business owners/Marketeers etc. would be excluded and this would allow places to shine that are of the beaten path for most travellers. jan (talk) 13:25, 23 January 2013 (UTC)

We shouldn't beat TripAdvisor? Maybe we cannot but why shouldn't we try? Once developed this sort of system could easily pull up the top 25 hidden gems in what ever region one wished. Travel Doc James (talk · contribs · email) 13:54, 23 January 2013 (UTC)

Why merely copy TA by inviting comments which only promote or denigrate one hotel? I'd suggest that any "review" comment on the entire destination (listing *every* venue visited and commenting upon each) instead of using TA's manipulation-prone format. A little like asking the author of a book report show they read the *()$#@& book. K7L (talk) 14:52, 23 January 2013 (UTC)

Many travellers like reading personalized reviews about a location / hotel / restaurant. This is why TA is ranked 282 by Alexa.

We could also or only apply this to entire destinations. This would be a lot simpler to implement. Maybe even by end of April. Travel Doc James (talk · contribs · email) 14:57, 23 January 2013 (UTC)

I never contribute to TA but i check them for hotels because of the personalized reviews. I need for a business trip a different hotel than for a weekend with my girl friend. I think that is one of our weaknesses that we cater in a single article for e.g. Kuta for different kind of traveller. Some blokes want fun at Jl. Legian, other are honeymooners and other are pensioneers on their first trip to Asia. I started a while ago Grand old hotels which had big interest and i guess after the major cleanup, we need to get a bit more personalized. One of the reasons i started here where the trip reports from Jani. That was 2005/6 a big inspiration for my first Asia trips. jan (talk) 20:55, 23 January 2013 (UTC)

One of the benefits of Wikidata is we could potentially allow our reader which 10 or 20 they wish to see. Do you want to see the backpacker option, the business option, different price ranges. Travel Doc James (talk · contribs · email) 21:12, 23 January 2013 (UTC)

Not to offend, James, but I am unsure you either understand or agree with the purpose or principles of this site. Do you contribute any travel knowledge at all?108.232.169.153 21:50, 23 January 2013 (UTC)

The fact that this "random" IP from the LA area doesn't think it is a good idea should be a vote in its favor :-) I remember we had half a dozen of these during the set up of this site. Travel Doc James (talk · contribs · email) 23:32, 23 January 2013 (UTC)

One of the things that we could do with the feedback option it to identify clearly unreliable listings. I'd really like to give the the read a report listing option, with an option to say I've stayed/eaten/entertained at this listing, and the description is fundamentally at odds with the nature of the experience there. Or something similar. --Inas (talk) 22:15, 23 January 2013 (UTC)

Personally, I think that TA is a disgusting site, plastered with intrusive ads, pop-ups, aggressive assertion of copyright and access to their information, etc. They do everything in their power to obscure content that could allow you to book outside of their own tools. But there isn't another viable, global alternative out there. Providing a feel-good, attractive, alternative backed with the resources our project has could compete for contributors IMO. --Peter ^Talk 00:15, 24 January 2013 (UTC)

While I fully agree with your assessment of TA, I have to say that they offer lots of fancy features (likes, cross-posting to Facebook, travellers' forum; not to mention the booking tool) that a WV/WMF-related project will never use. As a first step, you can try to beguile some people from their forum (they are good experts, by the way). I don't think it will work, so I am not very optimistic about this possible competition. --Alexander (talk) 00:29, 24 January 2013 (UTC)

The nature of the problem

I don't think we have any issues with our logistic sections. Get in, around, and next. Our Cultural sections, Talk, Understand, we seem to be getting under control. See and Do, tend to be fairly exhaustive lists in most destinations - occasionally some touting, but mostly straight up and down. Eat, Drink and Sleep have real issues. How do we currently decide what restaurants, nightlife and accommodation we list, and how we describe them?

If a business owner has employed a competent SEO consultant, we generally have a listing together with positive attributes.
If a business owner has found the site, they generally add a listing.
If a Wikivoyager has stayed, eaten or drunk somewhere, they may add a listing with their single experience from their perspective.
Within a the fiefdom, the Wikivoyager familiar with the feifdom usually maintains the lists.

The end result is these sections usefulness is compromised. 'Cause you don't know if they are true, biased, or meet your needs. You're much better off with a tripadvisor, a yelp, or an eatability. It is much harder for business owners to skew those sites, and your more likely to find someone with a similar profile to you to recommend. I can understand people not wanting to be a directory, or a review site. I don't really want to go down that path either. But I'd like us to at least acknowledge that we have a real problem in this area, and we need a solution. Then we can start discussing what form the solution will take. --Inas (talk) 22:15, 23 January 2013 (UTC)

Agree whole heartidly. I think there needs to be a way for people to contribute subjective content (a list of 20 most notable is defiantly subjective) or avoid eat/drink/sleep all together. Another option would be to link to services that best meet this need in the various locals (Yelp works great in SF, Openrice is a good choice in Hong Kong).--Ceazar77 (talk) 23:35, 23 January 2013 (UTC)

Yes, we have this problem, because it fundamentally exists. TA, booking.com and others have this problem as well. I have seen multiple ways of improving hotel ratings on booking.com (positive reviews submitted by owners, discounts to users who submit positive reviews, etc.), so I no longer trust their numbers. Unfortunately, there is no solution other than finding a real person who stayed in the hotel or dined in the restaurant. Our present strategy at Wikivoyage is in fact very close to this option, because we require people to write informal descriptions that most hotel managers can not manage. I wonder if we can further improve it, but our strategy seems to be viable. --Alexander (talk) 23:17, 23 January 2013 (UTC)

What are peoples thoughts on linking to others who are currently doing this? Travel Doc James (talk · contribs · email) 23:39, 23 January 2013 (UTC)

We've come down pretty firmly against linking to (other) review sites, which effectively would just be TA, since they're global. --Peter ^Talk 00:03, 24 January 2013 (UTC)

And to be clear, I think one of the two main reasons for doing this is to address the problem Inas describes. If our eat, drink, sleep content is unreliable, we could remedy this with a potential army of casual reviewers, and links to those reviews. The reviews not only would provide readers with something else to check, but also our editors with a way to judge whether a listing really belongs in our guides. --Peter ^Talk 00:18, 24 January 2013 (UTC)

What Inas says is very true. As things stand, I would never trust this site for accurate "Sleep," "Eat" or "Drink" sections. Instead, I have tended to use Yelp and Chowhound for eateries and lately, I've been either using Homeaway or just showing up and inspecting hotel rooms before deciding whether to stay somewhere or not. My clear impression so far has been that this site is full of self-interested listings by business owners or people employed by them. However, I think there's room for doubt about whether some kind of review sub-site would remedy that. What strategies would be employed to dilute the influence of business owners and their various employees and sock puppets on such a sub-site? Ikan Kekek (talk) 08:40, 24 January 2013 (UTC)

You mean on the site proposed? We would allow established editors to remove poor quality comments. Comments would be either supported or not by fellow readers where the support of those who are established is worth more than someone who isn't. Just a couple options. I am sure there are more. Travel Doc James (talk · contribs · email) 14:32, 24 January 2013 (UTC)

Hopefully some of this can be done through an algorithm similar to they way Yelp and TA filter reviews, although I don't know if this can be done with the feedback tool. In 2011 Cornell CS students developed an algorithm that identified 90% of fake reviews (also useful for when you are reading review sites yourself). I don't think there is a way to totally remove the influence of those with a vested interest, but I have no doubt that the influence can be mitigated somewhat. —The preceding comment was added by ‎64.54.16.254 (talk • contribs)

Great we should ask them to join us. Travel Doc James (talk · contribs · email) 19:28, 24 January 2013 (UTC)

Expanding our pool of contributors

This is the other big reason I see to do this. Sites like Yelp and TA have no trouble attracting people from all sorts of different backgrounds, whereas we are incredibly homogenous in everything except geography. (Other wikis, most famously Wikipedia, have the same issue.) We're not getting very diverse input. And I hate trying to write about women's clothing stores, and would much like to farm that type of reviewing work to a review database ;) --Peter ^Talk 00:22, 24 January 2013 (UTC)

Peter, on one side your are right but families and women do read our articles, they simple don't contribute. I was yesterday with a bunch of new employees out for lunch. Everybody knew WT (know everyone has to change to WV...) and especially the females (young backpackers or moms) are eager users of our articles. I got the feedback that the main hurdle the women told me are, they were either afraid to do something wrong or they had very specific interests we don't cater for. Men usually just start and think they do great stuff ;-) Like you made for The Wire an outstanding article, we need to have such things for Twilight. A young lady forced her boy friend to spent a week in Oregon of their annual holiday just to visit all the places and i remember dozens of Japanese fans in Ross (Tasmania) that just visited that village due to a Japanese TV series. jan (talk) 08:59, 24 January 2013 (UTC)

That's one of the most frustrating things. From my conversations with fellow travelers, I see the same thing. It seems that women are much more likely to use our guides than men (although it's possible that I'm deliberately talking more to women...), but a casual perusal of our contributor base will reveal that almost all our registered contributors are male. --Peter ^Talk 23:42, 24 January 2013 (UTC)

Legal issues

Legal problems concerning restaurant and hotel ratings have been discussed on devoy already and should be sorted out in the first place. --Aschmidt (talk) 10:14, 24 January 2013 (UTC)

These issues already exist with respect to the feedback tool (as an example someone can say a person is bad via the feedback which could be libel.)

The WMF simple hosts content and is not liable for what people add. Same as Google can not be sued just because it links to a site that contains copyright infringement. The internet would fall apart if it worked like this.

Travel Doc James (talk · contribs · email) 14:29, 24 January 2013 (UTC)

These issues already exist on Wikivoyage too. If a review were to step over the line, we can just remove it (as we do already). --Peter ^Talk 23:38, 24 January 2013 (UTC)