How would you set up a review system?

[beerinator]

Ok, I have a teaser image for you from an upcoming feature expansion here on the site (click for bigger);



Basically, that "bubble" containing the location reviews will popup when a map user clicks on a link in the regular information bubble that is currently on the maps.  The "reviews" that you see in that image are actually reviews that are currently stored in the database (there are only two reviews there currently).  The scores on those reviews are being pulled from the database as well, but those are just temporary fillers for the scoring system that we will hope to develop for our system (possibly in this thread!). There is currently no review submission setup (not even a test setup) but this shouldn't be a horribly difficult thing to accomplish.

The difficult thing that is looming on the horizon of this project is review calculation and how locations should be scored.  What do you value in a beer location?  I don't want to be a copy of some of the other beer sites out there so I was thinking that we should try to do things differently.  Here are some of my ideas, please feel free to contribute yours or comment on mine.

thumbs up system
At one point, I was thinking that a "digg" type option would be cool.  A user would have three scoring options: thumbs up, thumbs down and neutral.  There would obviously be a place for you to type out a full review, but there wouldn't be a whole lot of options for specific voting.  Each person that voted would need to make a comment of a certain length and you would not be able to vote for one specific place more than once.  This simplified voting system would allow us to basically tally each vote on a point system and locations in an area that had more "points" could be featured a bit more prominently on the maps.  Locations that went a certain number of points into the negative could also be removed from the maps.

regular review system
We could also have a simple 10 point scale for an overall review.  You would write up a review like normal, and simply give the location a score (example: 7 out of 10) as an overall score, rather than having specific individual scores that add up or are calculated to get your overall impression.  This in my opinion is easier for the user to keep up with how they rank their individual reviews, but it's fairly simple and not very deep.  We could still promote better reviewed locations and "demote" those locations that really shouldn't stay on the maps.

in-depth review system
This is more like what is already set up at beeradvocate or ratebeer.  You would review a location based on various things like "selection" or "quality" and these totals would be calculated to come up with your overall score for this location.  If this option is chosen, I'm going to ask for some beta testers or people that consider themselves interested in math to help devise an algorithm that will calculate these scores.  This algorithm will be published on the site somewhere, because I think it's necessary for people to know these things.

In all three options, there would be a minimum character requirement, because I feel that each location (no matter how good or bad) deserves a certain level of respect.  If you use a site that uses a different type of system (and you like or dislike the system), please let me know about it.  How would you build a review system?

[brewcrew]

I prefer the in-depth system. I can see how others that stumble upon beermapping and are as serious about good beer might prefer the simplicity of the other ranking systems. The reason I like the in-depth system is if I go to a place that has a great atmosphere, an excellent selection and everything is good but the service was bad, I find it hard to give an accurate score of this place using the 1st two mentioned systems. I might give it a neutral rating based on the thumb scale or a 6 overall based on 1-10 scale. This would negatively changed the overall rating but with the in depth review I could give all aspects a good score and service a low score. Then when someone else is reading reviews and most reviews have a good service rating and 1 bad rating pops up people might see this as an anomaly. A good example would be the reviews of the Map Room recently on another site. Just my thoughts.

[afatty]

I favor an in-depth system as well.  Other things that should be considered include smoking/non-smoking, proximity of public transportation.. Philly for example, Standard Tap is a hike from the T but SP Taproom is only about a block...

[beerinator]

Ok, so for those of you who favor the in-depth system, what categories do you think are needed?

• selection
• atmosphere
• ..what else..?

[beerist]

Ok, so for those of you who favor the in-depth system, what categories do you think are needed?

• selection
• atmosphere
• ..what else..?

• service
• overall

I never cared for the "quality" part on BA and feel like an "overall" would usually work better for me.  I'd also like to see some check boxes that would list things like:

• indoor smoking
• outdoor smoking
• proper glassware
• dedicated parking lot
• close to public transit
• full service food
• (frequent) real ale
• etc...

...just my $.02

[afatty]

and a good $0.02 it was.. saved me typing out most all of those.

Outdoor seating available
Food/ beer costs

[beerinator]

I never cared for the "quality" part on BA and feel like an "overall" would usually work better for me. 

I feel like if we did this and gave the user the opportunity to pick the "overall" score, it would have to be a solid score that went toward the overall total for the location.  And the other scores that you picked would be averaged with other scores of the same type to create an average like they have at ratebeer.  Rather than the BA system where your "selection", "atmosphere" scores actually averaged together to form the "overall" score for your experience.

Also, I like the checkboxes thought and I will try to implement something like that (I think they have a similar system at ratebeer).

[Izzy]

I'm voting for the likes of selection, food, atmosphere, service and overall.

Assuming the check boxes are just useful info and not counting against a rating. I think this is a great component, just wouldn't want it to count against the best pub in town. Could we add categorization to that? (whether it's a restaurant, beer bar, brew pub, etc.)

I've also noticed that when people are given the choice of, say, 1=low and 5-high, they tend to score a 3 in a non-commital and unhelpful way. Having a scale of 1-4 for rating choices in each area might combat that tendency: 1=poor; 2=lackluster; 3=pretty good; 4=amazing

Also, I've seen a spot elsewhere to have a brief rundown of Pros and Cons, which might be good.

[BeerSox]

I feel there should be something that indicates the price of the place, which would probably need to be included in the in-depth system. However, the price shouldn't factor into the overall score. If the place is really good, but it is really pricey, it may be well worth the price.

[beerinator]

I just see price as being way more relative than any of the other things that we could review.  I can include it, but a 10 dollar burger is seen in such a different light depending on who you are.

[beerist]

Assuming the check boxes are just useful info and not counting against a rating. I think this is a great component, just wouldn't want it to count against the best pub in town.

That's what I was thinking - just for informative reasons.  It wouldn't count in the actual "rating" of the place.  I'd just be less likely to go to a place with poor parking AND indoor smoking if there were a similarly rated place that was close with good parking and no indoor smoking.  However, smokers and bicyclists might disagree with me!

I just see price as being way more relative than any of the other things that we could review.  I can include it, but a 10 dollar burger is seen in such a different light depending on who you are.

Price to me is a crapshoot.  I'm more likely to spend more on vacation than I would regularly, plus as you say, prices can vary pretty wildly depending on where you are.  What's "normal" in one context (or to 1 person) would be pretty abnormal for another person.  Maybe some kind of indicator that would give an average price of "regular" (i.e., not BA, hi-grav, limited release) draft beers.

• <$3
• $3-5
• $5-7
• $7+

Doesn't really account for food, but you're probably not going to get a heaping pile of NACHOS for $5 at a place that charges $8 for a pint of Pale Ale.

[BeerSox]

I wouldn't want the prices to count against the overall rating of the place. It's just a note so somebody knows if they are going to empty their wallet or get a few pints at a decent rate.

[beerinator]

I wouldn't want the prices to count against the overall rating of the place. It's just a note so somebody knows if they are going to empty their wallet or get a few pints at a decent rate.

I totally agree with this, as I think most people do that have suggested prices as an option.  I'll go to a place regardless of the price of their sirloin if I am just going for the beer.  The problem is that many people will vote that a place is expensive based upon the food prices.

I like the suggestion that beerist gave that would be a label above the "how are the prices" section of a review that would attempt to tell the reviewer that this is based on beer pricing not on food pricing.

This is not foodmapping.com  This is beermapping.com!   

[kp]

My initial response upon seeing this subject was to do it the Google way - use and encapsulate someone else's content.

More I think about it the more that I see the value of having your own data and doing it yourself. Being able to filter and show only the places that achieve a high score would be very valuable.

One the subject of just an overall score vs. individual scores, I believe more data is better. Given the extra data you can the search/filter on the criteria that is important to you. I can even see eventually being able to assign personal importance to the categories - for example I'd rate selection as a 10, service as a 3, and beer price as a 4, because selection is all important to me. My personal preference would then affect the ranking that I see for the places. Someone with different personal preferences would see different rankings for the same places.

On the subject of checkboxes - don't forget WiFi as an important feature.

[BeerSox]

On the subject of checkboxes - don't forget WiFi as an important feature.

Ha! Bring your laptop to a brewpub and upload your review instantly!