PDA

View Full Version : About the stars rating system


gnz
12-08-2004, 07:12 AM
New forum... Here Im reposting my previous suggestion. The original thread (http://www.cgtalk.com/showthread.php?t=182547) has some interesting replies.


Hi, After spending some time rating threads on the finished work forums, Ive come to the conclusion that the current stars rating system is somewhat flawed. Here is the reasoning:

We currently have:
***** Excellent
**** Good
*** Average
** Poor
* Terrible

In theory this would work fine as it gives a whole range to rate the quality of the thread. But it could only work if everyone actually rated every thread they viewed.

The reality is (I think) that if someone takes the effort to go and click on the rating menu is because he/she thinks the content is avobe the average quality and deserves to stand out. I mean, if I think some thread is really "terrible" or even "average" I will simply won't care about rating it. Does this make sense?

What I suggest is having something like:
* Average
** Good
*** Very Good
**** Excellent
***** Sublime <- probably some other calificative would be better suited here :)

I think with a scheme similar to what I suggest the stars rating system would be much more effective. What do other cgTalkers think?

--
Just my 2c
Gnz

Garma
12-08-2004, 11:32 AM
I think a "bad" rating should stay, it happens now and then that people rate something very good when I honestly think it is bad. maybe remove the "terrible"? like:

bad
average
good
very good
excellent

?

Ithilien
12-08-2004, 08:17 PM
Just throwing in my 2 cents... I see your points and empathize.

In my opinion (and mine only, you're welcome to disagree) I think that they should stay because they provide an easy and fair way for people to quickly review work. I rarely click on any artwork in the gallery that does not have star ratings - even if it has 1,2,3 star ratings, I will click on it to see what the hullaballoo is all about!

Yes you're right. It is misused. People give 1 star ratings just to piss off the artist. But for the most part, I think that it is a fair means of testing public opinion on an artwork.

My 2 cents...

gnz
12-08-2004, 09:21 PM
I think a "bad" rating should stay, it happens now and then that people rate something very good when I honestly think it is bad.

I see your point. But the idea of my suggestion is to have a thread with at least one star be something worth a look at. Having any stars representing bad work is what its currently not working.

How about having plain stars without labels? ... let the users interpret if one star is worth looking at or not.

What my suggestion is for in the end, is to provide a more ample range to express our liking of a particular piece... I mean, I already like it, that's why Im taking the trouble to rate it... Reading the "average", "poor" and "terrible" labels is what stops me from giving a thread three, two or at least one star, which I think all would do good to it.

Now, what we could do about the issue of over-rated threads (Those threads that are nice but very far from perfect and that the author and his dog have it currently rated *****)? How about listing them like this:

Thread title *****(10 votes)
Another thread ***(86 votes)

The number between pharenteses being the number votes it has received. That would give me a much better idea of wether a thread is accurately rated or not.

Does this make sense?

jussing
12-13-2004, 12:54 PM
OK, after long consideration, I've come to my final argument about this:

THE SINGLE BIGGEST problem with the rating system is, that NOT ENOUGH PEOPLE USE IT.

No matter how good or perfect you design the system, it's all for nothing when so few people vote, that a single malicious down-voter can topple the entire rating with a one-star rating.

Plus, if people get 4 or 5 friends to register and give their work 5 stars, that will also comlpetely annihilate the fair "overall" rating.

So no. 1 priority regarding rating, is to get more people to rate! Only then will it represent a true rating.

Right now, the rating is nothing but a battle between down-voters, and the artist's friendly up-voters. And in the middle, vastly outnumbered, are the few people who use the rating system as it is meant to be used. -In my opinion.

Cheers everybody,
- Jonas

jussing
12-13-2004, 12:58 PM
...not to mention, that any rating system where you can see the result before voting is, inherently, not working, and never will.

-Even a lot of honest voters (=not evil down-voters or biased friends) will not use the full scale, but rather use the maximum and minimum rating, for "arrow up" and "arrow down", respectively. -Meaning that, mathematically, the last votes will have more impact on the final result than the first votes. :shrug: So it's a lost race already.

jussing
12-13-2004, 10:31 PM
Thread title *****(10 votes)
Another thread ***(86 votes)

The number between pharenteses being the number votes it has received. That would me a much better idea of wether a thread is accurately rated or not.

Does this make sense? Yes, that makes a lot of sense to me, particularly in regards to the issues I have just adressed. You will then know if a thread has enough votes to show a "fair" rating.

As an alternative, you can raise the minimum number of ratings (before the rating shows) from 3 to 20 or something. But, this will also require more people to vote, or almost no ratings will show.

A difficult challenge indeed. This whole rating thing is actually, IMHO, the singlemost important thing to be adressed of all the things currently posted in this suggestions forum. Because the rating is a bloody good idea, but it is presently not working, and getting it to work is going to be tricky...

Cheers,
- Jonas

gnz
12-13-2004, 10:41 PM
THE SINGLE BIGGEST problem with the rating system is, that NOT ENOUGH PEOPLE USE IT.Yes I think too this is the root of the problem.

I suggest giving the thread rating menu a better placement and design, make it hard to resist casting a vote.

I have some ideas of layouts for the thread pages, Im going to produce a test one and post it for comment. Im a web designer/developer BTW.

Garma
12-13-2004, 11:12 PM
I suggest giving the thread rating menu a better placement and design, make it hard to resist casting a vote.

or force people to rate when they leave the thread :twisted:

j/k :D

gnz
12-13-2004, 11:20 PM
or force people to rate when they leave the thread :twisted:
j/k :Dheh :) imagine the next time you go http://www.cgtalk.com :
Sorry, no CGTalk for you! you need to do your homework and rate the following threads you have visited: *threads*
Thank you.

gnz
12-13-2004, 11:29 PM
heh :) imagine the next time you go http://www.cgtalk.com :
Sorry, no CGTalk for you! you need to do your homework and rate the following threads you have visited: *threads*
Thank you.Hey but seriously, now that I read my own post I think it could actually work. Not exactly like that of course, but having an unobstructive box on your home page that invites you to quickly cast a vote on threads you have recently visited. Kind of what netflix (http://www.netflix.com) does with movies you have rented from them.

salmonmoose
12-14-2004, 02:01 AM
Perhaps a different system altogether... take a look at www.bash.org.

Now, once you stop giggilng, each post there has a + and a - link, which you can easily click on. If you click + it adds to the score, if you click - it takes from it.

It cuts down on the process a bit, either you enjoy a thread or not, simple really.

You could even compare the votes against other things like the age of the thread, how many visits it's had and have new ways to sort things.

gnz
12-16-2004, 12:55 AM
Perhaps a different system altogether... take a look at www.bash.org.

Now, once you stop giggilng, each post there has a + and a - link, which you can easily click on. If you click + it adds to the score, if you click - it takes from it.

It cuts down on the process a bit, either you enjoy a thread or not, simple really.

You could even compare the votes against other things like the age of the thread, how many visits it's had and have new ways to sort things.Thats a good site :) I think I had seen it ages ago when I was more into IRC.

Its a very interesting system, but being realistic, implementing a whole different way of rating threads would be much more technically difficult given the size of cgTalk. We might have a better chance of having something implemented if its not very complicated.

Maybe if some time in the future there is a big revamp planned for the forum then we can suggest things like this.

GOTgraphic
12-18-2004, 12:06 AM
I don't like the current star-rating system either. I'm not going to rate anyone a 1 or 2 or 3 star. As far as I'm concerned "average" is bad enough. Artists know if their work is lousy, so giving them a 1 or 2 star rating is ridiculous. Giving them NO STARS means is really bad or not worth bothering to click the button (hence really bad). So...

I think Gns's suggestion of:

* Average
** Good
*** Very Good
**** Excellent
***** Sublime

would be a much better rating system. If it were up to me I would go this route:

* Average
** Good
*** Wonderful
**** Fantastic
***** Perfect

I definitely think it should be changed.

jussing
12-18-2004, 06:58 PM
I think five "good" rates is too much. I say just shave off the bottom two, so it's

"Average"
"Good"
"Excellent"

Five rates is better when browsing, 'cause it gives a better picture of the variation in the work, BUT...

-When rating, how do you judge between FIVE different rates of good? I mean, who is to tell the difference between "excellent and "sublime", or "wonderful" and "fantastic"?

Cheers,
- Jonas

GOTgraphic
12-18-2004, 07:11 PM
Perhaps dropping the definitions of the stars would be good but keep the 5 star system and call it, ôRate the image on a scale of 1-5 stars, * being average and ***** being perfect".

I like the 5 star rank but like you pointed out Jussing, the terminology can be confusing and or ambiguous.

gnz
12-18-2004, 07:53 PM
Five rates is better when browsing, 'cause it gives a better picture of the variation in the work, BUT...

-When rating, how do you judge between FIVE different rates of good? I mean, who is to tell the difference between "excellent and "sublime", or "wonderful" and "fantastic"?
I say five levels are fine, let's just lose the labels... how about:

() no stars
*
**
***
****

That way if an overrated thread doesn't even deserve even one star, enough people can vote it "no stars" and get its non-deserved stars removed.

I think stars with no labels are just the simplest and easiest way to have an effetive rating system, just like in elementary school with stars on you forehead :)

GOTgraphic
12-18-2004, 08:59 PM
Yes, lose the labels. I think the current labels are the biggest problem.

That way if an overrated thread doesn't even deserve even one star, enough people can vote it "no stars" and get its non-deserved stars removed. However, the potential for that to get out of control and turn into a three-ring-circus of rating-wars might be very unfavorable and cause a lot of hardship and anxiety. It would be quite entertaining at first though :bounce: :wip:

Zack
12-18-2004, 11:00 PM
I'd say it depends on the area.

In the gallery area, I'd suggest having them be more positive:
* Needs improvement
** Getting There
*** Great Work
**** Excellent

I don't think artwork, besides that which is truly nasty, should ever be insulted. Constructive criticism is much better. In the general discussion and cg news area's I think it's much better to be able to say what you think.

DDS
12-29-2004, 09:21 AM
In the gallery forums, anyone who replies should be FORCED to vote when they click "submit reply" IMO

I like GOT's second rating system too :thumbsup:

CGTalk Moderation
01-20-2006, 02:00 AM
This thread has been automatically closed as it remained inactive for 12 months. If you wish to continue the discussion, please create a new thread in the appropriate forum.