CGTalk > CGTalk Support Forums > Suggestions
To minimize the ads you see on this page create a CGTalk account and log in HERE
Thread Closed share thread « Previous Thread | Next Thread »
 
Thread Tools Search this Thread Display Modes
Old 12-08-2004, 08:12 AM   #1
gnz
Capitán Hook
 
gnz's Avatar
portfolio
Gonzalo De la Peña Andreu
NURBS Patcher
Mexico City, Mexico
 
Join Date: Feb 2003
Posts: 216
About the stars rating system

New forum... Here Im reposting my previous suggestion. The original thread has some interesting replies.


Hi, After spending some time rating threads on the finished work forums, Ive come to the conclusion that the current stars rating system is somewhat flawed. Here is the reasoning:

We currently have:
***** Excellent
**** Good
*** Average
** Poor
* Terrible

In theory this would work fine as it gives a whole range to rate the quality of the thread. But it could only work if everyone actually rated every thread they viewed.

The reality is (I think) that if someone takes the effort to go and click on the rating menu is because he/she thinks the content is avobe the average quality and deserves to stand out. I mean, if I think some thread is really "terrible" or even "average" I will simply won't care about rating it. Does this make sense?

What I suggest is having something like:
* Average
** Good
*** Very Good
**** Excellent
***** Sublime <- probably some other calificative would be better suited here

I think with a scheme similar to what I suggest the stars rating system would be much more effective. What do other cgTalkers think?

--
Just my 2c
Gnz
__________________

"The secret to creativity is knowing how to hide your sources." -Albert Einstein
 
Old 12-08-2004, 12:32 PM   #2
Garma
No light please, thanks
 
Garma's Avatar
It will stay hidden.
Student
Delft, Netherlands
 
Join Date: Apr 2003
Posts: 3,396
I think a "bad" rating should stay, it happens now and then that people rate something very good when I honestly think it is bad. maybe remove the "terrible"? like:

bad
average
good
very good
excellent

?
 
Old 12-08-2004, 09:17 PM   #3
Ithilien
Guest
 
Posts: n/a
Just throwing in my 2 cents... I see your points and empathize.

In my opinion (and mine only, you're welcome to disagree) I think that they should stay because they provide an easy and fair way for people to quickly review work. I rarely click on any artwork in the gallery that does not have star ratings - even if it has 1,2,3 star ratings, I will click on it to see what the hullaballoo is all about!

Yes you're right. It is misused. People give 1 star ratings just to piss off the artist. But for the most part, I think that it is a fair means of testing public opinion on an artwork.

My 2 cents...
 
Old 12-08-2004, 10:21 PM   #4
gnz
Capitán Hook
 
gnz's Avatar
portfolio
Gonzalo De la Peña Andreu
NURBS Patcher
Mexico City, Mexico
 
Join Date: Feb 2003
Posts: 216
Quote:
Originally Posted by Garma
I think a "bad" rating should stay, it happens now and then that people rate something very good when I honestly think it is bad.


I see your point. But the idea of my suggestion is to have a thread with at least one star be something worth a look at. Having any stars representing bad work is what its currently not working.

How about having plain stars without labels? ... let the users interpret if one star is worth looking at or not.

What my suggestion is for in the end, is to provide a more ample range to express our liking of a particular piece... I mean, I already like it, that's why Im taking the trouble to rate it... Reading the "average", "poor" and "terrible" labels is what stops me from giving a thread three, two or at least one star, which I think all would do good to it.

Now, what we could do about the issue of over-rated threads (Those threads that are nice but very far from perfect and that the author and his dog have it currently rated *****)? How about listing them like this:

Thread title *****(10 votes)
Another thread ***(86 votes)

The number between pharenteses being the number votes it has received. That would give me a much better idea of wether a thread is accurately rated or not.

Does this make sense?
__________________

"The secret to creativity is knowing how to hide your sources." -Albert Einstein

Last edited by gnz : 12-13-2004 at 11:42 PM. Reason: grammar correction
 
Old 12-13-2004, 01:54 PM   #5
jussing
VFX supervisor
 
jussing's Avatar
portfolio
Jonas Ussing
Freelance VFX supervisor
Jussing VFX
Copenhagen, DK
 
Join Date: Dec 2002
Posts: 2,366
OK, after long consideration, I've come to my final argument about this:

THE SINGLE BIGGEST problem with the rating system is, that NOT ENOUGH PEOPLE USE IT.

No matter how good or perfect you design the system, it's all for nothing when so few people vote, that a single malicious down-voter can topple the entire rating with a one-star rating.

Plus, if people get 4 or 5 friends to register and give their work 5 stars, that will also comlpetely annihilate the fair "overall" rating.

So no. 1 priority regarding rating, is to get more people to rate! Only then will it represent a true rating.

Right now, the rating is nothing but a battle between down-voters, and the artist's friendly up-voters. And in the middle, vastly outnumbered, are the few people who use the rating system as it is meant to be used. -In my opinion.

Cheers everybody,
- Jonas
__________________
 
Old 12-13-2004, 01:58 PM   #6
jussing
VFX supervisor
 
jussing's Avatar
portfolio
Jonas Ussing
Freelance VFX supervisor
Jussing VFX
Copenhagen, DK
 
Join Date: Dec 2002
Posts: 2,366
...not to mention, that any rating system where you can see the result before voting is, inherently, not working, and never will.

-Even a lot of honest voters (=not evil down-voters or biased friends) will not use the full scale, but rather use the maximum and minimum rating, for "arrow up" and "arrow down", respectively. -Meaning that, mathematically, the last votes will have more impact on the final result than the first votes. So it's a lost race already.
__________________
 
Old 12-13-2004, 11:31 PM   #7
jussing
VFX supervisor
 
jussing's Avatar
portfolio
Jonas Ussing
Freelance VFX supervisor
Jussing VFX
Copenhagen, DK
 
Join Date: Dec 2002
Posts: 2,366
Quote:
Originally Posted by gnz
Thread title *****(10 votes)
Another thread ***(86 votes)

The number between pharenteses being the number votes it has received. That would me a much better idea of wether a thread is accurately rated or not.

Does this make sense?
Yes, that makes a lot of sense to me, particularly in regards to the issues I have just adressed. You will then know if a thread has enough votes to show a "fair" rating.

As an alternative, you can raise the minimum number of ratings (before the rating shows) from 3 to 20 or something. But, this will also require more people to vote, or almost no ratings will show.

A difficult challenge indeed. This whole rating thing is actually, IMHO, the singlemost important thing to be adressed of all the things currently posted in this suggestions forum. Because the rating is a bloody good idea, but it is presently not working, and getting it to work is going to be tricky...

Cheers,
- Jonas
__________________
 
Old 12-13-2004, 11:41 PM   #8
gnz
Capitán Hook
 
gnz's Avatar
portfolio
Gonzalo De la Peña Andreu
NURBS Patcher
Mexico City, Mexico
 
Join Date: Feb 2003
Posts: 216
Quote:
Originally Posted by jussing
THE SINGLE BIGGEST problem with the rating system is, that NOT ENOUGH PEOPLE USE IT.
Yes I think too this is the root of the problem.

I suggest giving the thread rating menu a better placement and design, make it hard to resist casting a vote.

I have some ideas of layouts for the thread pages, Im going to produce a test one and post it for comment. Im a web designer/developer BTW.
__________________

"The secret to creativity is knowing how to hide your sources." -Albert Einstein
 
Old 12-14-2004, 12:12 AM   #9
Garma
No light please, thanks
 
Garma's Avatar
It will stay hidden.
Student
Delft, Netherlands
 
Join Date: Apr 2003
Posts: 3,396
Quote:
Originally Posted by gnz
I suggest giving the thread rating menu a better placement and design, make it hard to resist casting a vote.


or force people to rate when they leave the thread

j/k
 
Old 12-14-2004, 12:20 AM   #10
gnz
Capitán Hook
 
gnz's Avatar
portfolio
Gonzalo De la Peña Andreu
NURBS Patcher
Mexico City, Mexico
 
Join Date: Feb 2003
Posts: 216
Quote:
Originally Posted by Garma
or force people to rate when they leave the thread
j/k
heh imagine the next time you go http://www.cgtalk.com :
Sorry, no CGTalk for you! you need to do your homework and rate the following threads you have visited: *threads*
Thank you.
__________________

"The secret to creativity is knowing how to hide your sources." -Albert Einstein
 
Old 12-14-2004, 12:29 AM   #11
gnz
Capitán Hook
 
gnz's Avatar
portfolio
Gonzalo De la Peña Andreu
NURBS Patcher
Mexico City, Mexico
 
Join Date: Feb 2003
Posts: 216
Quote:
Originally Posted by gnz
heh imagine the next time you go http://www.cgtalk.com :
Sorry, no CGTalk for you! you need to do your homework and rate the following threads you have visited: *threads*
Thank you.
Hey but seriously, now that I read my own post I think it could actually work. Not exactly like that of course, but having an unobstructive box on your home page that invites you to quickly cast a vote on threads you have recently visited. Kind of what netflix does with movies you have rented from them.
__________________

"The secret to creativity is knowing how to hide your sources." -Albert Einstein
 
Old 12-14-2004, 03:01 AM   #12
salmonmoose
Moose Administrator
 
salmonmoose's Avatar
portfolio
Anton Fletcher
Gluing captions to cats
Ballistic Media
Australia
 
Join Date: Aug 2003
Posts: 4,172
Send a message via ICQ to salmonmoose Send a message via AIM to salmonmoose Send a message via MSN to salmonmoose Send a message via Yahoo to salmonmoose
Perhaps a different system altogether... take a look at www.bash.org.

Now, once you stop giggilng, each post there has a + and a - link, which you can easily click on. If you click + it adds to the score, if you click - it takes from it.

It cuts down on the process a bit, either you enjoy a thread or not, simple really.

You could even compare the votes against other things like the age of the thread, how many visits it's had and have new ways to sort things.
__________________
Get off my lawn.

Twitter: salmonmoose
 
Old 12-16-2004, 01:55 AM   #13
gnz
Capitán Hook
 
gnz's Avatar
portfolio
Gonzalo De la Peña Andreu
NURBS Patcher
Mexico City, Mexico
 
Join Date: Feb 2003
Posts: 216
Quote:
Originally Posted by antrent
Perhaps a different system altogether... take a look at www.bash.org.

Now, once you stop giggilng, each post there has a + and a - link, which you can easily click on. If you click + it adds to the score, if you click - it takes from it.

It cuts down on the process a bit, either you enjoy a thread or not, simple really.

You could even compare the votes against other things like the age of the thread, how many visits it's had and have new ways to sort things.
Thats a good site I think I had seen it ages ago when I was more into IRC.

Its a very interesting system, but being realistic, implementing a whole different way of rating threads would be much more technically difficult given the size of cgTalk. We might have a better chance of having something implemented if its not very complicated.

Maybe if some time in the future there is a big revamp planned for the forum then we can suggest things like this.
__________________

"The secret to creativity is knowing how to hide your sources." -Albert Einstein
 
Old 12-18-2004, 01:06 AM   #14
GOTgraphic
As Effective as Floss
 
GOTgraphic's Avatar
portfolio
Mark Spencer Reynolds
His Graphicness
GOT GRAPHIC™
NEBRASKA, USA
 
Join Date: Nov 2004
Posts: 685
Send a message via ICQ to GOTgraphic Send a message via AIM to GOTgraphic Send a message via MSN to GOTgraphic Send a message via Yahoo to GOTgraphic
I don't like the current star-rating system either. I'm not going to rate anyone a 1 or 2 or 3 star. As far as I'm concerned "average" is bad enough. Artists know if their work is lousy, so giving them a 1 or 2 star rating is ridiculous. Giving them NO STARS means is really bad or not worth bothering to click the button (hence really bad). So...

I think Gns's suggestion of:

* Average
** Good
*** Very Good
**** Excellent
***** Sublime

would be a much better rating system. If it were up to me I would go this route:

* Average
** Good
*** Wonderful
**** Fantastic
***** Perfect

I definitely think it should be changed.
__________________
GOT GRAPHIC
NOTHING IS TOO BEAUTIFUL -Bugatti
UGLINESSS DOESN'T SELL -Loewy
 
Old 12-18-2004, 07:58 PM   #15
jussing
VFX supervisor
 
jussing's Avatar
portfolio
Jonas Ussing
Freelance VFX supervisor
Jussing VFX
Copenhagen, DK
 
Join Date: Dec 2002
Posts: 2,366
I think five "good" rates is too much. I say just shave off the bottom two, so it's

"Average"
"Good"
"Excellent"

Five rates is better when browsing, 'cause it gives a better picture of the variation in the work, BUT...

-When rating, how do you judge between FIVE different rates of good? I mean, who is to tell the difference between "excellent and "sublime", or "wonderful" and "fantastic"?

Cheers,
- Jonas
__________________
 
Thread Closed share thread


Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
CGSociety
Society of Digital Artists
www.cgsociety.org

Powered by vBulletin
Copyright ©2000 - 2006,
Jelsoft Enterprises Ltd.
Minimize Ads
Forum Jump
Miscellaneous

All times are GMT. The time now is 01:11 PM.


Powered by vBulletin
Copyright ©2000 - 2016, Jelsoft Enterprises Ltd.