Here are my ideas for a revamped rating system here on Protagonize. Share your ideas.
I had a discussion with NickB recently in the forums about ratings. It is obvious to me after that discussion that the current rating system is lacking. Here's my idea for a revamp.
First of all, I think that there are too many ratings. To help discourage popularity rating, I would suggest that there be only two ratings: good and bad. If someone did at least a decent job, it is rated good. If someone did a great job, it is rated good as well.
Second, for the ratings to mean something, for people to learn from them, more information needs to be gleamed. So, I think there ought to be more than one category. I thought of three: grammar, content, and entertainability.
Grammar would include ... well, grammar - the use of commas, periods, sentence structure, nouns and verbs, other parts of speech, spelling, etc ...
Content would include things such as thought process, organization of ideas, higher forms of speech (metaphor), the use of illustrations, the level of detail, etc ...
Entertainability: was it funny, sad, boring, insulting, etc ...
Third, poetry should not have the same ratings as prose. I wouldn't divide it any further, unless you wanted to throw in a third category for neither.
Fourth, require more than just a good or bad rating. It might be easier to describe how this might be done instead of a general description.
You read a piece. You want to rate it. In the grammar section, you thought commas were bad but everything else was good. So, you mark commas as bad and leave everything else good (the default setting). In content, you thought that there should have been more description, so you mark that bad and leave everything else as good. In the entertainability section, you mark it as boring.
In the end, there is a grammar rating, which is an average of each part of grammar. So, having bad spelling, for example, won't necessarily hurt you in the end if other things are marked as good.
The three ratings are averaged to give you a final rating, which is an average of everything. There are probably ways of preventing (discouraging) someone from giving a bad rating just for the sake of giving a bad rating.
Now, the above rating system would require a lot of data for the server to keep track of. Instead of one number for each story from each person who rates, there would be multiple numbers from each person.
Programming shouldn't be too hard; server load would be the key factor. However, I think the advantages might out weigh the difficulty of keeping the system going. (I know ... money and all!)
Each member could have a statistics page. On that page, a member could learn that a particular story or all their stories in general are marked as having bad commas. They know, then, that they need to work on commas.
I think this system would be less about numbers, but there could still be numbers to rank people. But, in the end, people are ranked on how well they actually write, not on how much they are liked.