Sometimes it can, although it's usually more accurate that the more red someone has, the 'less helpful' other people believe they are.
However, if you have 19 green (rating: 5) votes, and then someone gives you less, such as a 4 rating, then your score will become mixed, like mine is.
The way to vote for someone is to hit the profile button underneath their post, and then you will see 'Rate this User' with a drop-down box. Just click on the vote you believe they deserve, and the job's done.
A few things to note:
You can change your voting at any time by repeating the process, so if you change your view, you can change your vote.
If you have 2 or fewer votes you will not have a live link beneath your avatar on each post. Once you get 3 votes, the link becomes live, and by clicking on it you can see who has voted for you, although you cannot see what they voted.
On the list of people who have voted for you, they are not in numerical order, but ranked. So moderators will always appear at the top, then senior members, then junior members, etc.