Rating vs. Ranking
For the majority of my coaching career, I had the privilege of also holding the position of physical education instructor. I taught everything from activity classes to lecture classes to practicum. The lecture classes got to be a little too much, even though they were basically a NSCA CSCS Exam prep course.
We ended up adding a classroom component to our practicum curriculum. Our physical education practicum was basically an internship program for internal candidates. Class credit from the host institution which made up about 80-90% of our interns. External candidate followed the same curriculum but depended on their institutions requirements on how they would be graded.
A lot of what I write is taken from the authors and coaches I have read and learned from. I do, on occasion, have some thoughts that I develop some principles from and I am pretty adamant about this one.
Rating < Ranking
The Problem
The first time I really was bothered by a rating system to evaluate was our instructor evaluation process in which our students would evaluate us and there were 3 issues which I am sure would appear on many evaluation forms.
1. The Numbers.
This particular evaluation had the student rank the instructor on a scale from 1 to 7. Whether you choose 1-5 or 1-10, it doesn't matter. The issue is what exactly is the difference between, let's say, a 6 and a 7? These are just arbitrary numbers.
One technique some forms utilize is adding words instead of numbers. But, again, can someone concretely tell me the difference between outstanding and excellent? There may be not enough variables.
Phrases start to help a little but can be comparative to a group or the past evaluated. Phrases like "one of the best instructors I've had" help, but again this becomes very subjective.
2. The Questions
Often times, questions are very subjective, which in turn defeats the porpoese of an objective evaluation. Poor questions choices often come up. Here are some examples from some of my evaluatins my students did of me:
Q1: The Instructor was knowledgeable about the subject.
The problem is the learner is being asked to determine if the person teaching them know their subject, Sure, you can always tell if someone knows their shit, but this is still a poor questions chose
Q2: The instructor increased my interest in the subject matter.
Is that really the instructors job and isn't most of that have to come back on the student?
Those are just two examples. Another problem happens when a questions that asks the evaluator to rate the instructor with a borderline yes or no question.
Possible Solutions
Rubrics, Matrix, and Rankings
Ok, so the problem with the rating system is there is sometimes no context, the numbers are too vague, and the rating lacks meaning. Here are some ways that the elevation can be improved.
Explanations of the Numbers
I have had evaluations of interns where there are explanations of each rating. For example, on a scale of one to five and the topic was problem solving; there would be a brief explanation and even examples of what a 5 would entail as opposed to a 4. This made the evaluation less arbitrary and more accurate for the evaluator.
More Suitable Questions
As hinted on before, the style of question can lead to a more accurate evaluation. Being specific can be difficult and can lead to an unnecessarily long evaluation process.
Questions that are specific to the tasks that the subject being evaluated on can help. Questions like:
The instructor gave timely feedback on all projects.
This is supposed to be a rating but really is a yes or no question. By adding value to each rating, which would not only add value to the time frame in which the instructor gave feedback but also the quality of the feedback.
Ranking
Sometimes this isn't feasible, but ranking a group of individuals will often get you a more accurate depiction of a group. For example, giving each of your interns a score will sometimes be misleading, especially if multiple categories are added or averaged to give an intern and overall ranking. This may be fine to compare from year to year, but ranking interns is a great way to really figure out who you would give a recommendation for.
Basically, it works great for athletics. This is also a great tool for cross-referencing with other coaches to rank athletes on subjective factors such as leadership and work ethic. By asking your assistants to rank the athlete on a certain team you do two things.
1.) Get more perspective from more sets of eyes to paint a more accurate picture of the athlete.
2.) You will get to see how cohesive your staff is and if there is anyone (including yourself) out of touch with the rest of the group.
You can also have student rank each other. Again, this works best for subjective tasks like participation, group projects and presentations.
Here are some examples of our KIC evaluations we've used as peer and self evaluations for our internship.
KIC Evaluations
- Basic Knowledge - Can They?
- Taking Initiative - Will They?
- Communication Skills - How Do They?
Level 0 - The intern does not know basic exercise technique
Level 1 - The intern know basic exercise technique and can a.) explain it thoroughly and b.) demonstrate it proficiently
Level 2 - The intern can visually identify proper technique execution and technique discrepancies of an athlete
Level 3 - The intern understands the direct or indirect causes of technique discrepancies and can suggest technique adjustments (for an acute affect) and help formulate corrective strategies (for a cumulative affect) with the strength coaches approval.
Taking Initiative
Level 0 - The intern does not interact with student athletes during training sessions
Level 1 - The intern will occasionally interact with student athletes or prepare for the next training group mostly when asked to by another coach and with set up
Level 2 – The intern will interact with student athletes most of the time and occasionally prepares for the next training group mostly when asked to by another coach and with set up
Level 3- The intern coaches athletes at every opportunity and consistently organizes equipment and starts training sessions without being asked to
Communication Skills
Level 0 - The intern has NOT built a positive rapport any student athletes
Level 1 - The intern has built a positive rapport with some student athletes base on social circumstances
Level 2 - The intern has built a positive rapport with most student athletes based on respect for the S&C program
Level 3 – The intern has built a positive rapport with all student athletes based on trust and respect for the student coach
TRAINING
SATURDAY
Log Clean & Press
200 x 5,3,1
Deadlift
375 x 1, 2, 3, 4, 3, 2, 1