Author Topic: A little help please  (Read 417 times)

Offline SCHSref

  • *
  • Posts: 256
  • FAN REACTION: +8/-9
  • In the land of the blind, the one eyed man is king
A little help please
« on: March 09, 2018, 07:58:39 AM »
Just curious how your state/local association conducts ratings for officials.  Any detailed info would be great.  Also, maybe some pros and cons of your system.

Thanks again and just a few more months till Friday Night Lights
If you didn't see it, you can't call it

Offline bossman72

  • *
  • Posts: 1181
  • FAN REACTION: +56/-22
Re: A little help please
« Reply #1 on: March 09, 2018, 08:21:16 AM »
PA has no formal system.

Sometimes you get evaluated, but that's not hard-tied to your assignments in any way.  If I grade out at a 92 for a playoff game and someone else grades out at an 80, the other person can get the championship game instead of you.  The evals are more for your personal edification.

Same thing for regular season.

Offline ChicagoZebra

  • *
  • Posts: 117
  • FAN REACTION: +2/-1
  • Without officials... it is only recess.
Re: A little help please
« Reply #2 on: March 09, 2018, 09:21:04 AM »
Chicago, Illinois:

State - the IHSA has a very convoluted "Power Rankings" system, but ultimately playoff assignments are more of a who-do-you-know thing. Power Rankings include: previous tournaments, IHSA rankings, Part II exam score, "Top 15" lists made by coaches, and coaches scores (1-6 scale).

Local - no formal evaluation system here. Lot of great mentors giving pointers on different topics though.
« Last Edit: March 09, 2018, 09:33:07 AM by ChicagoZebra »

Offline Ralph Damren

  • *
  • Posts: 2502
  • FAN REACTION: +303/-27
  • Without officials... it is only recess.
Re: A little help please
« Reply #3 on: March 09, 2018, 09:25:20 AM »
The evolution of our rating system came about to apply officials for a pool used by the state association in state games. Our state establishes available officials from a combination of coach's votes and those in the top 25% of their chapter via rating. Chapter assigners and the state football committee gather and fill 28 slots (7 man crews for 4 classes). An official can only work a championship game every other year.

Each chapter can develop their own rating system, ours is :

30% NFHS Part II taken on website.
40% Fellow official's rating - four components:
   (1) Rule knowledge
   (2) Game control
   (3) Teamwork
   (4) Appearance and hustle
20% Meeting attendance
 8% Seniority
 2% Evaluation of fellow officials

Our chapter's assignor may use your rating in providing assignments. If you are rated #36 and there are only 7 games (5-man crews) ,you may not get a game unless someone above you is sick/dead/in jail.

Most find it fair :) , some don't  >:( , some wouldn't find anything fair :puke: it's really time for some lobster chowder !!

Offline FLAHL

  • *
  • Posts: 677
  • FAN REACTION: +41/-8
Re: A little help please
« Reply #4 on: March 09, 2018, 10:48:31 AM »
The state assigns us a rating of 1, 2, or 3 based on years of experience and test score.  Only 1 & 2 rated officials are eligible to work playoff games.

Each association submits a list of playoff crews to the state.  We usually have 5 or 6 crews, and they are ranked 1 through X based on evaluations throughout the year.  Each crew is evaluated twice, and the evaluators are (mostly) guys who have been with the association for quite a while but no longer work on the field.  We evaluate the things you'd expect to see - mechanics, pre game activities, hustle, uniforms, communication, rules, etc.  The biggest complaint that we have is consistency (or lack of consistency) among the evaluators, but the process is subjective so, as Ralph said, "Most find it fair, some don't, and some wouldn't find anything fair."

If I could change anything about our process, I'd have the evaluations done like olympic scoring - everybody starts with a 10, and then may have deductions for things that are not up to par.  But there's no perfect way to do this.  A couple of years ago, there was a lot of moaning and groaning among the NFL officials about the guy selected as the WH.  I remember thinking that if they can't get this right at their level, we have no chance at ours.
"Never argue with stupid people.  They will drag you down to their level and then beat you with experience." - Mark Twain

Offline SouthGARef

  • *
  • Posts: 255
  • FAN REACTION: +10/-15
Re: A little help please
« Reply #5 on: March 09, 2018, 02:18:32 PM »
We'll let ya know when we figure it out. :)

Offline prab

  • *
  • Posts: 539
  • FAN REACTION: +33/-37
  • Wherever you go, there you are!
Re: A little help please
« Reply #6 on: March 09, 2018, 03:54:35 PM »
Wisconsin - Coaches may rate officials.  Some do, some don't.  Officials can find out which schools submitted ratings, but can not learn who gave what rating.  Ratings are not even provided in a generic form.  Even if ratings are accurate, it is impossible for officials to use them as a basis for improvement.  I believe that the ratings are used solely as a CYA for the state association to assign officials to play off games.  Coaches are discouraged, but still allowed to rate officials who have not worked a game for that school during the season.

Online Magician

  • *
  • Posts: 490
  • FAN REACTION: +98/-4
Re: A little help please
« Reply #7 on: March 09, 2018, 04:59:24 PM »
Indiana uses a system that is 50% coach/AD vote and 50% other factors. The other factors are things like years licensed, years of playoff experience, playoff advancement, association attendance, and games worked (self reported scheduled since it's done as part of your application at the beginning of the year). Test score used to be a part of it, but they removed it last year. You still have to pass the test to remain eligible. Everything is on a scale of 1-5 and weighted. For example, you get full credit (5 points) for years licensed if the top 3 guys on your crew average 10 years of experience. It then scales down. I think max for playoff experience is 6 years for the top 3. Meeting attendance is 8 for football. Most associations will have 10-12 meetings and the state offers 2-4 online meetings that count. The summer clinic counts as well. You need to have 8 varsity games to get full credit, and we have 9 regular season weeks. To get full credit for playoff advancement your top 3 guys need to have worked a state final. A semi-final is 4, regional final is 3, sectional final is 2.

Most crews get full credit except for the advancement portion. That means the real differentiator is still the coach/AD vote. The coaches are the ones that are supposed to vote, but the application is accessed by the AD account so it could be one or the other or both that complete the vote. All schools are required to vote and are asked to only vote for the crews they had in the past 3 years. The IHSAA has no way to track who those schools are though so they let every school vote for every crew. Some crews get 80-100 votes. Most get 25-35. The general thought is the crews with 80-100 votes are getting a lot of extra votes from people who know them for various reasons (i.e. they have guys who work other sports especially basketball and baseball, they have guys are/were coaches/administrators, they have been doing this so long they know a LOT of people) and all those votes would be positive. This greatly increases their average score. The 80-100 vote crews will get a 4.7/4.8 average while the 25-35 vote crews will usually top out around 4.3/4.4 with many lower. They may have the same number of 1/2/3 votes but the extra 5s bring up the average much higher.

Once all the votes are in the crews are rated 1-x depending on how many there are. There are 125-130 playoff games week 1 and last year we had 142 crews apply. Week 2 has 96 games, week 3 has 48 games and so forth to the finals with 6 games. Starting with the sectional final round (48 games) at least half the crews have to have never worked a state final. That is determined by the referee. It's possible there is a crew with 3-4 guys who have worked a state final but now have a different referee because their state referee retired. I call these recycled crews. They still get full credit on the points because they have 3 state final officials, but they are on the "new" side because their current referee wasn't the white hat for their state final game. Of the 24 "new" sectional final crews it seems 6-8 of them are recycled crews. As they advance to the regional and semi-state rounds the percentage of recycled crews gets higher. It's not impossible for a truly "new" crew to advance, but it can be very difficult. They have very little margin for error in the vote.

A brand new crew can only work 2 rounds but usually only get 1 because they don't know enough coaches/ADs to get enough votes. A recycled crew is still considered new for advancement, but the can work up to 3 rounds. Any crew on their second year could advance to the third round (sectional final). From there you can't work a later round unless you worked the prior round in a previous year. Once you have worked a state final you can only advance to the sectional final the following year and then one additional round each year. That means a state final crew can only work a final every 4 years. Of the 24 crews that work a state final over a 4-year period, I would estimate 16-18 of them return in the next 4 years. It's hard to break into that group, but if you keep most of the guys together you'll likely return as long as you stay together.

It's a really bad system because quality of crew doesn't really play into it. It's not really an old boys network because it's not based on an individual or group getting together to decide ranking. It's based largely on the collective vote and the more people you know that vote the more likely you are to advance. So the ranking isn't a measure of quality. It's just a number assigned to each crew so the state can assign the crews.

I agree with FLAHL. No matter what system is used the same number of crews will be upset. There really isn't a way to create a completely subjective method of evaluation. I've heard the same issues with NFL ratings. They are graded on every play so theoretically the highest graded officials are working the playoffs. But I was told once they know by mid-season who they want to work the Super Bowl and all the sudden all their 50/50 calls are graded as correct.

Control what you can control. Work the games you are assigned to the best of your ability. And have fun doing this. Bad assignment processes aren't personal, and they aren't intended to be a reflection of your performance. Everybody wants a process that puts them in a state final, and there are only so many spots there.

Offline the clown

  • *
  • Posts: 23
  • FAN REACTION: +5/-17
  • Without officials... it is only recess.
Re: A little help please
« Reply #8 on: March 09, 2018, 09:55:20 PM »
Henry Ford said it best, "It's not what you know.. It's who you know."  In California we're required to be nominated by our association and then submit a picture so they can tell what, "Shape" your in.  However, some believe that the picture is for diversity... Not that there is anything wrong with that.  They would like the officials to look a bit like the players on the field... I believe we've only had a "state Championship" for football for 10 years now.  We have 52 league Championships, 26 state regional championships, 13 State championships.  What makes the system work is that it's your Chapter / Association that nominates you, one person per 7-man crew position, so most groups will not renominate someone who has already worked a State Championship.  All seven can get a State game or none.  We have averaged two or three per year.  Out of our 80 member group we have 14 who have worked a State Championship game.  We take greater pride in the nomination then working a "State" game.  Rumor has it that we are switching to crews next year.  Your association nominates a 7 man crew and that crew works one of the regional finals and hopefully a Championship final.  I like the system.

Offline Ralph Damren

  • *
  • Posts: 2502
  • FAN REACTION: +303/-27
  • Without officials... it is only recess.
Re: A little help please
« Reply #9 on: March 15, 2018, 01:54:41 PM »
IMHO, the two most important components of any rating system is :
 (1) Transparency - you've got to be able to see why your rating is what it is. This will enable you to see where you can improve. Example :
    You have a rating of 92 and are rated 10th;
    You scored 90 on NFHS test (worth 30%);
    Each miss costs you 0.3 points;
    You're supplied with everyone's rating (without names);
    You notice that a 95% would jump you to 7th;
    You now have a goal = put more time into the test.

 (2) Simplistic - you've got be able to understand it's rationale and formula. There'll be a reason for every factor and it's good know, if not agree, with them.

IMHO, school votes are another issue. They are our employer, so many feel they deserve a say. When I first started in Maine (1971) we supplied self-addressed/stamped postcards to each coach for rating purposes. That stopped when it was realized that 90% of those returned were from losing coaches >:(. Now our state association asks each school to vote for 12 officials for state championship games They get a high return , and that helps to form the pool. For playoffs leading up to that, our assigner asks each school on post-season to supply a list ranking their top 20 in our chapter (we have a membership of +45). He will pare the list of opponents up and try to only assign names on both lists. Seems to work...I think.....some don't think it works...and some don't think at all...


Offline bama_stripes

  • *
  • Posts: 2217
  • FAN REACTION: +69/-24
Re: A little help please
« Reply #10 on: Yesterday at 06:43:16 AM »
Thankfully, coaches here have no say about playoff officials.  The problem with allowing their input is that it encourages officials to cultivate a "buddy-buddy" approach with coaches, rather than nut up & make the tough call in a tough situation.

Offline TampaSteve

  • *
  • Posts: 1270
  • FAN REACTION: +13/-10
Re: A little help please
« Reply #11 on: Today at 03:56:06 PM »
while i'm with those who do not want any input from any coach to evaluate officials.....
Round these parts, it's impossible to see any coach whom you see during the regular season during playoffs.