By BLR Founder and CEO Bob Brady
Several years ago, the HR department of the city of New Haven, Connecticut, was presented with a very difficult choice: It could accept the results of a recent promotion exam and risk the ire of the majority of city residents, or ignore them and risk a lawsuit by the firefighters who would be denied promotions.
In the end, prodded by public opinion and politics, the city decided to throw out the results of the test. (Note: The city didn’t decide to promote lower-scoring applicants—it just decided to start over again.)*
And then the trouble began. The union representing the firefighters sued and, in a well-publicized case, the U.S. Supreme Court ultimately decided against the city.
The events that led up to the case are an example of a “perfect storm” where several factors come together to yield a big, bad result:
- An inflexible union contract that specified who could sit for a promotion exam and how promotions were to be made;
- A city charter that absolutely mandated paper and pencil tests for determining who was best qualified;
- Two competing “good old boy” networks, and
- City political factions that were heavily racial and ethnic in their orientation—each determined to protect its constituency.
I’m glad I wasn’t the HR manager charged with maneuvering through this minefield. *
See what everyone’s talking about! Check out BLR’s remarkable everything-you-need-for-HR website, HR.BLR.com, at no cost or risk, and get a complimentary special report! Get more info.
Tried Their Hardest
The combination of these factors made it nearly impossible for HR to function effectively, but city HR professionals tried their hardest. The city charter required that a test be used to determine promotion, so the city hired a well-known, highly qualified firm to develop one, paying them $100,000.
Both the test developer and the city followed a rigorous protocol designed to ensure that the test was good and that it was not compromised. For example, no one in HR was allowed to see any test question before it was administered because of past accusations that questions had been leaked.
The test was administered and, when the results came back, no African Americans—and only one Hispanic—made it into the top of the list, though they were heavily represented in the rank and file. It wasn’t that a good number didn’t pass the test, demonstrating considerable competence. However, under the rules, only the highest-scoring individuals made it into the hiring pool.
This put city administrators in a very precarious position: Use the test scores and face a very unhappy minority community and probably a lawsuit on the basis of “adverse impact,” or ignore the scores and face a lawsuit because of “reverse discrimination.”
The Civil Service Commission, which had to certify the results, voted 2 to 2 (the fifth member did not participate because she was the sister of one of the white firefighters). Under the rules, a tie meant that the results were not certified, and that they had to go back to the drawing board. The high-scoring applicants were understandably outraged. They’d studied long and hard and now their efforts were for naught. They decided to sue and were recently vindicated by the Supreme Court.
But Who Are We Kidding?
Do we really think that a paper and pencil test—or for that matter, any test—is really the best way to determine who will be best able to lead firefighters? As this test was devised, it measured ability to memorize and reason. It is important—essential—for a leader to know procedures and best practices, but that is just part of what is required.
The test does not measure the ability to lead firefighters under pressure or deal with complex community issues in high stress situations. Any of us who have worked with similar instruments have seen cases where the highest-scoring individuals were not necessarily the best qualified. Tests measure job skills, and they do so in a particularly “academic” way. They don’t measure life skills, which are hugely important in leadership positions. (** Please give us your opinion on this question. See survey below.)
But HR’s hands were tied. The city charter required tests and the union was inflexible. No doubt, the requirement was put in place to counter political appointments, whereby supporters received the spoils of victory, which was probably even worse. That is the argument made by test proponents today.
Have you road-tested the biggest bargain in HR? Try HR.BLR.com at no cost or risk and receive a special report that’s yours to keep no matter what you decide. Get the details.
Who Is Right?
As the tone of this column probably makes clear, I don’t think that a test is a very good way to decide who gets promoted in a fire department. On the other hand, political bosses, be they 19th century WASPS and Irish, or 21st century African Americans, probably have a different view of “merit” than HR managers.
If I had to choose, I’d choose a test, but I would make sure that the test takers’ pool includes all potentially qualified individuals. I would put everyone who passes the test into a promotion pool, and then rely on department managers and HR managers to make the all-important subjective assessments. Civil Service officials would have the power to review and approve. You can be assured that politics would enter into any such equation, but this is government. Politics will intrude. It is different from a private organization, where delivering a product and making money is the goal. The Fire Department’s “product” is public safety. That calls for more than just putting out fires.
Remember the saying, “You hire people for what they know, and you fire them for who they are.” A test doesn’t measure that.
Anyway, that’s my e-pinion. I’d love to hear yours. E-mail me at Rbrady@blr.com.
* Full disclosure: I should note that the city’s HR director at the time the decision was made, Tina Burgett, is a personal friend, though we’ve never discussed the case. (We share a passion for cabaret singing.) My judgment may be colored by friendship, though I hope it isn’t.
** How would you have decided this case?
Please take a few minutes to answer our four-question survey. We will report on the results in a column in the next few weeks.
Ahh yes, yet there’s one thing that you left out of the story. Who was the deciding judge on the Supreme Court who got the tests thrown out? Can you all say Sotomayor? And note by the way that caucasion’s are quickly becoming the “minority” in this country. But, because they’re white… Well, you get the government that you vote for I guess. If you can’t pass the test, throw the test out! Talk about politics. Makes me sick!
I agree that a test is a good way to ensure the firefighters have a solid understanding of there jobs and it also shows who is really committed to being prepared and doing there best. I would have had a second phase to this promotion process…looking at the leadership skills…and also followership the applicant had. If the team doesn’t believe in their leader it doesn’t matter how well you do on a test.
My opinion is that..YES ..Test scores should count and NOT color of skin…If you are not competent to pass a test, then you should not be promoted because of the color..You have to pass a written and physical test to become a firefighter, why shouldn’t you have to take a test to get promoted??
As a tax paying citizen, I would want the highest scoring lieutenant or captain leading his men to save lives or mine…
I am glad they have sued the City of New Haven and its officials…Enough is enough already!! We do not owe anyone anything!!!!!
I spent some time in the US Army, on active duty and in the reserves, and participated in a few of the schools run by the military. It occurs to me that the company that was hired to create the test did not take into account the real purpose behind the test, was not given effective direction on what to test for, took the easy way out by having only a pencil and paper test, or some other issue other than really doing its job of being a test developer and developing a test for the end result desired by the client. If this was truly a leadership test, and there is reasoning involved, it would seem that some manner of role-playing would be included, to see how the applicants react in real situations. The “in-box” exercise under a time constraint is a simple example of what could be considered a “real-life” situation. Firefighters have outside training facilities, and so some apsects that require physical action could have been included. Look at Ranger school for examples of leading under adverse but real conditions. Anyway, that is my input,
Well done and you certainly did not hear about the conflicting requirements, i.e., city charter and union, from the ‘talking heads’.
I have to agree with you; though, I’ve never really liked the idea of testing period. This situation didn’t leave much room for anything else; however, placing all qualified applicants into the pool would have been the best solution. I firmly believe that managers and supervisors should be capable of selecting the right candidate for a job, given they ask the right questions. If we do not trust our managers and supervisors to make those critical decisions, then why do we place anyone into those positions to begin with?
That’s my opinion; thanks for asking for it.
When a candidate who is dyslexic and desireous of competing for the position to the extent that he has the study material put in an audio format, I would say that there is a level of committment and desire that, if the candidate passes the test, he would be an excellent firefighter because he has already demonstrated effort and personal qualitites that are critical to success in any endeavor.
This is what it is all about – if some of the other candidates put that kind of effort into the process there probably would not have been a case.
In the organization where I work, we have promotional opportunities based on either 1) appraisal of promotability or 2) appraisal of promotability and written/oral test (interview). The appraisal of promotabilility is a document the current supervisor completes documenting the employee’s current and past responsibilities, skills and their ability to excel in the higher level position. This evaluation, provided by the current supervisor, is reviewed by a hr analyst and scored. The supervisor doesn’t provide a score. Some positions also require a written or oral test which is weighted as part of the exam process. All persons who pass the exam process with a score of 70 or above are placed on an eligible list by band/level. As the bands are depleted from the highest to lowest, employees can be chosen.
I haven’t hired that many people, but I have hired some from the lower test bands who have been very good employees and who have since excelled and been promoted to other positions, so hiring someone solely on the basis of an exam score is not the most effective means of choosing an employee who fits the position or determining the success of an individual. I have encountered many people in my organization of whom I wonder how they got their position or even passed the exam. Some people are very good at taking exams, some have very good interview skills, but that should not be the only determining factor for placing someone in a position.
I agree that the top should be the first to be chosen for the interview process, but I don’t think others who pass the exam should be excluded.
Also, the list expires after a certain amount of time, so the applicants would need to re-apply and be re-tested and pass to get on the list again, but hopefully they will achieve a better score and be placed in a higher band.
By the way, our exams do include questions on supervision, policy and procedure, as well as basic academic skills.
The City of New Haven needs to revamp their promotional process.
I really enjoy my daily HR Advisor, but this one was way off the mark. First and foremost, the reason many are disagreeing with the Supreme Court ruling is because this ruling takes Affirmative Action into the 21st Century. No longer can less than qualified candidates get positions solely because of race-which has become the norm due to fear of reprisals and lawsuits. Many have forgotten the REAL reason of Affirmative Action. Affirmative Action came into being because EQUALLY qualified applicants were being passed over for positions because of their race. Affirmative Action was then enacted to ensure that workplaces became equally diversified as the environment they thrived in. Over the next 2 decades, Affirmative Action has become so muddied with interpretations, that now the Supreme Court has made it perfectly clear – you do not pass over qualified individuals because of race, and that is what happen with the New Haven Firefighters.
There is no doubt about it that those test scores should have been certified. You managed to skip over the fact that those who failed, failed miserably because they didn’t study. You also managed to fail to acknowledge that the panel of mixed personnel, also failed some of the candidates in the oral part of the exam, for lack of knowledge for the essential parts of these positions.
In the police and fire departments, it is crucial to know everything about their jobs. For anyone to use these positions as a political tool to further their career is ludicrous and that is what the Mayor of New Haven chose to do. I really suggest you read the Supreme Court ruling, as I have, and you will find that in fact, there were dealings going on behind the scenes because of one specific race.
http://adversity.net/newhavenfd/default.htm
In almost all professional positions, written tests are required to be passed before the professional can be licensed and they must continue to keep up that license with ongoing courses. Yet as firefighters and police officers, these tests are not necessary to begin with. So when it comes to one leading this group, they better damn well know what their job is all about. Having seen some of the written results at the scores of 49 and 50 is dumbfounding.
So in any other industry, where a person’s leadership skills excel but knowledge of the components is weak, that person may still be able to get by on his leaderships skills. However when it comes to the lives of human beings, you better damn well put those that know what they are doing, and the best person for the job in charge, no matter what race or ethnicity.
Thank you
Our experience has been that you are absolutely correct! Using a written test as the sole tool to determine promotions is a not the way to obtain a quality employee. Having knowledge that you can put on paper is only one aspect of the job that requires numerous capabilities.
We utilize a series of tests, practical exercises and interviews to choose individuals for promotion.
In SC we are an “at will’ state and Unions are not recognized so we don’t have the union issues that have created no win situations and strained environments between labor and management in some areas. No doubt this has benefited us in the end albeit contrary to what union organizers may argue.
Our promotion process includes:
A general knowledge written examination approved and prepared by an outside firm that provides candidates with a listing of the materials to study. The score on this test is not to see how much you know but how capable you are of taking study material and being able to study it and retain the material. Basically, is the candidate a good learner and are they willing to put forth the effort to learn and study when that is required. This job is an ever learning field and officers must be capable of continued learning. The test scores are then scored with a point value 90 to 100 (5) 80 to 90 (4) 70 to 80 (3) 60 to 70 (2) 50 to 60 (1)
A written examination that is created in house that covers the policies and procedures that are part of our organization. Scored in the same manner as the general knowledge written examination.
Scenarios are provided where the candidate is required to make a decision based upon the organizations policies and this is put down in writing. We typically include an employee problem, a management problem and an emergency incident problem. These are scored as follows Proper decision and Per policy (5) Questionable Decision but Per Policy (3) Bad decision and not per policy (1)
Table top scenarios that deal with handling emergency incidents based upon training the candidates have received over the course of their employment. We typically have a major structure fire, a Haz-Mat incident, a natural disaster and a mass casualty incident. Proper Incident Command and proper Strategy and Tactics (5) Proper Incident Command and acceptable Strategy and Tactics (4) Acceptable Incident Command and Strategy and Tactics (3) Poor Incident Command and Poor Strategy and Tactics (2) Candidate Completely Failed Scenario (1)
An oral interview conducted by HR that utilizes peer professionals and HR personnel from outside of our organization. They ask a series of general pre-chosen questions approved by HR. If the response to a particular question is unusual the interviewers may ask the candidate to further explain their answer. The HR administrator is present to ensure that the prepared questions are adhered to and that no inappropriate follow-up questions are asked. This interview is scored based upon guidelines provided to the interviewers in an attempt to make the scoring as objective as possible.
Example:
Came properly dressed for a promotion interview? Yes Perfect (5) Less than perfect but acceptable (3) No (1) Properly dressed means business attire shirt tie slacks dress shoes or a Class A uniform minus jacket and hat. Candidates are advised to come properly dressed for the interview as previously indicated. A suit coat is not required for business attire.
The Scores are tallied and each candidate is rated based upon the scores they obtained.
The candidates are brought before a panel of in house senior officers and HR administrator who also conduct and interview based upon a series of prepared questions. Any officers that have worked closely with or have supervised a candidate must refrain from evaluating the candidate. This is done to prevent favoritism and/or negativity towards the candidate. The responses are scored as above and the panel then examines the candidates’ awards, accomplishments, training above and beyond that required by the department, any significant disciplinary actions within the last two years, demeanor, attitude and opinions. This panel makes a recommendation as to the first, second and third rated candidates.
The tallied scoring results and the Officer panel scores are then given the Chief of the department and he along with input form the HR Administrator decide who receives the promotion.
All candidates are given a questionnaire to voluntarily complete and the answers are used to determine areas where additional training may be needed.
A survey of our department personnel was taken as to what their perceptions and opinions of the promotional process were. The personnel overwhelmingly thought the process was very objective and fair and hey understood that the City was attempting to find the best qualified and capable candidate for the position.
I won’t say its perfect but we feel that in the end it has provided us with the best and most desirable candidates for the job. As far as any issues with sex or race they appear to be non-existent as we have promoted females as well as individuals of different races, ethnicities and sexual orientation.
I will say that has our department grows that the officer panels have become extremely objective and effective in their ratings. We initially were concerned that the officer panel might simply resort to favoritism but the decision to eliminate supervisors and close job associates appears to have prevented that.
Now I am sure you can tell us areas of our process where we may be setting ourselves up for some bad promotional situation that may come back and bite us one day.
The test scores should have been taken into consideration for the position or promotion. Testing can be a valuable vehicle for a selection process. Other tests, vehicles and instruments could have been incorporated in the process to make sure that all aspects or competencies relating to the position were tested appropriately. Testing is one of the most valuable tools for hiring and selection, people development, etc., yet is it rarely used effectively by most HR departments.
The results of the test were valid and the Courts finally got it right in upholding the results. Imagine that a black scored highest and a white firefighter objected…disparate impact???
Secondly the the numbers involved are not statistically significant to show disparate treatment, which is always a weak argument allowing courts to arrive at any decision they want.
A fireman needs to know how to enter a burning building, how to turn a water hose on, what to do about flammable liquids, etc…whether or not the fireman was from the inner city!