Saturday, January 22, 2011

The Sky Is Not Falling

The Sky Is Not Falling
A Response To “Why Our Best Officers Are Leaving" by Tim Kane
Some ideas never seem to fall out of fashion. One of these is the notion that our “best” officers leave the military service at extraordinarily high rates and the effects of said exodus are monumental. Tim Kane’s article which was recently published in The Atlantic tosses another log on the fire and stokes the flame of what passes for conventional wisdom. I think it is important to understand the facts from the anecdotes. A little bit of economic theory will go a long way in understanding why the military promotion system evolved into what it is today. A lot of what Kane wrote tangentially hits on some important themes but is off the mark in other ways. First, let me critique the actual article and then come back around to offer a different explanation of the world and why it is so.
To begin, let us not put too much faith into a poorly done “survey” of 250 current and former Army officers. Opinions are an impression of reality not reality in and of itself. I don’t mean to dismiss or discount what people believe but cognitive biases and cascading can lead us all over the place. Lets reread carefully what Kane wrote about how he conducted his survey.
In a recent survey I conducted of 250 West Point graduates (sent to the classes of 1989, 1991, 1995, 2000, 2001, and 2004), an astonishing 93 percent believed that half or more of “the best officers leave the military early rather than serving a full career.” By design, I left the definitions of best and early up to the respondents. I conducted the survey from late August to mid-September, reaching graduates through their class scribes (who manage e-mail lists for periodic newsletters). This ensured that the sample included veterans as well as active-duty officers.    
First and foremost, this is an article about Army officers and Army promotions. We should be careful not to extrapolate Kane’s conclusions into the other services. This tells us nothing about what Air Force officers believe about their promotion systems.
Secondly, separating this paragraph from the rest of the article it is abundantly clear that Kane’s survey doesn’t hold up to any rigorous scientific standards. How well do the respondents represent their cohort populations? What was the response rate to his survey? How many people did not respond to his survey? Was there any systematic self-selection bias as to who did or did not answer his survey? What I’m getting at is if Kane emailed this survey to six different West Point graduating classes, that is potentially  around 6,000 people. If only 250 responded that is a take rate of about 4%. If you could think of at least one reason that some people answered his email and others did not then we have a biased sample. If the 250 respondents represent a random sampling of the pool of candidates then there is no problem. However if higher proportions of officers with negative or pessimistic views about the Army answered the survey then we are left with nothing useful.
Thirdly, Kane only interviews West Point graduates…why not Army ROTC and OCS commissioned graduates? Might they see the world differently than their USMA peers? West Point produces roughly one-third of the required commissioned officers every year. Again the sample is likely to be biased and only tells us what a minority of Army officers think about their service promotion system.
Finally, leaving the terms “early” and “best” up to the respondent seems innocuous enough but its highly problematic to do that. What if the 250 officers had in their mind that leaving the service early was equivalent to less than thirty years of service? Then we would get the exact same response if all the best officers served twenty five years but left before thirty. Are we really in dire straits if the best officers serve at least twenty five years but stop short of thirty? Most likely that is not what the respondents had in mind but the point is that we have no idea what they had in mind when they answered.
If boredom has not caused you to stop reading this blog then I congratulate you and ask you to do one more thing. Reread Kane’s article but skip over or black out any paragraph or sentence that draws upon his survey results. Stripping away all the thoughts and feelings of what is likely a non-representative set of Army officers we can get to the core of the issue. The US Military has a very unique set of parameters to deal with regarding personnel that companies like Intel, Google and IBM do not, and it optimizes accordingly.
Let me start at the beginning with some basic labor economics. In classic labor theory, wages are based on productivity. A worker and boss agree upon a price for each widget or unit of production. If the worker produces more he makes more salary. More productive workers will earn higher salaries but everyone gets paid the same amount per unit of production. That sounds great and simple so why are the best officers not paid more than their less productive peers? Wouldn’t that keep them in the service longer and more satisfied?
Unfortunately, this  theory does not explain wages and salary schemes when we cannot define a unit of production. The US Military produces national defense, we know that, but we also have no way of measuring aggregate national defense production nor any one individual’s contribution to the overall sum. How many units of national defense does a particular Quartermaster officer produce and how does that compare to an Infantry officer? Or to an F-16 pilot? We have no way to know these things. Now, the military does signal the relative importance of certain career specialties through the use of incentive pay. Pilots, Navigators, Submarine officers, and so forth receive somewhat more compensation than their peers but there is no differentiation within these careers. The person considered to be the finest F-15 driver is paid exactly the same as the guy that nobody wants to fly with(controlling for service time and rank).
Even within a specific career specialty it is an incredibly grey issue as to who is more productive than who. It may be clear to everyone in the 101st Airborne Division who is the best platoon leader but how can I objectively demonstrate that? We could use measurements like enemy soldiers killed or insurgents captured but those create perverse incentives. Do we really want to incentivize our infantrymen to pump up their body count numbers or to be less discriminate about who they pickup off of the battlefield?
I suggest a great paper by John T. Warner and Beth J. Asch with a long-ish title “A Theory of Compensation and Personnel Policy in Hierarchical Organizations with Application to the United States Military”. It is on JSTOR and if you do not have access I can email it to you. They hit the nail on the head that real source of all our problems is the idiosyncrasy that there is no lateral entry into the service. That is to say, everyone who enters (except for some specialties like medicine or law) must do so at the bottom of the ladder. All officers start out as Second Lieutenants and it matters not if you are entering with an MBA from Harvard and 8 years of management and operational experience or are twenty-two years old and straight out of ROTC. We can debate the efficacy of this policy on another day but taking it as given one can see right away the Army’s problem.

The problem is this i) a military officer’s performance and production is almost unobservable and unquantifiable and ii) there is no lateral entry into the ranks. What does this mean? It means that the military cannot afford to enact policies that favor the “best” and risk alienating the large mass of officers. To do so would be suicidal. The military is highly dependent on the cohort of officers at every level to shoulder the burden of leadership. Numbers matter a lot! If large numbers of lower and mid-rated officers leave because they are not the “chosen” ones or are not favored then the Army or military cannot go into the market to recruit more Captains, Majors or Lieutenant Colonels. In reverse, IBM and Microsoft can immediately weed out their weaker performers and go recruit people at any level of the organization. So for Kane’s example of Lt. Col Nagle it is too bad that he got out of the service but a promotion policy that benefits him would most likely come at the expense of the vast middle. 

Gen Colin Powell wrote in his autobiography that when he attained his first star and became a Brigadier General all the newly promoted were gathered into a room and addressed by a higher ranking general. This person told them that for as special as they were, not to let it go to their heads. If a plane crash wiped them all out tomorrow the Army could easily replace them with the very highly talented Colonels that did not make the selection cut. The military can do without any one of us but it cannot do without the great majority of our peers.         

From above you may still be unconvinced of my assumption i) that military officer performance is incredibly difficult to measure in either absolute or relative terms. If you do not believe me consider the role of advanced education in the Air Force officer promotion process. Advanced education is parlance for master’s degree, MBA, PhD etc.. . Anyone who has spent more than a few weeks in the Air Force will pick up right away on our obsession and fetish with advanced education for officers. On 2 Feb 2005, the then-Chief of Staff of the Air Force Gen. John Jumper sent all of us a policy memorandum stating that the Air Force’s emphasis on master’s degrees was out of control (I’m paraphrasing) and that for a lot of people it was a waste of time and served no other purpose than to “check-the-box” in our careers. He decided to hide education information on promotion boards up to a certain rank. This meant that it did not matter if an officer came from the Air Force Academy, the University of Texas, had a master’s degree or not. They would only be judged on their job performance and evaluations. Even though Gen Jumper’s policy was eventually reversed there were three years of promotion cycles where education data was unavailable to the promotion board. And do you know what happened in those three years?
The effect was most pronounced for the promotion boards to Lt. Col but one can see this in Major and Colonel promotions as well. As the education information became unavailable to the people deciding promotions the promotion rates for those without advanced degrees shot way up. Officer promotion rate to Lt. Col in 2001-2004 was around 12% for those with only bachelor degrees. That rate rose to nearly 50% in the years 2005-2007. Why?
Part of the reason is that officers withdrew from their online or distance learning master’s degree programs when they learned that this step was no longer considered for promotion (including yours truly). The other part of the answer is that the promotion boards had historically used advanced degrees as a discriminator to decide whom to promote among the great undifferentiated mass of officers. Without that discriminator it became much more difficult for the board to make promotion decisions and a many officers who would not have been promoted in the past were able to slip by the board.
An advanced education degree can have at least two effects. First it can increase an officer’s human capital and make them more productive than they otherwise would have been. Maybe the skills and knowledge obtained make them more productive and better officers and that helps them shine and stand out from their peers. On the other hand an advanced education degree can be useful as a signal of higher cognitive ability. Hypothetically, not everyone can complete one so regardless of its direct value to defense production it can be useful to help screen higher ability officers from lower ability ones. The problem is that in order for a signal to be useful it has to be difficult to obtain. That is not the case in the Air Force today. An entire industry has sprung up out of the ground to sell this signal and the educational value of these degrees is of dubious value. Essentially the market is saturated with advanced education degrees because they are cheap and easy to obtain giving them very little value in discriminating high ability officers from low ability officers. In the years 2005-2007, the guys who slipped through the promotion system and into a higher rank without advanced education degrees essentially proved the point that officer performance does not change with an advanced degree. The non-advanced degree officers should have had lower productivity than the guys with advanced degrees. Email me and I’ll send the data I have. I asked the Air Force for all the micro level data to study this but needless to say they were not interested in any results of such an investigation and slammed the door in my face. I think they knew the answer.
Coming full circle, this is all to say that officer promotions is hard, very hard. It is tough to distinguish between all the officers in a peer group because we cannot accurately measure their production of national defense nor can we do a great job of comparing people between or within career fields. The Air Force’s poorly placed emphasis on advanced education as a promotion discriminator is a great example of this.
Where Kane is on the mark is with our assignment systems. On this point I agree that more flexibility and less centralized control of our internal labor market would yield some big gains in retention and career satisfaction. If you haven’t yet read the publication that Kane references from the Strategic Studies Institute you should. There are a series of papers from West Point economists that advocate for a change in policy regarding talent development and retention. One regarding retention is here ( I highly recommend to all Field Grade Officers to read the entire series. We may be in the Army’s shoes one day.
Surprisingly Kane does not mention a game changing strategy implemented by the Army that far exceeds anybody’s expectation and in my mind blows out of the water the notion that the Army cannot innovate. Economists and policy makers from the Office of Economic and Manpower Analysis (OEMA) at West Point designed a system of incentives to offer ROTC and USMA cadets. These incentives fall under the general title of Officer Career Satisfaction Program and the link above gives more details. In short, cadets can exchange three more years of service on top of the baseline commitment in order to obtain one of three incentives. The incentives are either fully funded graduate school, base of choice and career specialty of choice. This creates a market for cadets to obtain what they really desire and pushes their active duty commitment to eight years(seven for most ROTC cadets). Something like 40% of the eligible cadets chose one of the three options and the Army has locked them down for eight years without spending a dime. The graduate school does not take place until after the extra three years of service have been paid. And after grad school the officer pays back six years of active duty service. The Army pays nothing until the extra commitment is complete and then reaps more years of service from those who attend grad school.
That’s right, without tossing lots of taxpayer money at officers the Army ceded a little bit of control over assignments and personnel decisions and the results have been through the roof. What the military has not grasped is that yes I would like another 10,000 dollars in my yearly salary but we are already generally well compensated. It is not about the money. What I cannot buy is the assignment I want, the location I want, or the educational opportunity I want. Previous to the implementation of this program there was no market or way to take ownership of one’s career. The Air Force could learn a lot from this program and get ahead of the talent recruitment competition.
Coming back around to Kane’s article, I think he substitutes a lot of biased opinions for facts and solutions. This is particularly disappointing when he has more concrete evidence available but either does not know about it or does not think it is worthwhile. I believe that the Army does have a serious retention problem but as far as we can tell it is among the junior officer corps. Our sister service should be applauded for its steps towards making the labor market more flexible and we should not wait until we have fill rates of 75% to do the same.

Asch, Beth J. and Warner, John T, A Theory of Compensation and Personnel Policy in Hierarchical Organizations with Application to the United States Military, Journal of Labor Economics, Vol. 19, No. 3 (July, 2001), pp. 523-562
Government Accountability Office, Military Personnel Strategic Plan Needed to Address Army’s Emerging Officer Accession and Retention Challenges, GAO-07-224 (January 2007)
Henning, Charles A., Army Officer Shortages: Background and Issues for Congress, Congressional Research Service (July 2006), Order Code RL33518
Lazear Edward P. and Shaw, Kathryn, Personnel Economics: The Economist’s View of Human Resources, Journal of Economic Perspectives, Vol. 21, No. 4 (Fall, 1997), pp.91-114
Wardynski, Casey, Lyle, David S., Colarusso, Michael J., Towards a U.S. Army Officer Corps Strategy For Success: Retaining Talent, Strategic Studies Institute (Jan. 2010), ISBN 1-58487-425-2


  1. Good work Tobe. Appreciate your critique and insight into what continues to be a thorny question -- how do the Services craft promotion systems that balance the need for volume with the expectation of meritocracy? I will carve out time to read the SSI series you reference. I wonder if you have read T.X. Hammes critique of the evaluation system? He devotes a chapter to it in The Sling and the Stone (chp 15, Where to from Here?). His assertion is the current career pyramid is designed to develop generalists instead of specialists. This is a problem in Hamme's estimation not because the military has no use for officers with broad experience, rather, because there exists a one-size-fits-all system where specialists are not permitted to flourish within their area of expertise. I found his take particularly interesting in light of what I saw in Tampa -- officers with regional expertise being wrested out of their niche in the name of staying competitive by taking on a "career broadening" tour, often against their will. I submit we would be better served by allowing those with an interest in specialization to remain competitive for promotion by allowing their expertise to serve as a discriminator equal in merit to those of the generalist (advanced degree, joint tour, exec time, etc.) vice equating limited scope of service (the dreaded stove pipe) to limited value.


  2. Believe it or not, I actually did read this blog! I have very little contact with the military, other than the Washington Special Olympics Summer Games, held at Joint Base Lewis McChord. I'm always impressed with the caliber of 90% of the officers I deal with.
    In another large bureaucracy that I AM familiar with, Boeing, it always seemed to me that upper level promotions were based less on a person's qualifications and more on who you know and other political factors. It's also very difficult to assess this level of mgmt's performance. They are too far removed from the "nuts and bolts" and are judged on "metrics" such as budget (most importantly), schedule performance of the organization and subjective things like group morale. I'm of the opinion that Boeing could indeed do without one or most of the middle/upper managers. Their structure has way too many levels of mgmt.

  3. From distinguished Georgetown University scholar and Army FAO Maj Keith Weidner...

    My only point of contention goes back to the advanced degree arguement. I don't agree that by removing that requirement you are attaining the same net effect. Yes you will promote the same amount of officers (roughly) from year to year because the services have quotas to fill. But in essence you have only lowered the standard.
    I do concede the fact that todays culture of degree aquisition has made the act less competitive. But if an advance degree accomplishes what you say it should, which is improves "human capital","productivity", and "cognitive skills" one has to determine whether or not these are desired skills of our senior officer. If your point is to say our Generals do not need to demonstrate these abilities then "yes" the promotion boards should not take them in to consideration. But I dont think thats your intention.
    I could make the same arguement by removing APFT (Physical Fitness Test) data from officer evaluations too. The end result would likely show similar results. Overall promotion numbers would be similar to years past with a surprising number of PT failures now being promoted. The consequences though would be apparent to every soldier from E-1 to O-10. The standard and overall physical fitness of a service would decline.
    I think the service (or any institution for that matter) needs to determine what the desired skills and traits should be for advancing its labor force. Yes - it does lead to "block check" approaches to our metrics. But whether I am passing a PT test or passing Graduate School, I would hope in the end its because these skills and abilities are a necessary asset for leaders in my instititution.

  4. Serving as an enlisted airman it was once relayed to me by my Flight Superintendent that I would never be promoted past E-6 without a bachelors degree. That degree could be in any specialty ranging from Engineering to Under Water Basket Weaving. For enlisted personnel that are looking for a career with good benefits, job security, and possibility for advancement - all without a degree - this is very disconcerting. I was one of those who had strong feelings against this mindset; however, I chose to circumnavigate this requirement for advancement and I am pursuing a degree in the hopes of entering the officer ranks. I say all this because it was clear to me that a bachelors degree was being used as a discriminator for enlisted personnel. This only makes since in today's military as many with bachelors degrees (or higher) are scattered through the enlisted ranks. Of course there are reasons for this - economic hardships, lack of civilian job opportunities, or even incentives to pay off student loans. Be that as it may, the amount of degree holding enlisted personnel has certainly raised the bar for upper-level promotion.
    Since a bachelor's degree is required for commissioned officers, the use of post-baccalaureate accomplishments for promotion is reasonable measure.

    Speaking specifically in regards to officer retention I can only speak from personal observations while serving with many exceptional officers. Simply put, the best officers stay because the very best officers have a commitment to their service. The pessimists may call this commitment misplaced idealism, complacency in their job security, or whatever they wish. But the best officers stay because they put greater emphasis on service and a dedication to their men rather than what incentives that they receive. Now, perhaps, you can call me the one with misplaced idealism. True, at some point the officer who has no control over their career - as Tobe related - must assume some degree of self-preservation. Be that as it may, I still stand by my assertion that the best officers stay regardless of career incentives. Whether I thought my officers' job performance was exceptional or less than exceptional (ARB) I will still stand by that assertion.

  5. Great column, and excellent comments, gents. I only have a little to add, but I'll add nonetheless. First, it's clear to me (as someone who falls into the HANDGRIPS - RAISE category) that the results of the survey Kane cites are so heavy with selection bias that they generate their own heat. Tobe, you do a fantastic job of taking the survey and his use of it to task, so I don't need to say any more.

    As for the evaluation system itself, Scott's idea of setting a goal not to punish specialists is a worthy one. In my opinion, it serves to highlight a larger problem with the service itself: the Air Force is a technical service, which requires a significant amount of technical training in all jobs. At the same time, however, it's a human service that requires leaders, and leadership is something that can neither be trained nor specialized in. An officer corps that can successfully cover both categories is inherently difficult to produce.

    It seems to me that this problem leads to an attempt by the service at the shotgun approach of training all officers for both generalist leadership and for technical specialty. In my opinion, trying to solve both problems solves neither. The system can neither select for leaders nor for specialists, nor does it train either as well as it could.

    This isn't helped, of course, by the mamby-pamby rating system, where "Integrity First" takes a holiday, and everyone has saved the world at least a half-dozen times in the last year. (Although now that I think about it, the cultural insistence that everyone must be capable of anything until they're not probably doesn't help this).

    In the end, I think Scott's onto something in allowing for specialization as well as generalization. It would encourage both self-selection as well as honest rating (and therefore chain of command selection) by not discouraging either career path.

    Of course, the cultural inertia behind certain things being "good" for one's career will be difficult to stop. Here's my thought on how to fix that: reduce PME for the specialists, expand it for the leadership track, and not only unmask advanced degrees, but unmask the source and result of the advanced degrees. An MBA from Box-Check U might, well, check a box, but a Public Policy degree from a real university might be more useful for a given career path.

    I don't know. It's my first thoughts on the subject. So much for short.

  6. A repost from Thomas Ricks, author of Fiasco and The Gamble, of a few officers reply to the Kane article.