Saturday, January 22, 2011

The Sky Is Not Falling

The Sky Is Not Falling
A Response To “Why Our Best Officers Are Leaving" by Tim Kane
       
Some ideas never seem to fall out of fashion. One of these is the notion that our “best” officers leave the military service at extraordinarily high rates and the effects of said exodus are monumental. Tim Kane’s article which was recently published in The Atlantic tosses another log on the fire and stokes the flame of what passes for conventional wisdom. I think it is important to understand the facts from the anecdotes. A little bit of economic theory will go a long way in understanding why the military promotion system evolved into what it is today. A lot of what Kane wrote tangentially hits on some important themes but is off the mark in other ways. First, let me critique the actual article and then come back around to offer a different explanation of the world and why it is so.
To begin, let us not put too much faith into a poorly done “survey” of 250 current and former Army officers. Opinions are an impression of reality not reality in and of itself. I don’t mean to dismiss or discount what people believe but cognitive biases and cascading can lead us all over the place. Lets reread carefully what Kane wrote about how he conducted his survey.
In a recent survey I conducted of 250 West Point graduates (sent to the classes of 1989, 1991, 1995, 2000, 2001, and 2004), an astonishing 93 percent believed that half or more of “the best officers leave the military early rather than serving a full career.” By design, I left the definitions of best and early up to the respondents. I conducted the survey from late August to mid-September, reaching graduates through their class scribes (who manage e-mail lists for periodic newsletters). This ensured that the sample included veterans as well as active-duty officers.    
First and foremost, this is an article about Army officers and Army promotions. We should be careful not to extrapolate Kane’s conclusions into the other services. This tells us nothing about what Air Force officers believe about their promotion systems.
Secondly, separating this paragraph from the rest of the article it is abundantly clear that Kane’s survey doesn’t hold up to any rigorous scientific standards. How well do the respondents represent their cohort populations? What was the response rate to his survey? How many people did not respond to his survey? Was there any systematic self-selection bias as to who did or did not answer his survey? What I’m getting at is if Kane emailed this survey to six different West Point graduating classes, that is potentially  around 6,000 people. If only 250 responded that is a take rate of about 4%. If you could think of at least one reason that some people answered his email and others did not then we have a biased sample. If the 250 respondents represent a random sampling of the pool of candidates then there is no problem. However if higher proportions of officers with negative or pessimistic views about the Army answered the survey then we are left with nothing useful.
Thirdly, Kane only interviews West Point graduates…why not Army ROTC and OCS commissioned graduates? Might they see the world differently than their USMA peers? West Point produces roughly one-third of the required commissioned officers every year. Again the sample is likely to be biased and only tells us what a minority of Army officers think about their service promotion system.
Finally, leaving the terms “early” and “best” up to the respondent seems innocuous enough but its highly problematic to do that. What if the 250 officers had in their mind that leaving the service early was equivalent to less than thirty years of service? Then we would get the exact same response if all the best officers served twenty five years but left before thirty. Are we really in dire straits if the best officers serve at least twenty five years but stop short of thirty? Most likely that is not what the respondents had in mind but the point is that we have no idea what they had in mind when they answered.
If boredom has not caused you to stop reading this blog then I congratulate you and ask you to do one more thing. Reread Kane’s article but skip over or black out any paragraph or sentence that draws upon his survey results. Stripping away all the thoughts and feelings of what is likely a non-representative set of Army officers we can get to the core of the issue. The US Military has a very unique set of parameters to deal with regarding personnel that companies like Intel, Google and IBM do not, and it optimizes accordingly.
Let me start at the beginning with some basic labor economics. In classic labor theory, wages are based on productivity. A worker and boss agree upon a price for each widget or unit of production. If the worker produces more he makes more salary. More productive workers will earn higher salaries but everyone gets paid the same amount per unit of production. That sounds great and simple so why are the best officers not paid more than their less productive peers? Wouldn’t that keep them in the service longer and more satisfied?
Unfortunately, this  theory does not explain wages and salary schemes when we cannot define a unit of production. The US Military produces national defense, we know that, but we also have no way of measuring aggregate national defense production nor any one individual’s contribution to the overall sum. How many units of national defense does a particular Quartermaster officer produce and how does that compare to an Infantry officer? Or to an F-16 pilot? We have no way to know these things. Now, the military does signal the relative importance of certain career specialties through the use of incentive pay. Pilots, Navigators, Submarine officers, and so forth receive somewhat more compensation than their peers but there is no differentiation within these careers. The person considered to be the finest F-15 driver is paid exactly the same as the guy that nobody wants to fly with(controlling for service time and rank).
Even within a specific career specialty it is an incredibly grey issue as to who is more productive than who. It may be clear to everyone in the 101st Airborne Division who is the best platoon leader but how can I objectively demonstrate that? We could use measurements like enemy soldiers killed or insurgents captured but those create perverse incentives. Do we really want to incentivize our infantrymen to pump up their body count numbers or to be less discriminate about who they pickup off of the battlefield?
I suggest a great paper by John T. Warner and Beth J. Asch with a long-ish title “A Theory of Compensation and Personnel Policy in Hierarchical Organizations with Application to the United States Military”. It is on JSTOR and if you do not have access I can email it to you. They hit the nail on the head that real source of all our problems is the idiosyncrasy that there is no lateral entry into the service. That is to say, everyone who enters (except for some specialties like medicine or law) must do so at the bottom of the ladder. All officers start out as Second Lieutenants and it matters not if you are entering with an MBA from Harvard and 8 years of management and operational experience or are twenty-two years old and straight out of ROTC. We can debate the efficacy of this policy on another day but taking it as given one can see right away the Army’s problem.

The problem is this i) a military officer’s performance and production is almost unobservable and unquantifiable and ii) there is no lateral entry into the ranks. What does this mean? It means that the military cannot afford to enact policies that favor the “best” and risk alienating the large mass of officers. To do so would be suicidal. The military is highly dependent on the cohort of officers at every level to shoulder the burden of leadership. Numbers matter a lot! If large numbers of lower and mid-rated officers leave because they are not the “chosen” ones or are not favored then the Army or military cannot go into the market to recruit more Captains, Majors or Lieutenant Colonels. In reverse, IBM and Microsoft can immediately weed out their weaker performers and go recruit people at any level of the organization. So for Kane’s example of Lt. Col Nagle it is too bad that he got out of the service but a promotion policy that benefits him would most likely come at the expense of the vast middle. 

Gen Colin Powell wrote in his autobiography that when he attained his first star and became a Brigadier General all the newly promoted were gathered into a room and addressed by a higher ranking general. This person told them that for as special as they were, not to let it go to their heads. If a plane crash wiped them all out tomorrow the Army could easily replace them with the very highly talented Colonels that did not make the selection cut. The military can do without any one of us but it cannot do without the great majority of our peers.         

From above you may still be unconvinced of my assumption i) that military officer performance is incredibly difficult to measure in either absolute or relative terms. If you do not believe me consider the role of advanced education in the Air Force officer promotion process. Advanced education is parlance for master’s degree, MBA, PhD etc.. . Anyone who has spent more than a few weeks in the Air Force will pick up right away on our obsession and fetish with advanced education for officers. On 2 Feb 2005, the then-Chief of Staff of the Air Force Gen. John Jumper sent all of us a policy memorandum stating that the Air Force’s emphasis on master’s degrees was out of control (I’m paraphrasing) and that for a lot of people it was a waste of time and served no other purpose than to “check-the-box” in our careers. He decided to hide education information on promotion boards up to a certain rank. This meant that it did not matter if an officer came from the Air Force Academy, the University of Texas, had a master’s degree or not. They would only be judged on their job performance and evaluations. Even though Gen Jumper’s policy was eventually reversed there were three years of promotion cycles where education data was unavailable to the promotion board. And do you know what happened in those three years?
The effect was most pronounced for the promotion boards to Lt. Col but one can see this in Major and Colonel promotions as well. As the education information became unavailable to the people deciding promotions the promotion rates for those without advanced degrees shot way up. Officer promotion rate to Lt. Col in 2001-2004 was around 12% for those with only bachelor degrees. That rate rose to nearly 50% in the years 2005-2007. Why?
Part of the reason is that officers withdrew from their online or distance learning master’s degree programs when they learned that this step was no longer considered for promotion (including yours truly). The other part of the answer is that the promotion boards had historically used advanced degrees as a discriminator to decide whom to promote among the great undifferentiated mass of officers. Without that discriminator it became much more difficult for the board to make promotion decisions and a many officers who would not have been promoted in the past were able to slip by the board.
An advanced education degree can have at least two effects. First it can increase an officer’s human capital and make them more productive than they otherwise would have been. Maybe the skills and knowledge obtained make them more productive and better officers and that helps them shine and stand out from their peers. On the other hand an advanced education degree can be useful as a signal of higher cognitive ability. Hypothetically, not everyone can complete one so regardless of its direct value to defense production it can be useful to help screen higher ability officers from lower ability ones. The problem is that in order for a signal to be useful it has to be difficult to obtain. That is not the case in the Air Force today. An entire industry has sprung up out of the ground to sell this signal and the educational value of these degrees is of dubious value. Essentially the market is saturated with advanced education degrees because they are cheap and easy to obtain giving them very little value in discriminating high ability officers from low ability officers. In the years 2005-2007, the guys who slipped through the promotion system and into a higher rank without advanced education degrees essentially proved the point that officer performance does not change with an advanced degree. The non-advanced degree officers should have had lower productivity than the guys with advanced degrees. Email me and I’ll send the data I have. I asked the Air Force for all the micro level data to study this but needless to say they were not interested in any results of such an investigation and slammed the door in my face. I think they knew the answer.
Coming full circle, this is all to say that officer promotions is hard, very hard. It is tough to distinguish between all the officers in a peer group because we cannot accurately measure their production of national defense nor can we do a great job of comparing people between or within career fields. The Air Force’s poorly placed emphasis on advanced education as a promotion discriminator is a great example of this.
Where Kane is on the mark is with our assignment systems. On this point I agree that more flexibility and less centralized control of our internal labor market would yield some big gains in retention and career satisfaction. If you haven’t yet read the publication that Kane references from the Strategic Studies Institute you should. There are a series of papers from West Point economists that advocate for a change in policy regarding talent development and retention. One regarding retention is here (http://www.strategicstudiesinstitute.army.mil/pubs/display.cfm?pubID=965). I highly recommend to all Field Grade Officers to read the entire series. We may be in the Army’s shoes one day.
Surprisingly Kane does not mention a game changing strategy implemented by the Army that far exceeds anybody’s expectation and in my mind blows out of the water the notion that the Army cannot innovate. Economists and policy makers from the Office of Economic and Manpower Analysis (OEMA) at West Point designed a system of incentives to offer ROTC and USMA cadets. These incentives fall under the general title of Officer Career Satisfaction Program and the link above gives more details. In short, cadets can exchange three more years of service on top of the baseline commitment in order to obtain one of three incentives. The incentives are either fully funded graduate school, base of choice and career specialty of choice. This creates a market for cadets to obtain what they really desire and pushes their active duty commitment to eight years(seven for most ROTC cadets). Something like 40% of the eligible cadets chose one of the three options and the Army has locked them down for eight years without spending a dime. The graduate school does not take place until after the extra three years of service have been paid. And after grad school the officer pays back six years of active duty service. The Army pays nothing until the extra commitment is complete and then reaps more years of service from those who attend grad school.
That’s right, without tossing lots of taxpayer money at officers the Army ceded a little bit of control over assignments and personnel decisions and the results have been through the roof. What the military has not grasped is that yes I would like another 10,000 dollars in my yearly salary but we are already generally well compensated. It is not about the money. What I cannot buy is the assignment I want, the location I want, or the educational opportunity I want. Previous to the implementation of this program there was no market or way to take ownership of one’s career. The Air Force could learn a lot from this program and get ahead of the talent recruitment competition.
Coming back around to Kane’s article, I think he substitutes a lot of biased opinions for facts and solutions. This is particularly disappointing when he has more concrete evidence available but either does not know about it or does not think it is worthwhile. I believe that the Army does have a serious retention problem but as far as we can tell it is among the junior officer corps. Our sister service should be applauded for its steps towards making the labor market more flexible and we should not wait until we have fill rates of 75% to do the same.

Asch, Beth J. and Warner, John T, A Theory of Compensation and Personnel Policy in Hierarchical Organizations with Application to the United States Military, Journal of Labor Economics, Vol. 19, No. 3 (July, 2001), pp. 523-562
Government Accountability Office, Military Personnel Strategic Plan Needed to Address Army’s Emerging Officer Accession and Retention Challenges, GAO-07-224 (January 2007)
Henning, Charles A., Army Officer Shortages: Background and Issues for Congress, Congressional Research Service (July 2006), Order Code RL33518
Lazear Edward P. and Shaw, Kathryn, Personnel Economics: The Economist’s View of Human Resources, Journal of Economic Perspectives, Vol. 21, No. 4 (Fall, 1997), pp.91-114
Wardynski, Casey, Lyle, David S., Colarusso, Michael J., Towards a U.S. Army Officer Corps Strategy For Success: Retaining Talent, Strategic Studies Institute (Jan. 2010), ISBN 1-58487-425-2