Deliver Us From Bean Counters

Background:
I’ve been ‘taking to task’ a hyperventilating commenter over at DoD Buzz who goes by the handle ‘Engineer_Economist’. He (I am assuming he is a ‘he’) is in a fully formed outrage over what he claims is the completely failed and doomed F-35 program. He bases his claims on what I see as  superficial knowledge of what might be the actual status of the program and what IMHO publically mimics a school-house awareness of the acquisition process (see for yourself here). It is entirely possible that he has worked one or more 'smallish' programs (as compared to the F-35), but doubtfully anywhere near a Program Manager level. This experience seems to have reinforced a dogmatic, and incorrect belief that the acquisition process must be implemented according to strict interpretation of a very rigid paradigm, and if not then there is a certainty that chaos and failure WILL result. When and if he someday runs a program, he will find there are many ways to ‘skin a cat’ within the bounds of the acquisition process. When and if he someday gets involved in a program with major political ramifications, then maybe he’ll see where real program chaos comes from.

In the linked-to comment thread, I gave him a challenge (my typos corrected):

I haven't avoided your arguments at all, just cast scorn on them as hybris-ridden rantings of a program manager wannabe. If you can't Grok that the objectives and goals of the processes are more important than the processes themselves, that we have rules and processes to foster success and order--not to DEFINE success and order, that is not my problem. Let me flip your arguments back on you: tell us EXACTLY WHY what is so overarchingly important to you, should be just as important to me, and EXACTLY WHAT will be the consequences if your worst fears come true if no one listens to you....and I will post a complete ‘fisking’ of it at my place.

'Engineer_Economist’s' response:

yes you have avoided the arguments and you are still avoiding them. you have faith in and are selling us on the unit cost estimates when development is far from complete. again, I have to explain to you, the F-35's latest SAR shows MS B as being rescinded, and all the dates for IOT&E, IOC, MS C, FOC, are all TBD. That means there is no schedule. They canceled the DAB that was supposed to determine these dates. They can't even hold the DAB the program is so screwed. If you'd pay attention I wouldn't have to REPEAT to you over and over again why this matters. F-35 continues the repeated failing pattern of late, overbudget, and underperforming programs. If we continue to reward such programs, the problem will just get worse and worse. We will fail to properly recapitalize our defense system portfolios, forcing us to operate legacy systems longer & longer and greater risk. If I just say "DMS" are you able to make the connection or am I going to have to explain THAT to you as well?

And now, I fulfill my promise
The only 'splaining' you've done is to yourself –and it is more appropriately called ‘rationalization’. Simply declaring you are ‘right’ about something without supporting facts or claiming doom without explaining how and why “B” will follow “A”. Stating what form it will take and assertion of doom with certainty) does not explain much less prove diddly-squat. The irony here is that if you had managed to judiciously insert a few ‘perhaps’ , ‘mights’, or ‘increased chances of’, I don’t think hardly anyone would have have had a problem with your comments. But instead you pretty much only insist repeatedly that the 'formalities must be observed' and declare that because they’re not occurring according to a script (as you believe they should be), then an epic program failure is happening all around us. And all of your outrage is based upon ... made-up and poorly understood (public) cost estimates (and expectations) as well as your frail outsider understanding of actual schedule drivers and impacts to-date, and all with no idea of the nature of program or what risks are behind it or what the risks are that sit ahead. The program does. DoD does. Political posturing aside, I believe even enough of Congress does. You don't, and are in a sophomoric snit about it.

Let the Fisking begin!
Your argument is sufficiently vague to ALMOST evade ‘fisking’, but not vague enough.
We’ll take your key ‘points’ in order.

>RE: ” …the F-35's latest SAR shows MS B as being rescinded, and all the dates for IOT&E, IOC, MS C, FOC, are all TBD. That means there is no schedule. They canceled the DAB that was supposed to determine these dates”

Such hyperbolic breathlessness! Schedules and progress don’t magically disappear based upon a declaration of a Milestone do-over or reprogramming, so rescinding MS B doesn’t mean there is no schedule or budget baseline. It DOES mean that the program is still working to a previous baseline while it is developing updated schedules and budget forecasts for a new baseline. 

>RE: “They can't even hold the DAB the program is so screwed…”.

No. It means that because the F-35 is a VERY large and complex program, nothing is easy - even bean counting. In fact, if you read around a bit, I believe you will find that the biggest driver in delaying the DAB is the reconciliation of the so-called ‘independent’ CAPE and program cost estimates. The PEO wants to get things right, and ensure as much actual program data is available so the DAB is now slated for sometime in the Fall. Good on him.

"It was decided: why don't we let a little bit more of the performance of the program both in test and production play out through the summer; why don't we let that integrated master schedule get finished, do a schedule risk assessment of it, present that to service leadership, let them ponder IOC [initial operating capability], let the operational test planning complete," said JSF program executive officer Vice Adm. David Venlet. "Then, rather than set a baseline now with a whole bunch of go-finish-your-homework assignments, we will go finish the homework and then present the new baseline for [Pentagon procurement chief] Dr. [Ashton] Carter's approval in the fall of this year."
….The delay is a matter of being thorough, not an indication of new problems, Venlet said.
"That is not a sign of alarm. It is, I think, a determination to continue in a deliberate fashion with good solid fundamentals applied to get things done," he said.

Thus we find the JSF program is evidently not “so screwed” as the ranting of a PM ‘wannabe’ might lead us to believe and insists MUST be true in spite of evidence otherwise.  

>RE: “…. If we continue to reward such programs, the problem will just get worse and worse. We will fail to properly recapitalize our defense system portfolios, forcing us to operate legacy systems longer & longer and greater risk.”

I think we may assume that you are using the term “reward” metaphorically for deciding to continue pursuit of a program instead of a literal and illogical ‘reward/punish’ paradigm. Mission need and the cost of meeting that need versus the risk of not being capable of meeting that mission need is all that matters in determining the viability of a program. To-date, the F-35 program is seen as executable by those responsible, and the mission need remains in the eyes of the users. As I’ve noted before, outside observers on the sideline don’t know diddly, never have – never will.

On CAPE estimates and comparisons for casual readers dropping in:
My ‘opponent’ gives much credence to external estimations (that employ varying assumptions, (many of which are irrelevant) that predict outlandish costs In the out-years, yet completely discounted the fact that LRIP 1-3 deliveries have beaten internal program cost estimates to-date and are trending downward. Like many, he chose to believe external numbers. I choose to believe my ‘own lying eyes’ and the information in front of them. [BTW: I suspect these same internal estimates are now showing adverse impacts to earlier LRIP 4 and 5 due to constantly imposed program ‘tweaks’, but I also predict they will STILL be significantly lower than the farcical external estimate methodologies, including CAPE’s.]
Since reconciling the CAPE estimates with actual costs to-date and internal cost projections is a sticking point that will remain with us for some time, I cannot pass up this opportunity to highlight some known aspects of the CAPE estimate -- and how much ‘relevance’ the government really (vs. publically) places on CAPE estimations… and we need not look any further than how the LRIP 4 negotiations were conducted [emphasis mine]:

Lockheed Martin officials take issue with the Pentagon’s APUC estimates, which come from the cost analysis and program evaluation (CAPE) office. “The CAPE model uses production costs derived from the F-18 and F-22 programs and extrapolates them out to 2037 using only a 50-percent confidence level,” Steve O’Bryan, vice president for F-35 business development, told AIN. “That model takes no account of our plans for lean production, nor our much lower supplier costs. For instance, we can buy an F-35 radar for half the cost of an F-22 radar. We can make the same order of savings on the electronic warfare system and other avionics, including the single-piece plasma screen in the cockpit.”
Moreover, he added, the APUC figure includes military construction necessary to support the F-35 and a portion of the support costs. The F-35 comes complete with radar, avionics, defensive systems and so forth, as well as weapons pylons, he noted.
“While the U.S. government uses the high independent estimates for program budgeting purposes, it holds the contractor to an entirely different standard in contracting for the actual aircraft,” another Lockheed Martin spokesman told AIN. Negotiations for LRIP Lot 4 (low rate of initial production) have been under way in the weeks leading up to this week’s Farnborough show. The government’s offer was 40 percent lower than its own CAPE estimate for this lot, while Lockheed Martin’s opening bid was 20 percent lower than the same estimate. “We’ll settle somewhere in the middle of that range,” the spokesman added.

And so a LRIP 4 production deal came to pass. [As a side note, I find it interesting how many of the major aerospace websites covered the same story at the time. They somehow seemed to have ‘missed’ some of the more ‘affirmative’ details concerning the F-35. (Just saying)]
The above quote also highlights the pivotal bookkeeping hook that ‘some’ are using to draw inappropriate (since the end purpose appears to be nefarious -- might I say “unfair”?) comparisons. Whereas F-16/F-18/AV-8 costs tossed around are essentially the URF [my note: ‘Unit Recurring Flyaway’ cost], we note that the F-35 numbers tossed around  by critics these days are estimated  “APUC”[my note: Average Procurement Unit Cost]. This is not trivial, so an example is in order:

Mod programs to bring early production aircraft up to final production standard have typically been booked against the APUC and are thus missing from F-18/F-16 and AV-8 URF numbers, But these costs ARE included in what is being thrown around concerning the F-35. These high jinks are not unprecedented: Pushing the APUC costs in the public mind has been an increasingly common tactic of incumbent programs (not just by contractors – do NOT underestimate entrenched DoD constituencies):

Comparisons between the acquisition costs of the F/A-18E/F and an attack capable version of the F-14D, such as the F-14D Quick Strike, have been highly conjectural because of conflicting estimates of development and production costs, and projected inventory requirements . In mid-1991, Department of Defense (DOD) officials estimated their flyaway unit costs in FY1990 dollars as $33 million for the F/A-18E/F vs. $44.5 million for the F-14D Quick Strike . F-14 supporters noted, however, that flyaway costs do not include development costs, which would be several billion dollars for the F/A-18E/F vs. several hundred million for the F-14D Quick Strike . They noted further that these flyaway costs assumed annual buys of 72 F/A-18s vs. 24 F-14s, and they argued that comparable 72-plane buys would reduce the cost of each F-14D Quick Strike to $34 .5 million, approaching the cost of an F/A-18E/F. “ (p.10)

When evaluating mercantile claims critical of ongoing programs, it is best to always consider “Cui bono?” While the incumbent contractor’s ties to their commercial interests are obvious, it still takes a certain willful ignorance to pretend those assailing an ongoing program are not serving what they perceive to be their own self-interests.

Additional Notes on the Use and Abuse of Cost Numbers
In a rational world, we could leave the question of whether or not the JSF program is progressing, is ‘worth it’ and has the majority of its ‘risk’ behind it with an answer of definitely “YES” . We can also answer a similar question as to if the JSF program still needs to be managed carefully to ensure the plans and program are successfully implemented in the years to come with an equally emphatic ‘YES” (I am aware of no one who disputes the second point) . But I think given the disconnect between the hysteria generated by JSF-detractors in their public-relations war against the F-35, and actual program status, a couple more points should also be made.

1. SDD costs are running higher than first projected. That anyone would think stretching the SDD to reduce program and production risk would not have adverse cost impacts speaks more about those who think such lunacy than the program itself. Squawking about increased SDD costs as if they were some unexpected anomaly without acknowledging the role that decisions made to ‘buy down’ risks had in the cost increase is disingenuous. I would blame decisions made solely by Lockheed Martin for exactly one year’s worth of schedule delays to-date, and that was because their plan was to fully staff the program faster than any program had been staffed before, but did not change any of their staffing processes or ground rules: you can’t get different results unless you do things differently. Ironically, this delay is about the only one that we can point to that contributed relatively little to the SDD cost growth: if you don’t hire the people to begin with, it doesn’t cost the program for people they don’t have to not do the work. Conversely, if you stretch your plan after the people are on board, it costs big $ to have a capability run in place. Conversely, if you elect to cut deliverables after your program is staffed up and running, you are burning dollars while running in place. I’d like to know what the costs of cutting the LRIP batches are. This would include not only the costs of LRIP production batches, but also the costs added by suppressing the learning curve needed to achieve full production rates. There’s also a PhD thesis ripe for the picking for some industrious candidate in this problem: Develop a methodology to trade off the costs of early production units having to be modified to later baseline configurations against the increased costs of production units with inhibited learning curves. The trick is to make the methodology transparent enough for innumerate policy makers to understand. Good Luck.

2. Given that the F-35 program is replacing (from the US point-of-view) 4 existing aircraft weapon systems, shouldn’t the SDD cost of replacing more than one weapon system (at least two?) be the standard by which the F-35 SDD program costs and schedules are declared “too high”? (some other thoughts on ‘perspective’ and other factors here) I submit that the SDD cost increase would have to be significantly greater than the current projected bill before the JSF SDD costs became anything less than a ‘tremendous bargain’.

About the worst that could ever happen at this point is for the program to ‘break even’ and that is only if some VERY major technological hurdle pops up that no one yet sees. I think the chances of such a big ‘uh-oh’ moment are highly unlikely this late in the program. (This not to say the weak sisters among us don’t try to manufacture a crisis every time a little hiccup comes along)

In closing, it must be mentioned that at the root of all the confusion, claim and counterclaim concerning costs, is the fact remains that no one has ever presented a complete line by line comparison and analysis of the cost numbers, or the legitimacy of their underlying rationale, being thrown about.
Nor is there a reason to – the decision makers have the numbers.