The gist of the last two paragraph in your post and the quote in the end is that professional evaluation and certification organizations, like OT&E in US and CEMILAC & ASTE in India, are doing meaningless excessive testing and just putting a a squadron through the paces is good enough to decide whether all capabilities are in place, and especially all safety parameters are met.
I never implied that and neither does the pilot quoted below. Safety and related testing is separate from Operational test and evaluation, and the gripe is with DOTE not with OT&E. Moreover the relationship and scope of responsibility is actually defined in law and procedures within the US. The services decide IOC, and FOC criteria, and then their own evaluation determines whether those have been met, exceeded, are un-met and if so what to do about it. The DOTE does not decide that and does not have the power to influence their decisions. The DOTE does not field a set of Subject matter experts required to do end to end testing but merely relies on the services to provide all that has its own bureaucracy to support as required. Needless to say they too much like the services can get things grossly wrong as they did quite recently when they lobbied hard (through document leaks, official testimony and through the regular media channels that get such information leaked) to get $50 or so million to conduct early trials on a particular Navy system earlier than had been planned by the Navy. The DOTE personally testified that he would be there in person to witness them and then chickened out at the last minute..the results of those DOTE monitored trials ran contrary to what the DOTE had argued would happen.
Where the DOTE has influence is not in meddling with the service's but with advising the Congress on milestone decisions. As per law, unless a waiver is granted the DOTE needs to sign off on on Milestone-C decisions which take a weapons system from Low Rate production to full rate production. Generally, to do so they need to conduct full operational testing (and keep in mind that OTE is just one of the few operational assesments that would have been conducted on any system - It would by all means not be the ONLY ONE) which for this program involved the JPO converting the entire DT fleet at Edwards into operationally representative 3F aircraft, and over and above that also providing one complete squadron worth of aircraft and assets to the OTE. Of course they will do so as is the law but what has ticked off Gilmore is that it will come in late. There is a perfect explanation for this - The JPO was confronted with two choices -
- Ask for more money to build up the development-test fleet in order to simultaneously have development aircraft to support 3I and 3F development, while also having operationally representative aircraft ready for OTE in time.
- Delay DT to OT conversion in order to avoid massive development slippage in terms of capability.
They chose option 2 because their first priority is to complete development with the shortest delay possible so as to minimize any disruption to individual service's bed down plans. They'll still give Gilmore his jets, just a year or so late. Congress could have stepped in, and just increased program funding to negate this delay but they did not because in the global context a year delay in OTE has really no impact on the actual process of induction of the aircraft unless you have a different goal than to complete development and field capability to the operator's satisfaction. What is also likely to happen (my guess) is that the Congress grants the program a milestone C waiver and it enters full rate production without OTE, with OTE following a year or so later. There are potentially billions of added costs if you add another LRIP block beyond the 11 or so planned as opposed to moving to the first FRP.
One can go down the list of the IOC criteria and even the DOTE couldn't conclusively prove that any mandated set has remained unaddressed or unmet prior to the Marines or the Air Force declaring their jets operational. What he is talking about in his report does not pertain to that capability but to a higher capability not expected to be introduced in the fleet till a year to 18 months from now.
There are 4 groups that the F35's out there fall in -
1* Those that support the developmental testing and will never be converted to higher standards
2* Developmental test aircraft currently running block 1 or 2 software and hardware, w/o any concurrency changes applied to them but expected to receive full 3F capability to support OT&E
3* Aircraft in testing, training and supporting non operational activities that are in the pipeline for receiving concurrency changes through the individual service run plans at the depot level.
4* Aircraft comming out of production now or those that have gone in and had the depot changes incorporated to them to get them to the block 3I standard
Out of these only the last one has operational relevance. Group one will be retired once they accomlish what they set out to do, or they'll be moving over to different roles that do not require upgrades. Group 2 and 3 are in transition to get to the same state as group 4, while group 2 holds on until it has fulfilled its critical mission of supporting development of the last SDD block (3F). Only group 4 aircraft, or those in group 3 that have reached the same standard as group 4 are opertional or will be declared operational in the future. At the moment the next couple of squadrons for the Marines and the USAF that IOC are all new ones, with aircraft comming out of LRIP8, 9 and 10. Eventually more and more Pre LRIP6 aircraft will get to full stanadard as per their internal plans and depot capacity.
If you want to look at how testing, safety and software stability pertains to operationally coded aircraft then you look at the versions and configurations these aircraft are flying with, not the ones that the development-test fleet is flying with which is not expected to be ported on to the operational fleet for at least another year.
http://aviationweek.com/combat-aircraft/us-air-force-f-35s-ready-first-overseas-deployment
Where DT pilots at Edwards were having trouble booting-up their aircraft about once out of every three flights, Palz says the F-35s at Hill have been able to maintain an “abort rate” of less than 2%. This level of reliability is impressive, he says, particularly compared to legacy aircraft.
“I could tell you, for my F-16 unit next door—that is not the case,” Palz says.
Meanwhile, the “break rate” for the Hill F-35s—how often the aircraft breaks after a flight—is just 5%, compared to about 13% for the F-16s here, Palz notes. The F-35 pilots rarely see shutdown events of the aircraft’s mission systems during flight, and so far there have been no instances where flights were diverted or aborted due to software glitches, he says.
One reason for the improvement is that Hill has Lockheed’s most recent F-35 production build, low-rate initial production lots (LRIP) 7 and 8, while the aircraft at Edwards are from LRIP 4, an earlier F-35 build.
“It’s almost a different version, so we have the most mature, vetted, capable, modified aircraft in the fleet, where your Eglin and Luke [AFBs] do not,” Palz says. “I couldn’t do half with those aircraft what I can do with these aircraft.”
Testing is uncovering potential fatal issues before its put into manufacturing or large scale operations.
Testing gives confidence to the pilot about the edges to which his plane can fly.
Which is the scope of the developmental testing and not operational testing that is left for the aircraft and which is the main thing being debated. You must first distinguish b/w operational assessment of a product with an assessment of its safety and whether its systems perform as per the desired requirements The former is done under the supervision of the DOT&E, while the DOT&E is only a passive observer on the latter. All safety related matters are handled during developmental testing since it would be unwise to be well into production, and commit a decade + of development funding to something to have to wait for it to be deemed safe at the last minute.
No one is arguing against the OTE or the DOTE (there is a distinction between the two since the DOTE staff is merely interpreting OTE actions and not doing independent testing of their own) is but one needs to be clear about what they do and what they do not do. They do not tell the USAF how it should assess a wepaons system. If the USAF goes through its process of vetting any DT or OT plan, and decides to eliminate test points they hold themselves accountable based on their established procedures to do so. The DOTE does not have the authority to overrule them. All the DOTE can do is suggest to the Congress and other civilian leaders that XYZ has been eliminated and ask it to re-consider reinstating these test points. The Congress listens to the service's pov and then can put money in for additional testing if it finds their argument is unconvincing. Similarly the DOT&E can go to the OSD and plead his/her case and convince the OSD to effectively overrule the service (s) or the program office. This was exactly what happened with the Navy system i cited earlier. Navy pushed one particualr trial to the right, DOTE (Gilmore) objected to the Secretary of defense, OSD rejected his request, DOTE objected infront of Congress and got them to add $50 million to move testing ahead citing his disagreement with the Navy's interpretations. Even if we ignore that Gilmore came out totally wrong it shows how the process is meant to work.
The DOTE can't unilaterally do so. The DOTE is not the ultimate arbitrator or the decision authority on the adequacy of testing, or even what should be eliminated or added as test plans change with maturing systems.
I think it would be better to look at who does the actual DT and OT, who decides the plans, procedures and scopes for the same, and what the role of the DOTE is in all this. They are all looking at the same data not two different conclusions being drawn based on two different test activities. Lastly, as previously mentioned OT&E completion impacts Milestone-C and the next production rate transition (from high LRIP to FRP). It does not impact IOC, FOC or affect operational readiness in any sort of way.
I think let's first understand that this is about conducting a timely (not to disrupt M-C) OT&E and not something that is relevant to safety in any shape or form since that is covered elsewhere and is not covered in the final OT&E . Let's also understand that the many assessments pertaining to developmental and operational scenarios conducted on the aircraft have been in accordance with procedures jointly developed with the OT&E community. The USAF and USMC have operationally assessed the aircraft and the block 3I standard. They declared it operational only after doing so through a well laid out process with decision authority residing with the Air Combat Command boss (USAF) and the Marine aviation boss. What the OT&E covers is the Operational assessment of full SDD capability so that the Government can conclude the development phase of the aircraft and enter full rate production and follow-on development. They don't wait till the fag end of development to put it through its paces in an operational context but do so incrementally whenever a new capability drop arrives. Formal OT&E is just one where everything is taken together as per the original SDD requirement or minus or addition of any modification to it over the years.
There have been programs that have gone from developmental testing, straight to combat, and have allowed enough real-world_combat assessment to convince the operator community to seek complete waivers from formal OT&E's that take away operational assets and cost a lot of money. One such system was the JSTARS. I'm not saying that this should happen but only highlighting that we need to put the final OT&E exercise in proper context and that it would be wrong to state that in the absence of the OT&E the F-35 will never complete development indefinitely as some in the media try to spin this as. In fact, if the JPO rushed DT jets for rapid conversion in support of the original OT&E schedule the Axe's of the world could have convincingly made a SAFETY argument since what the DT jets are currently doing is testing out changes to see whether they are safe or not before they are implemented on the fleet. Take away this crucial role by putting aircraft in the depot fro weeks to months and you have to rely more heavily on synthetic means to verify fixes which would no doubt outrage Gilmore more than anyone else.
Moreover if one did an autopsy of bad programmatic decisions on the JSF and earlier programs you'd realize that what the DOTE does addresses at the most 10% of those. Its actually the ICE process that has a much higher positive impact than the DOTE empowerment does. All the DOTE does is advise the Congress which then balances what they see in their report, with what the operator community tells them and decides. The problem with the JSF is/was never about insurmountable technical challenges but more about faulty, over ambitious cost and schedule assumptions.
. For them no DOTE Stamp = No Milestone C = Not Working and if you get waivers and even use it in two wars to the operator's satisfaction it still means = Not Working. 
was killed The bomber had no AAM
