Wednesday, May 4, 2022

Boeing Getting Set to Try Starliner Again

It's probably too optimistic to mark your calendars yet, but Boeing is in the final stages of preparation to try their Starliner capsule's uncrewed flight test on May 19th - no time has been given.  

It has been nine months since the end of last July when a routine test of the capsule in preparation for launch found troubles with valves on board.  Eventually, investigators determined that 13 of 24 oxidizer valves within Starliner's propulsion system were stuck.  It took months with the capsule atop its Atlas V launcher at the pad, then back in a hanger for Starliner, then with even more capability needed, to determine the root cause of the valve failures.  The cause was diagnosed as being that dinitrogen tetroxide oxidizer that had been loaded onto the spacecraft 46 days prior to launch had combined with ambient humidity to create nitric acid, which had started the process of corrosion inside the valve's aluminum housing. Who would have expected humidity on an Atlantic beach in July - in Florida, where humidity remains one of our biggest exports? 

It's worth mentioning without dwelling on the story that this was, again, July of 2021, a year and seven months of troubleshooting systems on spacecraft after their nearly disastrous first attempt at an orbital test in December of 2019.  The bigger picture is that they've really been working on Starliner much longer than that

The company has been working on the vehicle since at least 2010, when it was called Crew Space Transportation-100, or CST-100. Starliner made its debut flight in December 2019, but problems cropped up just minutes after liftoff, when the spacecraft captured the wrong "mission elapsed time" from its Atlas V launch vehicle. It also had difficulty communicating with ground stations. Flight controllers at NASA and Boeing were able to restore communications with Starliner and help it reach orbit. However, because of the propellant spent during these activities, Starliner was unable to complete its primary objective, demonstrating a safe docking with the International Space Station.

The whole flight was a disaster in slow motion, with the vehicle almost being lost more than once.  It was practically a miracle they got the capsule safely down.  In particular, there was a catastrophic software bug that would have fired the capsule's thrusters to fire incorrectly.  Luckily, it was found in time to fix the bug before the craft was lost.  

These problems led NASA to declare the first Starliner test flight a "high visibility close call" and set off a years-long investigation and deep dive into Starliner's software problems. Boeing agreed to pay for a second test flight at a cost of $410 million and eventually readied the Orbital Flight Test-2 mission that reached the pad in the summer of 2021. Then, the vehicle had its sticky valve issue. Finally, after all of that, the company has Starliner back on the pad, ready for a do-over of the do-over launch.

NASA, of course, has SpaceX and Crew Dragon to ferry crews to and from the Space Station.  It's prudent, however, to have a backup system should some disaster happen and the fleet need to be grounded.  The intent of the commercial crew program itself was always to have two vehicles.  That desire for a backup is only given more urgency with the Russian threats of abandoning the ISS.   

The flight Starliner capsule being prepared for rollout to the launch pad.  Boeing photo. 

That underlines that there's intense interest in this mission throughout the manned spaceflight offices.  In April of '20, four months after the disastrous Starliner flight, Boeing was removed from further contract award consideration by NASA's acting chief of human spaceflight, Ken Bowersox.  I don't know how long such policies last or if it's in effect now, but I can imagine it being re-instituted or made permanent if they fail at this mission, too. 



12 comments:

  1. That whole program reeks of "Make Work" and pork. Even my Sweet Little Wife has been praising Mr. Musk lately, and she's not really "into" commercial space....

    ReplyDelete
  2. Over time. Over budget. Underperforming. Overpriced.
    Has the capability of ISS re-boost. Lah-dee-dah. If it can't get there, what good is it??
    Let's get the Starchaser up and running for 1/10th the cost. Let's get SpaceX's dragon capsule to do reboosts - doesn't have to be manned, does it??
    Quit pi**ing our money and time away already, Mr. Bill...

    ReplyDelete
    Replies
    1. One of those things that doesn't get much attention is that there's a split in congress over "cost plus" contracts, with some congress critters wanting to keep them and others saying it's wasting money. I have to assume they want "cost plus" because since they control the money, they'll get more power.

      The surprise is that Mr. Bill has come out against cost plus.

      "You get it done with that competitive spirit. You get it done cheaper, and that allows us to move away from what has been a plague on us in the past, which is a cost-plus contract, and move to an existing contractual price."

      Delete
    2. As I remember, there is a type of contract between full on cost-plus and firm fixed price. In a development program like this, there can be unknown unknowns at the start of the program that have to be resolved when they are discovered. If done under a firm fixed price contract, the contract would have to be overpriced to be able to provide for resolution of such issues and then the contractor would be accused of overcharging if nothing really bad happens. The type of contract I am thinking of requires the contractor to come begging for more funding with needed justification.

      Delete
  3. As much as I hate it, I hope the flight works. Better than having to use Russkie equipment.

    But, of course, Starship could probably reboost the ISS into Mars orbit...

    ReplyDelete
  4. Exposure of the NTO to ambient moisture makes no sense at all. When NTO is loaded into propellant tanks an inert gas of either helium or nitrogen is used to pressurize the tank. The tank system is then completely sealed as it must be due to the toxic nature of NTO. Furthermore to protect the technicians working around the spacecraft any valves which could lead to external leakage must be dual fault tolerant. So, a fill valve, for example, has three separate sealing mechanisms. There is a way which moisture can enter the system but this has to occur prior to the NTO loading operation. It is quite common for spacecraft to undergo vibration testing as part of the acceptance test process. To simulate the propellant mass each propellant tank is loaded with an equivalent weight of water. Upon the completion of the vibe tests the propellant tanks must be drained of all water and the entire “wetted” propellant tank system dried to a high level. Having done this process myself I can assure you this is an absolute bitch to do. If I was to speculate what happened to this spacecraft my best guess would be the system wasn’t dried sufficiently after a vibe test.
    CP

    ReplyDelete
    Replies
    1. Let me just paste the explanation from the source article here, and tell me if you think it's full of it.

      "On Tuesday, during a teleconference with reporters, officials from Boeing and NASA discussed the steps they have taken to ameliorate the problem for Starliner's upcoming test flight. Michelle Parker, vice president and deputy general manager of Boeing Space and Launch, said the valves remain the same on the vehicle but that technicians have sealed up pathways by which moisture might get inside the propulsion system. They are also purging moisture from the valves using nitrogen gas and loading propellants onto Starliner closer to launch."

      The way you describe it agrees with my very limited knowledge of the stuff. When they say, "sealed up pathways by which moisture might get inside the propulsion system", I say any path the allows moisture in works both ways and allows N2O4 out and They Don't Want to Do That." Seals work both ways. Because of that, this explanation kind of sucks to me.

      Back during the last Crew Dragon splash down, the talking heads read something about "waterproofing the capsule" in the sense of making sure water doesn't make its way into the crew area. My first thought was sealing methods that keep the air from leaking out of the cabin will keep water from leaking in.

      Delete
    2. First let me say this SiGraybeard - you have an excellent website full of great info on space related topics. I enjoy reading it everyday.

      Typically a spacecraft propulsion system is designed to isolate the propellant storage tanks from the actual thrusters until the spacecraft is in space. This is because the propellants are toxic to the technicians who work around the spacecraft and launch vehicle prior to launch. (Please note that the spacecraft is loaded with propellants prior to it being stacked onto the launch vehicle). A typical propulsion system design will have propellant tanks attached to fill and drain valves - these valves are solely used to fill, or empty, the tanks. Those valves are exposed to the exterior and can be a source of leaks - so they have triple seals to prevent such leakage. Two of those seals can fail and the valves will still not leak. Between the tank and the thrusters are isolation valves. In some cases isolation valves are pyro-valves - these provide the best isolation as there is no fluid path thru them until they are activated and they are only activated once the spacecraft is in space. More commonly used isolation valves are known as latching valves which are electronically activated and can be latched open or closed. It is quite common to test these valves during the acceptance test process (ATP) to insure they work. For those ATP tests the propellant system is dry and filled with an inert gas (Helium or nitrogen). Once the tanks are full of propellant the latching valves are never opened until the spacecraft is in space and the propellant system is ready to be activated. To do otherwise would allow propellant to reach the thruster valves which would be a no-no because of the leakage hazard to the humans on the ground. I cannot foresee any way that NASA would allow propellants to reach the thrusters prior to launch. So I have a hard time believing the ambient humidity at the Cape was responsible for the valve problem they experienced. It just doesn't add up.
      Having said all that I will say I've never seen the propulsion system schematic of the Starliner nor have I seen any of the details of this problem other then what appears in the public domain.
      CP

      Delete
    3. Thanks for that, CP. Great stuff!

      My last job before retiring was in commercial avionics. We were a big subcontractor to Boeing, so I've worked with their design engineers and engineering managers before. Those guys were professional and left me with a good overall impression of the company.

      Starliner ruined that for me - to the point of wondering if they kept anyone with experience away and just let new grads do everything. The software mistakes were so egregious, so awful, that it's a miracle they got the capsule down in one piece. I can't imagine a real software quality assurance effort, or software design review process was followed. If they did reviews, that means the experts didn't pay attention or weren't invited, and that's even more frightening.

      This stuff, with the likely problems not aligning with how it's typically really done, really doesn't do much to help me respect them again.

      Delete
    4. Great post and comments.

      Delete
  5. On a somewhat unrelated topic. Somewhat...

    I tripped over this podcast over that TWIT.TV (This Week In Tech). They do a number of tech-related podcasts, mostly about computers, phones, etc. but they have added This Week In Space.

    I haven't looked at it, but they Tariq Malik from Space.com, and Rod Pyle from The Planetary Society as the moderators. There is certainly a lot to talk about.

    I know Twit.TV mostly from their long-standing show on computer security, Security Now. Anyway, I thought you might be interested

    ReplyDelete
    Replies
    1. Thanks, Deb. I used to watch TWiT back when I had cable, so several years ago. They were an hour long show with Leo Leporte as the host. Did some fun and some interesting stuff.

      I'll set the latest episode or so to run while I'm working on other stuff and see if it's worth it.

      Delete