This is the large issue with writing software that can put people’s lives in danger. The cost of writing “bug free software” is huge but the cost of a human life is even greater. But the likelihood of a failure at 98% vs. 99% is actually worth the risk.
Rumor has it that space shuttle software costs $1,500 per line to develop. When asked about the price tag, Norvig said “I don’t know if it’s optimal. I think they might be better off with buggy software.” At some point it’s certainly not optimal. If it doubles the price of a project to increase your probability of a successful mission from 98% to 99%, it’s not worth it; you’re better off running two missions with a 98% chance of success each.
By the way if you are a computer engineer/scientist/programmer I can’t make a larger suggestion then purchasing a copy of Coders at Work.