If a rocket costs 100 gigadollars to construct, and 20% of launches fail, then your rocket really costs 120 gigadollars. That's a pretty big difference. Is the work to get it up to 99% really costing 20 gigadollars?

No, and I'm asserting that it's costing >1,000 gigadollars to get it up to 99%. I analyzed it like so:

If 1 out of 5 attempts fail, and I launch 'x' number of rockets, my odds of every launch being failures are (1/5)^x, which is less than 1% after just 3 rocket launches. So introducing a quality assurance program that costs 10 times as much to guarantee a 99% success rate doesn't make sense when you could launch 10 rockets without the QA program to get an effective success rate of 99.99%+.

If your craft is reusable, it's even worse. Because then you spent all that money on a craft that's only good for 5 launches, on average.

I would say 4 launches, on average. So if it costs more than 4 times as much money to implement a reusable craft, it's not worth it.

What about manned craft? If the average rocket has a crew of 5, you now have a death toll of one human per launch.

Nooo, you do not skimp on costs when human life is involved. Not arguing that one, you institute the best QA program possible.