>If the data says "C is better than Ada" but it's 50% confidence, >doesn't that mean there's a 50% chance Ada is better than C ? The usual null hypothesis is "no difference", so 50% confidence would mean "C may be better than Ada, or it may not be any better, and the data are equally consistent with either hypothesis". If Ada is only a tiny bit better, it will take a large N to prove conclusively that tiny difference is real. But who would care? OTOH, if Ada is a lot better, it would be both more important and easier to prove. I'd sure like a stronger endorsement than "once we managed to find some Ada experts, and built some tools, we found Ada modestly better than ...".