>We decided to test two identical versions of our homepage against each other... we saw that the new variation, which was identical to the first, saw an 18.1% improvement. Even more troubling was that there was a “100%” probability of this result being accurate.

Wow. Cool explanation of one-tailed, two tailed tests. Somehow I have never run across that. Here's a link with more detail (I think it's the one intended in the article, but a different one was used): http://www.ats.ucla.edu/stat/mult_pkg/faq/general/tail_tests...