When your data corrupts, you are witnessing a violation of the Hamming distance. When your compression algorithm bloats instead of shrinks, you are witnessing low entropy.
[ H = -\sum_{i=1}^{n} p_i \log_2(p_i) ]
[ h(x) = -\log_2(p) ]
By Steven Roman (Inspired by his lifelong work in mathematical literacy) Introduction To Coding And Information Theory Steven Roman
