Calculating entropy with Python.
Fun fact: There is more than one kind of entropy out there.
If you've been through high school chemistry or physics, you might have learned about thermodynamic entropy, which is (roughly speaking) the amount of disorder in a closed system. Alternatively, and a little more precisely, thermodynamic entropy can be defined as the heat in a volume of space equalizing throughout the volume. But that's not the kind of entropy that I'm talking about.
Information theory has its own concept of entropy. One way of explaining information theory is that it's the mathematical study of messages as they travel through …
Read more...