Thermodynamics

Entropy, Properly

A count of the microstates consistent with what you can measure from outside.

Updated May 2026

Entropy is usually introduced as disorder, and that description does not survive contact with the definition. A messy room has no more entropy than a tidy one in any sense a thermodynamicist would recognise.

Definition

Boltzmann entropy

S = k_{B} ln Ω

where $Ω$ counts the microstates compatible with the macrostate. Entropy measures how many ways the inside of a system could be arranged without changing anything you can measure from outside.

The logarithm is not cosmetic. Microstate counts multiply when systems combine, $Ω_{12} = Ω_{1} Ω_{2}$ , and we want entropy to add. The logarithm is the only function that converts one into the other.

Why gas fills a room

Open a bottle of perfume in the corner. The molecules spread out. Not because they repel each other, and not because of any force at all. They spread because there are overwhelmingly more arrangements with molecules everywhere than arrangements with molecules in one corner.

For $N$ molecules and two halves of a room, the number of ways to have them all on one side is $1$ , while the number of ways to have them roughly evenly split is about $2^{N} / N$ . For $N \sim 1 0^{23}$ that ratio is not merely large; it is large enough that "never" is a fair description.

Free energy is what actually gets minimised

Systems in contact with a reservoir do not maximise their own entropy. They minimise free energy, which balances their entropy against the reservoir's.

F = U - T S (fixed T, V) G = H - T S (fixed T, p)

This is why anything ordered can form at all. A crystal has lower entropy than the liquid it grew from, but it dumps enough heat into the surroundings that the total still rises. Life does the same trick, continuously and at considerable expense.

Maxwell relations

Because $U$ , $F$ , $H$ and $G$ are state functions, their mixed second derivatives commute. Each potential yields one identity:

(\frac{\partial T}{\partial V})_{S} = - (\frac{\partial p}{\partial S})_{V} (\frac{\partial S}{\partial V})_{T} = (\frac{\partial p}{\partial T})_{V}

These are the workhorses of practical thermodynamics. The second one in particular converts an entropy derivative, which you cannot measure, into a pressure derivative, which you can read off an equation of state.

Information

Shannon's entropy of a probability distribution,

H = - i \sum p_{i} ln p_{i},

reduces to $ln Ω$ for a uniform distribution over $Ω$ outcomes. It is the same quantity.

That correspondence has teeth. Landauer's principle says erasing one bit of information must dissipate at least $k_{B} T ln 2$ of heat, a thermodynamic cost for a logical operation. It is also what finally killed Maxwell's demon: the demon can sort molecules, but it must record which way each one went, and eventually it has to clear its memory.

Aside

The cost is absurdly small, and it still matters

At room temperature $k_{B} T ln 2 \approx 3 \times 1 0^{- 21}$ J. A modern processor dissipates something like a billion times that per logical operation, so Landauer's bound is nowhere near the binding constraint on your laptop. Its interest is that the bound exists at all: it says computation is a physical process with a thermodynamic price, not an abstraction floating free of the universe running it.

Entropy is not a property of the system alone

The subtlety that trips people up: $Ω$ depends on which macrostate you chose to describe the system with, and that choice is yours.

Take a box divided by a partition, gas on both sides, same species, same pressure. Remove the partition. Has the entropy increased? If the molecules are genuinely identical, no. The macrostate did not change, and nothing observable happened. If they differ in any way you could in principle detect, yes, by $2 N k_{B} ln 2$ .

The discontinuity as the two gases become more and more similar is the Gibbs paradox, and it is only a paradox classically. Quantum mechanically, identical particles are identical exactly, with no continuum in between: two atoms of the same isotope in the same state are not merely indistinguishable in practice, they are the same kind of thing. The paradox was an early sign that classical statistics was overcounting.