0.4 MACs, key derivation and randomness

Integrity is not authenticity

A hash function gives every message a short fingerprint. If you download a file and its SHA-256 matches the published value, you know the bytes were not corrupted on the way. That is integrity against accidents.

It does nothing against an attacker who can change the traffic. They change amount=100 to amount=900, compute the SHA-256 of the new message and send both. The check passes, because a hash contains no secret and anyone can compute it. What you want is authenticity: proof that the message was produced by someone who holds a particular key.

Think of a wax seal instead of a checksum. Anyone can count the letters in a letter, but only the holder of the signet ring can press the seal. A message authentication code is that ring, shared by sender and receiver.

Both sides share the same key, so a MAC proves the message came from one of the key holders, not which one. When a third party must be able to check who wrote something, you need a digital signature instead.

Why not just hash the key and the message?

The obvious construction is $t = \mathrm{SHA\text{-}256}(k \,\|\, m)$: put the key in front of the message and hash. With SHA-256 this is broken. SHA-256 processes its input in 64-byte blocks and its output is the complete internal state after the last block. Someone who sees a valid tag can load that state and keep hashing, producing a valid tag for $k \,\|\, m \,\|\, \text{padding} \,\|\, m'$ without ever learning the key. This is the length-extension property described in 0.2 Hash functions. An attacker can append &to=mallory to a signed request and the tag still verifies.

HMAC (RFC 2104, FIPS 198-1) avoids this by hashing twice with two derived keys:

Here $\text{ipad}$ and $\text{opad}$ are fixed byte patterns (0x36 and 0x5c repeated) and $\oplus$ is XOR. The inner hash absorbs the message; the outer hash, keyed separately, seals the result, so the tag is never an internal state an attacker can continue from. HMAC stays secure even if the hash function has weaknesses such as length extension. Hashes built differently, like SHA-3 and BLAKE3, do not have length extension and offer keyed modes of their own, but HMAC-SHA256 is the one every library and protocol supports.

HMAC in code

Python's standard library has everything needed. Note the last step: the receiver compares tags with hmac.compare_digest, not ==.

import hashlib
import hmac

key = b"server-side secret key, 32 bytes"
message = b"amount=100&to=alice"

tag = hmac.new(key, message, hashlib.sha256).hexdigest()
print("tag:", tag)

def verify(key: bytes, message: bytes, tag_hex: str) -> bool:
    expected = hmac.new(key, message, hashlib.sha256).hexdigest()
    return hmac.compare_digest(expected, tag_hex)  # constant-time comparison

print(verify(key, message, tag))                   # True
print(verify(key, b"amount=900&to=alice", tag))    # False: message changed
print(verify(b"wrong key", message, tag))          # False: wrong key

tag: 6165adfb5f3ab46008569e0b8f1761efd136828ce2c220c8154fd03dfe0424bf
True
False
False

A normal comparison stops at the first byte that differs, so a wrong tag with a correct first byte takes a little longer to reject than one with a wrong first byte. Over many requests that timing difference can reveal a valid tag one byte at a time. A constant-time comparison looks at every byte regardless, so its running time says nothing about the input.

Key derivation: one secret, many keys

A protocol rarely needs just one key. A secure connection wants separate keys for each direction and for each purpose, and it should never reuse one key for two jobs. Yet a key exchange produces a single shared secret, and that secret may not even be uniformly random (a Diffie–Hellman output, for example, has structure). A key derivation function (KDF) solves both problems.

TLS 1.3 derives its entire key schedule this way (RFC 8446, section 7.1): the handshake secret, the traffic keys for each direction and the resumption secrets all come out of HKDF with different labels. When a connection uses a hybrid post-quantum key exchange, the two shared secrets are simply concatenated and fed into the same HKDF.

from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.kdf.hkdf import HKDF

# A shared secret, e.g. the output of a key exchange (fixed here so the output is reproducible).
shared_secret = bytes.fromhex("4a5d9d5ba4ce2de1728e3bf480350f25e07e21c947d19e3376f09b3c1e161742")
salt = b"handshake salt"

def derive(label: bytes) -> bytes:
    return HKDF(algorithm=hashes.SHA256(), length=32, salt=salt, info=label).derive(shared_secret)

client_key = derive(b"client write key")
server_key = derive(b"server write key")
print("client key:", client_key.hex())
print("server key:", server_key.hex())

client key: 2a902d8ec29330bc927116327d4f3b55697d119d93954ad7931e785eed3338f7
server key: 2554ef26d0d3eb3f743d5122fb1daa6ed224a77c87c39a99f8b1df7126ad9740

Passwords are different

HKDF assumes its input is already a strong secret, such as 32 random bytes. A password is not. People choose from a small, predictable space, so the real threat is guessing: if a database of password hashes leaks, an attacker hashes likely passwords and looks for matches. A fast hash helps them. A single modern GPU computes billions of SHA-256 hashes per second.

A password hash defends against guessing in three ways:

• Salt. A random value, unique per user, stored next to the hash and mixed in. Two users with the same password get different hashes, and precomputed tables of common passwords become useless because every salt needs its own table.
• Slowness. The function is deliberately expensive, tuned so one check takes tens to hundreds of milliseconds. A user logs in once; an attacker needs billions of guesses, so the cost falls almost entirely on them.
• Memory hardness. Argon2 and scrypt also need a lot of memory per guess. Memory is expensive to put on GPUs and custom chips, which removes most of the attacker's hardware advantage.

The widget above shows the slowness at work: each step up in PBKDF2 iterations divides the number of guesses per second by the same factor. Current recommendations:

The settings come from the OWASP Password Storage Cheat Sheet and RFC 9106, section 4. Libraries store the algorithm, parameters and salt inside the hash string, as the Argon2 example below shows, so you can raise the cost later and rehash each user at their next login.

from argon2 import PasswordHasher
from argon2.exceptions import VerifyMismatchError

# OWASP baseline for Argon2id: 19 MiB of memory, 2 passes, 1 lane.
ph = PasswordHasher(time_cost=2, memory_cost=19456, parallelism=1)

stored = ph.hash("correct horse battery staple")   # a fresh random salt every time
print(stored)

print(ph.verify(stored, "correct horse battery staple"))
try:
    ph.verify(stored, "correct horse battery stapler")
except VerifyMismatchError:
    print("wrong password")

$argon2id$v=19$m=19456,t=2,p=1$yzugsBbzS6jGpljaJcOh+A$42j8UztRwKcl4/tdPrhLYmiOAdo+GPWONgUlKiWDOvw
True
wrong password

The salt (yzugsBbzS6jGpljaJcOh+A in this run) is random, so your output will differ every time. A pepper is an extra secret mixed into every password hash but stored elsewhere, for example in a hardware security module or a secrets manager. Unlike a salt it is the same for all users and must stay secret; it helps only if the database leaks without the pepper.

Where random numbers come from

Every key, salt and nonce on this page has to be unpredictable, and general-purpose random functions are not. Python's random module uses the Mersenne Twister, whose entire internal state can be reconstructed from 624 consecutive 32-bit outputs; after that, every future value is known. JavaScript's Math.random gives no security guarantee at all. Both are fine for games and simulations and wrong for secrets.

Use the operating system's cryptographically secure pseudorandom number generator (CSPRNG), which is seeded from hardware events and designed so that its outputs reveal nothing about past or future outputs. In Python that is the secrets module or os.urandom; in the browser, crypto.getRandomValues; in Node.js, crypto.randomBytes.

import secrets

key = secrets.token_bytes(32)        # 256-bit key from the OS CSPRNG
nonce = secrets.token_bytes(12)      # 96-bit nonce, e.g. for AES-GCM
reset_token = secrets.token_urlsafe(32)

print(len(key), len(nonce), len(reset_token))

32 12 43

Keys, salts and nonces all look like random bytes, but they have different jobs:

Nonce reuse is one of the most damaging mistakes in practice, and 0.3 Symmetric encryption shows why.

Common mistakes

• Sending a plain hash as proof of authenticity. Anyone can recompute it. Use HMAC or a signature.
• Building a MAC as $H(k \,\|\, m)$. With SHA-256 it is open to length extension. Use HMAC.
• Comparing tags with ==. Use a constant-time comparison.
• Using one key for several purposes. Derive a separate key per purpose with HKDF and distinct labels.
• Storing passwords with a fast hash, even salted SHA-256. Use Argon2id, scrypt, bcrypt or high-iteration PBKDF2.
• Generating keys or tokens with random or Math.random. Use the OS CSPRNG.

Further reading

• OWASP, Password Storage Cheat Sheet: current parameters for Argon2id, scrypt, bcrypt and PBKDF2.
• RFC 2104 (HMAC), RFC 5869 (HKDF) and RFC 9106 (Argon2).
• David Wong, Real-World Cryptography (Manning, 2021), chapters on MACs, randomness and password hashing. Cited from memory.

Next: 0.5 Public-key cryptography, where two parties who have never met agree on the shared secret that HKDF turns into keys.