Vector and Matrix Notation for Lattices: Row vs Column, Norms, Gram

Almost every formula on lattices.io is built from a handful of linear-algebra symbols. This page lists them with how to read them aloud, what they mean here, a small example, and the chapters that use them. The one thing to check before reading any formula is the basis convention: the site uses two, depending on the section.

Vectors

A vector is written as a plain lowercase letter such as $x$ , $v$ or $b_i$ , not in bold and not with an arrow. Its entries are listed in parentheses: $x = (x_1, \dots, x_n)$ .

Symbol	Read as	Meaning on this site	Example	Used in
$b_i$	“b sub i”	The $i$ -th basis vector. Indices start at 1 in the articles; g6k-rs code starts at 0.	$b_1, \dots, b_n$	1.5, 4.1, 4.2, 2.2, 1.2
$x_i$	“x sub i”	The $i$ -th entry (coordinate) of the vector $x$ .	$x = (3, -4)$ has $x_2 = -4$	7.1, 1.6
$e_i$	“e sub i”	The $i$ -th standard unit vector: 1 in position $i$ , 0 elsewhere. In LWE and CVP chapters $e$ without an index is an error vector instead.	$e_2 = (0, 1, 0)$	7.2, 7.5
$0$ , $\mathbf{1}$	“zero”, “the all-ones vector”	The zero vector, and the vector with every entry equal to 1.	$A\mathbf{1}$ is the vector of row sums of $A$	7.5
$\mathbb{Z}^n$ , $\mathbb{R}^n$	“Z to the n”, “R to the n”	All vectors with $n$ integer (or real) entries. $\mathbb{Z}^n$ is also the simplest lattice.	$(2, -1) \in \mathbb{Z}^2$	1.6, 1.2, 1.3, 2.4, 6.2

Rows or columns: the two basis conventions

A basis $b_1, \dots, b_n$ is stored as one matrix $B$ . You can put the basis vectors in its columns or in its rows, and the site does both:

Convention	Basis vectors are	Lattice	Change of basis	Used in
Column	columns of $B$	$\Lambda(B) = \{ Bx : x \in \mathbb{Z}^n \}$	$B' = BU$	1.x, 2.x, 6.1
Row	rows of $B$	$L(B) = \{ xB : x \in \mathbb{Z}^n \}$	$B' = UB$	7.x, 4.6 (g6k-rs, fplll)

The two matrices are transposes of each other, and the lattice is the same set of points. In the column convention the integer coefficients $x$ form a column vector multiplied on the right; in the row convention they form a row vector multiplied on the left. The column chapters also write $B = [b_1 \mid b_2 \mid \cdots \mid b_n]$ , where the bars separate columns.

Worked example

Take $b_1 = (2, 0)$ and $b_2 = (1, 3)$ , and the coefficients $x = (2, -1)$ , so the lattice point is $2b_1 - b_2 = (3, -3)$ .

\text{column: } B = \begin{pmatrix} 2 & 1 \\ 0 & 3 \end{pmatrix}, \quad Bx = \begin{pmatrix} 2 & 1 \\ 0 & 3 \end{pmatrix} \begin{pmatrix} 2 \\ -1 \end{pmatrix} = \begin{pmatrix} 3 \\ -3 \end{pmatrix}

\text{row: } B = \begin{pmatrix} 2 & 0 \\ 1 & 3 \end{pmatrix}, \quad xB = (2,\ -1) \begin{pmatrix} 2 & 0 \\ 1 & 3 \end{pmatrix} = (3,\ -3)

Same point, same determinant ( $\det B = 6$ either way), transposed matrix. When you copy a formula from one section into code for another, transpose $B$ first.

Matrices

Symbol	Read as	Meaning on this site	Example	Used in
$\mathbb{Z}^{m \times n}$	“m by n integer matrices”	Matrices with $m$ rows and $n$ columns and integer entries ( $\mathbb{R}^{m \times n}$ for real entries). Rows come first.	$A \in \mathbb{Z}^{k \times n}$ : $k$ equations in $n$ unknowns	1.2, 2.1, 2.2, 7.4, 7.5
$A^\top$	“A transpose”	Rows and columns swapped. On a vector, $a^\top s$ is the inner product of $a$ and $s$ . Some sources write $A^T$ ; same thing.	$\begin{pmatrix} 2 & 1 \\ 0 & 3 \end{pmatrix}^{\!\top} = \begin{pmatrix} 2 & 0 \\ 1 & 3 \end{pmatrix}$	3.1, 7.5, 2.3, 1.2, 6.1
$A^{-1}$	“A inverse”	The matrix with $AA^{-1} = I$ . For a polynomial, $f^{-1} \bmod q$ is its inverse in the ring (see 2.6).	$\begin{pmatrix} 2 & 1 \\ 0 & 3 \end{pmatrix}^{-1} = \tfrac16 \begin{pmatrix} 3 & -1 \\ 0 & 2 \end{pmatrix}$	6.1, 1.2
$B^{-\top}$	“B inverse transpose”	$(B^{-1})^\top$ . Its columns (column convention) or rows (row convention) are a basis of the dual lattice.	For the column basis above: $\tfrac16 \begin{pmatrix} 3 & 0 \\ -1 & 2 \end{pmatrix}$	6.1, 1.2
$I_n$ , $I$	“the n by n identity”	1 on the diagonal, 0 elsewhere. $qI$ is $q$ on the diagonal.	$I_2 = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}$	7.5, 2.6, 1.6
$\operatorname{diag}(d_1, \dots, d_n)$	“diagonal matrix”	The matrix with $d_1, \dots, d_n$ on the diagonal and zeros elsewhere.	$\operatorname{diag}(2,3) = \begin{pmatrix} 2 & 0 \\ 0 & 3 \end{pmatrix}$	6.1
$\begin{pmatrix} I & H \\ 0 & qI \end{pmatrix}$	“block matrix”	A matrix assembled from smaller matrices. $0$ stands for a block of zeros of whatever size fits. Embeddings are almost always written this way.	NTRU basis in 2.6; Kannan embedding in 7.4	7.1, 7.5, 2.2, 2.6, 7.4
$[\,A \mid B\,]$	“A next to B”	Matrices placed side by side (same number of rows). $[b_1 \mid \cdots \mid b_n]$ lists the columns of a basis.	$[\,I_n \mid N a\,]$ in the kernel embedding	1.1, 1.2, 4.1

Inner products, norms and distances

All lengths on the site are Euclidean unless a subscript says otherwise. The examples use $x = (3, -4)$ and $y = (1, 2)$ .

Symbol	Read as	Meaning on this site	Example	Used in
$\langle x, y \rangle$	“inner product of x and y”	$\sum_i x_i y_i$ . Also written $x^\top y$ ; in 7.1 as $a \cdot x$ .	$\langle x, y \rangle = 3 - 8 = -5$	2.2, 1.2, 6.1, 3.4, 7.1
$\cdot$	“times” or “dot”	Usually ordinary multiplication ( $2^{0.292 \cdot n}$ , $N \cdot 271$ ). Between two vectors, as in $a \cdot x$ , it is the inner product.	$(1,2) \cdot (3,-4) = -5$	3.4, 1.3, 7.1, 1.5, 3.2
$\\|x\\|$	“norm of x”, “length of x”	Euclidean length $\sqrt{\langle x, x \rangle}$ , the same as $\\|x\\|_2$ . Lattice reduction minimises this norm.	$\\|x\\| = \sqrt{9 + 16} = 5$	7.4, 1.5, 2.2, 1.4, 3.3
$\\|x\\|^2$	“norm of x squared”	The squared length $\sum_i x_i^2$ . Algorithms and code work with it to avoid square roots; g6k-rs `norm_sq` returns it.	$\\|x\\|^2 = 25$	1.4, 7.5, 7.4, 1.5, 1.2
$\\|x\\|_\infty$	“infinity norm”, “max norm”	The largest absolute entry, $\max_i \|x_i\|$ . Signature schemes bound this norm (“every coefficient is small”).	$\\|x\\|_\infty = 4$	3.3, 2.5, 2.2, 2.4, 2.6
$\\|x\\|_1$	“one norm”	The sum of absolute entries, $\sum_i \|x_i\|$ . For a 0/1 vector it counts the ones.	$\\|x\\|_1 = 7$	3.3
$\ell_2$ , $\ell_\infty$	“ell two”, “ell infinity”	Names for the norm a problem is measured in, as in “SVP in the $\ell_\infty$ norm”.	—	2.1, 2.2
$\|x_i\|$ , $\max_i$	“absolute value”, “max over i”	Absolute value of a number, and the largest value of an expression as $i$ varies.	$\max_i \|x_i\| = 4$	7.3, 7.1
$\\|t - v\\|$	“distance from t to v”	Euclidean distance between two points.	$\\|x - y\\| = \\|(2, -6)\\| = \sqrt{40}$	2.2, SVP & CVP, 1.4
$\mathrm{dist}(t, \Lambda)$	“distance from t to the lattice”	$\min_{v \in \Lambda} \\|t - v\\|$ : how far $t$ is from the closest lattice point.	$\mathrm{dist}((0.4, 0.3), \mathbb{Z}^2) = 0.5$	2.2

Determinants, span and change of basis

Symbol	Read as	Meaning on this site	Example	Used in
$\det(B)$	“determinant of B”	The signed determinant of a square matrix. Applied to a lattice, $\det(\Lambda)$ means its volume, which is always positive (see 8.3 Lattice notation).	$\det \begin{pmatrix} 2 & 1 \\ 0 & 3 \end{pmatrix} = 6$	1.3, 1.4, 1.6, 1.2, 7.5
$\|\det(B)\|$	“absolute determinant”	The volume of the parallelepiped spanned by the basis. Swapping two basis vectors flips the sign of $\det B$ but not this.	$\|\det(b_2, b_1)\| = \|-6\| = 6$	1.2, 4.6, 1.4, 1.3, 6.2
$\sqrt{\det(BB^\top)}$	“root of det B B transpose”	Volume of a lattice given by a non-square row basis (fewer vectors than coordinates). With a column basis the same quantity is $\sqrt{\det(B^\top B)}$ .	rows $(1,0,5), (0,1,3)$ : $\sqrt{35}$	7.1, 7.2
$\mathrm{span}(b_1, \dots, b_k)$	“span of b one to b k”	All real combinations of the vectors: a line, plane or subspace. The lattice is only the integer combinations.	$\mathrm{span}((1,0))$ is the $x$ -axis	4.2, 1.3
$\dim$ , rank	“dimension”, “rank”	The number of linearly independent vectors in a span or lattice. A lattice is full-rank when this equals the number of coordinates.	the solutions of $a \cdot x = 0$ in $\mathbb{Z}^3$ have rank 2	1.3, 7.5, 7.1
$U$ , $\det(U) = \pm 1$	“a unimodular matrix”	An integer matrix with determinant $\pm 1$ , so its inverse is also an integer matrix. Multiplying a basis by $U$ gives another basis of the same lattice.	$U = \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}$ replaces $b_2$ by $b_1 + b_2$	1.2, 6.1
$\mathrm{GL}_n(\mathbb{Z})$	“G L n over Z”	The set (group) of all $n \times n$ unimodular matrices.	$U \in \mathrm{GL}_2(\mathbb{Z})$	1.2

Gram matrices and projections

Symbol	Read as	Meaning on this site	Example	Used in
$G = B^\top B$	“the Gram matrix”	All pairwise inner products of the basis vectors: $G_{ij} = \langle b_i, b_j \rangle$ . With columns it is $B^\top B$ ; with rows it is $BB^\top$ . The matrix is the same.	for $b_1 = (2,0)$ , $b_2 = (1,3)$ : $G = \begin{pmatrix} 4 & 2 \\ 2 & 10 \end{pmatrix}$	1.2, 1.4
$\pi_i(v)$	“pi i of v”	Orthogonal projection of $v$ away from $\mathrm{span}(b_1, \dots, b_{i-1})$ : keep only the part of $v$ perpendicular to the first $i-1$ basis vectors. $\pi_i(\Lambda)$ is the projected lattice BKZ works in.	$\pi_2((1,3)) = (0,3)$ when $b_1 = (2,0)$	4.2, 4.4
$\tilde b_i$	“b i tilde”	The $i$ -th Gram–Schmidt vector, $\pi_i(b_i)$ . Many papers write $b_i^*$ ; this site always uses the tilde.	$\tilde b_2 = (0, 3)$ in the example	2.2, 4.1, 1.5, 4.2, 1.2

The Gram–Schmidt coefficients $\mu_{ij}$ and the quantities built from them are covered in 8.3 Lattice notation.

Same symbol, different meaning

Symbol	Meaning	Where
star on a lattice, $\Lambda^*$	The dual lattice. Some papers write $\Lambda^\vee$ .	6.1, 1.2, 4.4
star on a matrix, $B^*$	Conjugate transpose (adjoint), as in the Falcon Gram matrix $G = BB^*$	3.6
star on a point, $v^*$	The closest lattice vector found by a CVP algorithm	2.2
$e$	error or noise vector (LWE, CVP); with an index, $e_i$ is a unit vector	2.3, 7.4; 7.2
$\oplus$ , $\otimes$	homomorphic addition and multiplication of ciphertexts, not direct sum or tensor product	3.2
$G$	Gram matrix in lattice chapters; a hash function in 3.1; an elliptic-curve generator point in 6.4	—

Converting between conventions

When a formula comes from a column-convention chapter and your code (g6k-rs, fplll, the 7.x guides) stores bases as rows, rewrite it with this table. Here $B_{\mathrm{c}}$ is the column basis and $B_{\mathrm{r}} = B_{\mathrm{c}}^\top$ the row basis of the same lattice.

Quantity	Column convention	Row convention
Lattice point	$B_{\mathrm{c}}\, x$ , $x$ a column	$x\, B_{\mathrm{r}}$ , $x$ a row
Change of basis	$B_{\mathrm{c}} U$	$U^\top B_{\mathrm{r}}$ (any unimodular $U$ on the left)
Gram matrix	$B_{\mathrm{c}}^\top B_{\mathrm{c}}$	$B_{\mathrm{r}} B_{\mathrm{r}}^\top$
Volume, full rank	$\|\det B_{\mathrm{c}}\|$	$\|\det B_{\mathrm{r}}\|$
Volume, any rank	$\sqrt{\det(B_{\mathrm{c}}^\top B_{\mathrm{c}})}$	$\sqrt{\det(B_{\mathrm{r}} B_{\mathrm{r}}^\top)}$
Dual basis	columns of $B_{\mathrm{c}}^{-\top}$	rows of $B_{\mathrm{r}}^{-\top}$
Kannan embedding of target $t$	$\begin{pmatrix} B_{\mathrm{c}} & t \\ 0 & M \end{pmatrix}$ (2.2)	$\begin{pmatrix} B_{\mathrm{r}} & 0 \\ t & M \end{pmatrix}$ (7.4)
$i$ -th basis vector	column $i$	row $i$ (index $i-1$ in g6k-rs)

With the example basis, $B_{\mathrm{c}}^{-\top} = \tfrac16 \begin{pmatrix} 3 & 0 \\ -1 & 2 \end{pmatrix}$ has columns $(1/2, -1/6)$ and $(0, 1/3)$ , and $B_{\mathrm{r}}^{-\top}$ has the same two vectors as rows. Each has inner product 1 with its partner basis vector and 0 with the other.

Conventions we follow

Vectors are plain lowercase letters, matrices plain uppercase letters; no bold, no arrows.
Chapters 1–2 and the dual-lattice guide put basis vectors in columns; chapter 4.6 and the embedding guides put them in rows. Each page says which it uses.
$\|\cdot\|$ without a subscript is always the Euclidean norm.
Indices in formulas start at 1; indices in Rust and Python code start at 0.
$\det$ of a matrix is signed; $\det$ of a lattice is its positive volume.

8.2 Vectors and matrices