Matrix Algebra Basics

Info sheet · Statistics for Psychology & Neuroscience

Info sheet 0.2 (draft) · Prerequisites: basic arithmetic; the idea of a vector · Give feedback ↗

Warning✎ Editing notes — to do / to check

Working notes for the author — not shown to students once collapsed; remove before publishing.

• [ ]

What you’ll get from this sheet

This kicks off a short side‑module on linear algebra — really matrix algebra, the building blocks you’ll lean on next term in the more mathematical brain‑science courses (and, quietly, all over statistics and programming). The goal isn’t mastery; it’s to plant the seed so these ideas don’t come as a shock later. By the end you should be able to:

1. Name a scalar, a vector, and a matrix, and add/subtract them.
2. Tell the three kinds of multiplication apart (element‑wise, dot, matrix).
3. Say when a multiplication is allowed (conformable) — and why order matters.

A scalar is a single number, a vector is a row or column of numbers, a matrix is a 2‑D grid. Add/subtract element‑wise (shapes must match). To multiply two matrices the inner dimensions must match — an (m×n)(n×p) product gives an (m×p) matrix — and order matters.

The vocabulary

A scalar is just a single number (\(a = 2\)); we write it lowercase. A vector is a one‑dimensional set of scalars — a single row or column — written lowercase bold. A matrix is a two‑dimensional set of numbers (at least two rows and two columns), written UPPERCASE; an individual entry gets two subscripts, row then column (no comma): \(A_{23}\) is row 2, column 3.

Scalar arithmetic behaves exactly as you’d expect, and it’s commutative — order doesn’t matter for addition or multiplication. A scalar combined with a vector or matrix just applies to every element: add 7 to a 5‑element vector and each element gains 7. Adding or subtracting two vectors (or two matrices) is element‑wise, so they must be the same shape and orientation — first‑with‑first, second‑with‑second, and so on. Addition is commutative; subtraction, as always, is not.

Three kinds of multiplication

Vectors can be multiplied two different ways. The Hadamard (element‑wise) product multiplies matching elements in place (same shape needed) and returns another vector — in R that’s *, in MATLAB you must write .* (the dot means “element‑wise”). The dot (inner) product needs two vectors of equal length but opposite orientation; you multiply pairwise and sum, so it returns a single number — %*% in R, dot() in MATLAB.

That summing rule is the whole engine of matrix multiplication: each entry of the result is the dot product of a row of the first matrix with a column of the second. Pick an output cell below and watch it being built.

md`**Which entry of the result to compute** (row, column)`

viewof cell = html`<select style="font-size:15px;padding:4px 8px">
  <option>1,1</option><option>1,2</option><option>2,1</option><option>2,2</option>
</select>`

matmulWidget = {
  const A=[[1,2,3],[4,5,6]], B=[[7,8],[9,10],[11,12]], C=[[58,64],[139,154]];
  const parts=cell.split(","), i=+parts[0]-1, j=+parts[1]-1;
  const mk=(M,hl)=>`<table style="border-collapse:collapse;border-left:2px solid #334;border-right:2px solid #334">`+
    M.map((row,r)=>`<tr>`+row.map((v,c)=>{const bg=hl(r,c);return `<td style="padding:5px 11px;text-align:center;font-variant-numeric:tabular-nums;${bg?`background:${bg};`:''}">${v}</td>`;}).join("")+`</tr>`).join("")+`</table>`;
  const tblA=mk(A,(r,c)=> r===i ? "#dbe6f2" : null);
  const tblB=mk(B,(r,c)=> c===j ? "#dcefe0" : null);
  const tblC=mk(C,(r,c)=> (r===i&&c===j) ? "#f6ddc9" : null);
  const terms=A[i].map((a,k)=>`${a}×${B[k][j]}`).join(" + ");
  const sum=A[i].reduce((s,a,k)=>s+a*B[k][j],0);

  const box=document.createElement("div");
  box.style.cssText="background:#f8fafc;border:1px solid #e2e8f0;border-radius:8px;padding:14px";
  box.innerHTML=
    `<div style="display:flex;align-items:center;gap:12px;flex-wrap:wrap;font-size:15px">`+
      `<div><div style="font-size:11px;color:#1F4E79;text-align:center">A (2×3)</div>${tblA}</div>`+
      `<span style="font-size:18px">×</span>`+
      `<div><div style="font-size:11px;color:#1E7A34;text-align:center">B (3×2)</div>${tblB}</div>`+
      `<span style="font-size:18px">=</span>`+
      `<div><div style="font-size:11px;color:#C0501E;text-align:center">C (2×2)</div>${tblC}</div>`+
    `</div>`+
    `<div style="margin-top:.7rem">C<sub>${parts[0]}${parts[1]}</sub> = ${terms} = <b>${sum}</b> — the dot product of <span style="color:#1F4E79">row ${parts[0]} of A</span> with <span style="color:#1E7A34">column ${parts[1]} of B</span>.</div>`+
    `<div style="color:#667;font-size:13px;margin-top:.3rem">The inner dimensions match (3 = 3), so the product exists; the result takes the <i>outer</i> dimensions → 2×3 · 3×2 = <b>2×2</b>.</div>`;
  return box;
}

Matrix–vector and matrix–matrix products follow the same rule and are not commutative — order matters, and swapping it usually gives either a different answer or an error. For a matrix times a column vector, an (m×n) matrix needs an (n×1) vector (so a 4×3 needs a 3×1). For two matrices, the second must have as many rows as the first has columns, and the result takes the first’s rows and the second’s columns: a 3×2 times a 2×4 gives a 3×4.

(And division? You just… don’t. Dividing one matrix by another isn’t really a thing at this level — there are related operations, but they’re beyond us here.)

See it in code

We finally get to MATLAB — short for MATrix LABoratory, and unsurprisingly excellent at this. Note MATLAB’s .* (element‑wise) vs * (matrix), the mirror image of R:

R

Python

MATLAB / Octave

A = [1 2 3; 4 5 6];        % 2x3  (semicolon separates rows)
B = [7 8; 9 10; 11 12];    % 3x2
A * B          % matrix multiplication -> 2x2

v = [1 2 3]; w = [4 5 6];
v .* w         % Hadamard (element-wise) — note the DOT before *
dot(v, w)      % dot product -> a single number

The R and Python tabs run live; MATLAB is a static reference. (A trailing ; in MATLAB just suppresses that line’s output.)

TipCheck your understanding

Can you multiply a 4×3 matrix by a 3×1 column vector? What shape is the result — and what about the other way round?

Yes: the inner dimensions match (3 = 3), so 4×3 · 3×1 gives a 4×1 vector (the outer dimensions). The other way round, 3×1 · 4×3, the inner dimensions are 1 and 4 — they don’t match, so that product is undefined and you’d get an error. That asymmetry is exactly why matrix multiplication isn’t commutative.

Getting orientations right is “a bit like plugging in a USB device — you’ll get it wrong on the first try,” so learn to read the errors. Three traps in particular: matrix multiplication is not commutative (AB ≠ BA, and one order may not even be defined); the dot product needs opposite orientation and returns a scalar, whereas the Hadamard product needs the same shape and returns a vector; and in MATLAB * is matrix multiplication while .* is element‑wise — mixing them up is the classic beginner error (in R it’s the reverse: * is element‑wise, %*% is matrix).

Where this shows up next

Next week: some neat properties and special operations of matrices; the week after, why you should care — including where this quietly powers regression (the normal equation) and PCA. See Appendix (Matrix Algebra) for the full treatment.

Related topics

• Special Matrix Operations — transpose, identity, inverse, determinant.
• Matrix Algebra in Data Analysis — covariance, the normal equation, vectorisation.
• Eigenvectors & Eigenvalues — where matrices meet PCA.
• Ordinary Least Squares: Finding the Best Line — regression, which the normal equation solves in one matrix line.
• Why Reduce Dimensions? — the payoff this side‑module builds toward.