Compute with arrays and explain numerical choices

An array calculation has three parts: the mathematical operation, the axis or shape over which it acts, and the meaning of its output. A correct expression can still answer the wrong question if any of these are unclear. This lesson consolidates universal functions, reductions and numeric checks before the mathematical chapters.

Elementwise operations and reductions

An elementwise operation transforms each compatible pair of values. A reduction combines values along an axis. Squaring each observation is different from squaring an aggregate, because nonlinear operations generally do not commute with averaging.

import numpy as np
values = np.array([1., 3.])
print((values**2).mean(), values.mean()**2)
assert (values**2).mean() == 5.0 and values.mean()**2 == 4.0

5.0 4.0

A universal function such as np.sqrt expresses an elementwise transformation. A summary such as mean reduces a collection. Say which operation your formula requires before combining them. Taking the square root of a mean of squares, for example, is not the same as averaging the original magnitudes in general.

Define weighting explicitly

A weighted mean is $\bar{x}_w=\sum_i w_i x_i/\sum_i w_i$ when the denominator is positive. Weights might represent compatible exposure counts or a sampling design; they are not arbitrary decorations. Explain why a weight belongs to the target quantity and check whether units and observation sets match.

rates = np.array([.9, .5])
registrations = np.array([10., 60.])
weighted = np.average(rates, weights=registrations)
print(weighted, rates.mean())
assert np.isclose(weighted, 39/70) and rates.mean() == .7

0.5571428571428572 0.7

The weighted result matches the pooled attendance proportion when the groups and counts represent compatible, non-overlapping registrations. Equal subgroup weighting answers a different question. Negative weights or a zero total require a different mathematical justification or a clear failure, rather than an unexplained result.

Normalize within the intended observation

Suppose each row records category counts within one institution. Dividing by a row total gives a composition within that institution. A zero row has no defined composition. Use a mask so that the undefined row remains visible.

counts = np.array([[2., 3.], [0., 0.], [1., 3.]])
totals = counts.sum(axis=1, keepdims=True)
shares = np.divide(counts, totals, out=np.full_like(counts, np.nan), where=totals != 0)
print(shares)
assert np.allclose(shares[[0,2]].sum(axis=1), 1)
assert np.isnan(shares[1]).all()

[[0.4  0.6 ]
 [ nan  nan]
 [0.25 0.75]]

The length-one dimension preserves the row structure for broadcasting. It is part of the calculation’s meaning: a column normalization would describe a different composition. The explicit unresolved row is preferable to inventing a distribution such as two zeros or two equal shares.

Respect domains, units and types

A logarithm of a positive quantity can make relative changes easier to study; it changes the scale and interpretation. Zero or negative inputs are outside the real logarithm’s domain. Adding a constant changes the transformation and needs a reason. A missing result is not automatically evidence that a computation failed: it can communicate that the stated operation is undefined for that input.

observed = np.array([0., 1., 4.])
logs = np.full_like(observed, np.nan)
positive = observed > 0
logs[positive] = np.log(observed[positive])
assert np.isnan(logs[0]) and logs[1] == 0
print(logs)

[       nan 0.         1.38629436]

Numeric storage also has limits. An integer dtype has a finite range; floating-point values have finite precision. Use a tolerance to check a calculated identity rather than requiring every decimal operation to be exact. Record the dtype when exporting a feature array and inspect finite values before a numerical algorithm.

Practice and transfer

Create a three-row table of non-negative counts, including a zero row. Compute row shares and column shares and explain both denominators. Compare a weighted mean with the pooled calculation from numerators and denominators. Finally, show a nonlinear transformation and aggregation whose order changes the result. Your explanation should identify the changed quantity, not call one expression “wrong” without defining the question.

Chapter exercisesWorked answers after an attempt