Guided project: audit a cultural collection
On this page
A museum educator wants to describe the historical coverage of a selected set of catalogue records. Deliver a validated table and an honest date-bound comparison. Your task is not to estimate the whole museum’s collection distribution or infer the date on which an uncertain work was completed.
The saved query requests the first 100 records matching painting. It is a convenience selection in API order, not a random collection sample. The extracted rows preserve catalogue IDs, titles, object types, date labels, earliest/latest years, artist identifiers and source links. Museum metadata are CC0; no artwork images are required.
Inspect and specify the contract
Read the source register and inspect the CSV. Define one observation as one catalogue object and id as its key. Identify the difference between a keyword query and an object-type classification. Check the actual type column rather than assuming the search term is a filter on that field. In this snapshot all 100 returned rows have Painting as their recorded type; a later query may differ.
Require IDs to be unique. Parse year bounds numerically, reject unresolved or reversed bounds into a separate table, and preserve source values. BCE or very old dates are not automatically errors; validity depends on the catalogue definition, not a modern-date preference.
Classify without inventing exact dates
For a cutoff of 1900, classify a valid object as before the cutoff when its latest possible year is below 1900; at or after the cutoff when its earliest possible year is at least 1900; otherwise retain a crossing interval. Define all boundaries before counting. The span between bounds records catalogue date uncertainty; it is not necessarily production duration.
Produce evidence and challenge the pipeline
Export validated and unresolved records, period counts and a JSON audit recording input and output counts. Create a labelled count plot with the selected-record denominator in its caption. Verify that the period counts sum to the validated observations and that the observation key remains unique.
Now work on a disposable copy: duplicate one ID, make one earliest year nonnumeric, and reverse one interval. Explain why a duplicate key should stop the pipeline while an unresolved date can enter a documented rejection table. Do not edit the raw snapshot to make a check pass.
Deliver and defend
Submit the source register, schema, validated table, unresolved table, period summary, diagnostic plot, executable code and a 200-word report. State the query selection, date-bound interpretation and limits on generalization. Explain why a high pre-1900 count would not establish the distribution of the whole collection.
Acceptance evidence: 100 input objects in the supplied snapshot; unique IDs; one disposition for every row; period counts equal validated rows; invalid fixtures detected; all outputs reproduced from the saved inputs. A different valid implementation may use explicit masks or functions; its semantics must match the stated cutoff rules.
Begin the experiment
Run from the extracted experiment folder, or use the notebook’s first path-setup cell. Inspect the source before implementing the analysis.
from pathlib import Path
import pandas as pd
source = Path('data/teaching/museum.csv')
records = pd.read_csv(source)
print(records.shape)
print(records.head())Check your work
Attempt the project first. Then compare with the complete worked solution. Preserve your own implementation and explain differences.