Independent capstone: a documented analysis from question to handoff
On this page
This capstone asks you to conduct an investigation, not reproduce a sequence of answers. Use the book’s saved World Bank population snapshot as a reliable starting resource, or propose another public dataset whose provenance and permission you can establish. The NumPy cultural-production project provides an earlier numerical checkpoint. Here you must integrate acquisition, validation, transformation, analysis and communication without a step-by-step solution.
Establish a question you can answer
For the supplied snapshot, a suitable question compares change in total population across the three selected countries between 2018 and 2023. Refine that into a precise question of your own: absolute change, relative change, year-to-year variation, or the effect of a missing year on a comparison. State the observation unit and inclusion rule. Do not present these three countries as a representative global sample.
The World Bank indicator metadata define what population means in this source. Use that definition and the actual reported units. A changing total can reflect several processes and revisions; these annual totals do not separately identify births, deaths, migration or the causal effect of a policy. A careful descriptive conclusion is a successful analysis when that is what the evidence supports.
Prepare an analysis brief before calculating
Submit the question, intended reader, proposed comparison, observation key, required fields, expected coverage and limitations. Define how you will handle missing values and unequal endpoint coverage. Specify what result would count as sufficient evidence for your descriptive answer. If you later change the question after inspecting the values, record the change rather than describing the new question as if it had always been specified.
For an alternative dataset from Kaggle or Hugging Face, add its original producer, licence, revision or version, download route and dataset card. Explain why it is suitable for this analysis. Avoid a choice that requires hidden access, private records or a new modelling technique merely to complete the basic investigation.
Inspect the starter without solving the investigation
The starter verifies the basic file structure and observation key. It intentionally does not calculate your final rankings or write a conclusion.
from pathlib import Path
import json
import pandas as pd
base = Path("data/completion/worldbank")
metadata = json.loads((base / "provenance.json").read_text())
data = pd.read_csv(base / "population.csv")
print(data.columns.tolist())
print(data.groupby("country_code")["year"].agg(["min", "max", "count"]))
assert not data.duplicated(["country_code", "year"]).any()
assert data["population"].notna().all()
assert data["population"].gt(0).all()['country_code', 'country', 'year', 'population']
min max count
country_code
DEU 2018 2023 6
FRA 2018 2023 6
GBR 2018 2023 6The saved data contain 18 country-year observations. That makes independent inspection possible; the purpose is to demonstrate disciplined reasoning rather than to impress with file size. Preserve the raw response and verify its hash before deriving new tables.
Complete the acquisition and quality audit
Produce a data dictionary and source register. Check keys, types, missingness, year coverage, units and plausible value constraints. Identify which checks are mechanical and which require the source documentation. Create a cleaning log even if no corrections are needed: “no change required after these checks” is more informative than an undocumented clean-looking table.
If you deliberately remove a year to test robustness, do so in a separate experimental copy. Explain what the incomplete table can no longer support. Do not fill the missing population with zero or silently compare different endpoints across countries.
Conduct and challenge the analysis
Calculate the comparison specified in your brief. Verify at least one result with independent arithmetic from the actual saved values. Compare absolute and relative change and explain why their rankings can differ. For year-to-year calculations, check ordering and consecutive years within each country.
Include one alternative representation or sensitivity check: a different baseline year, a restricted complete-period comparison, or a clear demonstration of what fails when coverage differs. A sensitivity check is useful when it tests a meaningful assumption; it is not an invitation to search until one preferred conclusion appears.
Choose summaries that respect the measure. Annual population is a stock at a reference time; summing six annual values does not count distinct people across the six-year period. A simple average across countries also answers a different question from a population-weighted measure. State the target quantity before selecting its calculation.
Communicate the evidence and prepare the next stage
Write a report that another reader can audit. Include the question, source and snapshot, inclusion rules, method, main result, supporting table, limitations and next information needed. Separate a measured difference from a proposed explanation. Credit the World Bank and indicator, and state the extraction and transformations you performed.
Prepare a chart-ready long table for Book 3. Give every field a name, meaning and unit; include country code, year and population plus any derived measure whose baseline is explicit. Propose two possible visual views and explain which question each would answer. You do not need advanced styling or an interactive dashboard to complete this book.
For a potential later model, prepare a separate handoff note rather than claiming that six annual observations per country are enough for a robust predictor. Define a possible target, prediction horizon, features available at prediction time and a defensible evaluation split. Identify the evidence you would need before proceeding. Recognizing insufficient data is part of readiness.
Deliverables and assessment
Submit the analysis brief; raw-source identity and provenance; validated and derived tables; data dictionary; executable notebook or script; quality and transformation log; analytical report; and chart/model handoff note. Include instructions that let another reader reproduce the work from the saved inputs without a secret credential or an undocumented manual step.
Assessment uses five equally weighted dimensions: question and observation definition; acquisition and traceability; validation and transformations; interpretation and limitations; and reproducibility and communication. A strong submission justifies its choices and survives its own checks. A runnable notebook with unsupported claims is incomplete, as is a thoughtful report whose numbers cannot be reproduced.
Before considering the capstone finished, restart and run the analysis, inspect its outputs, verify a calculation independently, and ask a second reader to explain your observation unit and main limitation. Keep the revision record. AI may offer feedback after your attempt, but you must defend every transformation and conclusion yourself.
What follows
You are prepared to study visualization when you can supply a tidy, documented table and explain the comparisons a figure should reveal. You are prepared to begin machine-learning and deep-learning study when you also understand feature-target alignment, evaluation boundaries and array dimensions. These are prerequisites for further learning, not a guarantee of modelling competence. The next books build that competence explicitly.