APIs & Web Scraping

Modern research relies on data that increasingly lives online — from newspaper archives and research portals to museum catalogs and digital libraries. Python offers two complementary ways to access this information:

1. APIs

Structured, machine-friendly endpoints that return data (often JSON) in a predictable format. When available, APIs are the safest, cleanest, and most reliable way to obtain data.

2. Web Scraping

Extracting information directly from a webpage’s HTML when no API exists. Scraping is more flexible, but requires careful parsing — and ethical responsibility.

This chapter introduces the full toolkit: - How HTTP works - How to fetch online content - How to parse JSON (API responses) - How to understand and interpret HTML - How to scrape webpages safely and responsibly - How to combine all of these techniques into real-world workflows

At the end, a complete mini-project will show all these concepts working together.