BeautifulSoup parse table: extract HTML rows into Python records

Select a timestamp to watch that moment on YouTube.

1. 00:00

   A page has two tables and only one contains the price data we need. If we collect every row on the page, unrelated records join the result. Predict what the first run will print. Beautiful Soup parses an HTML document already supplied to it; it does not download the page. We use a fixed local string so you can reproduce every step and focus on selectors, headers, cell text, and row validation. BeautifulSoup parse table is a common next step after downloading an HTML page. Beautiful Soup parses the markup; it does not fetch the page for you. Start with a small local HTML string so selection and extraction can be tested

2. 00:41

   without internet access. Install beautifulsoup4 for local Python, create BeautifulSoup(html, 'html.parser'), and select the specific table by id or another stable attribute. A broad `find_all('tr')` on the entire document may collect rows from unrelated tables. A page may contain layout tables, historical data, and the table you actually need. Parse one known HTML string with BeautifulSoup and select the target by an id or another stable attribute before reading rows. This prevents a global `find_all('tr')` from mixing unrelated records.

3. 01:23

   It also makes the first failure obvious when the target table is absent. The broken run asks the whole soup object for every tr. It prints Old and Tea together. No exception warns us that the first row belongs to a different table. That kind of quiet data mixing is more dangerous than a syntax error because the output can look plausible. Before iterating, identify a stable marker for the table you actually need. Here we use id=prices, which is simple enough to inspect in the fixture. A table row can contain th header cells, td data cells, or both. Read the header from the chosen table, then map each subsequent row to columns.

4. 02:06

   `get_text(' ', strip=True)` inserts spaces between nested text nodes and trims surrounding whitespace. It does not convert numbers, dates, or currency automatically. If a row has fewer cells than headers, do not silently zip away missing columns. If it has extra cells, decide whether to reject or handle them explicitly. This guide rejects nonmatching data rows for a predictable teaching contract. Beautiful Soup accepts HTML supplied by your program; it does not download a URL. Keeping the fixture in this lesson local avoids network variability.

5. 02:46

   On a real site, inspect the returned HTML, not only the rendered browser view: JavaScript may add rows after the static response, or the data may be paginated. If the HTML lacks the rows, another data source is needed. Now select #prices first and search inside it. We see only its header and data row. Read th for column names and td for values; get_text with a space separator and strip=True handles extra whitespace around nested text. It still returns strings, so a value such as 2.50 is not yet a float. Decide where numeric conversion belongs after structure is valid.

6. 03:28

   On a real site, inspect the returned HTML because JavaScript may add rows later and a static parser may see none. Use `table.select('tr')` only after selecting the correct table; a nested table can still yield descendant rows. For a straightforward flat table fixture, this is enough. For an uncontrolled site, inspect direct children or identify a specific tbody, and examine actual markup in developer tools. Never assume every site provides semantic table markup: some use div-based grids, JavaScript rendering, or pagination. A static parser cannot invent rows that are not present in the HTML response.

7. 04:12

   A header row usually uses th cells, while data rows use td. Read those separately and compare the number of data cells with the number of headers. `get_text(' ', strip=True)` joins nested text with spaces and removes surrounding whitespace, preserving a usable string. It does not infer numeric types; the price 2.50 is still text at this stage. The corrected function scopes selection to the requested table, checks for a header, reads direct child th and td cells, and compares row widths before zipping them into dictionaries. This rejects a row that would otherwise lose a column silently.

8. 04:52

   It also checks duplicate or empty headers because either would make dictionary keys ambiguous. These choices form one clear extraction contract for a flat table. A nested or irregular table may need different traversal rules. The browser lesson runs Beautiful Soup on bundled HTML and checks a function that returns dictionaries. It does not scrape a live site, so outputs are deterministic and there is no site- specific crawling policy hidden in the example. Compare the parsed text with the source and confirm that header names are exactly what you expect before converting cells. For a real site, obtain permission or follow its terms, use reasonable request

9. 05:35

   rates, and prefer a documented data API when available. The offline web crawler project covers page discovery; this lesson focuses narrowly on extracting a table from one HTML document. The example uses direct child cells. This avoids accidentally taking a cell from a nested table when one appears inside another cell. For an uncontrolled document, inspect tbody and table structure before assuming a simple direct row selection. The browser task rejects rows with missing or extra cells instead of silently truncating them with zip. The final run supplies a ragged row and gets ValueError.

10. 06:18

    Change the fixture to add a second complete row and predict the record list. For live acquisition, this parsing function belongs after a separate HTTP client with timeouts, modest request rates, and attention to site terms. A documented API is often better than scraping unstable markup. This lesson is about one HTML document in memory; the linked offline crawler lesson covers navigation and page discovery without hitting third-party websites. Once a flat table's headers and cells are validated, dict(zip(headers, values)) creates a record for each row. Keep the original string values until a separate conversion stage can report

11. 07:01

    invalid prices or dates with row context. An empty table, missing header, duplicate header, or ragged row should have an explicit policy instead of producing a misleading partial result. For a live scraping workflow, add a responsible acquisition layer outside this function: an HTTP client, timeouts, rate limits, and the site's usage rules. Prefer a documented API when one exists. This extraction function is deliberately limited to an HTML document already in memory, which makes its tests quick and stable. Compare the returned records to the original markup after any site redesign.