Count and Group Data with Counter and defaultdict

Select a timestamp to watch that moment on YouTube.

1. 00:00

   Imagine a small issue tracker exporting labels such as bug, idea, and question. We need a frequency report and, later, a way to collect the actual messages under each label. A plain dictionary can do both jobs, but we have to decide what a missing key means. On the first encounter with bug, there is no count to increment. If we simply add one to a dictionary entry that does not exist, Python raises KeyError. That failure is useful because it shows us the exact assumption our code made. In this lesson we will run the broken code, choose Counter for frequencies, and use defaultdict when each category needs a growing list.

2. 00:41

   We will also test an empty input and a label that never appeared. Both tools live in the standard-library collections module, so no package installation is needed. The companion lesson on PythonLessonLab has a runnable browser console, worked examples, a checked coding task, a quiz, and common mistakes. Follow along by predicting each output before we run it. A counting task may look trivial, but the same pattern appears in log summaries, product inventories, word frequencies, and data cleaning. The key is to make our missing-key policy explicit instead of relying on an accidental dictionary state.

3. 01:21

   The very first label is bug. `counts['bug']` attempts to retrieve a value before `+= 1` can store an updated one. Because the dictionary is empty, that lookup raises KeyError. The loop never gets to idea or question, and the dictionary remains empty. One valid manual repair would be `counts[label] = counts.get(label, 0) + 1`. Another would test `if label not in counts` and initialize the count first. Those are useful techniques, especially when you need custom behavior. For a straightforward tally, Counter expresses the intent more directly. It can consume an iterable of labels and build counts for us.

4. 02:03

   Do not respond to the error by wrapping the entire loop in a broad exception handler and returning an incomplete report. The real problem is the missing initial value, so correct that rule where the update happens. Notice also that counting is different from grouping. A count tells us how many bug labels occurred. A group might store the complete bug messages. If we use one tool for both jobs without thinking about the stored value, the code becomes harder to understand. We will inspect each abstraction separately before combining them into a useful report. Counter is a dictionary subclass designed for tallying hashable items.

5. 02:44

   It stores two bugs and one idea. Reading a key that was never counted returns zero, which is natural for a frequency question. That read does not mean a count was inserted into the underlying mapping. A defaultdict behaves differently. We supplied the factory `list`, without parentheses. On a missing-key read, it calls that factory, stores a new empty list under the key, and returns it. That is why `groups['missing']` prints an empty list and the final membership check prints true. This side effect matters. If all you want to know is whether a category exists, test membership before reading it, or use `groups.get(label)` with a suitable fallback.

6. 03:27

   Do not create empty categories merely by inspecting them. Counter values can also be changed, but its purpose is counts; defaultdict can use factories such as list, set, or int for other accumulation patterns. The factory must be callable. Passing `list()` would supply one already-created list rather than the list constructor, and defaultdict would reject it. Decide whether you need numbers or collections first. Then the code can name the appropriate intent clearly for a future reader. This code makes two separate promises. `counts` answers how many issues have each normalized category. `titles_by_category` answers which original titles belong under that category.

7. 04:12

   We use strip and lower so a category written as space, Bug, space is counted together with lowercase bug. Counter builds the frequency map from those normalized values, while defaultdict creates a fresh title list for each category. Returning ordinary dictionaries gives the caller predictable lookup behavior; a missing category in the returned groups raises KeyError rather than silently creating an empty group. The counts dictionary also needs an explicit get call for a missing label because it no longer has Counter's special zero-on-missing lookup. That conversion is part of the website task's contract.

8. 04:52

   We make the category list once, then pair each issue with its normalized category using zip. The function does not edit any issue dictionary or title. The prepared example on PythonLessonLab uses this exact function name and expects these two lines, so you can run the code and then use the practical checker. Try changing feature to FEATURE and predict the new grouping. A correct solution handles first occurrences without a special-case error and preserves the order in which titles arrive within each group. These checks are more valuable than merely seeing a plausible count on the happy path. Two bug titles remain in their input order after the categories are normalized, and the feature list stays separate.

9. 05:38

   The source issue dictionaries remain unchanged. An unseen category is not inserted into either returned dictionary; to read its frequency as zero, the caller uses `counts.get('missing', 0)`. Empty input returns two empty ordinary dictionaries rather than raising an exception or inventing a category. Appending a new bug title does not affect the feature list, confirming that defaultdict created distinct lists per key before conversion. When you complete the PythonLessonLab task, run its checker and quiz, then experiment with your own categories. Consider Bug and bug with extra spaces: the explicit strip and lower policy combines them.

10. 06:22

    That is a domain decision, not something Counter decides by itself. A missing category field in a record would still need a separate input policy. You might reject that record, map it to Unknown, or skip it, depending on the application. Counter and defaultdict remove repetitive initialization code; they do not remove the need to define what counts and groups mean. If a future program fails on a missing dictionary key, ask whether you need a safe lookup, a numeric tally, or an automatically created collection. The right answer follows from the data you intend to store.