PythonLessonLab
LESSON 44 · PRACTICE PROJECT28 min

Project: Analyze Application Logs

Turn multiline log text into severity counts, error messages, and a count of malformed rows.

Before you start: Basic String Operations · Loops · Dictionaries · Functions

Example check: Worked and guided code samples are checked against CPython 3.13. Practice uses Pyodide 0.29.5 (Python 3.13) in your browser. How we check lessons

WATCH

Video lesson: Project: Analyze Application Logs

The preview is stored on this site. YouTube loads only when you press Play. Watch on YouTube

Jump to a chapter

Read the video transcript

Select a timestamp to watch that moment on YouTube.

  1. 00:00

    Imagine that a small Python service wrote a few log lines while handling a request. One line says the worker started, another says a connection timed out, and a later line says a retry failed. Reading five lines is easy. Reading five thousand is not. In this mini project we will turn log text into a report: the number of entries at each severity level, a list of error messages in the order they appeared, and a count of malformed rows we could not safely interpret. Before we write code, predict the sample result: two errors, one information message, one warning, and one broken row. We will work with a string in the website's browser console.

  2. 00:42

    Nothing is uploaded from your computer, and we are not claiming to parse every possible logging format. Our explicit contract is a timestamp, a level, and a message separated by pipes. The interesting bug comes when the message itself contains a pipe. Can a simple split still tell us which part is the message? Let us run the naive parser and see. The run failed before it reached the print statement. Python split the line at every pipe, so it produced four pieces. On the left side we supplied only three names: timestamp, level, and message. That mismatch is exactly what the ValueError reports. The log line itself is not necessarily wrong.

  3. 01:24

    The service may legitimately include a pipe in a free-form message, for example to distinguish a retry from its attempt number. Our parser made an incorrect assumption about how many separators could occur. We could catch ValueError and discard this row, but then we would lose an important error event. We could also take the first three pieces and ignore the rest, but that would truncate the message. The better fix follows the format: only the first two pipes separate fields. Every later pipe belongs to the message. When debugging a parser, look at the intermediate pieces before changing multiple lines of code. The exception tells us where the assumption broke; inspecting the split result tells us how to repair it.

  4. 02:10

    The second argument to split is a maximum number of splits. With a maximum of two, Python separates the timestamp, then the level, and leaves the rest as one message. We strip surrounding whitespace from each part because the human-readable format places spaces around the separators. This is a format-specific choice, not a universal parser. We must still verify that there really are three fields and that none is empty. A row containing just broken text has one part. A row ending with a pipe has an empty message after stripping. Neither should silently count as an event. Blank lines are different: they often separate blocks in pasted text, and our chosen rule is to ignore them.

  5. 02:54

    For malformed nonblank lines, we will increment a skipped counter. That counter makes missing data visible in the final report. A new severity such as DEBUG should be counted too; hard-coding only INFO and ERROR would make the tool brittle. Our dictionary can create a count on first sight with get(level, zero) plus one. We now have one pass over the text. splitlines gives one candidate record at a time. We ignore blank lines, split a nonblank line at most twice, strip each field, and reject a row with the wrong shape. The timestamp variable is present to enforce the three-field contract, but this small lesson does not validate whether it represents a real date.

  6. 03:38

    For a valid row we normalize the level to uppercase, then increment its dictionary count. The first ERROR starts at zero and becomes one. The second reads one and becomes two. When the normalized level is ERROR, we append its message to the errors list. A list preserves the order in which errors appeared; sorting the dictionary keys at print time only makes the counts easier to scan. Run the code and compare every printed value to your prediction. Notice that Retry | attempt 2 remains intact, and the broken row appears in Skipped instead of becoming a fake severity. Returning the values rather than printing inside the function makes the analyzer reusable.

  7. 04:22

    A good sample report is not enough to trust a parser. The first assertion verifies that empty input yields an empty summary and no skipped rows. The second verifies two behaviors at once: lowercase error becomes the ERROR key, and a later pipe remains in the message. The third checks that a new DEBUG level is accepted while a malformed row increments skipped. All three assertions passed in the real Python run. On the companion page, the starter code leaves the main parsing loop unfinished. Implement that loop, run the report, and select Check task. The checker also tries fields with extra whitespace, empty data, a new level, and an error message containing a pipe.

  8. 05:06

    That means simply hard-coding the sample counts will not pass. Finish the quiz after the task to mark the mini project complete. For a follow-up, try reading a local text file line by line. A serious log analyzer also needs a precise format, timestamp validation, and a policy for multiline stack traces, but this project gives you the key habit: verify input shape before trusting a summary.

THE IDEA

Understand the concept

Application logs are useful only when you can find the events that matter. This project accepts log text in memory and summarizes lines shaped like timestamp | level | message. It counts each severity level, collects ERROR messages in source order, and records malformed nonblank rows separately. You can inspect every intermediate value in the browser console; no server logs or private files are uploaded.

Use splitlines() to process one record at a time. Splitting a row with split('|', 2) makes at most three fields, so a message such as Retry | attempt 2 stays in the message field. Strip whitespace from each field and require all three to be nonempty. Normalize the level with upper() before using it as a dictionary key. A fixed set of levels is unnecessary: the same accumulator can count INFO, WARNING, ERROR, and a new DEBUG level.

The return value separates data from presentation: a counts dictionary, an ordered list of error messages, and a skipped-row count. The caller sorts dictionary keys for a stable report. This teaching parser checks only the three-field shape, not whether a timestamp is a valid date. A production analyzer would define a formal log format, validate timestamps, consider multiline stack traces, and stream large files instead of loading all text into memory.

  • str.splitlines processes each physical line
  • split('|', 2) preserves extra separators in a message
  • dict.get accumulates counts for any severity level
  • A returned summary keeps parsing separate from printing
WORKED EXAMPLES

See it step by step

Read the code, predict the output, then compare it with the result.

01. Keep separators inside the message

PYTHON
line = "09:10 | ERROR | Retry | attempt 2"
parts = [part.strip() for part in line.split("|", 2)]
print(parts)
print(len(parts))
EXPECTED OUTPUT
['09:10', 'ERROR', 'Retry | attempt 2']
3

The maximum of two splits produces three fields. The second pipe inside the message stays there instead of creating a fourth field.

02. Count levels dynamically

PYTHON
levels = ["INFO", "ERROR", "INFO", "DEBUG"]
counts = {}
for level in levels:
    counts[level] = counts.get(level, 0) + 1
for level in sorted(counts):
    print(level, counts[level])
EXPECTED OUTPUT
DEBUG 1
ERROR 1
INFO 2

get supplies zero for an unseen level, so the same loop also counts a new DEBUG event. Sorting affects display, not the stored counts.

03. Separate malformed rows from empty lines

PYTHON
text = "t | INFO | Ready\n\nt | ERROR | \nbroken"
skipped = 0
for line in text.splitlines():
    if not line.strip():
        continue
    parts = [part.strip() for part in line.split("|", 2)]
    if len(parts) != 3 or not all(parts):
        skipped += 1
print(f"Skipped: {skipped}")
EXPECTED OUTPUT
Skipped: 2

The blank separator line is ignored. The missing message and the line without separators are malformed nonblank records.

GO DEEPER

A closer look

Follow the reasoning, inspect each result, then try the suggested changes in the console below.

01 / 03

Read the record format before writing a parser

A log line has three fields in this project: a timestamp, a severity level, and a message. The pipe character separates fields. Before counting anything, split one representative line and inspect the pieces. The message may itself contain a pipe, so use a maximum of two splits. A plain split('|') would produce four values for the first example below and make three-variable unpacking fail.

Trim each field after splitting because human-readable log formats commonly put spaces around separators. Check that exactly three nonempty parts remain. This exercise treats any nonempty timestamp text as a field; it does not claim to verify date syntax. Keeping that limit explicit avoids silently promising a production-grade parser.

PYTHON
line = "2026-10-13 09:01 | ERROR | Retry | attempt 2"
parts = [part.strip() for part in line.split("|", 2)]
print(parts)
print(len(parts) == 3 and all(parts))
EXPECTED OUTPUT
['2026-10-13 09:01', 'ERROR', 'Retry | attempt 2']
True
Follow the reasoning
  1. The first split isolates the timestamp and the second isolates the level.
  2. Everything after that second separator remains in the message.
  3. Try an empty message, then explain why all(parts) becomes false.
02 / 03

Count every valid level and keep errors in order

Once a row is valid, convert its level to uppercase so error and ERROR refer to the same category. A dictionary maps each level to a count. counts.get(level, 0) returns the previous count or zero for a new level; adding one handles both the first and later occurrences without a list of allowed levels.

Keep the error messages in a separate list. append preserves the order in which ERROR lines appeared, which is useful when reviewing a sequence of failures. Sorting the dictionary only when printing provides a stable report while leaving the messages in event order. A new DEBUG row works without a code change.

PYTHON
rows = [("info", "Started"), ("ERROR", "Timed out"), ("error", "Retry failed"), ("DEBUG", "Probe")]
counts = {}
errors = []
for level, message in rows:
    level = level.upper()
    counts[level] = counts.get(level, 0) + 1
    if level == "ERROR":
        errors.append(message)
for level in sorted(counts):
    print(level, counts[level])
print(errors)
EXPECTED OUTPUT
DEBUG 1
ERROR 2
INFO 1
['Timed out', 'Retry failed']
Follow the reasoning
  1. Both spellings of error increment the same ERROR key.
  2. Only ERROR messages go into the ordered review list.
  3. Add a WARNING row and predict the report order before running.
03 / 03

Make malformed data visible without inventing events

Real input has blank separators, truncated records, and unexpected content. This project ignores blank lines but counts malformed nonblank rows in skipped. That separate count matters: simply discarding bad rows could make a summary look complete when it was not. Validate before unpacking, then continue to the next line when a record fails the contract.

The complete task returns (counts, errors, skipped), which lets a caller print a report or inspect values directly. For a huge local log, reading the file line by line would use less memory than receiving all text at once. For multiline exception traces, a more formal grammar is needed; this three-field demonstration intentionally treats each physical line as one candidate record.

PYTHON
text = "t | INFO | Ready\n\nt | ERROR | \nbroken"
skipped = 0
valid = []
for raw_line in text.splitlines():
    if not raw_line.strip():
        continue
    parts = [part.strip() for part in raw_line.split("|", 2)]
    if len(parts) != 3 or not all(parts):
        skipped += 1
        continue
    valid.append(parts)
print(valid)
print(f"Skipped: {skipped}")
EXPECTED OUTPUT
[['t', 'INFO', 'Ready']]
Skipped: 2
Follow the reasoning
  1. The empty line is ignored rather than marked malformed.
  2. The missing ERROR message and separator-free line each increment skipped.
  3. Change the final row to t | ERROR | Retry failed and predict the new counts.
HANDS-ON

Try it in Python

Edit the example and run it. Python starts in your browser the first time you click Run.

Python console

Ready to run
Need input()? Add one value per line
Ctrl / ⌘ + Enter to run
OUTPUT
Your output appears here.
Your turnComplete summarize_log(log_text). Ignore blank lines. Split each nonblank line into at most three parts on |, strip every part, and count it as skipped if the timestamp, level, or message is missing. Normalize the level to uppercase, count each valid level, and append messages from ERROR rows. Return (counts, errors, skipped). The prepared report must show ERROR: 2, INFO: 1, WARNING: 1, Skipped: 1, and both error messages in order.
Need a hint?

Try parts = [part.strip() for part in raw_line.split('|', 2)]. After checking len(parts) == 3 and all(parts), unpack timestamp, level, message. Set level = level.upper(); use counts.get(level, 0) + 1; append message only when level == 'ERROR'.

Complete the task and select Check task to verify your code.

    KNOWLEDGE CHECK

    Quick quiz

    Three questions. You can change your answers and try again.

    01.Why use split('|', 2) instead of split('|') for this format?

    02.What should a blank line do in this project's summary?

    03.Why normalize the level before counting it?

    DEBUG BETTER

    Typical mistakes

    Everyone meets these errors. See what causes them and how to fix them.

    Splitting every pipe

    COMMON MISTAKE
    parts = line.split("|")
timestamp, level, message = parts
    FIX
    parts = [part.strip() for part in line.split("|", 2)]
if len(parts) != 3 or not all(parts):
    raise ValueError("Malformed log row")
timestamp, level, message = parts

    What happens: A message containing a pipe creates four parts and unpacking raises ValueError.
    Limit the split to two separators and validate the three fields before unpacking.

    Replacing a previous level count

    COMMON MISTAKE
    counts[level] = 1
    FIX
    counts[level] = counts.get(level, 0) + 1

    What happens: The report says ERROR: 1 even after several ERROR records.
    Add one to the previous subtotal rather than assigning the initial value every time.

    Treating malformed rows as events

    COMMON MISTAKE
    level = raw_line.split("|")[1].strip()
    FIX
    parts = [part.strip() for part in raw_line.split("|", 2)]
if len(parts) != 3 or not all(parts):
    skipped += 1
    continue

    What happens: A row without a separator raises IndexError; an empty level can create a misleading category.
    Validate the record shape and required fields before using their positions.

    Browse this lesson’s problem fixes

    PROJECT SOURCE

    Explore the runnable code

    The GitHub project includes setup instructions and source code you can run on your computer. Use this lesson for the guided examples, browser task, and quiz.

    View this project on GitHub

    VERIFY THE STORY

    Sources and further reading