LESSON 105 · DEEPER PYTHON30 min

Compress Text and Bytes with Python gzip

Compress UTF-8 text into gzip bytes and recover the original text.

Before you start: Understand Unicode Text and UTF-8 Bytes · Functions · Exceptions

Example check: Worked and guided code samples are checked against CPython 3.13. Practice uses Pyodide 0.29.5 (Python 3.13) in your browser. How we check lessons

WATCH

Video lesson: Compress Text and Bytes with Python gzip

The preview is stored on this site. YouTube loads only when you press Play. Watch on YouTube

Jump to a chapter

Read the video transcript

Select a timestamp to watch that moment on YouTube.

00:00
What happens when gzip.compress receives the text string hello? Run the call. The error will show why compression starts with bytes and why the decompressed result later needs text decoding. The broken run raises TypeError before producing a gzip member. gzip.compress cannot choose a text encoding on behalf of the caller. Calling str on whatever compressed bytes would eventually result would only create a Python debugging representation, not the original text or a useful wire payload. Imagine receiving a JSON field containing café and saving a compressed response. JSON parsing gives Python a str; that value describes characters, while a gzip
00:44
member must represent bytes. A deliberate text-to-bytes policy belongs between those steps. For UTF-8, the accented letter occupies two bytes even though a person sees one character. If two services choose different character encodings, their compressed members can differ while both appear to contain the same visible word. The TypeError in this tiny run forces us to make the policy explicit before a sender and receiver disagree. The diagnosis encodes hello with UTF-8, prints bytes and the value b'hello', then compresses those bytes. The first two compressed bytes appear as 1f8b in hex, gzip's file signature.
01:28
This makes the boundary clear: character encoding supplies bytes; gzip transforms bytes into another byte sequence. The hex signature identifies the beginning of a gzip container, but says nothing about whether the rest is complete. Five input bytes can yield a larger member because gzip adds a header, compressed body, and trailer. Try a long repeated phrase beside equally long unpredictable bytes: repetition gives the compressor patterns to exploit, while high-entropy data may barely shrink. That is why a compression claim needs representative inputs and measured byte counts. The setting mtime equals zero removes a varying header timestamp from this sample.
02:12
It does not establish that the recovered content is trustworthy or that two different compressor versions produce identical bytes. A file ending in dot gz is only a naming convention; the actual bytes, beginning with the signature shown here, tell us whether it plausibly contains gzip data. The corrected pack_text function uses UTF-8 before compression and sets mtime to zero in the gzip header for a reproducible sample. Decompression reverses the binary transformation, and UTF-8 decoding recovers café. The terminal prints the accented word, demonstrating a real round trip rather than a test that only works for ASCII.
02:55
Count the text boundary carefully: café contains four Unicode characters but five bytes once UTF-8 encodes the accented letter. The compressor sees those five bytes; it has no concept of four characters. Decompression returns the same five bytes, and only then does UTF-8 decoding reconstruct the four-character string. If the original input were an image, the second decode step would be wrong: the recovered bytes should remain binary. Likewise, if a JSON API expects text, the compressed bytes need a separate documented transport representation, such as Base64.
03:35
Keep the order of transformations in the protocol; changing that order cannot be repaired by a decoder guessing later. The verifier constructs a snowman character with chr(0x2603), compresses the text, and prints True when the decoded result matches the source. It also tries to decompress the bytes not gzip; gzip.BadGzipFile is caught and the terminal prints invalid gzip. A real service must additionally bound the decompressed size, because a small compressed input can expand far beyond the memory budget. The snowman adds three UTF-8 bytes, so this round trip exercises non-ASCII text.
04:18
The not-gzip sample fails at the container layer. A different corrupt input might begin with a valid header but end early, producing EOFError; damage to compressed blocks can produce a lower-level decompression error. A valid gzip member can still contain bytes that are invalid UTF-8. For example, compress a single FF byte and decompression succeeds, but the subsequent text decode fails. These are separate checks, so handle decompression and character decoding at their own boundaries. gzip's trailer includes a CRC32 checksum and original- size field to detect accidental corruption.
04:59
Neither authenticates an adversarial payload, because a sender can recompute both. For untrusted input, avoid expanding an arbitrary member into memory at once: use a streaming reader with an application- enforced output limit, then validate the recovered format. A valid image should remain bytes; only a declared text payload should go through UTF-8 decoding.

THE IDEA

Understand the concept

A small report must be sent as gzip-compressed bytes, then reconstructed exactly by another Python process. Compression is a byte operation, while the report begins as text. The two formats need an explicit UTF-8 boundary so that accents and other Unicode characters survive the trip.

The browser starter encodes text but returns the uncompressed bytes. Passing those bytes to gzip.decompress raises gzip.BadGzipFile because they have no gzip header. The video shows a different broken call: passing str directly to gzip.compress raises TypeError because the compressor expects bytes. The source text must be encoded before compression and decoded after decompression.

The diagnosis prints the UTF-8 bytes and their type before asking gzip to compress them. That is the point where text encoding ends and binary compression begins. We use the reverse order on the way back: decompress first, decode second. Use gzip.compress(text.encode('utf-8'), mtime=0) to make a reproducible sample payload, and gzip.decompress(payload).decode('utf-8') to recover the text. The mtime setting affects gzip header metadata; it is not a security feature and does not promise equal compression across every Python or zlib version.

A byte string that is not a gzip member raises an error during decompression, so blindly decoding arbitrary input is unsafe. Test a malformed payload as well as the normal round trip. For untrusted compressed data, also impose a reasonable uncompressed-size limit in the surrounding application. Compression can drastically expand during decompression for specially prepared input. The tiny classroom example does not enforce a production memory budget. Handle invalid data and size limits where the bytes enter a real service, and never treat gzip as encryption.

A small report must be sent as gzip-compressed bytes, then reconstructed exactly by another Python process.
Passing str directly to gzip.compress raises TypeError because the compressor expects bytes.
Use gzip.compress(text.encode('utf-8'), mtime=0) to make a reproducible sample payload, and gzip.decompress(payload).decode('utf-8') to recover the text.
A byte string that is not a gzip member raises an error during decompression, so blindly decoding arbitrary input is unsafe.

WORKED EXAMPLES

See it step by step

Read the code, predict the output, then compare it with the result.

01. Round-trip UTF-8 text

PYTHON

import gzip
source = 'café'
packed = gzip.compress(source.encode('utf-8'), mtime=0)
print(gzip.decompress(packed).decode('utf-8'))

EXPECTED OUTPUT

café

The text survives both boundaries, including its accented character.

02. Inspect the gzip signature

PYTHON

import gzip
packed = gzip.compress(b'hello', mtime=0)
print(packed[:2].hex())
print(len(gzip.decompress(packed)))

EXPECTED OUTPUT

1f8b
5

The magic bytes identify gzip data; decompression restores five source bytes.

03. Reject an invalid member

PYTHON

import gzip
try:
    gzip.decompress(b'not gzip')
except gzip.BadGzipFile:
    print('invalid gzip')

EXPECTED OUTPUT

invalid gzip

The decoder raises on data that does not contain a gzip header.

GO DEEPER

A closer look

Follow the reasoning, inspect each result, then try the suggested changes in the console below.

01 / 03

Observe the first result

A small report must be sent as gzip-compressed bytes, then reconstructed exactly by another Python process. Compression is a byte operation, while the report begins as text. The two formats need an explicit UTF-8 boundary so that accents and other Unicode characters survive the trip.

The text survives both boundaries, including its accented character. The diagnosis prints the UTF-8 bytes and their type before asking gzip to compress them. That is the point where text encoding ends and binary compression begins. We use the reverse order on the way back: decompress first, decode second.

PYTHON

import gzip
source = 'café'
packed = gzip.compress(source.encode('utf-8'), mtime=0)
print(gzip.decompress(packed).decode('utf-8'))

EXPECTED OUTPUT

café

Follow the reasoning

Predict the first sample's printed value and type.
Run it, then compare with the expected output.

02 / 03

Change the rule deliberately

The browser starter encodes text but returns the uncompressed bytes. Passing those bytes to gzip.decompress raises gzip.BadGzipFile because they have no gzip header. The video shows a different broken call: passing str directly to gzip.compress raises TypeError because the compressor expects bytes. The source text must be encoded before compression and decoded after decompression.

The magic bytes identify gzip data; decompression restores five source bytes. Use gzip.compress(text.encode('utf-8'), mtime=0) to make a reproducible sample payload, and gzip.decompress(payload).decode('utf-8') to recover the text. The mtime setting affects gzip header metadata; it is not a security feature and does not promise equal compression across every Python or zlib version. Encode text, then call gzip.compress(encoded, mtime=0). Do not decode the compressed result.

PYTHON

import gzip
packed = gzip.compress(b'hello', mtime=0)
print(packed[:2].hex())
print(len(gzip.decompress(packed)))

EXPECTED OUTPUT

1f8b
5

Follow the reasoning

Identify the line that changes the rule.
Run the corrected sample with the same input.

03 / 03

Verify the boundary

The diagnosis prints the UTF-8 bytes and their type before asking gzip to compress them. That is the point where text encoding ends and binary compression begins. We use the reverse order on the way back: decompress first, decode second. Use gzip.compress(text.encode('utf-8'), mtime=0) to make a reproducible sample payload, and gzip.decompress(payload).decode('utf-8') to recover the text. The mtime setting affects gzip header metadata; it is not a security feature and does not promise equal compression across every Python or zlib version.

The decoder raises on data that does not contain a gzip header. A byte string that is not a gzip member raises an error during decompression, so blindly decoding arbitrary input is unsafe. Test a malformed payload as well as the normal round trip. For untrusted compressed data, also impose a reasonable uncompressed-size limit in the surrounding application. Compression can drastically expand during decompression for specially prepared input. The tiny classroom example does not enforce a production memory budget. Handle invalid data and size limits where the bytes enter a real service, and never treat gzip as encryption.

PYTHON

import gzip
try:
    gzip.decompress(b'not gzip')
except gzip.BadGzipFile:
    print('invalid gzip')

EXPECTED OUTPUT

invalid gzip

Follow the reasoning

Try the documented edge case.
Explain why the output satisfies the function contract.

HANDS-ON

Try it in Python

Edit the example and run it. Python starts in your browser the first time you click Run.

Python console

Ready to run

import gzip

def pack_text(text):
    return text.encode('utf-8')

print(gzip.decompress(pack_text('café')).decode('utf-8'))

Need input()? Add one value per line

Input lines

Ctrl / ⌘ + Enter to run

OUTPUT

Your output appears here.

Your turnMake pack_text(text) return gzip-compressed UTF-8 bytes using mtime=0. The prepared call should print café after decompression; the result must begin with the gzip magic bytes.

Need a hint?

Encode text, then call gzip.compress(encoded, mtime=0). Do not decode the compressed result.

Complete the task and select Check task to verify your code.

DEBUG BETTER

Typical mistakes

Everyone meets these errors. See what causes them and how to fix them.

The visible failure

COMMON MISTAKE

gzip.compress('hello')

FIX

gzip.compress('hello'.encode('utf-8'), mtime=0)

What happens: Passing str directly to gzip.compress raises TypeError because the compressor expects bytes.
Use gzip.compress(text.encode('utf-8'), mtime=0) to make a reproducible sample payload, and gzip.decompress(payload).decode('utf-8') to recover the text. The mtime setting affects gzip header metadata; it is not a security feature and does not promise equal compression across every Python or zlib version.

Missing the edge condition

COMMON MISTAKE

gzip.decompress(b'not gzip').decode('utf-8')

FIX

validate and limit the compressed input before consuming it

What happens: A byte string that is not a gzip member raises an error during decompression, so blindly decoding arbitrary input is unsafe.
Compression can drastically expand during decompression for specially prepared input. The tiny classroom example does not enforce a production memory budget. Handle invalid data and size limits where the bytes enter a real service, and never treat gzip as encryption.

Browse this lesson’s problem fixes

VERIFY THE STORY

Sources and further reading

Python gzip documentation