Pack Binary Records with Python struct
Pack an unsigned identifier and status byte into a stable three-byte record.
Example check: Worked and guided code samples are checked against CPython 3.13. Practice uses Pyodide 0.29.5 (Python 3.13) in your browser. How we check lessons
Video lesson: Pack Binary Records with Python struct
The preview is stored on this site. YouTube loads only when you press Play. Watch on YouTube
Jump to a chapter
Read the video transcript
Select a timestamp to watch that moment on YouTube.
- 00:00
Our packet format has a two-byte identifier and one status byte. What will struct.unpack with format greater-than H B do with a buffer containing only one byte? Run it and inspect the length error. The broken program calls unpack with a one-byte buffer. The terminal raises struct.error because the format requires exactly three bytes. The missing bytes are not optional fields, and Python does not invent zeros. That strictness is useful when detecting a truncated packet before using a status that was never received. This failure can occur even when the sender intended to transmit a valid record.
- 00:40
A network read can return only the first byte of a longer stream because message boundaries and read boundaries differ. The receiver must accumulate the declared record length before parsing. If the stream ends after that first byte, it is genuinely truncated; if the remaining two bytes arrive later, the record can still be complete. Do not pad the short input with zeros to make the exception disappear, because that would change the identifier or invent a status the sender never transmitted. The diagnosis calls calcsize on greater- than H B and prints three required bytes beside one received byte. The greater-than sign selects big-endian order; H consumes an unsigned two-byte
- 01:23
value and B consumes an unsigned one-byte value. Those choices define a portable protocol layout instead of relying on the host machine's native alignment. Let calcsize drive the exact number of bytes to buffer before parsing. If a future protocol revision adds another field, updating its format string then updates the expected record length too. A network receiver should keep reading until it has that many bytes or reaches a definite end of stream; one read call is not a record guarantee. Byte order is a separate contract. To see why the greater-than marker matters, interpret the first two bytes of
- 02:04
an example packet in both orders: their numeric meanings differ even when the byte count is correct. The unsigned H field allows zero through 65535, while a signed h would assign negative meanings to values with the top bit set. Choose those conventions from the protocol specification, not from the machine running the reader. The fixed pack_record takes identifier 1023 and status 1. The terminal prints hex 03ff01 and then the recovered tuple 1023, 1. In hex, 03ff is the identifier and 01 is the status.
- 02:45
Packing and unpacking use the same explicit format, so the byte layout can be checked independently of the current computer. An independent reader can check the packet without using the same struct format twice. Taking the first two bytes and calling int.from_bytes with big-endian order recovers 1023; treating those same bytes as little-endian would produce 65283. That is an interoperability test, not merely a round trip between two Python calls that might share the same mistaken convention. Test a zero identifier and the maximum identifier too. Packing numbers outside the H and B ranges raises struct.error, while a represented
- 03:29
status may still violate a narrower business vocabulary. Document both the byte layout and the allowed status meanings. The boundary run packs the largest values for these fields, 65535 and 255, and reports a three-byte packet that unpacks to the same pair. Then it removes one byte and catches struct.error. Range checks and length checks are application duties; a binary packet format also needs versioning and framing when used beyond this tiny demonstration. This test separates two forms of validity. At the wire level, 65535 fits in H and 255 fits in B, so both can be packed into
- 04:13
exactly three bytes. At the application level, a status vocabulary might permit only zero for pending and one for complete; the structurally valid 255 would then be rejected by a different validator. Identifier zero might also be reserved even though it fits the field. When messages are concatenated, unpack requires an exact three-byte slice, whereas unpack_from can read at a documented offset in a larger buffer. The receiver needs to know where each record begins. A version field or length prefix can support later format changes, and an
- 04:54
integrity check can detect damage beyond the short-buffer case. Test transport framing, numeric ranges, and business rules independently so that a passing pack-unpack round trip cannot conceal a mismatch.
Understand the concept
A device packet contains a two-byte unsigned identifier followed by a one-byte status. Python's struct module turns those fields into an exact byte layout. The format string must state both the field widths and byte order so a reader on another machine can decode the same record.
The browser starter tries bytes([1023, 1]), which raises ValueError because 1023 cannot fit in one byte. A separate video demo unpacks a one-byte buffer with a three-byte format and raises struct.error. Both failures show why the protocol's field widths and total length must be explicit. Assuming that a truncated packet will be padded or partially decoded would hide data loss.
struct.calcsize('>HB') reports three bytes: big-endian unsigned short followed by unsigned byte. The greater-than prefix makes byte order explicit instead of depending on the current machine. Inspect the encoded bytes with bytes.hex() and check the full length before attempting unpack. Use struct.pack('>HB', identifier, status) to build the packet and struct.unpack('>HB', packet) to recover a tuple. Range-check integers against the format before packing if user input is involved. An unsigned short supports 0 through 65535 and an unsigned byte supports 0 through 255.
The verifier attempts to decode a truncated record and deliberately catches struct.error. That failure is preferable to silently inventing the missing status. It also checks the hex layout for identifier 1023 and status 1, making the selected byte order visible. A binary protocol must define length, byte order, version and validation explicitly. Native struct formats may add platform-dependent padding, so avoid an implicit native format for data exchanged between systems. The example is a local three-byte protocol, not a network framing implementation.
- A device packet contains a two-byte unsigned identifier followed by a one-byte status.
- Unpacking a one-byte buffer with a three-byte format raises struct.error.
- Use struct.pack('>HB', identifier, status) to build the packet and struct.unpack('>HB', packet) to recover a tuple.
- The verifier attempts to decode a truncated record and deliberately catches struct.error.
See it step by step
Read the code, predict the output, then compare it with the result.
01. Calculate the protocol size
import struct
print(struct.calcsize('>HB'))
print(struct.pack('>HB', 1023, 1).hex())
3
03ff01Three bytes hold the two fields, with the most significant identifier byte first.
02. Decode a packet
import struct
packet = bytes.fromhex('03ff01')
print(struct.unpack('>HB', packet))
(1023, 1)The same format string recovers the identifier and status as a tuple.
03. Reject a truncated packet
import struct
try:
struct.unpack('>HB', b'\x03')
except struct.error:
print('truncated packet')
truncated packetA short record fails clearly rather than supplying invented field values.
A closer look
Follow the reasoning, inspect each result, then try the suggested changes in the console below.
Observe the first result
A device packet contains a two-byte unsigned identifier followed by a one-byte status. Python's struct module turns those fields into an exact byte layout. The format string must state both the field widths and byte order so a reader on another machine can decode the same record.
Three bytes hold the two fields, with the most significant identifier byte first. struct.calcsize('>HB') reports three bytes: big-endian unsigned short followed by unsigned byte. The greater-than prefix makes byte order explicit instead of depending on the current machine. Inspect the encoded bytes with bytes.hex() and check the full length before attempting unpack.
import struct
print(struct.calcsize('>HB'))
print(struct.pack('>HB', 1023, 1).hex())
3
03ff01- Predict the first sample's printed value and type.
- Run it, then compare with the expected output.
Change the rule deliberately
The browser starter tries bytes([1023, 1]), which raises ValueError because 1023 cannot fit in one byte. A separate video demo unpacks a one-byte buffer with a three-byte format and raises struct.error. Both failures show why the protocol's field widths and total length must be explicit. Assuming that a truncated packet will be padded or partially decoded would hide data loss.
The same format string recovers the identifier and status as a tuple. Use struct.pack('>HB', identifier, status) to build the packet and struct.unpack('>HB', packet) to recover a tuple. Range-check integers against the format before packing if user input is involved. An unsigned short supports 0 through 65535 and an unsigned byte supports 0 through 255. Import struct. The format >HB means big-endian, unsigned short, then unsigned byte.
import struct
packet = bytes.fromhex('03ff01')
print(struct.unpack('>HB', packet))
(1023, 1)- Identify the line that changes the rule.
- Run the corrected sample with the same input.
Verify the boundary
struct.calcsize('>HB') reports three bytes: big-endian unsigned short followed by unsigned byte. The greater-than prefix makes byte order explicit instead of depending on the current machine. Inspect the encoded bytes with bytes.hex() and check the full length before attempting unpack. Use struct.pack('>HB', identifier, status) to build the packet and struct.unpack('>HB', packet) to recover a tuple. Range-check integers against the format before packing if user input is involved. An unsigned short supports 0 through 65535 and an unsigned byte supports 0 through 255.
A short record fails clearly rather than supplying invented field values. The verifier attempts to decode a truncated record and deliberately catches struct.error. That failure is preferable to silently inventing the missing status. It also checks the hex layout for identifier 1023 and status 1, making the selected byte order visible. A binary protocol must define length, byte order, version and validation explicitly. Native struct formats may add platform-dependent padding, so avoid an implicit native format for data exchanged between systems. The example is a local three-byte protocol, not a network framing implementation.
import struct
try:
struct.unpack('>HB', b'\x03')
except struct.error:
print('truncated packet')
truncated packet- Try the documented edge case.
- Explain why the output satisfies the function contract.
Try it in Python
Edit the example and run it. Python starts in your browser the first time you click Run.
Python console
Ready to runNeed input()? Add one value per line
Your output appears here.
Need a hint?
Import struct. The format >HB means big-endian, unsigned short, then unsigned byte.
Complete the task and select Check task to verify your code.
Quick quiz
Three questions. You can change your answers and try again.
Typical mistakes
Everyone meets these errors. See what causes them and how to fix them.
The visible failure
struct.unpack('>HB', b'\x03')if len(packet) == struct.calcsize('>HB'): struct.unpack('>HB', packet)What happens: Unpacking a one-byte buffer with a three-byte format raises struct.error.
Use struct.pack('>HB', identifier, status) to build the packet and struct.unpack('>HB', packet) to recover a tuple. Range-check integers against the format before packing if user input is involved. An unsigned short supports 0 through 65535 and an unsigned byte supports 0 through 255.
Missing the edge condition
struct.pack('HB', identifier, status)struct.pack('>HB', identifier, status)What happens: The verifier attempts to decode a truncated record and deliberately catches struct.error.
A binary protocol must define length, byte order, version and validation explicitly. Native struct formats may add platform-dependent padding, so avoid an implicit native format for data exchanged between systems. The example is a local three-byte protocol, not a network framing implementation.