LESSON 110 · DEEPER PYTHON30 min

Benchmark Tiny Python Functions with timeit

Compute a per-call benchmark from repeated timeit runs without claiming universal speed.

Before you start: Functions · Loops · Basic Operators

Example check: Worked and guided code samples are checked against CPython 3.13. Practice uses Pyodide 0.29.5 (Python 3.13) in your browser. How we check lessons

WATCH

Video lesson: Benchmark Tiny Python Functions with timeit

The preview is stored on this site. YouTube loads only when you press Play. Watch on YouTube

Jump to a chapter

Read the video transcript

Select a timestamp to watch that moment on YouTube.

00:00
This single sum of ten numbers prints a tiny, machine-dependent duration. Is one call enough evidence to rank methods? Run it and inspect what was actually measured. The broken program measures a single tiny operation. It prints a plausible floating-point duration, but the number can be dominated by scheduling noise and timer granularity. A precise-looking timestamp does not tell us the per-call cost of a repeatable workload or whether another implementation does equivalent work. The measured region is so small that timer-call overhead can be a substantial fraction of the decimal on screen. Running it once also gives no view of variation from task switching, CPU
00:43
frequency changes, cache state, or warmup. The operation under test returns the same integer each time; only the elapsed observation fluctuates. Before interpreting any time, state what workload it represents: a single one-off call, repeated lookups, or a batch that includes setup. These can favor different designs. For a tiny operation, repeating many equivalent calls makes each sample large enough to compare more sensibly, while several samples reveal whether the environment is stable. The goal is not a magically exact number; it is a measurement tied to a clearly defined question.
01:23
Two perf_counter calls surround one execution, so their overhead and other process activity sit inside this tiny interval. The result is a real observation, but one sample cannot separate the operation from interruptions. Run it twice; the digits may differ without code changes. The diagnosis uses timeit.repeat and prints three samples, then confirms they are nonnegative. Every sample includes several calls. To compare methods, divide each sample's total by the same call count. The minimum often reflects a run with fewer interruptions; it is still an observation on this interpreter and machine.
02:04
The repeat call produces totals for three separate groups of one thousand lambda invocations, not three measurements of a single call. The lambda's own invocation cost is included, so do not claim that the result isolates the mathematical sum perfectly. A minimum sample can reduce the influence of interruptions; the spread among samples tells us whether the measurement is stable enough to compare. We print only count and nonnegativity because exact seconds depend on hardware and current system load. If two alternatives do different work or use different setup, dividing both totals by one thousand does not make the comparison fair.
02:45
The timed lambda has a call cost of its own, which matters when the body is as small as sum of a short range. Try timing a no-op lambda as a rough baseline, but do not blindly subtract one noisy run from another. Put one-time setup, such as building test data, outside the measured callable unless that setup is part of the real task being compared. The corrected per_call function divides a measured total by a validated positive number of calls. The deterministic terminal example prints 0.00025 seconds per call. Before timing two implementations of a small computation, first assert that both
03:25
return the same result. Otherwise the faster-looking method might simply skip required work. The arithmetic example isolates a reporting step: a quarter second across one thousand calls is 0.00025 seconds per call, or 0.25 milliseconds, or 250 microseconds. That deterministic answer does not mean Python executed the measured function in that time on this computer. Keep seconds and converted units in the label so a factor-of-one-thousand mistake cannot look like an optimization. The helper requires a built-in positive integer count; a bool or float is rejected
04:08
even if it appears numerically positive. Pass it the same number used in the actual timing call. Consider illustrative totals from two different call counts. A takes 0.02 seconds for one hundred calls, or 200 microseconds each. B takes 0.15 seconds for one thousand calls, or 150 microseconds each. B has the larger total but the shorter per-call time. Normalize each total using its own number of calls before comparing. These values illustrate the arithmetic; they are not timings captured from this machine. The verifier divides a three-second total across six calls, then checks an invalid
04:50
zero count. It prints 0.5 and invalid call count. A production benchmark should state input size, Python version, repeats and hardware; neither the demo nor a single run proves a universal speed ranking. Consider comparing membership in a list with membership in a set. If the set benchmark builds a set on every invocation while the list is already built, the result includes construction for only one contender. That may answer an end-to-end one-off query, but it is a different question from comparing repeated lookups on existing collections. Put setup outside both timed callables for the latter question, and confirm they
05:31
return identical answers. Reset mutable state if earlier iterations could alter later work. Record the Python version and input sizes, then inspect the spread across repeated samples instead of declaring a universal winner from one minimum. The final zero-call rejection in this video is an API guard: division by zero would produce no meaningful per-call estimate. Clear units and a documented workload matter more than extra decimal places. The verifier tests a positive count and rejects zero, so a later edit cannot silently divide by an invalid number.
06:12
This arithmetic test is repeatable even when actual clock readings vary. With mutable inputs, reset starting state between benchmarks; otherwise later repetitions may measure a different task from the first. Record what setup happened outside the timer so readers can reproduce the comparison.

THE IDEA

Understand the concept

A developer times one tiny sum(range(10)) call. A wall-clock reading around one execution is dominated by timer resolution and unrelated system activity. timeit repeats a callable many times and reports elapsed time, making tiny operations measurable. This lesson focuses on measurement method, not a promise that one approach always wins.

Calling time.perf_counter once before and once after a single tiny operation gives a number, but its precision does not make it trustworthy. A one-run result can vary with CPU scheduling, cache state, interpreter startup, and other work. Reporting the raw total for many iterations as if it were one call is another common error.

Use timeit.repeat with a callable, a fixed number of calls per repeat, and several repeats. Divide each total by the call count to get seconds per call. The smallest repeat is often used as an estimate of a less-interrupted run, but compare methods in the same environment and verify they produce equivalent output first. Write per_call(total_seconds, number) to divide a measured duration by a positive integer call count. The function is simple by design: it keeps the unit conversion honest, while the demonstration shows how timeit supplies each total. A benchmark needs a stated interpreter, machine and input size before drawing conclusions.

Zero calls cannot yield a meaningful per-call number and would divide by zero. Reject nonpositive counts with ValueError. The final run shows a deterministic arithmetic check of per_call rather than asserting that a noisy measured duration equals a fixed constant. Benchmark results are observations, not language guarantees. The local demo does not prove which implementation is faster on the user's machine or production workload. Start by confirming both functions return the same value; then measure realistic inputs and report enough context to interpret the numbers.

A developer times one tiny sum(range(10)) call.
Calling time.perf_counter once before and once after a single tiny operation gives a number, but its precision does not make it trustworthy.
Write per_call(total_seconds, number) to divide a measured duration by a positive integer call count.
Zero calls cannot yield a meaningful per-call number and would divide by zero.

WORKED EXAMPLES

See it step by step

Read the code, predict the output, then compare it with the result.

01. Calculate seconds per call

PYTHON

total = 0.25
calls = 1000
print(total / calls)

EXPECTED OUTPUT

0.00025

A repeated-call total must be divided by the number of calls before it is presented as per-call cost.

02. Run a repeatable measurement API

PYTHON

import timeit
samples = timeit.repeat(lambda: sum(range(10)), number=1000, repeat=3)
print(len(samples))
print(all(value >= 0 for value in samples))

EXPECTED OUTPUT

3
True

The exact durations vary; the stable output checks that three nonnegative measurements were returned.

03. Check equivalent work before timing

PYTHON

def loop():
    return ''.join(['a', 'b', 'c'])
def literal():
    return 'abc'
print(loop() == literal())

EXPECTED OUTPUT

True

Timing methods with different results would answer the wrong question.

GO DEEPER

A closer look

Follow the reasoning, inspect each result, then try the suggested changes in the console below.

01 / 03

Observe the first result

A developer times one tiny sum(range(10)) call. A wall-clock reading around one execution is dominated by timer resolution and unrelated system activity. timeit repeats a callable many times and reports elapsed time, making tiny operations measurable. This lesson focuses on measurement method, not a promise that one approach always wins.

A repeated-call total must be divided by the number of calls before it is presented as per-call cost. Use timeit.repeat with a callable, a fixed number of calls per repeat, and several repeats. Divide each total by the call count to get seconds per call. The smallest repeat is often used as an estimate of a less-interrupted run, but compare methods in the same environment and verify they produce equivalent output first.

PYTHON

total = 0.25
calls = 1000
print(total / calls)

EXPECTED OUTPUT

0.00025

Follow the reasoning

Predict the first sample's printed value and type.
Run it, then compare with the expected output.

02 / 03

Change the rule deliberately

Calling time.perf_counter once before and once after a single tiny operation gives a number, but its precision does not make it trustworthy. A one-run result can vary with CPU scheduling, cache state, interpreter startup, and other work. Reporting the raw total for many iterations as if it were one call is another common error.

The exact durations vary; the stable output checks that three nonnegative measurements were returned. Write per_call(total_seconds, number) to divide a measured duration by a positive integer call count. The function is simple by design: it keeps the unit conversion honest, while the demonstration shows how timeit supplies each total. A benchmark needs a stated interpreter, machine and input size before drawing conclusions. Validate type(number) is int and number > 0, then return total_seconds / number.

PYTHON

import timeit
samples = timeit.repeat(lambda: sum(range(10)), number=1000, repeat=3)
print(len(samples))
print(all(value >= 0 for value in samples))

EXPECTED OUTPUT

3
True

Follow the reasoning

Identify the line that changes the rule.
Run the corrected sample with the same input.

03 / 03

Verify the boundary

Use timeit.repeat with a callable, a fixed number of calls per repeat, and several repeats. Divide each total by the call count to get seconds per call. The smallest repeat is often used as an estimate of a less-interrupted run, but compare methods in the same environment and verify they produce equivalent output first. Write per_call(total_seconds, number) to divide a measured duration by a positive integer call count. The function is simple by design: it keeps the unit conversion honest, while the demonstration shows how timeit supplies each total. A benchmark needs a stated interpreter, machine and input size before drawing conclusions.

Timing methods with different results would answer the wrong question. Zero calls cannot yield a meaningful per-call number and would divide by zero. Reject nonpositive counts with ValueError. The final run shows a deterministic arithmetic check of per_call rather than asserting that a noisy measured duration equals a fixed constant. Benchmark results are observations, not language guarantees. The local demo does not prove which implementation is faster on the user's machine or production workload. Start by confirming both functions return the same value; then measure realistic inputs and report enough context to interpret the numbers.

PYTHON

def loop():
    return ''.join(['a', 'b', 'c'])
def literal():
    return 'abc'
print(loop() == literal())

EXPECTED OUTPUT

True

Follow the reasoning

Try the documented edge case.
Explain why the output satisfies the function contract.

HANDS-ON

Try it in Python

Edit the example and run it. Python starts in your browser the first time you click Run.

Python console

Ready to run

def per_call(total_seconds, number):
    return total_seconds

print(per_call(0.25, 1000))

Need input()? Add one value per line

Input lines

Ctrl / ⌘ + Enter to run

OUTPUT

Your output appears here.

Your turnReturn elapsed seconds per call by dividing total_seconds by a positive integer number. Reject zero, negative and non-integer counts with ValueError. The prepared call prints 0.00025.

Need a hint?

Validate type(number) is int and number > 0, then return total_seconds / number.

Complete the task and select Check task to verify your code.

DEBUG BETTER

Typical mistakes

Everyone meets these errors. See what causes them and how to fix them.

The visible failure

COMMON MISTAKE

time.perf_counter() around one tiny call

FIX

timeit.repeat(callable, number=many_calls, repeat=several_runs)

What happens: Calling time.perf_counter once before and once after a single tiny operation gives a number, but its precision does not make it trustworthy.
Write per_call(total_seconds, number) to divide a measured duration by a positive integer call count. The function is simple by design: it keeps the unit conversion honest, while the demonstration shows how timeit supplies each total. A benchmark needs a stated interpreter, machine and input size before drawing conclusions.

Missing the edge condition

COMMON MISTAKE

return total_seconds

FIX

return total_seconds / number after validating number > 0

What happens: Zero calls cannot yield a meaningful per-call number and would divide by zero.
Benchmark results are observations, not language guarantees. The local demo does not prove which implementation is faster on the user's machine or production workload. Start by confirming both functions return the same value; then measure realistic inputs and report enough context to interpret the numbers.

Browse this lesson’s problem fixes

VERIFY THE STORY

Sources and further reading

Python timeit documentation