LESSON 165 · DEEPER PYTHON30 min

asyncio.gather: run tasks together and handle errors

Run independent coroutines together and turn returned exceptions into clear per-item results without losing input order.

Before you start: Start Concurrent I/O with asyncio · Exceptions

Example check: Worked and guided code samples are checked against CPython 3.13. Practice uses Pyodide 0.29.5 (Python 3.13) in your browser. How we check lessons

WATCH

Video lesson: asyncio.gather: run tasks together and handle errors

The preview is stored on this site. YouTube loads only when you press Play. Watch on YouTube

Jump to a chapter

Read the video transcript

Select a timestamp to watch that moment on YouTube.

00:00
asyncio.gather schedules several awaitables and waits for their results in a single await. It is useful when independent I/O operations can overlap: for example, fetching several pages that do not depend on each other's responses. It does not make CPU-bound Python loops run in parallel. Results are returned in the same order as the awaitables supplied, even when a later job finishes first. Today we will test the claim with a tiny example, inspect a misleading first attempt, and finish with a verification case. Run independent coroutines together and turn returned exceptions into clear per- item results without losing input order.
00:42
With the default return_exceptions=False, the first raised exception is propagated to the coroutine awaiting gather. Other already-scheduled awaitables are not automatically cancelled merely because one failed; their later behavior depends on the surrounding event loop and caller. This is a different failure policy from TaskGroup, which coordinates sibling cancellation. Choose the policy before using gather rather than assuming all remaining work stops. The first run is intentionally incomplete or incorrect. Independent waits run one after another. Read the actual output before editing the code; a familiar-looking command can still
01:23
answer the wrong question. The main reason to use asyncio.gather is to overlap independent waits. Each call to an async function creates a coroutine; supplying those coroutines to gather lets the event loop drive them while one is waiting. If a later task has no dependency on the first task's result, serial awaiting wastes that opportunity. Network requests can benefit, though this page uses short sleeps so no network is needed. The faster job in this example completes first, yet the result list reads slow then fast. gather preserves the order of the awaitables passed to it.
02:03
That positional promise is useful when matching responses back to request IDs. It does not mean that the jobs executed in the same order or at the same time. We use a second run to make the cause visible. The printed values are small enough to check by hand. If they disagree with your prediction, pause and identify the assumption that changed. The default gather behavior propagates the first exception to its awaiter. That is often right when any failed job should stop the calling operation. Sometimes a batch report needs to show which requests succeeded and which failed. Pass return_exceptions=True to receive an exception object at the failed job's
02:45
position and ordinary values at successful positions. Do not call a string method on every result without inspection. In this sample, ValueError occupies the middle position. Use isinstance to recognize expected errors and decide how to report them; another exception type may indicate a programming defect that should still be raised. Keeping the original input beside each result makes the final error message meaningful. Now compare the corrected run with the earlier one. The important change is the rule encoded in the operation, not merely the final display. Run the code and inspect the returned values as well as their interpretation.
03:29
When every result position matters, return_exceptions=True puts an exception object in the corresponding position. Inspect each item and handle expected failures explicitly. In the task, a label beginning with ! raises ValueError. The collector returns an error label for that item and a normal uppercase label for successes. It checks ValueError specifically; a production program should decide which errors are recoverable instead of treating every exception as a harmless result. With the default policy, an exception propagates immediately, but gather does not automatically cancel every other submitted awaitable merely because one
04:09
child failed. In this example the caller created a sibling task, catches ValueError, and then awaits that sibling. It finishes. A surrounding asyncio.run shutdown can cancel leftover tasks if the main coroutine exits, so the broader application structure still matters. TaskGroup offers a different coordinated failure policy: an unhandled child error cancels unfinished siblings as the group exits. Use that tool when such shutdown is required. For this lesson's collector, return_exceptions=True is deliberate because every input needs a status.
04:50
Test both an empty list and duplicated labels to ensure positional mapping does not depend on uniqueness or at least one task. The final run checks a different condition so a special-case answer cannot pass unnoticed. Try the linked browser task and use the guide when you need a smaller step. The source links on the lesson page give the complete API behavior. The browser exercise uses asyncio.sleep(0) to yield control without network requests. That makes the output deterministic while showing real coroutine scheduling. The task keeps duplicate labels and an empty input intact.
05:30
Build one coroutine per input, await gather once, then zip inputs and results to produce the final list. Avoid awaiting inside a loop before gather, which would serialize independent work. Imagine three independent page requests where the middle endpoint fails. If the report must include a result for every requested URL, positional results let you attach the failure to its exact URL. If any failure should invalidate the entire batch, use the default exception path and handle cleanup at the caller level. Do not turn every possible exception into a success-looking string. An unexpected TypeError may indicate a bug rather than a routine unavailable page.
06:15
Finally, remember that gather is an I/O coordination tool; it does not speed up a heavy calculation that never yields control to the event loop.

THE IDEA

Understand the concept

asyncio.gather schedules several awaitables and waits for their results in a single await. It is useful when independent I/O operations can overlap: for example, fetching several pages that do not depend on each other's responses. It does not make CPU-bound Python loops run in parallel. Results are returned in the same order as the awaitables supplied, even when a later job finishes first.

With the default return_exceptions=False, the first raised exception is propagated to the coroutine awaiting gather. Other already-scheduled awaitables are not automatically cancelled merely because one failed; their later behavior depends on the surrounding event loop and caller. This is a different failure policy from TaskGroup, which coordinates sibling cancellation. Choose the policy before using gather rather than assuming all remaining work stops.

When every result position matters, return_exceptions=True puts an exception object in the corresponding position. Inspect each item and handle expected failures explicitly. In the task, a label beginning with ! raises ValueError. The collector returns an error label for that item and a normal uppercase label for successes. It checks ValueError specifically; a production program should decide which errors are recoverable instead of treating every exception as a harmless result.

The browser exercise uses asyncio.sleep(0) to yield control without network requests. That makes the output deterministic while showing real coroutine scheduling. The task keeps duplicate labels and an empty input intact. Build one coroutine per input, await gather once, then zip inputs and results to produce the final list. Avoid awaiting inside a loop before gather, which would serialize independent work.

gather preserves argument order
independent awaits can overlap
return_exceptions=True returns exception objects
gather and TaskGroup have different failure behavior

WORKED EXAMPLES

See it step by step

Read the code, predict the output, then compare it with the result.

01. Result order differs from completion order

PYTHON

import asyncio
async def job(label, delay):
    await asyncio.sleep(delay)
    return label
async def main():
    print(await asyncio.gather(job('slow', 0.01), job('fast', 0)))
asyncio.run(main())

EXPECTED OUTPUT

['slow', 'fast']

The fast job finishes first, but the result list follows argument order.

02. Keep a failure in its result position

PYTHON

import asyncio
async def job(value):
    await asyncio.sleep(0)
    if value < 0:
        raise ValueError('negative')
    return value * 2
async def main():
    results = await asyncio.gather(job(2), job(-1), job(3), return_exceptions=True)
    print([type(value).__name__ if isinstance(value, Exception) else value for value in results])
asyncio.run(main())

EXPECTED OUTPUT

[4, 'ValueError', 6]

An exception object occupies the failed job's original position.

03. Default error does not by itself cancel a sibling

PYTHON

import asyncio
async def slow():
    await asyncio.sleep(0.01)
    return 'finished'
async def fail():
    await asyncio.sleep(0)
    raise ValueError('bad')
async def main():
    sibling = asyncio.create_task(slow())
    try:
        await asyncio.gather(sibling, fail())
    except ValueError:
        print('caught', sibling.cancelled())
    print(await sibling)
asyncio.run(main())

EXPECTED OUTPUT

caught False
finished

The caller catches the error and still awaits the already-scheduled sibling.

GO DEEPER

A closer look

Follow the reasoning, inspect each result, then try the suggested changes in the console below.

01 / 03

Run independent waits together

The main reason to use asyncio.gather is to overlap independent waits. Each call to an async function creates a coroutine; supplying those coroutines to gather lets the event loop drive them while one is waiting. If a later task has no dependency on the first task's result, serial awaiting wastes that opportunity. Network requests can benefit, though this page uses short sleeps so no network is needed.

The faster job in this example completes first, yet the result list reads slow then fast. gather preserves the order of the awaitables passed to it. That positional promise is useful when matching responses back to request IDs. It does not mean that the jobs executed in the same order or at the same time.

PYTHON

import asyncio
async def job(label, delay):
    await asyncio.sleep(delay)
    return label
async def main():
    print(await asyncio.gather(job('slow', 0.01), job('fast', 0)))
asyncio.run(main())

EXPECTED OUTPUT

['slow', 'fast']

Follow the reasoning

Predict which job finishes first.
Compare completion timing with the printed result order.

02 / 03

Keep expected failures in place

The default gather behavior propagates the first exception to its awaiter. That is often right when any failed job should stop the calling operation. Sometimes a batch report needs to show which requests succeeded and which failed. Pass return_exceptions=True to receive an exception object at the failed job's position and ordinary values at successful positions.

Do not call a string method on every result without inspection. In this sample, ValueError occupies the middle position. Use isinstance to recognize expected errors and decide how to report them; another exception type may indicate a programming defect that should still be raised. Keeping the original input beside each result makes the final error message meaningful.

PYTHON

import asyncio
async def job(value):
    await asyncio.sleep(0)
    if value < 0:
        raise ValueError('negative')
    return value * 2
async def main():
    results = await asyncio.gather(job(2), job(-1), job(3), return_exceptions=True)
    print([type(value).__name__ if isinstance(value, Exception) else value for value in results])
asyncio.run(main())

EXPECTED OUTPUT

[4, 'ValueError', 6]

Follow the reasoning

Run the three-job batch.
Identify the failed input from the result's position.

03 / 03

Choose a failure policy explicitly

With the default policy, an exception propagates immediately, but gather does not automatically cancel every other submitted awaitable merely because one child failed. In this example the caller created a sibling task, catches ValueError, and then awaits that sibling. It finishes. A surrounding asyncio.run shutdown can cancel leftover tasks if the main coroutine exits, so the broader application structure still matters.

TaskGroup offers a different coordinated failure policy: an unhandled child error cancels unfinished siblings as the group exits. Use that tool when such shutdown is required. For this lesson's collector, return_exceptions=True is deliberate because every input needs a status. Test both an empty list and duplicated labels to ensure positional mapping does not depend on uniqueness or at least one task.

PYTHON

import asyncio
async def slow():
    await asyncio.sleep(0.01)
    return 'finished'
async def fail():
    await asyncio.sleep(0)
    raise ValueError('bad')
async def main():
    sibling = asyncio.create_task(slow())
    try:
        await asyncio.gather(sibling, fail())
    except ValueError:
        print('caught', sibling.cancelled())
    print(await sibling)
asyncio.run(main())

EXPECTED OUTPUT

caught False
finished

Follow the reasoning

Run the sibling example.
Explain why its final line can print finished after another job failed.

HANDS-ON

Try it in Python

Edit the example and run it. Python starts in your browser the first time you click Run.

Python console

Ready to run

import asyncio

async def label_job(label):
    await asyncio.sleep(0)
    if label.startswith('!'):
        raise ValueError('invalid label')
    return label.upper()

async def collect_labels(labels):
    return [await label_job(label) for label in labels]

print(asyncio.run(collect_labels(['ada', '!bad', 'grace'])))

Need input()? Add one value per line

Input lines

Ctrl / ⌘ + Enter to run

OUTPUT

Your output appears here.

Your turnUse asyncio.gather in collect_labels(labels) to run all label_job coroutines together with return_exceptions=True. Return an uppercase value for successes and 'ERROR:<original label>' for a ValueError. Keep the original input order and duplicates; empty input returns []. The sample must print ['ADA', 'ERROR:!bad', 'GRACE'].

Need a hint?

Build a coroutine for each label, await asyncio.gather(*jobs, return_exceptions=True), then pair each result with its original label.

Complete the task and select Check task to verify your code.

DEBUG BETTER

Typical mistakes

Everyone meets these errors. See what causes them and how to fix them.

Serial awaiting

COMMON MISTAKE

[await job(x) for x in inputs]

FIX

await asyncio.gather(*(job(x) for x in inputs))

What happens: Independent waits run one after another.
Submit the independent awaitables before waiting for their collection.

Treating exceptions as strings

COMMON MISTAKE

result.upper()

FIX

Check isinstance(result, ValueError) before formatting it.

What happens: With return_exceptions=True, a result can be an exception object.
Keep expected failure handling explicit and leave unexpected exceptions visible.

Browse this lesson’s problem fixes

VERIFY THE STORY

Sources and further reading

Python asyncio.gather documentation