Build Query Strings with urllib.parse

Select a timestamp to watch that moment on YouTube.

1. 00:00

   Imagine that a learner has a search page URL and wants to replace the search phrase with red fox. The tempting solution is string concatenation: add a question mark and the new phrase. But this URL already has a query, already contains another parameter, and already has a fragment. We need to preserve all of those pieces. Today we will run the naïve approach and inspect exactly what goes wrong. Then we will split the URL into components, treat its query as an ordered list of pairs, replace one key, and put the URL back together. This lesson does not send a request or contact the example domain.

2. 00:40

   It is purely about constructing a predictable URL string. At the end we will test a blank parameter, two occurrences of an unrelated key, a fragment, and a Unicode search term. Those tests matter because a function that works only for a neat one-parameter example is very easy to write and very easy to trust too early. First, predict what happens if we add a second question mark to a URL that already has one. The first line may look like a URL with two searches, but the parser tells a more useful story. The query is still q equals old and lang equals en. Everything after the hash is part of the fragment, including the extra question

3. 01:21

   mark and red fox. A fragment is not part of the server query, so this operation did not update the search value at all. It also left a raw space in the text, which should be encoded in a query value. The important diagnosis is structural: a URL is not one undifferentiated string. The order of query and fragment markers changes the meaning of the same characters. Adding another question mark is not a general update operation, and adding text at the end is even less reliable when a fragment is present. We need a way to access the existing query without disturbing the path or fragment.

4. 02:02

   Python's urllib.parse supplies urlsplit for this job. It does not decide whether a URL is trusted, but it gives us the components needed for a deterministic transformation. Let us inspect those components before trying the repair. urlsplit gives us a path, a query string, and a fragment as separate fields. parse_qsl converts the query into a list of key and value pairs. Notice that tag appears twice. That is legal query data and may be intentional, for example when a filter accepts multiple tags. If we immediately convert this list to a dictionary, one tag value disappears.

5. 02:42

   The blank empty parameter also survives because we explicitly requested keep_blank_values. Without that option, a blank query value can be dropped during parsing. So our replacement policy should be precise: remove every pair whose key is the target, keep the other pairs in order, and append one new target pair. That handles repeated old q values without deciding that unrelated duplicates are errors. We also need to reassemble the original scheme, host, path, and fragment after the edit. When the pairs become a query again, urlencode will encode the values. This is where a space, ampersand, or accented character should be handled;

6. 03:24

   manually concatenating those characters would be error-prone. Now let us implement the function and look at the exact result. We have one query marker, the unrelated language parameter remains, and the fragment is still at the end. The old q value is gone, replaced with a single q equals red plus fox. In this query encoding, plus represents the space between the words. Read through the function from top to bottom. urlsplit names the components. parse_qsl gives us pairs and retains blanks. The list comprehension removes every pair with the exact requested key, while leaving all other pairs in their previous order. append adds the new value once.

7. 04:08

   urlencode turns the pair list into valid query syntax, and urlunsplit reconstructs the URL with the original nonquery components. This function has a useful, narrow contract. It preserves decoded parameter values and sequence, but it may normalize the spelling of existing percent escapes when it decodes and re-encodes the original query. If an application needs the original query bytes preserved exactly, that is a different requirement and needs a different editor. For a normal decoded query transformation, this sequence is easy to test and review. Let us now challenge it with awkward inputs.

8. 04:48

   The first result keeps both tag values, keeps the empty value, replaces q, and leaves the fragment after the query. The second shows that a Unicode accent becomes UTF-8 percent encoding, the space becomes plus, and the ampersand inside the value becomes percent two six rather than acting as a separator between parameters. If one of these results surprises you, inspect the pair list before urlencode. That tells you whether the bug came from your filtering rule or from your expectations about encoding. One final boundary is important: a parsed URL is not automatically safe to request. urlsplit can identify a scheme and host without checking whether your application

9. 05:31

   should access that destination. If you later accept a user-supplied URL for fetching, validate its scheme and destination separately and account for redirects. Our function never contacts the network; it only edits a chosen URL string. Try the checked task on the lesson page, then change one test input yourself. Predict the exact pair order before running the code. A good URL helper is more than a short function: it is a contract that survives the blank, duplicate, Unicode, and fragment cases you just watched.