Skip to content

requests: Add ssl argument to pass a verifying SSLContext. - #1175

Open
joshdoesthis wants to merge 1 commit into
micropython:masterfrom
joshdoesthis:requests-ssl-context
Open

joshdoesthis wants to merge 1 commit into
micropython:masterfrom
joshdoesthis:requests-ssl-context

Conversation

@joshdoesthis

@joshdoesthis joshdoesthis commented Nov 13, 2026 •

Copy link
Copy Markdown

Summary

requests builds its own tls.SSLContext and sets verify_mode = tls.CERT_NONE, and a caller can't change that. So HTTPS requests accept any server certificate (#838).

This adds an optional ssl argument to request() and the method helpers. It takes an SSLContext, as umqtt.simple does.

  • The socket is wrapped with the given context, with the URL's host as server_hostname. A CERT_REQUIRED context with a root CA loaded then verifies both the chain and the hostname.
  • Redirects to https:// are wrapped with the same context, against the redirect's own host.
  • With ssl given, an http:// URL, or a redirect to one, raises ValueError instead of sending the request unencrypted.
  • When ssl is None, requests are sent as before.

The socket is now closed on any exception, not only OSError. A certificate that fails verification raises ValueError, and running out of memory raises MemoryError; both left the socket open. The same change also stops leaks that already existed, for example on a 401 that has a Location header.

The README covers what a verifying context needs (CERT_REQUIRED, a root, a set clock, an mbedTLS port) and what it raises. The version goes to 1.2.0.

import ssl, requests

ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
ctx.verify_mode = ssl.CERT_REQUIRED
ctx.load_verify_locations(cafile="ca.der")
with requests.get("https://example.com", ssl=ctx) as r:
    print(r.status_code)

Part of #838 is still open after this. Verifying by default needs a source of root certificates on the device (for example #633, or the ESP-IDF certificate bundle discussed there), so this PR refers to the issue rather than closing it.

Testing

test_requests.py gains six tests. The last three fail without this change.

  1. Without ssl, the request still uses CERT_NONE.
  2. A given context wraps the socket, with the right server_hostname.
  3. A redirect to another https host keeps the context.
  4. ValueError, MemoryError and AttributeError from wrap_socket() close the socket.
  5. ssl with an http:// URL raises before any socket is opened.
  6. A redirect to http:// raises, and the first socket is closed.

I also ran a path test against real TLS servers. It runs on a board through mpremote, or on the unix port, with test servers using only CPython's standard library and certificates made by openssl: https://gist.github.com/joshdoesthis/03410378b6b2a9b837abb11510661892. It covers:

  • Certificates: valid, wrong name, self-signed, expired, untrusted root, and an IP-address SAN.
  • Redirects: same host, relative, another port, to a server that fails verification, and to http://.
  • Errors: connection refused, a handshake that stalls (with timeout), a bad status line, and exceptions from wrap_socket().
  • Bodies: Content-Length, chunked, and read until close.

Results on stock MicroPython 1.29.0 release images:

Case Expected unix port RPI_PICO2_W ESP32_GENERIC_S3 ESP32_GENERIC_C6 ESP32_GENERIC_C3
valid certificate accepted yes yes yes yes yes
wrong name ValueError yes yes yes yes yes
self-signed ValueError yes yes yes yes yes
expired ValueError yes yes yes yes yes
untrusted root ValueError yes yes yes yes yes
name only in an IP SAN accepted yes yes yes yes yes
without ssl: V2-V5 accepted, as before yes yes yes yes yes
CERT_REQUIRED, no root loaded OSError yes yes yes yes yes
ssl with an http:// URL ValueError yes not rerun yes yes not rerun
redirect, same host verified yes yes yes yes yes
redirect, relative verified yes yes yes yes yes
redirect, other port verified yes yes yes yes yes
redirect to a failing certificate ValueError yes yes yes yes yes
redirect to http:// ValueError yes not rerun yes yes not rerun
connection refused OSError, closed yes yes yes yes yes
ValueError from wrap_socket() closed yes yes yes yes yes
MemoryError from wrap_socket() closed yes not rerun yes yes not rerun
OSError from wrap_socket() closed yes yes yes yes yes
ssl=True AttributeError, closed yes not rerun yes yes not rerun
handshake stalls, timeout=2 OSError after ~2 s yes yes yes yes yes
bad status line ValueError, closed yes yes yes yes yes
300 or 401 with Location closed yes not rerun yes yes not rerun
non-str header value TypeError, closed yes not rerun yes yes not rerun
data= and json= together AssertionError, closed yes not rerun yes yes not rerun
Content-Length body read yes yes yes yes yes
chunked body read yes yes yes yes yes
read until close_notify read yes yes yes yes yes
one context, 8 requests no growth yes yes yes yes yes

Every case ran on every target with the PR's previous revision. The cases this revision changes (ssl with http://, the redirect to http://, and the socket closing on other exceptions) were rerun on this revision on the unix port, the ESP32-S3 and the ESP32-C6. On the Pico 2 W and the ESP32-C3 they show "not rerun".

Memory on an ESP32-C3 without PSRAM (ESP32_GENERIC_C3 1.29.0), from a reset, 3 runs each:

ESP-IDF heap free before During the TLS session Handshake
CERT_REQUIRED, nothing else loaded 101.8 KB 76.2 KB 1.0-1.2 s
CERT_NONE, nothing else loaded 101.5 KB 76.8 KB 0.7-0.9 s
CERT_REQUIRED, 50 KB of Python heap held 36.3 KB 10.6 KB 1.0-1.9 s
CERT_NONE, 50 KB of Python heap held 36.3 KB 11.3 KB 0.7-1.0 s

Verification itself costs about 0.8 KB of ESP-IDF heap for the context with one root loaded, and about 0.7 KB more during the session. All 12 handshakes succeeded. With 85 KB held, both verifying handshakes I tried failed with OSError: ENOMEM once and ETIMEDOUT once, as in micropython/micropython#19946. I didn't run CERT_NONE at 85 KB.

Not tested: esp8266 and other axTLS builds, which can't verify certificates (their verify_mode setter is a no-op). I haven't tested other ports either.

Trade-offs and Alternatives

  • Code size: the .mpy for requests goes from 3178 to 3247 bytes (+69), measured with mpy-cross 1.29.0.
  • Verification stays opt-in. Callers who don't pass ssl still accept any certificate, so existing code keeps working but isn't any safer.
  • Stricter than CPython on redirects. CPython requests follows an https→http redirect. Here, with ssl given, it raises: a caller who asked for verification would otherwise get a plain-HTTP response, with its auth= header sent in the clear, and no way to tell. Without ssl, redirects behave as before.
  • The argument's name. CPython requests has no ssl argument; the equivalent there is verify, which takes True, False or a path to a CA bundle. I used ssl= to match umqtt.simple, and because a context can also hold roots that are only in memory. If you'd rather keep requests a strict subset of CPython, I can switch to verify="ca.pem" and verify=False. verify=True would have to raise, since there's no default CA store. Which would you prefer?
  • ssl=True. asyncio.open_connection() and aiohttp also accept ssl=True. Here ssl must be a context or None; anything else raises AttributeError, with the socket closed.
  • Found while testing, not changed here: these problems are already in master, and I can send them as a separate PR.
    • A redirect drops timeout and parse_headers.
    • After a 301/302/303, the redirected GET keeps the POST's Content-Length and Content-Type.
    • json= followed by a 307/308 raises AssertionError.
    • Authorization from auth= is sent to any redirect target.
    • Redirects have no limit.
    • head(...).content waits for a body that never comes.

Generative AI

I used generative AI tools when creating this PR, but a human has checked the code and is responsible for the code and the description above.

Refs #838.

@joshdoesthis
joshdoesthis force-pushed the requests-ssl-context branch 4 times, most recently from f55bb25 to 90deb4e Compare November 17, 2026 02:23
requests builds its own tls.SSLContext with verify_mode CERT_NONE and
offers no way to change it, so HTTPS requests accept any certificate.

Add an optional ssl argument to request() and the method helpers that
takes an SSLContext, as umqtt.simple does. When given, the socket is
wrapped with it and the host passed as server_hostname, and redirects
to https keep using it. An http URL, or a redirect to one, raises
ValueError instead of sending the request unencrypted. When ssl is
None, requests are sent as before.

Close the socket on any exception, not only OSError: a certificate that
fails verification raises ValueError, and running out of memory raises
MemoryError. This also stops the socket leaking on other errors, such
as a non-redirect status with a Location header.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Joshua Wilson <joshua@qualithm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant