HTTP Request Signatures, (Mon, Sep 8th) SANS Internet Storm Center, InfoCON: green

This weekend, I noticed three related headers being used in requests to some of our honeypots for the first time [1]: 

This weekend, I noticed three related headers being used in requests to some of our honeypots for the first time [1]:

• Signature-Input
• Signature-Agent
• Signature

These headers are related to a relatively new feature, HTTP Message Signatures, which was standardized in RFC 9421 in February last year [2].

First, what is the problem that HTTP Request Signatures attempt to solve? According to the RFC, there are quite a few problems. However, the main use case appears to be authenticating bots. Most well-behaved bots add specific user agents. Like, for example, “Googlebot”, to identify requests originating from the bot.

On the other hand, users have long figured out that setting your user agent to “Googlebot” may get you past some paywalls. To counter this, Google ensured that its IP addresses reverse-resolve to “googlebot.com” hostnames. This may work for a large organization like Google, but in modern architectures, the client IP address is often lost in the proxy bucket brigade, or if it is present, may not be communicated in a trustworthy manner.

HTTP Request Signatures are supposed to fix that.

Here is an example request received by one of our honeypots:

GET / HTTP/1.1
Host: [redacted]
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 (compatible; SitesOverPagesBot/1.0; +https://sitesoverpages.com/bot)
Signature-Agent: "https://www.sitesoverpages.com/.well-known/http-message-signatures-directory"
Signature-Input: sig1=("@authority" "signature-agent");created=1757301130;keyid="xqKvFlGdMZzhV2MwO6-HkYRqOeN3LCji2Srt1MkJN50";alg="ed25519";expires=1757301430;nonce="9x4l0vMMtKW1-9NOvR54NEpru2wCE-aLk0w3RoVp0oI";tag="web-bot-auth"
Signature: sig1=:EMJxWyWvkB136cU27jANCpbWz2bsK7mvaxcC56lRtVmsI6ngTo6K4saml+71n3GYMzTjCbaENyaJ6581VbfRBg==:
Accept: */*
Accept-Endogin: gzip, deflate

The “Signature-Agent” defines a URL to retrieve the public keys. In our case, the URL returns:

{
  "keys": [
    {
      "kty": "OKP",
      "crv": "Ed25519",
      "x": "Ypbt4rAoxgKaJdTaT-O65MN1Hq4JL57wTGJiOrpaFUA"
    }
  ]
}

A JSON-formatted document with the key, a key type (“Octet Key Pair”, typical for EC Keys), an Ed25519 curve (“crv”), and the actual key (“x”).

Next, the Signature-Input header defines a signature label (“sig1”) and lists all the fields included in the signature. Only the host name (“Authority”) and the Signature-Agent header are signed. Next, it lists the created and “expires” time, the “keyid” and algorithm used, as well as a nonce and the purpose of the key (web-bot-auth).

Finally, the signature value is included in the “Signature” header.

A server may retrieve the keys for any bots it trusts and only allow bots with valid signatures to send requests. Cloudflare offers some integration [3] and Zuplo, which also has a good primer at [4].

[1] https://isc.sans.edu/weblogs/allheaders.html
[2] https://www.rfc-editor.org/rfc/rfc9421.html
[3] https://developers.cloudflare.com/bots/reference/bot-verification/web-bot-auth/
[4] https://zuplo.com/blog/identify-ai-agents-with-http-message-signatures

—
Johannes B. Ullrich, Ph.D. , Dean of Research, SANS.edu
Twitter|

(c) SANS Internet Storm Center. https://isc.sans.edu Creative Commons Attribution-Noncommercial 3.0 United States License. 

​Read More