Visual RFC.
ALL RFCs / RFC 9309
Read the RFC ↗
RFC 9309 / ROBOTS EXCLUSION PROTOCOL

Use the most specific matching path

03

After selecting the group, compare each Allow or Disallow pattern with the requested URI beginning at its first path byte. A longer matching rule can override a shorter one, regardless of file order.

RFC 9309 · § 2.2.2EXAMPLE
3 / 3
Use the most specific matching pathThe requested docs/public/page.html path matches two prefixes. A six-octet docs restriction and a thirteen-octet docs/public Allow are measured separately; the longer Allow wins, independent of rule order. A third, separate same-path fixture has equivalent Allow and Disallow patterns and explicitly gives the recommended SHOULD Allow tie result. The byte rulers describe these ASCII examples, not a wildcard-specificity formula.The most specific matching prefix winsAllow: /sameDisallow: /sameequivalent patternsChoose Allow · SHOULDRecommendation ≠ unconditional MUST
03
Use the recommended Allow result for an equal tie.

For /same, the Allow and Disallow patterns are identical. Allow SHOULD be used; this is a recommendation, not an invented unconditional MUST.

Source for this drawing: RFC 9309 · Read RFC section 2.2.2 ↗

Protocol participantThreat or exceptionProtocol controlSimplified drawings · precise section references
New to this? Start hereA quick picture + helpful background

A message needs an address.

Think of a packet as a small addressed envelope. Devices pass it onward; transport protocols decide what delivery promises an application gets.

Helpful first, if these ideas are unfamiliar

These pictures are analogies to help you learn. The linked RFC explains the actual rules.

Terms in this drawing
Octet
One byte; rule specificity is expressed in octets.
Prefix match
Matching starts at the beginning of the path rather than anywhere inside it.
Related RFCs and changes

“Updates” changes part of an earlier RFC. “Obsoletes” replaces it. This list shows selected related documents.

KEY POINT

The most specific matching rule MUST be used: the one with the most octets. Matching MUST start at the first path octet and SHOULD be case sensitive. Equivalent Allow and Disallow rules SHOULD resolve to Allow.

  • The most specific matching rule MUST be used: the one with the most octets. Matching MUST start at the first path octet and SHOULD be case sensitive. Equivalent Allow and Disallow rules SHOULD resolve to Allow.
  • In the worked example, /docs/ has 6 ASCII octets, while /docs/public/ has 13. Both match /docs/public/page.html, so Allow: /docs/public/ wins over Disallow: /docs/.
  • Reordering these two lines does not change their lengths or the required winner. This is not a first-match or last-match firewall list.
  • For equal /same patterns, the recommendation is to choose Allow rather than Disallow. Keep that SHOULD distinct from the MUST to choose the most specific match.
  • If no rule matches, or the selected group has no rules, the URI is allowed by this protocol. Under recommended case-sensitive matching, /Docs/ and /docs/ are different prefixes. Duplicate rules MAY be deduplicated.
DETAILS

The byte counts here are independently checked ASCII examples, not character counts for every Unicode string or a general wildcard-specificity formula. The example is derived directly from section 2.2.2; it does not repeat the inconsistent filename example in section 5.2 or adopt reported erratum 7124 as a new normative rule.

Read the RFC section ↗

All chapters in RFC 9309

Drawing view

Scroll the drawing to see every part. Playback controls stay above it.

RFC 9700 / CHAPTERS

Chapters

For full details, tables, algorithms, and exceptions, read the original RFC.

RFC status and corrections ↗