Making Your MCP Server Findable: Link Headers, HTML Tags, and RFC 9728

Having a working MCP server is step one. Step two — the one most people skip — is making sure anything can actually find it.

We shipped our public MCP endpoint at /mcp a few days ago. It worked perfectly: any client that knew the URL could connect, list tools, and query our knowledge base. But "any client that knew the URL" was doing a lot of heavy lifting. In practice, that meant us — manually configuring the endpoint in Claude Code. No external AI agent visiting mcsoftsolution.com had any way to discover the MCP server existed.

This post covers the three mechanisms we added to fix that, and the OAuth discovery problem we hit along the way.

The Problem: A Server Nobody Can Find

Picture an AI agent that a user points at https://mcsoftsolution.com. The agent fetches the homepage, reads the HTML, maybe checks robots.txt. It sees a web development company. It has no idea there's a live MCP server at /mcp with structured, queryable data about services, blog posts, and contact info.

The agent would need to guess /mcp as a path — and even then, its MCP client library would likely try OAuth discovery, hit our HTML 404 page, fail to parse it as JSON, and give up.

We needed to solve two problems: pointing (how does the agent find the URL?) and probing (once found, how does it know auth isn't required?).

Fix 1: HTTP Link Header

The simplest, most universal signal. Every HTTP response from our server now includes:

Link: <https://mcsoftsolution.com/mcp>; rel="mcp"

This follows the standard Link header format from RFC 8288. An agent doesn't need to parse HTML — the discovery information is right there in the response headers of any page it fetches.

In Axum, this is a single middleware layer using tower-http:

use tower_http::set_header::SetResponseHeaderLayer;

let mcp_link = SetResponseHeaderLayer::appending(
    axum::http::header::LINK,
    HeaderValue::from_static(
        r#"<https://mcsoftsolution.com/mcp>; rel="mcp""#
    ),
);

let app = routes::build_router(state.clone())
    .merge(mcp::build_mcp_router(state))
    .layer(mcp_link);

Every response — homepage, blog posts, contact page, even 404s — carries the pointer.

Fix 2: HTML Link Tag

For agents that parse HTML (most of them), we added a <link> tag to the base template:

<link rel="mcp" href="https://mcsoftsolution.com/mcp">

Same pattern as RSS autodiscovery (rel="alternate"), favicon declaration (rel="icon"), or canonical URLs (rel="canonical"). Any agent reading the <head> section finds the MCP endpoint alongside the other machine-readable metadata it already knows how to process.

Fix 3: OAuth Protected Resource Metadata (RFC 9728)

This is the one that fixes the probing problem. Even after an agent finds /mcp, its MCP client library will typically check whether authentication is required before connecting. The standard way to answer that question is RFC 9728 — OAuth 2.0 Protected Resource Metadata.

When a client discovers an MCP endpoint, it checks:

GET /.well-known/oauth-protected-resource/mcp

Our response:

{
  "resource": "https://mcsoftsolution.com/mcp"
}

The key detail: no authorization_servers field. Per the spec, this signals that the resource is publicly accessible — no OAuth flow needed. The client skips authentication entirely and connects directly.

For the admin endpoint, we return different metadata:

{
  "resource": "https://mcsoftsolution.com/mcp/admin",
  "bearer_methods_supported": ["header"]
}

This tells clients: auth is required, Bearer token via header.

The JSON 404 Problem

Before this fix, any /.well-known/* request fell through to our general 404 handler — which returned an HTML page. MCP client libraries tried to parse that HTML as JSON, crashed, and reported the server as unreachable.

We added a catch-all for the .well-known namespace:

.route("/.well-known/{*rest}", get(well_known::not_found))

pub async fn not_found() -> impl IntoResponse {
    (StatusCode::NOT_FOUND, Json(json!({ "error": "not_found" })))
}

Every .well-known path now returns JSON — either real metadata or a clean 404 that won't break a parser.

The Discovery Flow

Here's what an AI agent experiences now, starting from nothing but the URL:

Three HTTP requests. Under a second. The agent goes from a bare domain to a fully connected MCP session with seven available tools.

What We Changed

Four files, about 60 lines of code total:

Change	File	Purpose
`<link rel="mcp">`	`templates/base.html`	HTML-based discovery
`Link` response header	`src/main.rs`	Header-based discovery
`/.well-known/oauth-protected-resource/*`	`src/routes/well_known.rs`	RFC 9728 auth metadata
`/.well-known/{*rest}` JSON 404	`src/routes/well_known.rs`	Prevent HTML parse crashes

The implementation is framework-agnostic in concept. If you're running an MCP server in any language, the same three mechanisms apply: add a Link header, add an HTML <link> tag, and serve RFC 9728 metadata at the .well-known path.

The Takeaway

Building an MCP server is the interesting engineering problem. Making it discoverable is the boring one. But boring infrastructure is what determines whether your server gets used by one agent (yours) or by any agent that visits your domain.

The three mechanisms layer on each other:

Link header — works for headless API clients that never parse HTML
HTML link tag — works for agents that read web pages (most of them)
RFC 9728 metadata — tells the client "no auth required, just connect"

All three together mean an AI agent can go from https://yoursite.com to a live MCP tool call without any prior knowledge, any configuration, or any human in the loop. That's the point.