Docs

Markdown and llms.txt

Agents and scripts can read every page as Markdown, and find pages through llms.txt and the sitemap.

Accept: text/markdown

Send Accept: text/markdown to any page and Peptilyze answers with the same page as Markdown: headings, text, links (absolute), lists and tables. Browsers get HTML as before. Every HTML answer carries Vary: Accept, so caches keep both forms apart.

Request

curl -sS -H 'Accept: text/markdown' https://peptilyze.com/compounds/bpc-157/

Response headers

HTTP/2 200
content-type: text/markdown; charset=utf-8
vary: Accept
link: </llms.txt>; rel="describedby"; type="text/plain", </sitemap.xml>; rel="sitemap"; type="application/xml"

An unknown address answers 404 in Markdown too, with links to the home page, the sitemap and llms.txt. The Markdown is a lossy view meant for reading; for data use the JSON API. The attribution rules apply to it as well.

Finding pages

/llms.txt: what the site is, when to use it, URL patterns, guides and the open data API.
/sitemap.xml: every indexable page.
HTML pages send a Link header pointing to both.

Content signals

The robots.txt declares how the content may be used by AI systems (contentsignals.org): search and AI answers may use it, training may not.

robots.txt

User-agent: *
Content-Signal: ai-train=no, search=yes, ai-input=yes
Allow: /