User-agent: * # No Disallow for /admin on purpose. Keeping it out of search results is the job # of the X-Robots-Tag: noindex header set on /admin and /admin/* in # next.config.js, and a crawler that is forbidden to FETCH the page never # receives that header — the URL can then still be listed from a link pointing at # it, which is the outcome the header exists to prevent. Blocking here and # noindexing there cancel each other out; the header alone is the stronger of the # two, so the path stays crawlable. Host: https://stubbs.pro/ Sitemap: https://stubbs.pro/sitemap.xml