robotsTxt
From Dontopedia, the open, paraconsistent wiki. (Last updated 2026-06-06.)
robotsTxt has 13 facts recorded in Dontopedia across 4 references, with 2 live disagreements.
Mostly:has note(3), has rules matched(2), is allowed(2)
Maturity scale
raw canonical shape-checked rule-derived certifiedInbound mentions (6)
Other subjects in dontopedia point AT this entity as a value. These are inverse relationships — e.g. "X motherOf this subject" — and answer questions the forward facts can't. Grouped by predicate.
hasRobotsTxtHas Robots Txt(2)
- Fetch Metadata
ex:fetch-metadata - Webfetch
ex:webfetch
hasRobotsTxtDataHas Robots Txt Data(1)
- Failure Result
ex:failure-result
hasRobotsTxtInfoHas Robots Txt Info(1)
- Fix Json
ex:fix-json
respectsRespects(1)
- Webfetch
ex:webfetch
respectsRobotsTxtRespects Robots Txt(1)
- Fetch Operation
ex:fetch-operation
Other facts (12)
The long tail: predicates that appear too rarely to warrant their own section. Filter or scroll to find a specific one. Each row links to its source.
| Predicate | Value | Ref |
|---|---|---|
| Has Note | This fetch respects robots.txt. Some resources may be omitted if disallowed by the site's robots.txt rules. | [1] |
| Has Note | This fetch respects robots.txt. Some resources may be omitted if disallowed by the site's robots.txt rules. | [2] |
| Has Note | This fetch respects robots.txt and the resource was blocked according to the site's robots.txt rules. | [3] |
| Has Rules Matched | Default: Allow (no matching rules) | [1] |
| Has Rules Matched | ["Default: Allow (no matching rules)"] | [2] |
| Is Allowed | true | [1] |
| Is Allowed | true | [2] |
| Evidenced by | Default: Allow (no matching rules) | [2] |
| Allowed | false | [3] |
| Has Rule Matched | Disallow: /*/raw | [3] |
| Allows Crawling | true | [4] |
| Crawl Delay | 2 | [4] |
Timeline
Timeline axis is valid_time — when each source says the fact was true in the world, not when Dontopedia learned about it. Retracted rows are kept for provenance; coloured stripes indicate the context kind.
References (4)
ctx:discord/blah/omega/part-820ctx:discord/blah/omega/part-1185ctx:discord/blah/omega/1177- full textomega-1177text/plain1 KB
doc:agent/omega-1177/f8361d7e-5f6d-4eb4-a153-fe20f900d6b3Show excerpt
[2026-02-28 19:41] omega [bot]: 🔧 2/2: webFetch ❌ Failed **Args:** ```json { "url": "https://git.unturf.com/engineering/unturf/uncloseai.com/-/raw/master/public/src/ago.js", "userAgent": "OmegaBot/1.0", "mode": "raw", "maxRedirects"…
ctx:discord/blah/unturf/35- full textunturf-35text/plain2 KB
doc:agent/unturf-35/b25dd54a-ef7f-44d2-9808-c2391c770b51Show excerpt
[2025-12-20 13:47] ajaxdavis: oh thats what you meant [2025-12-20 13:47] ajaxdavis: my blog is on ajaxdavis.dev [2025-12-20 13:47] ajaxdavis: thanks though lol [2025-12-20 13:47] ajaxdavis: is that your hosting service or just a custom setu…
Keep researching
Missing something or suspicious of what's here? Kick off a research session — a Claude agent will investigate, cite its sources, and file new facts into a dedicated context you can review before accepting into the shared view.