{
  "@context": "https://schema.org",
  "@type": "Article",
  "schema_version": "1.1",
  "content_item_id": "briefing-perplexity-crawler-roles.en",
  "translation_group_id": "briefing-perplexity-crawler-roles",
  "locale": "en",
  "type": "briefing",
  "section": "briefings",
  "slug": "perplexity-crawler-roles",
  "title": "Perplexity separates search crawling from user-triggered fetching",
  "description": "Why PerplexityBot and Perplexity-User must be counted separately and how to avoid treating logs as citation proof.",
  "direct_answer": "Perplexity documents PerplexityBot for search crawling and Perplexity-User for fetching a page in response to a user request. A matching User-Agent and official IP help classify the fetch, but neither role alone proves that the page appeared as a link in an answer.",
  "sections": [
    {
      "heading": "What is documented",
      "paragraphs": [
        "The two agents have different documented purposes. PerplexityBot concerns search crawling, while Perplexity-User fetches a page in the context of a user action. Combining both under “AI crawler” destroys useful signal: routine discovery and a request-specific fetch become indistinguishable.",
        "Separating roles supports a narrower question: what kind of access are we observing? Bot indicates the documented search crawl, while User indicates a request-triggered product fetch. Neither describes the private index, ranking, or the complete path from query to answer."
      ],
      "source_ids": [
        "perplexity-bots"
      ]
    },
    {
      "heading": "How to verify an agent",
      "paragraphs": [
        "Do not trust a User-Agent alone; it can be copied. Match the address against the current official IP endpoint and preserve list version and verification time. If DNS verification is used, perform reverse and forward-confirmed reverse lookup. A mismatch remains claimed or unknown rather than entering verified-bot statistics."
      ]
    },
    {
      "heading": "What to record",
      "paragraphs": [
        "A minimum event stores timestamp, URL, status, response size, complete User-Agent, IP-verification result, and role. Raw IPs are unnecessary in long-term analytics: after network verification they can be truncated and HMAC-hashed under the privacy policy. PerplexityBot and Perplexity-User aggregates remain separate.",
        "A short reproducible check can use one URL: send a normal HEAD/GET, record the server response, match the User-Agent and IP against the current official list, and repeat a week later. A changed list or response should create a new verification version rather than silently rewriting the old one."
      ],
      "source_ids": [
        "perplexity-bots"
      ]
    },
    {
      "heading": "What the log does not prove",
      "paragraphs": [
        "A successful 200 response proves delivery to that request. It does not reveal index inclusion, retrieval selection, use of a fact, a displayed link, or a human referral. Citation requires an archived answer with visible attribution to the specific URL."
      ]
    },
    {
      "heading": "Worked example",
      "paragraphs": [
        "If the server sees a request with the PerplexityBot user agent, first record the time, URL, status, and verified IP range. That is an observed crawl. If a later Perplexity answer links to the same URL, that is a separate observation: the match does not prove that this crawl produced the citation. Keep two rows in the report rather than one conversion."
      ]
    },
    {
      "heading": "Action checklist",
      "paragraphs": [
        "Before classifying a request, check the user agent, the official range list, forward and reverse DNS confirmation, URL, and response code. Separate crawler, user fetch, and unknown automation. Do not block an agent by name alone; record the rule and risk first. Support any citation claim with a separate archived answer."
      ]
    },
    {
      "heading": "What remains unknown",
      "paragraphs": [
        "Perplexity's documentation explains the roles of PerplexityBot and Perplexity-User, but it does not reveal which fetched documents entered a particular answer's candidate set or why one link became visible while another did not. A verified Perplexity-User request therefore remains a user-triggered fetch unless the answer and its exact URL are preserved separately."
      ]
    },
    {
      "heading": "Reporting decision",
      "paragraphs": [
        "Keep separate fields in analytics for verified crawl, user fetch, referrer, and observed citation. A combined number is acceptable only as a technical request count, not as an answer-influence metric. Each week, test classification rules against several real log entries."
      ]
    }
  ],
  "published_at": "2026-09-11",
  "modified_at": "2026-09-11",
  "data_through": "2026-09-11",
  "next_review_at": "2026-09-18",
  "author": "GeoAeoAle Editorial",
  "origin": "editorial",
  "publisher": "GeoAeoAle Editorial",
  "license": "https://creativecommons.org/licenses/by/4.0/",
  "canonical_url": "https://geoaeoale.com/en/briefings/perplexity-crawler-roles/",
  "claims": [
    {
      "claim_id": "perplexity-two-agents",
      "text": "Perplexity publishes separate descriptions and IP endpoints for PerplexityBot and Perplexity-User.",
      "status": "observed",
      "confidence": "high",
      "source_ids": [
        "perplexity-bots"
      ],
      "publication_status": "public",
      "confidentiality": "public"
    }
  ],
  "sources": [
    {
      "source_id": "perplexity-bots",
      "canonical_url": "https://docs.perplexity.ai/docs/resources/perplexity-crawlers",
      "title": "Perplexity crawlers",
      "publisher": "Perplexity Docs",
      "source_type": "official",
      "locale": "en",
      "published_at": null,
      "checked_at": "2026-09-11",
      "sha256": "75930d803650ae046c37ed3529840f7cbf1a6599c1bf2e41cc70bf83cc5b6470",
      "license": "Source terms apply",
      "visibility": "public"
    }
  ],
  "related_slugs": [
    "crawler-access-vs-citation",
    "ai-visibility"
  ],
  "limitations": [
    "IP ranges and agent purposes can change; recheck documentation before changing a firewall or report."
  ],
  "corrections": []
}