What is documented

Perplexity's official page describes rules for its web crawlers and connects them to how the service obtains web pages. That gives us a useful boundary: we can verify the agent name, request, and server response, but not the internal order of the index or model.

This profile does not turn documentation into a growth promise. Any visibility claim should point to an observation with a date, prompt, and archived answer.

Section sources:[1] Perplexity Docs

PerplexityBot and Perplexity-User

Do not merge automated crawling with a user request. PerplexityBot is an agent identifier to verify in logs; Perplexity-User refers to a separate access category described in the documentation. Neither string means that a particular visitor read your page.

Keep the User-Agent, IP verification, URL, response code, and date and time. A name in the User-Agent alone is not proof of authenticity.

Section sources:[1] Perplexity Docs

Technical access

Start with robots.txt, DNS, TLS, redirects, authentication, and the final HTML. For each page, record whether an allowed crawler can receive a 200 response and see the main text without executing JavaScript.

Access is necessary for observability, but it is not a selection condition. Hidden data, incorrect canonicals, and temporary failures can break the chain after a single successful request.

Section sources:[1] Perplexity Docs

Crawling is not indexing

A server-log line answers ‘who requested this URL and what did it receive?’ It does not answer whether the document was retained, which version was chosen, or whether it will be found for a question. These levels must not be collapsed into one metric.

For change analysis, compare page snapshots, publication dates, and server responses, not only crawler-request counts.

Section sources:[1] Perplexity Docs

How the system finds material for an answer

For a particular prompt, the system searches for and selects material that may help form the answer. An outside observer usually cannot see every page considered or reconstruct the selection reason from one answer.

Test a stable prompt set and archive the exact answer with its links. Change one variable at a time; otherwise prompt effects cannot be separated from platform effects.

Section sources:[1] Perplexity Docs

A displayed source link and a site visit

A source link belongs to one particular answer. A site visit occurs when a person follows that link and appears in analytics. An unlinked mention, a domain link, and an actual user visit are three different results, each requiring different evidence.

Store the URL, answer excerpt, date, and link type in an observation record. Do not call a visit proof that the model used that exact paragraph.

Section sources:[1] Perplexity Docs

How to read logs

Separate verified PerplexityBot requests, ordinary users, and unknown automation. Reverse-DNS and official-range checks reduce misclassification, but do not reveal what the service did with requested data.

Aggregate pages by URL and period: status, bytes, latency, error share, and repeat requests. Retain raw IP data minimally and for a limited period.

Section sources:[1] Perplexity Docs

A workable measurement model

Build a sequence: page access → confirmed bot visit → possible use in an answer → displayed source link → visit by a person. The first two stages can be supported by your logs; the later stages require archived answers and analytics and remain sampled.

Publish the total count, period, language, prompt set, and limitations. Treat any content-preparation effect as a hypothesis until a repeated controlled experiment supports it.

Section sources:[1] Perplexity Docs

What this does not prove

  • The source describes crawler rules, but does not disclose crawl completeness, indexing algorithms, candidate selection, ranking, or source link probability. Interfaces and policies may change; repeat observations with their dates.

Sources

  1. 1
    Perplexity crawlersPerplexity Docs · official · 11 Sept 2026

Correction history

No material corrections have been published.