From discovery to citation: inspect each stage

Use a diagnostic model, not a promise

Discovery, access, retrieval, interpretation, and citation are useful inspection stages. They are not a published universal architecture for every AI product. A page can pass an earlier check while never appearing in an answer. When someone says “the model cannot see us,” ask which observable stage they mean.

Inspect access before rewriting prose

Start with the exact URL. Can an unauthenticated visitor open it? Does it return the expected article rather than a login page, error message, or challenge? Is the important explanation present in readable text? Ask the person responsible for the website to review crawler rules and infrastructure controls; do not disable protections without understanding their purpose.

OpenAI documents separate controls for OAI-SearchBot search crawling and GPTBot model-training crawling. Allowing or denying one does not express the same preference as allowing or denying the other. Use the current crawler documentation when making that decision, and keep the business owner involved.

Inspect the document the reader actually receives

Write down the page title, canonical topic, main answer, publication date, and source links. If the page is a product document, also record the product version. Check a narrow paragraph in isolation: does “it” still refer to a clear thing, and does the numeric claim retain its units and conditions?

A simple worksheet helps keep separate diagnoses from being merged:

Stage	Observation	Next responsible action
Access	Visitor sees a login wall	Confirm whether the page should be public
Content	Main instruction is only in an image	Publish an equivalent text explanation
Evidence	A number has no date or method	Locate the source or remove the claim
Answer	URL appears in one recorded answer	Save the context and verify the cited passage

Interpret a missing citation carefully

No citation in a single response does not identify the failed stage. The question may not need your material, another source may be more relevant, the product may behave differently across runs, or your observation may be incomplete. Log what happened instead of manufacturing a crawler explanation.

Finish with a falsifiable next step

Assign one owner to one check: for example, “verify that this public help page returns its full procedure without signing in.” Record the before state and the result. After a content change, observe again using the same prompt context, but do not present a before-and-after difference as proof that the change caused citation.