Purpose
Runs technical SEO and public website audits of dental-practice websites — on demand when a DentalPresence user requests a score, and on a scheduled basis to maintain the public Dental Index and published research benchmarks.
Crawler documentation
DentalPresence Audit Bot
DentalPresence operates this crawler to evaluate publicly accessible dental-practice websites — when a user requests a DentalPresence Score or website audit, and on a scheduled basis to maintain the public Dental Index and the national website-readiness study.
Bounded access
Each audit reads robots.txt and sitemap metadata, then analyzes no more than three public HTML pages per website. The same per-site bound applies to on-demand audits and scheduled index crawls.
Transparent identity
Requests use the published user agent and cryptographic Web Bot Auth signatures.
Crawler identity
- Operator
- DentalPresence.in
- User agent
- Mozilla/5.0 (compatible; DentalPresenceAuditBot/1.0; +https://dentalpresence.in/crawler)
- Contact
- hello@dentalpresence.in
Crawl policy
- Reads robots.txt before requesting site content and does not request paths disallowed for this crawler.
- Honors applicable crawl-delay directives and keeps each audit to a small, bounded set of requests.
- Accesses only public pages. It does not sign in, submit forms, make purchases, or attempt to bypass access controls.
- Does not intentionally collect patient information. DentalPresence audits public business and marketing signals only.
- Uses Web Bot Auth signatures so compatible protection services can verify that requests originate from DentalPresence.
- Website owners can opt out of scheduled index crawls by disallowing the published user agent in robots.txt, and can request a correction, re-crawl, or removal from any published practice page — no account required.
Website-owner requests
To report unexpected crawler behavior or request investigation, email hello@dentalpresence.in with the affected hostname and approximate request time.