Provisioning
What each family of DataSync holds, field by field. Each family lists the fields Intelligence names in its part of a domain's record, with the labels Intelligence gives them.
Basic and DNS
A domain's DNS records, such as its name servers, mail servers, addresses and TXT records. Who hosts its DNS and its addresses, in which country, and whether the domain is in its zone file.
| Field | What it holds |
|---|---|
DNS hostdns.dns_hosting_company |
|
IP hostdns.ip_hosting_company |
|
IP host countrydns.ip_hosting_company_country |
The country's two-letter code in lower case, such as gb, never a name. |
Name server domainsdns.ns_list |
Each name server's registrable domain, such as cloudflare.com, never a host such as ns1.cloudflare.com. |
MX recordsdns.mx_list |
Each mail server's full host name, such as aspmx.l.google.com. |
IPv4 addressesdns.a_list |
One IPv4 address per value, such as 192.0.2.1. |
IPv6 addressesdns.aaaa_list |
One IPv6 address per value, in its short form, such as 2001:db8::1. |
CNAME targetsdns.cname_list |
|
TXT recordsdns.txt_list |
|
IPv4 address countdns.a_count |
|
IPv6 address countdns.aaaa_count |
|
CNAME countdns.cname_count |
|
Mail server countdns.mx_count |
|
Name server countdns.ns_count |
|
TXT record countdns.txt_count |
|
DNS responsedns.response_status |
|
Zone filedns.zonefile_membership |
|
Zone file datedns.zonefile_date |
WHOIS
A domain's registrar, and the organization, state and country of whoever registered it. The dates it was created, last updated and expires. The statuses and name servers in its WHOIS record, and whether it uses DNSSEC.
| Field | What it holds |
|---|---|
Registrar namewhois.registrar_name |
The registrar's name in lower case, as each registry writes it, so one registrar appears under several names. |
Registrar IANA IDwhois.registrar_iana_id |
The registrar's IANA ID, such as 146 for GoDaddy.com, LLC. |
Registrar (by IANA ID)whois.registrar_name_from_iana_id |
The name IANA gives the registrar's ID, in lower case. |
Createdwhois.creation_date |
|
Expireswhois.expiration_date |
|
WHOIS updatedwhois.update_date |
|
Registry updatedwhois.record_update_date |
|
Registrant countrywhois.registrant_country |
The country's English name in lower case, such as united kingdom, never a code. |
Registrant statewhois.registrant_state |
|
Registrant organisationwhois.registrant_organization |
|
WHOIS statuseswhois.statuses |
Each status in lower case as one word, such as clienttransferprohibited. |
Registration statuswhois.registration_status |
|
WHOIS errorwhois.error |
|
Partial WHOIS reasonwhois.partial_response_reason |
HTML and HTTP
What a domain's website answered when it was crawled: its status codes, its redirects and the address it ended on. What the home page says about itself: its title, description, text and language.
| Field | What it holds |
|---|---|
HTTP statushttp.final_status_code |
The HTTP status code the visit ended on, such as 200 or 404. Some are a host's own, such as Cloudflare's 522. |
Final URLhttp.final_url |
The whole address the visit ended on, with its scheme, path and query. |
Final URL SLDhttp.final_url_sld |
The label before the final address's suffix, such as hugedomains. |
Final URL TLDhttp.final_url_tld |
The final address's suffix without a dot, such as com or co.uk. |
Redirected to same TLDhttp.is_redirected_to_same_tld |
|
Redirected to same SLDhttp.is_redirected_to_same_sld |
|
Redirect counthttp.redirect_count |
|
Redirect status codeshttp.redirect_status_codes |
The HTTP status code of each redirect on the way, such as 301 or 302. |
Crawl statushttp.crawl_status |
|
Titlehtml.title |
|
Descriptionhtml.description |
|
Page texthtml.text |
|
Keywordshtml.keywords |
|
Authorhtml.author |
|
Generatorhtml.generator |
|
Twitter cardhtml.twitter_card |
|
Google site verificationhtml.google_site_verification |
|
Page languagehtml.site_language |
|
Encodinghtml.apparent_encoding |
|
Valid robots.txthtml.has_valid_robots |
|
Page fingerprinthtml.fingerprint |
Forty lower-case hex characters. |
Google Analytics IDhtml.google_analytics_id |
|
GA4 measurement IDhtml.ga4_measurement_id |
|
Tag Manager IDhtml.google_tag_manager_id |
Main Applications
The technology a website runs on, by category, such as its content management system, analytics and web server.
| Field | What it holds |
|---|---|
Any applicationapplications.all |
|
A/B testingapplications.a/b_testing |
|
Advertising networksapplications.advertising_networks |
|
Affiliate programsapplications.affiliate_programs |
|
Analyticsapplications.analytics |
|
Appointment schedulingapplications.appointment_scheduling |
|
Artificial intelligenceapplications.artificial_intelligence |
|
Blogsapplications.blogs |
|
Browser fingerprintingapplications.browser_fingerprinting |
|
Cache toolsapplications.cache_tools |
|
Captchasapplications.captchas |
|
Cart abandonmentapplications.cart_abandonment |
|
CDNapplications.cdn |
|
CMSapplications.cms |
|
Cookie complianceapplications.cookie_compliance |
|
CRMapplications.crm |
|
Databasesapplications.databases |
|
Document management systemsapplications.document_management_systems |
|
Ecommerceapplications.ecommerce |
|
Emailapplications.email |
|
Feature managementapplications.feature_management |
|
Form buildersapplications.form_builders |
|
Fundraising and donationsapplications.fundraising_and_donations |
|
Hosting panelsapplications.hosting_panels |
|
JavaScript frameworksapplications.javascript_frameworks |
|
Live chatapplications.live_chat |
|
Livestreamingapplications.livestreaming |
|
LMSapplications.lms |
|
Mapsapplications.maps |
|
Marketing automationapplications.marketing_automation |
|
Miscellaneousapplications.miscellaneous |
|
Operating systemsapplications.operating_systems |
|
PaaSapplications.paas |
|
Page buildersapplications.page_builders |
|
Payment processorsapplications.payment_processors |
|
Personalisationapplications.personalisation |
|
Photo galleriesapplications.photo_galleries |
|
Programming languagesapplications.programming_languages |
|
Recruitment staffingapplications.recruitment_staffing |
|
Referral marketingapplications.referral_marketing |
|
Reservations and deliveryapplications.reservations_and_delivery |
|
Retargetingapplications.retargeting |
|
Shopify appsapplications.shopify_apps |
|
Social loginapplications.social_login |
|
Static site generatorapplications.static_site_generator |
|
Tag managersapplications.tag_managers |
|
Translationapplications.translation |
|
Video playersapplications.video_players |
|
Web frameworksapplications.web_frameworks |
|
Web serversapplications.web_servers |
|
Widgetsapplications.widgets |
|
Wikisapplications.wikis |
|
WordPress pluginsapplications.wordpress_plugins |
|
WordPress themesapplications.wordpress_themes |
Web Usage
Whether a domain's website is developed, undeveloped, parked, a redirect or inactive. Why it got that label, and the parking company where there is one.
| Field | What it holds |
|---|---|
Web usageusage.web_usage |
|
Web usage reasonusage.web_usage_reason |
The reason behind usage.web_usage. Only some domains have a reason so far, so a count by reason comes out lower than the count by usage.web_usage. A developed site never has one. |
Parking companyusage.parking_company |
|
Inactive statususage.inactive_status |
Inferred
What Domainsbot works out from the rest of a domain's record. The company that provides its website, whether the domain is registered, and its creation date, from WHOIS or from when Domainsbot first saw it.
| Field | What it holds |
|---|---|
Website providerinferred.website_provider |
|
Created (inferred)inferred.creation_date |
SSL
A domain's certificate: its status and type, who issued it, and when it starts and stops being valid. Why the check failed, where it did.
| Field | What it holds |
|---|---|
SSL statusssl.status |
|
SSL expiresssl.expiration_date |
|
SSL valid fromssl.valid_from_date |
|
Certificate typessl.certificate_type |
|
Wildcard certificatessl.has_wildcard_cert |
|
Certificate issuerssl.issuer_common_name |
|
Issuer organisationssl.issuer_organization_name |
|
Issuer countryssl.issuer_country_name |
The country's English name in lower case, such as united states of america. |
SSL error typessl.error_type |
|
SSL error libraryssl.error_library |
|
SSL errorssl.error_description |
|
Certificate fingerprintssl.fingerprint_sha256 |
Lower-case hex pairs separated by spaces, such as 71 5f a0 7c. |
History, Bulk Crawling and quotes
DataSync's history goes back to January 2024.
- Choose your families
- A quote covers the families you choose.
- Bulk Crawling
- Send Domainsbot a list of domains, and Domainsbot has fresh data for every domain on it within 72 hours.
- Ask for a quote
- Ask for a quote on the Provisioning page, and Domainsbot will be in touch.