Domainsbot MeridianBeta

Sign in

Provisioning

What each family of DataSync holds, field by field. Each family lists the fields Intelligence names in its part of a domain's record, with the labels Intelligence gives them.

Basic and DNS

A domain's DNS records, such as its name servers, mail servers, addresses and TXT records. Who hosts its DNS and its addresses, in which country, and whether the domain is in its zone file.

FieldWhat it holds
DNS host
dns.dns_hosting_company
IP host
dns.ip_hosting_company
IP host country
dns.ip_hosting_company_country
The country's two-letter code in lower case, such as gb, never a name.
Name server domains
dns.ns_list
Each name server's registrable domain, such as cloudflare.com, never a host such as ns1.cloudflare.com.
MX records
dns.mx_list
Each mail server's full host name, such as aspmx.l.google.com.
IPv4 addresses
dns.a_list
One IPv4 address per value, such as 192.0.2.1.
IPv6 addresses
dns.aaaa_list
One IPv6 address per value, in its short form, such as 2001:db8::1.
CNAME targets
dns.cname_list
TXT records
dns.txt_list
IPv4 address count
dns.a_count
IPv6 address count
dns.aaaa_count
CNAME count
dns.cname_count
Mail server count
dns.mx_count
Name server count
dns.ns_count
TXT record count
dns.txt_count
DNS response
dns.response_status
Zone file
dns.zonefile_membership
Zone file date
dns.zonefile_date

WHOIS

A domain's registrar, and the organization, state and country of whoever registered it. The dates it was created, last updated and expires. The statuses and name servers in its WHOIS record, and whether it uses DNSSEC.

FieldWhat it holds
Registrar name
whois.registrar_name
The registrar's name in lower case, as each registry writes it, so one registrar appears under several names.
Registrar IANA ID
whois.registrar_iana_id
The registrar's IANA ID, such as 146 for GoDaddy.com, LLC.
Registrar (by IANA ID)
whois.registrar_name_from_iana_id
The name IANA gives the registrar's ID, in lower case.
Created
whois.creation_date
Expires
whois.expiration_date
WHOIS updated
whois.update_date
Registry updated
whois.record_update_date
Registrant country
whois.registrant_country
The country's English name in lower case, such as united kingdom, never a code.
Registrant state
whois.registrant_state
Registrant organisation
whois.registrant_organization
WHOIS statuses
whois.statuses
Each status in lower case as one word, such as clienttransferprohibited.
Registration status
whois.registration_status
WHOIS error
whois.error
Partial WHOIS reason
whois.partial_response_reason

HTML and HTTP

What a domain's website answered when it was crawled: its status codes, its redirects and the address it ended on. What the home page says about itself: its title, description, text and language.

FieldWhat it holds
HTTP status
http.final_status_code
The HTTP status code the visit ended on, such as 200 or 404. Some are a host's own, such as Cloudflare's 522.
Final URL
http.final_url
The whole address the visit ended on, with its scheme, path and query.
Final URL SLD
http.final_url_sld
The label before the final address's suffix, such as hugedomains.
Final URL TLD
http.final_url_tld
The final address's suffix without a dot, such as com or co.uk.
Redirected to same TLD
http.is_redirected_to_same_tld
Redirected to same SLD
http.is_redirected_to_same_sld
Redirect count
http.redirect_count
Redirect status codes
http.redirect_status_codes
The HTTP status code of each redirect on the way, such as 301 or 302.
Crawl status
http.crawl_status
Title
html.title
Description
html.description
Page text
html.text
Keywords
html.keywords
Author
html.author
Generator
html.generator
Twitter card
html.twitter_card
Google site verification
html.google_site_verification
Page language
html.site_language
Encoding
html.apparent_encoding
Valid robots.txt
html.has_valid_robots
Page fingerprint
html.fingerprint
Forty lower-case hex characters.
Google Analytics ID
html.google_analytics_id
GA4 measurement ID
html.ga4_measurement_id
Tag Manager ID
html.google_tag_manager_id

Main Applications

The technology a website runs on, by category, such as its content management system, analytics and web server.

FieldWhat it holds
Any application
applications.all
A/B testing
applications.a/b_testing
Advertising networks
applications.advertising_networks
Affiliate programs
applications.affiliate_programs
Analytics
applications.analytics
Appointment scheduling
applications.appointment_scheduling
Artificial intelligence
applications.artificial_intelligence
Blogs
applications.blogs
Browser fingerprinting
applications.browser_fingerprinting
Cache tools
applications.cache_tools
Captchas
applications.captchas
Cart abandonment
applications.cart_abandonment
CDN
applications.cdn
CMS
applications.cms
Cookie compliance
applications.cookie_compliance
CRM
applications.crm
Databases
applications.databases
Document management systems
applications.document_management_systems
Ecommerce
applications.ecommerce
Email
applications.email
Feature management
applications.feature_management
Form builders
applications.form_builders
Fundraising and donations
applications.fundraising_and_donations
Hosting panels
applications.hosting_panels
JavaScript frameworks
applications.javascript_frameworks
Live chat
applications.live_chat
Livestreaming
applications.livestreaming
LMS
applications.lms
Maps
applications.maps
Marketing automation
applications.marketing_automation
Miscellaneous
applications.miscellaneous
Operating systems
applications.operating_systems
PaaS
applications.paas
Page builders
applications.page_builders
Payment processors
applications.payment_processors
Personalisation
applications.personalisation
Photo galleries
applications.photo_galleries
Programming languages
applications.programming_languages
Recruitment staffing
applications.recruitment_staffing
Referral marketing
applications.referral_marketing
Reservations and delivery
applications.reservations_and_delivery
Retargeting
applications.retargeting
Shopify apps
applications.shopify_apps
Social login
applications.social_login
Static site generator
applications.static_site_generator
Tag managers
applications.tag_managers
Translation
applications.translation
Video players
applications.video_players
Web frameworks
applications.web_frameworks
Web servers
applications.web_servers
Widgets
applications.widgets
Wikis
applications.wikis
WordPress plugins
applications.wordpress_plugins
WordPress themes
applications.wordpress_themes

Web Usage

Whether a domain's website is developed, undeveloped, parked, a redirect or inactive. Why it got that label, and the parking company where there is one.

FieldWhat it holds
Web usage
usage.web_usage
Web usage reason
usage.web_usage_reason
The reason behind usage.web_usage. Only some domains have a reason so far, so a count by reason comes out lower than the count by usage.web_usage. A developed site never has one.
Parking company
usage.parking_company
Inactive status
usage.inactive_status

Inferred

What Domainsbot works out from the rest of a domain's record. The company that provides its website, whether the domain is registered, and its creation date, from WHOIS or from when Domainsbot first saw it.

FieldWhat it holds
Website provider
inferred.website_provider
Created (inferred)
inferred.creation_date

SSL

A domain's certificate: its status and type, who issued it, and when it starts and stops being valid. Why the check failed, where it did.

FieldWhat it holds
SSL status
ssl.status
SSL expires
ssl.expiration_date
SSL valid from
ssl.valid_from_date
Certificate type
ssl.certificate_type
Wildcard certificate
ssl.has_wildcard_cert
Certificate issuer
ssl.issuer_common_name
Issuer organisation
ssl.issuer_organization_name
Issuer country
ssl.issuer_country_name
The country's English name in lower case, such as united states of america.
SSL error type
ssl.error_type
SSL error library
ssl.error_library
SSL error
ssl.error_description
Certificate fingerprint
ssl.fingerprint_sha256
Lower-case hex pairs separated by spaces, such as 71 5f a0 7c.

History, Bulk Crawling and quotes

DataSync's history goes back to January 2024.

Choose your families
A quote covers the families you choose.
Bulk Crawling
Send Domainsbot a list of domains, and Domainsbot has fresh data for every domain on it within 72 hours.
Ask for a quote
Ask for a quote on the Provisioning page, and Domainsbot will be in touch.