What is the InfoTigerBot ?
InfoTigerBot is our web robot (bot) for the infotiger search engine's web crawler. Basicall, the InfoTigerBot is browsing the web and retrieving web pages, that might be parsed and possibly indexed in the infotiger search engine, later on.
How to Authenticate InfoTigerBot
InfoTigerBot is usually identified by the following User-Agent string:
Mozilla/5.0 (compatible; InfoTigerBot/1.9; +https://infotiger.com/bot)
the source IP addresses of our crawlers do resolve similar to:
You could verify authenticity by comparing the reverse DNS (ip->name) and forward DNS (name->ip), which should be matching the original IP address:
# nslookup 22.214.171.124
126.96.36.199.in-addr.arpa name = crawler-40-88-143-91.infotiger.com.
# nslookup crawler-40-88-143-91.infotiger.com
Please be aware that User-Agent strings can be easily spoofed, so not every request with InfoTigerBot in it's name, might actually come from our crawler.
Controlling the InfoTigerBot
InfoTigerBot is always (tryin to be) polite, and limits the number of requests per minute to a single host.
InfoTigerBot is honoring robots.txt rules as defined by the robots exclusion standard.
additionally, InfoTigerBot is respecting robots meta tags in HTML header, specifically noindex and nofollow.
If you do not want your web pages indexed by the infotiger search engine, you can easily exclude our InfoTigerBot by editing your robots.txt or robots meta tags appropriately.
How can I add my URL to the infotiger index?
You can submit your URL to be crawled by the InfoTigerBot on our add URL page
Should you have any questions or want to report a problem with the InfoTigerBot, please drop us a message at contact (at) infotiger (dot) com.