Virtual Robots.txt
Virtual Robots.txt automatically creates a robots.txt file for your site. Your robots.txt file can be…
Plugin Tag: crawler
WP Broken Link Status Checker
Link checker of all your content links and images, looking for broken links, check link…
Robots.txt rewrite
Provide the easy managment of your robots.txt from admin side. It propose you the advanced…
WordPress Robots.txt optimization (+ Sitemap) – Website traffic, ranking & SEO Booster + Woocommerce
Better Robots.txt is an all in one SEO robots.txt plugin, it creates a WordPress virtual…
PHPEnkoder
Encodes mailto: links and e-mail addresses with JavaScript to stifle webcrawlers. Automatically turns plaintext e-mails…
WordPress Meta Keywords
This plugin gives you the ability to set Meta Keywords in WordPress. Under pages it…
WordPress Meta Description
With this plugin you can control the text which will be shown as description on…
Robots.txt Editor
Robots.txt for WordPress
bodi0`s Bots visits counter
Counts the visits from web spiders, crawlers and bots in your blog, with ability to…
DB Robots.txt
DB Robots.txt automatically creates a virtual file robots.txt including special rules for Yandex.
Hide Email Address
Protect email address from being collected by web crawlers.
Block Archive.org via WordPress robots.txt
Adds lines to the virtual robots.txt file to block the archive.org bots.
JaspreetChahal’s wordpress bot detector lite
This plugin detects a few bots (e.g. Google, Google ads, Alta vista etc) and sends…
rexCrawler
rexCrawler is a regular expression web-crawler plugin. Able to parse websites, save data and show…
Page Modified
Display the results of your Page Modified crawls right within your WordPress Admin.