RapidBot Support

RapidBot FAQ

RapidBot Manual

RapidBot FAQ - Why can't I see the robot.txt file in Preview?

RapidBot creates a robots.txt file that should not be read from others then Web crawlers.
That's why pushing the "Preview" button you'll get this message:

Hide the RapidBot page from the menu. Go to Page Inspector > General and un-check "Show in menu".


RapidBot FAQ - What is a robots.txt file?

A robots.txt file restricts access to your site by search engine robots that crawl the web. These bots are automated, and before they access pages of a site, they check to see if a robots.txt file exists that prevents them from accessing certain pages. Though all respectable robots will respect the directives in a robots.txt file, some may interpret them differently.

You need a robots.txt file if your site includes content that you don't want search engines to index.

See Wikipedia robots.txt sample file for reference.

RapidBot Manual - Introduction | What is RapidBot?

RapidBot Banner.png

RapidBot is the incredibly easy-to-use RapidWeaver plugin to generate the robots.txt file for your site.

A robot (also known as Spider or Web Crawler) is a program that automatically traverses the Web's hypertext structure by retrieving a document and recursively retrieving all documents that are referenced.

You decide what spiders will visit in your site. Choose a crawler among our presets or a custom one. Then type a folder name or select it using default link picker. RapidBot is perfectly integrated with RapidWeaver!

Don't Get lost

Provide specific indexing rules thanks to RapidBot. Enable or disable files and folders crawling in a natural way: RapidBot will translate your directives into something that is understood by search engine bots.

RapidBot search engines

Teach Bots how you want to be reached

Don't forget that instructing search engines bots help you adding visibility to your site and let people reach you in a more efficient way excluding irrelevant contents.
RapidBot is a must-have SEO tool! Only if search engines know what to do with your pages, they can give you a good ranking.

Discover features included in RapidBot!

Next:Requirements and installation

RapidBot Manual - Introduction | RapidBot requirement and Installation

Previous:*What is RapidBot* ##RapidBot requirements * RapidBot requires RapidWeaver 6.x or higher and an Intel-based Mac. You can buy your RapidWeaver license from Realmac Software store (RapidWeaver 6.x requires Mac OS X 10.9 or greater, 50 MB disk space).

##RapidBot installation After downloading the RapidBot plugin from our website, follow the instructions on how to install the plugin in the General FAQ section.
RapidBot is a shareware: please test it before buy it and check if it suits your needs. When you decide to purchase a RapidBot license on omnidea store, follow the instruction on how to register in the General FAQ section.

Next:* Adding a RapidBot page*

RapidBot Manual - Introduction | Adding a RapidBot page to your project


Previous: RapidBot Requirement and Installation

Open your existing RapidWeaver Project or create a new one(1) by clicking the plus button on the left-bottom corner of RapidWeaver panel. Select RapidBot icon (2) among third party plugins and push "Choose" button (3).


RapidBot panel will show.


Next: Adding a rule

RapidBot Manual - RapidBot GUI | Adding a rule


Previous: Adding a RapidBot Page

Robots.txt file is made of simple rules, made of one or more directive(s). Each directive can enable or disable access to specific content.

Add a new rule clicking on + button on the bottom left corner.

From right pane, you'll be able to specify:

Which bot(s) rule apply to.

* means every bot. You may want to set a custom user-agent or choose from the pre-defined list. A comprehensive list of known bots is available here.


Adding a directive to the rule:

Add a new directive clicking on + button on the bottom right corner.

Choose directive type:


Choose Allow or Disallow for content:


Choose a content filter:

All content means every file on site, or you can set more specify filters based on files (and folder) name and extensions.



Enter a text comment that will be included in robots.txt rule. Bots won't read this information.

Clicking on - button will remove the selected rule or directive.

Next: RapidBot settings

RapidBot Manual - Settings | Page Options > General


Previous: RapidBot GUI | Adding a rule


Enter the sitemap.xml URL of your site on this field if you want to include it in your Robots.txt file.

The sitemap URL should be an absolute path.


Pushing Online validation... opens an online tool to validate your robots.txt file.

Next: RapidBot settings | About

RapidBot Manual - Settings | Page Options > About


Previous: Page Options >General

This tab will let you know which version of RapidBot you're running and other license details (such as name and email address linked to the license).

If you like to stay up-to-date, push the "Check for updates" button. (Remember that RapidWeaver also checks for add-ons updates at start).

Clicking on RapidBot icon will land you to the official RapidBot page on omnidea site.