Bots & Crawlers???

[Drop-Forged]

Actually I have two questionsà¢Ãƒ ¢Ã¢â‚¬Å¡Ã‚ ¬Ãƒâ€šÃ‚ ¦.

First:

Has anyone either moded the bots mod, or found a better way of inserting the bots.


When I started my site it was easy to keep up, but then after a while I was overwhelmed with Google, Inktomi, and Yahoo Bots, (Yahoo the lesser of the three).

I tried wildcards, but still have too many inserts to ad any more manually.

I have hundreds, and hundreds of bots to add.

Can a select all, add all be adapted???

Even if it would be a couple hour process like rebuilding the search, it would definetly be worth it.




My second question is:

I use to have a script from scriptwiz on my sites that indexed all my public forums and made them into fake .html docs with topic urls.

Im guessing the reason that I cant get it to work correctly anymore is because of the added security measures since the hacking episodes.

Does anyone know a good script for this, or something I can change to get the scriptwiz version to work without creating a security hole???