Comments on: 10 common mistakes using robots.txt on your website http://www.thomsonchemmanoor.com/10-common-mistakes-using-robotstxt-on-your-website.html Useful Information for Web site development & Search Engine Optimization Marketing Fri, 29 Jan 2010 12:30:14 -0700 http://wordpress.org/?v=2.9.1 hourly 1 By: Website Designing Hyderabad http://www.thomsonchemmanoor.com/10-common-mistakes-using-robotstxt-on-your-website.html/comment-page-1#comment-64895 Website Designing Hyderabad Tue, 01 Dec 2009 08:08:19 +0000 http://www.thomsonchemmanoor.com/?p=209#comment-64895 Thanks for valuable information We will follow your rules to create robots txt file for our websites. Thanks for valuable information We will follow your rules to create robots txt file for our websites.

]]> By: Web development lucknow http://www.thomsonchemmanoor.com/10-common-mistakes-using-robotstxt-on-your-website.html/comment-page-1#comment-64847 Web development lucknow Wed, 16 Sep 2009 10:49:22 +0000 http://www.thomsonchemmanoor.com/?p=209#comment-64847 Thanks for providing this info. Thanks for providing this info.

]]> By: Analog Designer http://www.thomsonchemmanoor.com/10-common-mistakes-using-robotstxt-on-your-website.html/comment-page-1#comment-64777 Analog Designer Fri, 12 Jun 2009 12:16:11 +0000 http://www.thomsonchemmanoor.com/?p=209#comment-64777 Hello, 27 (out of 100) of my site links have been blocked by robots.txt file. Would it be a problem for getting traffic to our site? If so, what should i do to unblock the links Hello,

27 (out of 100) of my site links have been blocked by robots.txt file. Would it be a problem for getting traffic to our site? If so, what should i do to unblock the links

]]> By: vetweb http://www.thomsonchemmanoor.com/10-common-mistakes-using-robotstxt-on-your-website.html/comment-page-1#comment-64392 vetweb Sun, 15 Mar 2009 11:49:54 +0000 http://www.thomsonchemmanoor.com/?p=209#comment-64392 This is awesome. Thanks for the list! This is awesome. Thanks for the list!

]]> By: blpgirl http://www.thomsonchemmanoor.com/10-common-mistakes-using-robotstxt-on-your-website.html/comment-page-1#comment-64387 blpgirl Sat, 14 Mar 2009 21:40:09 +0000 http://www.thomsonchemmanoor.com/?p=209#comment-64387 You have come up with a very nice list, great information. Just one thing, google do support the "Allow:"command, one use they make is if you want to disallow your pages to be crawled but you want to keep showing Adsense ads on those pages then you do a disallow for all bots and allow for the google bot that crawls pages with ads. You have come up with a very nice list, great information.

Just one thing, google do support the “Allow:”command, one use they make is if you want to disallow your pages to be crawled but you want to keep showing Adsense ads on those pages then you do a disallow for all bots and allow for the google bot that crawls pages with ads.

]]>