Home
About me
Advertise
Sitemap
Wordpress SEO
Contact
Hire me
SEO jargon busters
RSS feeds
Daily SEO Blog
Jul
26

Ensure effective Google crawling on blogs

If you don’t know nothing about robots.txt and controlling robots through Meta tags -and don’t want to take the pain of editing the txt file in your root folder and disallowing each search engine crawler manually – worry not! There is an easy way to get this done in Wordpress.

Goto Dashboard> Options> Privacy settings.

You will find two options here.

1 – I would like my blog to be visible to everyone, including search engines
2 – I would like to block search engines, but allow normal visitors

Now, it’s obvious that the first option is what you need.Ensure that the second option is not selected.

What happens when you select the first(Correction – Second) option?
It causes the - meta name=’robots’ content=’noindex,nofollow’ to appear automatically into the page head section of every page. Thus it will not allow Google or any other search engine to crawl the site at all.

It also creates a robots.txt file in the root folder(Only if the WP installation is in the root folder, and there is no existing robots.txt file), in which the command for disallowing all crawlers into the entire directory.

Ensure that you select the first(I would like my blog to be visible to everyone, including search engines) radio button. Be a smart SEO. :)

Category: SEO Tips | Author: Mani Karthik
SEO Wordpress Themes
  1. StevenNo Gravatar

    Your posts makes no sense to me. At first you tell us to not select the second option because the first option is the one we need. See here:

    “Ensure that the second option is not selected.”

    Then you tell us the first option blocks Google or any other search engines from crawling our site at all (opposite of your previous statement) and tell us to select the 2nd radio button. I think the second half of your post is incorrect.

    Reply

  2. adminNo Gravatar

    Steven , thanks for notifying me. Actually i mean the second option.(See correction)

    Sometimes, even an excessive dose of caffeine can’t stop you from typos.

    Cheers!
    Mani

    Reply

  3. anonymouseNo Gravatar

    How do yo place ads randomly in the post?.
    Do you use some wordpress adsense plugins?
    I’ve disabled Javascript ,now ads are no longer visible to me.

    Reply

  4. SEO Elite ReviewNo Gravatar

    Disabling Javascript will make ur ads disappear. Javascript Ads are used by adsense and other suppliers as well.. They need JS enabled or else tragedy :(

    Reply

  5. DineshNo Gravatar

    My Blog has 300 plus link, but google does not crawl all the link. any idea for google crawl ma all the links

    Reply





Click to cancel reply

  • Daily SEO Blog Sponsors

  • Subscribe to SEO Blog

    Enter your email address:

  • SEO Blogging tips

    • Ask Google how tall Nate Robinson is and it will tell you
    • Notes from the Google webmasters “Tricks and Treats” event
    • Use AllinOne SEO plugin to it's fullest potential
    • 6 free SEO friendly blogger templates
    • Get serious with off site optimization and incoming links
    • Article Submission - Do they work at all ?
    • Search Engine Optimization for images -Illustration
    • Boost your traffic with correct use of meta description tag
    • Forget all other SEO plugins, here's a brand new one to replace
    • Prepare your blog for this Google PR update
  • twitter software
  • Recent SEO blog posts

    • 5 Humble Reasons why I think Bing is actually better than Google !
    • Find your Twitter personality – Try the new Yahoo Mojo
    • 5 Cool SEO iPhone Apps for checking SEO stats and reports
    • 10 Cool Twitter Custom Button Generators & Badges
    • 5 SEO Mistakes I Learned from as an Online Marketer !
    • Twitter is for ? The Google suggest way
    • Google’s secret project is looking for feedback from testers – Interested ?
    • 8 Habits of Successful & Popular Twitter users for Inspiration
    • New WordPress theme “Bold” from Elegant Themes
    • Is shortening posts by excerpts on the homepage, good SEO ?
    • 10 Excellent WordPress Themes powered with Ajax & JQuery
    • SEM is all about blowing up your money, just because you have a budget !
    • Check design compatibility on various screen resolutions & devices like the Wii browser
    • WordPress Plugins to Display AdSense ads only to Google visitors
    • Top 20 SEO Blog feeds you should subscribe to ( Keep updated with Google )
  • Active users @ SEO Blog

    • Nihar
    • darrenmonroe (Darren S...
    • Jacques Snyman | Web D...
    • Anish K.S
    • Darren Scott Monroe
  • Tags @ Daily SEO Blog

    ads adsense backlinks blog blogger Blogging blogosphere blogs categories check crawler crawling crowd Excel google image images indexing india indian instances internet javascript keywords links pages rank robots search engine optimization seo tips serps sitemap submit tags templates theme themes upload video wordpress Wordpress SEO wordpress seo wordpress tips yahoo youtube
  • Popular at Daily SEO blog

    • 10 Twitter tools to effectively manage your followers (101)
    • How to get indexed by Google in 48 hours (93)
    • How to get free backlinks (68)
    • Free SEO Wordpress theme – SEO Blog (67)
    • 9 Twitter Tips that will help you gain respect in the Twitterverse (Like @Zaibatsu) (61)
    • The 50 Most Influential & Top Twitter Users in India (51)
    • New Wordpress theme - "SEO Green" from DailySEOblog (49)
    • 25 SEO Gurus you should follow on Twitter (48)
    • 30 Web Design SEO Tips for Web Masters (45)
    • SEO for Wordpress - A quick guide (44)
  • Blogger Podcasts

    Monika Mundell - Blogger(thewritersmanifesto.com) Daniel Scocco - Blogger (Dailyblogtips.com)
  • Social Media Profiles

Basic SEO Tips

  • Change the way your site appears on Google SERPs
  • Control Google's crawl frequency to your site
  • How does Google see your site?
  • Importance of footer text in SEO
  • Optimizing header images
  • SEO tips for MSN search engine
  • Optimizing blog titles for Google
  • SEO friendly layout
  • What is an SEO friendly site structure?
  • Importance of keywords - SEO Tips
  • Anchor text importance - SEO Tip
  • SEO Tips for blogs and bloggers
  • Absolute links or relative links is good for SEO?
  • Important SEO tips for wordpress

SEO Tips for blogs

  • 24 Must have SEO plugins for Wordpress
  • SEO Tips - Copy writing guide
  • How to get quality backlinks?
  • Social media and link building tips
  • SEO Tips - Landing page Optimization
  • How to increase traffic to your blog?
  • How to increase your Page Rank?
  • Image optimization tips for blogs
  • 7 steps to your blog's SEO
  • How and where to find incoming links?
  • SEO tips for Google
  • SEO metrics to track
  • How to build a sitemap for blogger
  • How to get Google sitelinks?
  • Find out when Google indexed you
Mani Karthik Hi, I'm Mani Karthik, Pro Blogger & SEO Consultant from India. I blog, I Tweet, help websites with SEO. Loves working from home office. Quit my day job as SEO consultant 4 years ago, to pursue full time blogging and consultation. Living the dream now.
>> More about me


© 2009 DailySEOblog.com Privacy policy