Welcome to Theadsensefamily Forum

Dear Guest Welcome To Theadsensefamily!

Biggest Internet Marketing Forum In Nigeria!

Please Sign up Below!

Register
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

What Is A Robots.txt File

Discussion in 'Questions And Answer Forum' started by Bala Hassan, Nov 8, 2020.

Tags:
Top Posters
  1. Bal

    Bala Hassan Elite Member

    what is a robots.txt file? and how do I create it
     
    Bala Hassan, Nov 8, 2020
    #1
  2. Theadsensefamily

    Theadsensefamily Jr.Vip Jr.Vip

    Best Answer
    Robots.txt is a text file on the host of a website that indicates to search engines which pages may and may not be indexed .

    Example

    User-agent: *
    Disallow: /?page=/
    Disallow: /find-new/
    Disallow: /account/
    Disallow: /attachments/
    Disallow: /goto/
    Disallow: /posts/

    Allow

    Sitemap: Theadsensefamily.com/sitemap.php

    Disallow here tells search engines and robots not to index pages after that...

    Allow does the opposite
     
    Theadsensefamily, Nov 8, 2020
    #2
    • Like Like x 1

Share This Page

Loading...
Loading...