# Robots.txt # All robots will spider the domain User-agent: * Disallow:/search Disallow:/search1 Disallow:/search01 Disallow:/search3 Disallow:/test Disallow:/index2.php