To prevent undesirable articles while in the look for indexes, webmasters can instruct spiders not to crawl selected information or directories from the standard robots.txt file in the root directory with the area. Furthermore, a webpage might be explicitly excluded from the online search engine's database through the use of https://news.rafeeg.ae