Each webmaster knows that in order for people to start accessing his resource from search engines, his indexing is necessary. We will talk about what site indexation is, how it is carried out, and what is its meaning, in this article.
What is indexing?
So, the word "indexation" in itself means entering into the register something, a census of materials that are available. The same principle applies to site indexing. In fact, this process can also be called entering information about Internet resources into the database of search engines.
Thus, as soon as the user drives another phrase into the Google search field, the script will return the result, including the title of your site and its brief description, which we see below.
How is indexing done?
Indexing itself (Yandex or Google doesnât play a role) is pretty simple. The entire web of the Internet, focusing on the base of IP addresses that search engines have, is scanned by powerful robots - âspidersâ that collect information about your site. Each of the search engines has a huge number of them, and they work automatically 24 hours a day. Their task is to go to your site and âreadâ all the content on it, while entering the data into the database.
Therefore, in theory, indexing a site is little dependent on the owner of the resource. The decisive factor here is a search robot that visits the site and explores it. This is what affects how quickly your site appears in search results.
Indexing Dates?
Of course, it is beneficial for each webmaster to have his resource appear in search results as quickly as possible. This will affect, firstly, the timing of the siteâs withdrawal to first positions, and, secondly, when the first stages of site monetization will begin. Thus, the sooner the search robot "eats" all the pages of your resource, the better.
Each search engine has its own algorithm for entering data about sites in its database. For example, indexing pages in Yandex is carried out in stages: robots crawl sites constantly, then organize information, after which the so-called update takes place, when all changes take effect. The company does not establish the regularity of such events: they are held every 5-7 days (as a rule), however, they can take place both in 2 and 15 days.
At the same time, indexing a site on Google is using a different model. In this search system, such "updates" (database updates) are held regularly, in this regard, it is not necessary to wait every time the robots enter information into the database, and then it will be ordered once every few days.
Based on the foregoing, we can draw the following conclusion: pages in Yandex are added after 1-2 updates (that is, 7-20 days on average), and on Google this can happen much faster - literally in a day.
In this case, of course, each search engine has its own characteristics of how indexing is carried out. Yandex, for example, has the so-called "speed of work" - a robot that can enter data into the results in a few hours. True, to make it go to your resource is not easy: this mainly concerns news and various high-profile events that develop in real time.
How to get into the index?
The answer to the question of how to enter information about your site in the search engine index is both simple and complex. Indexing pages is a natural phenomenon, and if you donât even think about it, but simply, say, keep your blog, gradually filling it with information, search engines will âswallowâ your content over time.
Another thing is when you need to speed up page indexing, for example, if you have a network of so-called âsatellitesâ (sites designed to sell links or advertisements, the quality of which, as a rule, is worse). In this case, it is necessary to take measures so that the robots notice your site. The following are considered common: adding a website URL to a special form (it is called âAddUrlâ); running a resource address through link directories; adding addresses to bookmark directories and much more. How each of these methods works, there are numerous discussions on SEO forums. As practice shows, each case is unique, and itâs difficult to find the reasons why one site was indexed in 10 days and the other in 2 months.
How to speed up getting into the index?
Nevertheless, the logic with which you can make the site get into the index faster is based on the placement of links to it. In particular, we are talking about setting the URL on free and publicly accessible sites (bookmarks, directories, blogs, forums); on the purchase of links on large and popular sites (using the Sape exchange, for example); as well as adding a sitemap to the addURL form. There may be other methods, but those that have already been listed can safely be called the most popular. Recall, in general, it all depends on the site and the luck of its owner.
Which sites are indexed?
According to the official position of all search engines, sites that go through a series of filters fall into the index. What requirements the latter contain, no one knows. It is only known that over time, all of them are being improved in such a way as to weed out pseudo-sites created for earning from selling links and other resources that do not carry useful information for the user. Of course, for the creators of these sites, the main task is to index pages as much as possible (to attract visitors, sell links, and so on).
What resources do search engines bany?
Based on the previous information, we can conclude which sites are most likely not to fall into the search results. The same information is voiced by official representatives of search engines. First of all, these are sites containing non-unique, automatically generated content that is not useful to visitors. These are followed by resources in which the minimum information created to sell links and so on.
True, if you analyze the search engine results, then you can find all these sites in it. Therefore, if we talk about sites that will not be present in the SERP, it should be noted not only non-unique content, but also a number of other factors - many links, incorrectly organized structure and so on.
Hide content. How to prevent page indexing?
Search engines crawl all the content on the site. However, there is a technique with which you can limit the access of search robots to a particular section. This is done using the robots.txt file, to which search engine spiders respond.
If you place this file in the root of the site, the indexing of the pages will take place according to the script that is written in it. In particular, you can disable indexing with a single command - Disallow. In addition to it, in the file you can also specify sections of the site to which this prohibition will apply. For example, to prohibit the entire site from entering the index, it is enough to specify one slash â/â; and in order to exclude the âshopâ section from the output, it is enough to indicate this characteristic in your file: â/ shopâ. As you can see, everything is logical and extremely simple. Indexing pages closes very easily. At the same time, search robots go to your page, read robots.txt and do not enter data into the database. So you can easily manipulate to see in the search for those or other characteristics of sites. Now let's talk about how the index is checked.
How can I check page indexing?
There are several ways to find out how many and which pages are present in the Yandex or Google database. The first - the easiest - is to enter the appropriate query into the search form. It looks like this: site: domen.ru, where instead of domain.ru you write down, accordingly, the address of your site. When you make such a request, the search engine will show all the results (pages) located at the specified URL. Moreover, in addition to simply listing all the pages, you can also see the total number of indexed material (to the right of the phrase âNumber of resultsâ).
The second way is to check page indexing using specialized services. There are a large number of them now, without a doubt you can call xseo.in and cy-pr.com. On such resources you can not only see the total number of pages, but also determine the quality of some of them. However, you only need this if you are more in-depth knowledge of this topic. These are usually professional SEO tools.
About "forced" indexing
I would also like to write a little about the so-called "forced" indexing, when a person tries to drive his site into the index using various "aggressive" methods. Optimizers do not recommend doing this.
Search engines at least, noticing the excessive activity associated with the new resource, may put in place some kind of sanctions that adversely affect the state of the site. Therefore, it is better to do everything so that the indexing of the pages looks as organic as possible, gradual and smooth.