The least squares method (least squares method) allows us to estimate various quantities using the results of many measurements containing random errors.
Characteristic of OLS
The main idea of this method is that as a criterion for the accuracy of the solution to the problem, the sum of the squared errors is considered, which they seek to minimize. Using this method, both numerical and analytical approaches can be applied.
In particular, as a numerical implementation, the least squares method involves performing as many measurements as possible of an unknown random variable. Moreover, the more calculations, the more accurate the solution. On this set of calculations (source data), another set of proposed solutions is obtained, from which the best one is then selected. If the set of solutions is parameterized, then the least squares method is reduced to finding the optimal value of the parameters.
As an analytical approach to the implementation of OLSs on a set of initial data (measurements) and the proposed set of solutions, a certain functional dependence (functional) is determined, which can be expressed by a formula obtained as a hypothesis that needs confirmation. In this case, the least squares method is reduced to finding the minimum of this functional on the set of squares of the errors of the source data.
Note that it is not the errors themselves, but the squares of errors. Why? The fact is that deviations of measurements from the exact value are often both positive and negative. When determining the average measurement error, a simple summation can lead to an incorrect conclusion about the quality of the estimate, since the mutual destruction of positive and negative values will reduce the sample power of the set of measurements. And, therefore, the accuracy of the assessment.
In order for this not to happen, and the squares of the deviations are summed. Even more so, in order to equalize the dimension of the measured quantity and the final estimate, the square root is extracted from the sum of the squares of the errors .
Some OLS applications
MNCs are widely used in various fields. For example, in probability theory and mathematical statistics, the method is used to determine such a characteristic of a random variable as the standard deviation, which determines the width of the range of values of a random variable.
In mathematical analysis and various fields of physics that use this apparatus to derive or confirm hypotheses, OLSs are used, in particular, to estimate the approximate representation of functions defined on numerical sets by simpler functions that allow analytic transformations.
Another application of this method is the separation of the useful signal from the noise imposed on it in filtering problems.
Another area of application for OLS is econometrics. Here, this method is so widely used that some special modifications were defined for it.
Most of the problems of econometrics, one way or another, comes down to solving systems of linear econometric equations that describe the behavior of some systems - structural models. The main element of each such model is a time series, which is a set of some characteristics whose values depend on both time and a number of other factors. In this case, there may be a correspondence between the internal (endogenous) characteristics of the model and external (exogenous) characteristics. This correspondence is usually expressed in the form of systems of linear economic equations.
A characteristic feature of such systems is the presence of relationships between individual variables, which, on the one hand, complicate it, and on the other, redefine it. What is the reason for the appearance of uncertainty when choosing a solution to such systems. An additional factor complicating the solution of such problems is the dependence of the model parameters on time.
The main goal of econometric problems is the identification of models, that is, the determination of structural relationships in the selected model, as well as the evaluation of a number of its parameters.
The restoration of dependencies in the time series that make up the model can be performed, in particular, using both direct least-squares method and some of its modifications, as well as a number of other methods. Special modifications of OLS in solving such problems are specially developed to solve certain problems arising in the process of numerical solution of systems of equations.
In particular, one of these problems is associated with the presence of initial restrictions on the parameters that need to be evaluated. For example, the income of a private enterprise can be spent on consumption or on its development. Consequently, the sum of the parts of the data of the two types of costs is obviously equal to 1. These parts can enter the system of econometric equations independently of each other. Therefore, it is possible to evaluate various types of expenses using the least-squares method, without taking into account the initial constraint, and then adjust the result. This solution is called the indirect least squares method.
The indirect least squares method (CMNC) is used for a well-defined structural model. The KMNK algorithm assumes the following actions:
1) the transformation of the structural model into a simpler, reduced form by introducing an additional dependence;
2) using conventional least squares estimation of the reduced coefficients for each equation of the simplified model;
3) the obtained coefficients of the simple form of the model are converted into the parameters of the original structural model.
It is worth noting that KVMKs are not used for super-identifiable systems, since in this case it is impossible to specify unambiguous estimates of the parameters of the structural model. For such models, one more modification of the least squares method can be used - the two - step least-squares method (DMSC).
The DMNA algorithm is as follows:
1) on the basis of a simplified model, calculate for an overidentifiable equation the values of the internal variables that are contained in the right side of the equation;
2) substitute the obtained values of the variables in the place of the corresponding actual variables in the original model and apply the usual OLS again.
A detailed description of the indirect and two-step least squares methods is given in many econometrics textbooks. The peculiarity of these methods, as well as the usual OLS, in their universality, which allows using them to estimate the coefficients of any structural model in any subject area.