The ideas of parallel computing and information processing have long been the lot of specialists and a rather significant problem in terms of implementation. They acquired special significance and mass interest not so long ago.
It can be argued that it was the development of Internet technologies that gave a new impetus and parallel programming acquired new consumer qualities. This led not only to the obvious progress of technologies and programming languages. This actually created an inverse effect on the understanding of the parallel process.
Parallel programming technologies have changed dramatically. The initial use of computer devices as calculators smoothly switched to using them as information processors. Tough architectural solutions gave way to semantics and flexible distribution of software functionality among “hardware executors”.
Parallel computing: meaning and implementation
Initially, the foundations of parallel programming were laid in the architecture of computing devices. A classification was proposed based on the concept of flow.
A sequence of instructions, data, functionally complete sequential algorithms was considered as an object that can be executed in parallel with another similar object.
With this approach, the essence of each object did not matter, but such a partition into parallel sections of code that could be executed independently, that is, the data at the input and output of each stream did not intersect, mattered. Each thread was independent of another thread, and if it needed data from another thread, it went into standby mode.
This idea led to four architectures:
- SISD - a simple stream of commands and a simple data stream;
- MISD - multiple command stream and simple data stream;
- SIMD - simple command stream and multiple data stream;
- MIMD is a multivariate instruction stream and multiple data stream.
These ideas existed for a relatively long time, but did not lead to special effects. Today is the story of a difficult beginning. But this beginning has laid the foundation for modern achievements.
Lack of architectural idea: lack of semantics
Like the construction of a residential building, the architecture of the computing system did not concern semantics. How builders will live in the building, how they can make repairs and what they decide to install furniture, never worried builders.
At the very beginning, parallel programming systems also did not attach importance to the algorithm that would be executed. The processor itself divided the code and data into sections that it executed in parallel. This gave a noticeable increase in productivity, but worried, in particular:
- the problem of sharing memory between processes;
- logic of waiting by one thread for the results of another thread;
- a mechanism for protecting the memory of one process from another process;
- the logic of interaction of independent processors, cores;
- logic switching between processes;
- on-the-fly data exchange between processes ...
Developers focused more on hardware mechanisms, which deprived parallel multi-threaded programming of the ability to have semantics and did not allow the programmer to control processes adequately to the task being solved.
Industrial application of parallelism
The first purpose of computers: complex mathematical calculations, industrial applications and everything that did not concern everyday life, mobility and the Internet. Naturally, when parallel programming tasks are so “limited", it is difficult to expect interesting achievements.
When computers became mass-produced, the Internet and mobile devices appeared, the requirements for parallelism changed dramatically and the developers had to radically change the style and speed of work.
The first signs were the idea of messaging between processes. The MPI messaging interface, concurrent programming, developer needs and consumer expectations have become an intermediate step.
Windows and similar systems consolidated this idea and actually made it the norm: parallelism and messaging are one for any multiprocessor , multi-core, but essentially for any information system.
From computing to information processing
Computing is a special case of information processing. From parallel architectures implemented in hardware to mobile software solutions: parallel programming languages have really become history. The modern language provides real parallel processes, but for this it is not necessary to have special operators in the syntax or additional libraries to the language.
“Industrial” thinking in programming, when parallel multi-threaded programming is the goal, and not the means, did not exist for long. It is hard to say what fundamental results it led. However, there is no doubt that programming, which was before the era of Internet programming, became the basis for great ideas and the good potential of modern languages and tools.
Hardware component
The first computers were monsters, occupied a quarter of the football field and emitted so much heat that it was possible to calmly warm a small village, and not spend money on building power plants.
The next generation of computers is personal. Personnel were placed on the desktop, and cell phones could be worn on the shoulder. Personals quickly changed and acquired a modern look, gave life to laptops, tablets and other devices, and cell phones turned into convenient multifunctional smartphones.
The manufacturer of electronic components has fully utilized the ideas of past years, and parallel programming now exists on any device, regardless of how one or another software developer relates to it.
Today, the number of processor cores, the number of processors, the level of technology, parallelism and functionality of the code are critical even for an uninitiated user.
Mathematical apparatus
The theory of graphs and mass service, as private options, calculations of straight lines and curves for the visual display of information, as the basis for video cards, led to a clear functionality of the hardware component, which acquired the status and quality of the standard.
We can talk about the number of cores in the processor of the device, but the processor for displaying information has long been alone and is doing its job. A video card can have more than one processor and more than one core, but the mathematical apparatus is implanted in it.
The computer processor only formulates simple commands for displaying information or retrieving it from video memory, the rest is the concern of the video card processor.
Actually, mathematical calculations have long been allocated from the main processor to the mathematical coprocessor. Currently, this is also the norm of things.
In fact, considering parallel programming at the hardware level, one can imagine a modern computer as a set of parallel working subsystems that provide the developer with everything necessary to implement all kinds of ideas for distributed and parallel information processing.
It is believed that the fundamental hardware resources for any information system are in perfect condition and are developing stably. The programmer can only write quality code.
Object oriented programming
In classical programming, the decision algorithm is a sequence of commands. With object-oriented programming, a decision algorithm is a collection of objects, each of which has its own data and its own methods.
Through methods, objects interact with each other, which means that the programmer is less concerned about how they will be executed by the hardware of the computer (device). However, the logic of the interaction of objects is the competence of the programmer.
An information system built on objects, as a rule, is a certain system of abstractions that allows various options for creating objects of various types and purposes. Being described at the level of abstractions, information systems can provide various combinations of objects, including the creation of the latter themselves.
Simply put, with object-oriented programming, it is difficult to bind the execution of an object to a kernel or processor to ensure its parallel execution. This will significantly slow down the overall process. One object can exist in a dozen instances , but this does not mean that the existence of everyone should wait for the existence of the previous one to end.
Clusters and Distributed Parallelism
Modern Internet programming for solving complex and unique problems offers the only possible solution: manual work! For everyday and commercial use, numerous and diverse site management systems are used.
A characteristic feature of Internet programming:
- uncertainty;
- plurality;
- simultaneity.
When creating a site, a programmer (usually a team of programmers) does not know how many visitors a web resource will have to accept, but he knows for sure that the site must provide the same and minimal response time to all clients for any action.
The obvious solution: to host a site on multiple servers or clusters on a territorial basis, and then each region will serve a specific server. But a modern site not only provides information, but also uses it. For example, an online store cannot sell air, and if one product was purchased in Moscow, it should disappear from the warehouse for a consumer in Vladivostok.
To do distributed information processing actually means to ensure parallel operation of the same functionality on different servers for different groups of consumers, provided that consumer actions are reflected in the system and do not contradict each other.
In this context, parallel programming takes on a completely different meaning. If earlier the developer put paramount the mechanism for implementing parallelism, not having in mind the task itself, today the developer is less concerned with how parallelism is implemented at the hardware or tool level, he is interested in parallelism at the client level, that is, the task itself, the scope of the web -resource.
Cluster as a parallel implementation option
It is generally accepted that a cluster is a form of distributed parallel information processing. This is a collection of computers connected by high-speed communication lines.
Typically, a cluster can consist of various computers that can be located in different places on the planet, but by definition: a cluster is a single whole. Cluster-based site management systems do not provide the ability to directly control the computers that make up the cluster, but they provide hidden parallel control of all processes at the level of the problem being solved.
A developer working with clusters can plan and implement his own functionality of parallel distributed information processing. This is a very significant progress in modern development.
"Life" of a modern facility
Today it is very difficult to find a web resource based on static or dynamic pages that are formed entirely. A modern site is a collection of dynamic pages that are filled in parallel using AJAX technology.
A modern dynamic page consists of various content, each part of the page can be loaded independently, depending on the behavior of the visitor. In this context, object-oriented programming shows that not all of its potential is revealed.
Indeed, client behavior results in a request to the server to refresh part of the page. The request is processed, a lot of objects are created, the server sends the result back. The next request ... again a lot of objects, again the result is sent. In fact, it turns out that with the modern approach, the server “does not remember” what, when and where it sent. With each call, he repeats the minimum necessary actions, creates all the same systems of objects.
A programmer cannot change the logic of the server, but he can easily emulate his own server at a physical level accessible to him. This will result in a completely new quality of distributed parallel information processing.
Own server will keep the desired system of objects in an updated state, which will significantly speed up the processing of requests from both one page and all pages open in the entire Internet space for a specific web resource.