M.M.Gorbunov-Possadov
|
|
Keldysh Institute for Applied Mathematics
M.M.Gorbunov-Possadov |
| Русский | Open Systems Journal, 1996, N. 4, pp. 65–70 (in Russian) |
Translation from Russian by Victor Sadikov
It has been noticed since a long time that a well practiced programmer hardly ever writes anything completely new; he mostly develops and improves what has already been written and debugged. Contrarily to this observation, the bulk of software tools available nowadays are aimed at composing a program once and for all. The tools for program evolution are so deficient and inadequate that the news about the necessity to modify a complex program seems to the software developer to be as unpleasant as a coming visit to the dentist. The situation can be improved by means of special constructs, which clear the way to the softness of program evolution. The softness of evolution means that the addition of a new module to a program should not call for the editing of the existing modules.
|
1. Introduction 2. An example in Borland C++ 3. #Horizon socket 4. An example in Delphi 5. The domain of softness Bibliography |
How rarely a programmer happens to have a day when he begins a new program "on a fresh screen"! What a felicity he feels on such a day, knowing for sure that any line he is typing cannot do harm to anything that was written by his colleagues or by himself sometime before.
On the contrary, what anguish the programmer suffers when a certain threshold level of the complexity of his program is achieved and he can't hear without pangs about new requirements to the program functionality because now it is next to impossible for him to lift his hand and make further modifications. Usually, the programmer easily finds out where and what is to be changed. However at that moment he suddenly recollects the recent changes in a couple of source code lines, which seemed to be quite insignificant. But these changes put the program out of operation for a week, and broke the date for an important planned run. Still, it is the intensively developed programs that can only survive in the modern world. So the programmer reluctantly sits down at his computer...
The dramatic collisions like this are extremely unpleasant, common and well known. Any attempt to make things in this field better would be greatly appreciated by the fellowship of programmers. Nevertheless, to all appearance, not many a designer of software tools asks himself from time to time the direct question "What have I done to facilitate the program evolution?"
One of the chief obstacles on the way to the softness of program evolution is the necessity of editing the previously written and already debugged source code. Just the same, in widespread programming environments, program evolution is almost impossible without making changes in the existing source code. The continual hard (unsafe) alterations to the debugged program are so usual that they even don't cause a due feeling of protest. Moreover, the user of such an environment is apt to think that either it is absolutely impossible to add a new part to the program without editing the previously written source code or it will need an exotic software development environment like some tools of artificial intelligence.
But in fact there exist rather unsophisticated instruments for a soft evolution of a program, which can be naturally integrated into a traditional software development environment. This environment only needs a little improvement by means of two or three additional simple constructs. As a result the work can be managed in such a way that the set of the existing source texts could be supplemented with the newly written parts of the program without any editing.
It is the editing of already debugged parts of a program that is the cause of crucial errors, which occur in the program during its evolution. The elimination of such editing would profoundly improve the reliability of the development process. It ensures keeping up the capacity for productive run of the previous version of the program. It is a major step towards the program extensibility. In other words, it realizes the aspirations of any programmer.
Consider a typical example. Let us assume that there is a debugged program for MS Windows. Let at the disposal of the programmer there be Borland C++ programming environment [1]. The program uses a menu, which consists of, say, six items. The task is to add a new item to the menu. For definiteness sake, let the new seventh item have the name About.
What shall be done in order to make this addition? The first thing to do is to determine all the locations dealing with the menu in the program. As the result of the detailed examination, at least five (!) locations will be found out. Each of them contains six homogeneous component parts, which realize the particular aspect of the six existing menu items. Accordingly, carrying out the addition, the programmer has to edit the source code in five different places. Every time he will add a new seventh component to the six existing ones (Fig. 1).
Let us try to understand why the editions are not localized in a single point. At first sight it would seem that the detected multiple connectivity of the implementation of a menu item is just a sign of inaccuracy in designing the program structure. It is pretty obvious that the addition or the removal of a simply connected component of the program is easier than that of a multiply connected one. This is especially important due to the fact that the easiness to manipulate the composition of the program is often a necessary ground for the reuse. The errors in editing may spoil the already debugged parts of the program nearby and thus put the capacity for productive run of the existing program at risk. If changes are made, not to one, but to five different places in the program than, roughly speaking, the possibility of an error will be five times greater. In addition, the simply connected implementation might be more readable and have a number of other advantages.
However, the more careful analysis shows that the division of the implementation of a menu item into five separate parts of source text is a quite deliberate act. Moreover, this division is dictated by the quite logical structural conventions of Borland C++ environment.
First of all, in this environment it is supposed that the resources of a program should be separated into an isolated part. The resources are program components dealing with the user interface. The resources are, for example, the visual elements composing the screen, the icon of the program, cursors, dialogue boxes, hot keys, fonts, messages etc. The resource is to be described in a specific language, which deliberately has nothing to do with the functional contents of the algorithm.
Of course, the resource declaration language provides the construct to define a menu item:
MENUITEM name, message
where name is to be shown to the user as the item title, and message is to be send to the program when the menu item is selected. (The precise syntax of the MENUITEM statement is a bit more complicated, but it doesn't matter now.) In order to add the new item About and to associate the message CM_ABOUT with it, the developer has to edit the declaration of resources by adding the new seventh line:
MENUITEM "About", CM_ABOUT
The advantages of separating the resource into an isolated object are evident. The main one is apparently that the modification of a user interface element is considerably facilitated. Instead of the application of the ill-fated editor to the algorithmic component of the program, which needs particularly careful handling, it involves only changes in the description of the resource. Furthermore, the resource language is an effective way of the rapid prototyping because it allows seeing an outline of the interface under construction without composing any line of source code. The resource to design can be created and modified both in the form of a resource language text and in the manner of visual programming, where its graphical image is directly displayed on the screen, and its size, location, type of appearance, and other exterior attributes can be altered interactively.
Thus, the simply connected realization of a menu item can't be achieved. With a view to make subsequent changes in the user interface easier one of the components of the realization should be written in the resource language, not in C++, and placed in the separate module. Besides, the rest of the realization splits into a number of other separate parts; and again the reasons for this are quite convincing.
The message sent to the program on selecting the new menu item was named for CM_ABOUT. This identifier must be bound to an integer number, because all the messages in MS Windows are numbered:
#define CM_ABOUT 213
The #define statement is placed — along with the similar statements for the six other items — in the header file, which is used by all the modules dealing with the menu. There are at least two such modules: the description of the resources and the algorithmic component in C++, so the header file is essential and the second separate component is to be detached from the realization of the menu.
Next, the designers of Borland C++ propose to form a kind of a table, where all the messages handled by the window containing the menu are put together. In this table the message CM_ABOUT is bounded to its handling function CmAbout:
EV_COMMAND ( CM_ABOUT, CmAbout )
Gathering all the processed messages together is of indubitable use: it is to solve some technical problems and to contribute to the readability of the program as well. So, the third separate component is broken away from the menu item realization and joins the group of six similar ones.
Yet, this is not all. The function CmAbout must be declared, along with six other declarations, as a method of the class maintaining the window with the menu. And finally, the definition of the function CmAbout itself, which is most important in terms of the algorithm, is required, of course.
In the long run we have counted at least five separate fragments of the implementation of a menu item. The isolation of each of them is induced by some important considerations. In a traditional software development environment this multiple connectivity of implementation means that in order to add a new menu item the programmer is forced to edit the text of the program at least in five different places, each time endangering the debugged source code. The millions of programmers have continuously to face such a technological nightmare at every step. Although, as it will be shown below, the release from this pain requires nothing extraordinary.
The outline of the solution to the problem is the following. One record is put into the database of the project for each menu item. The fields of the record are: Text — title of the item, MessageName — name of the message, MessageNumber — number of the message, FunctionName — name of the handling function and so on. In this way the set of the homogeneous records is formed. Let this homogeneous set be named OurMenu.
Next, the special #Horizon socket is inserted into each of the five separated places involved in the implementation of the menu item. The #Horizon socket defines the pattern for the generation of the homogeneous text elements for the menu items, on the basis of the relevant information from the database. For example, the following #Horizon socket is used in place of the message table:
#Horizon OurMenu EV_COMMAND (#MessageName, #FunctionName), #End_of_horizon
After the preprocessing these three lines are turned into the seven usual EV_COMMAND lines, so the input to the compiler is the same.
The syntax and the semantics of the #Horizon socket are quite obvious. The first line is the header, defining the set of homogeneous records (OurMenu) to be extracted from the database of the project. The following text is the body of the #Horizon socket. It contains references to the fields of the records. The references are marked with the special symbol (#). The socket is terminated by the #End_of_horizon statement.
In fact, the #Horizon socket represents a compile-time loop. The body is repeated in the generated text as many times as many records for the homogeneous set stated in the header there are in the database at the moment. Each time the next record is got, the body of the #Horizon socket is inserted into the output of the preprocessor, with the references replaced by the respective fields of the record.
What are the advantages of the #Horizon socket? The most visible one is a decrease in the length of the source code. The decrease may be considerable, but, from a viewpoint of the softness of further program evolution, it is not of much interest.
A much more significant result is the increase in the readability of a program. The programmer who uses a typical software development environment is at the dilemma. If he implements all the aspects of a menu item in one readable continuous segment of text, he will lose all the advantages of the multiply connected realization mentioned above. Otherwise the programmer puts up with multiple connectivity and is condemned to a hard search for all the fragments to be changed every time a new menu item is to be added or removed. The #Horizon socket combines the benefits of both solutions: the implementation of a menu item is localized in a single record in the database of the project, as well as all the advantages of a multiple connectivity are preserved, since after the preprocessing of the #Horizon sockets the extended source code is identical with the plain, multiply connected, one.
The weakness of the traditional source code lies in its one-dimensional structure, which is in conflict with the two-dimensional nature of the menu. Menu items form vertical lines and particular aspects of their implementation form horizontal ones. Once upon a time a source code was thought of as the text of a book, so its one-dimensional structure was taken for granted. But now, in the time of the wide spread of hypertext, DBMSs and other many-dimensional tools, the one-dimensional structure of the source code reflecting a two-dimensional structure of the program has the look of an anachronism.
The modern software development environment is simply obliged to provide the programmer with the freedom to express the two-dimensional structure of a program in full measure. Having put the implementations of the menu items in the shape of a set of homogeneous database records, the programmer has a right to see both the horizontal (all fields having the same name) or the vertical (full implementation of a menu item) layers of the formed structure, and this possibility should not reduce the efficiency of the implementation. The support of this facility must also include the means of reviewing all the #Horizon sockets belonging to a particular set in form of both original text and its extension.
However, the most attractive effect of applying the #Horizon socket appears in adding new parts to a program. Now, it is possible to satisfy the softness criterion of program evolution. A new item can be added to the menu without any editing of the existing debugged source code. It needs only putting a new record into the database of the project and declaring it as belonging to the homogeneous set OurMenu. The removal of a menu item can be done equally softly, without any use of a text editor. As a result, the source of frequent errors, which casts the hard shadow of ill-being on the everyday process of a program development, is avoided.
It will be recalled that in the traditional environment the first step of the addition of a new menu item to the program is the search of numerous locations where the changes are to be made. This search goes with the obligatory analysis of the objective of each of the location-candidates, because a detected group of the six components related to the six menu items does not necessarily mean that the new seventh component has to appear in this location. The suggestion that this group is formed by an occasional coincidence can't be mechanically ruled out. The use of the #Horizon socket, which canonizes the connections of the implementation of a menu item with the rest of the program, relieves the programmer of these troubles. The programmer can add a new item to the menu without analyzing the #Horizon sockets placed in the source code. He should only fill in the fields of the appropriate structure in the database of the project.
So, making use of the #Horizon socket ensures the softness of adding a new item to the menu. However the question might occur to the thoughtful reader: is it worthwhile to invent a whole new construct of the program development environment? In other words, how often do the situations like that take place in common programs? Let's try to demonstrate that such circumstances are quite typical, and for this end let's consider another popular product of the same firm, Delphi [2, 3], a translator for Pascal language.
Nowadays Delphi is one of the most well known software development environments based on the concept of visual programming. A typical Delphi programming session begins with selecting various buttons, input boxes, scrollbars, main menu, popup menus, and other Windows control elements, which are necessary for the program under design, from the special menu; and placing them as components on originally blank form. Next, having clicked any component, the programmer can change its properties or write down the algorithm implementing the response of this component to a particular event. In the latter case the programmer is provided with an edit window initially containing the empty framework of procedure, consisting only of header, begin and end.
In Delphi the above-stated problem of adding a new item to the menu is solved much more technologically. (However, there are also some facilities of visual programming in new releases of Borland C++, but they are irrelevant to our investigation.) The programmer uses the mouse in the manner of visual programming to expand the menu. As a result he is presented with a table to input the desired attributes of the newly created menu item, for instance the name About. This table is more or less similar in its structure to the project database record invented in the previous section. Then, as mentioned above, the programmer is to define the actions corresponding to this menu item on the procedure framework presented by Delphi.
Unfortunately, the described picture of programming in Delphi does not look so idyllic after a more attentive examination. The result of the keenness on visual approach is that while formerly the programmer had to write source code before he can see the designed screen, now he must first design (and see) the screen and only after that he can write the code.
For example, it is not at all difficult to select a visual object and get an access to any aspects of its implementation. Moreover, apart from the source text of the algorithms the programmer is always provided with the table containing the attributes of the object. The trouble is that when browsing the source code the programmer arrives at the implementation of another object, the table will still contain the attributes of the first one, and after returning to the visual mode the new object won't be current.
In this case of the evidently secondary role of the source code, you could suppose that the designers of Delphi would take the trouble to keep up the integrity of the pair: a visual object — its program implementation. Unfortunately, this integrity is unprotected from any ill editing. The procedure framework presented by Delphi to input a program segment is only an initial basis. Later on it indistinguishably merges with the inputted text. So the programmer can equally easily edit both the text inputted by him afterwards and the program constructs initially presented by Delphi to implement the visual objects. The effect of deleting or editing the latter is hardly predictable and mostly unpleasant.
The absence of any opposition between the text generated by Delphi and that inputted by the programmer hampers programming to such an extent that one of the manuals [3] even recommends to type Pascal reserved words in capital letters (BEGIN) in order to differentiate them from that belonging to the frameworks generated by Delphi and displayed in small ones (begin). What is the obstacle that has prevented the designers of Delphi, who courageously made a considerable advancement in the scantily explored sphere of visual programming, from extending Pascal with a relatively simple framework construct? It is not improbable, that the reason lies in the poor theoretical grounding and in the lack of experience of the wide use of the constructs of this kind. The legalization of the framework must lead to the introduction of something like the #Horizon socket from the previous section. However, for the present the required associative retrieval from the project database is taken for an eccentric trick disallowed to the designer of everyday programming tools.
To sum up, having put our task of adding a menu item in Delphi environment, we obtain two interesting results. On the one hand, the solution of the task is notably improved: the number of separate fragments of the implementation is less, besides the fragments are connected (although in only one direction), that is there is possibility of getting the table of the attributes and source code for the given visual menu item. On the other hand, to put the program representation in good order, it is necessary to have available something like the #Horizon socket again.
At the same time, the described tools of Delphi cover only the restricted collection of the control elements and their modifications. All the others parts of the program must be written in the old manner and the problem of dealing with two-dimensional structures is still so actual. An example from the mentioned Delphi manual [3] can convincingly illustrate the last sentence.
The first chapter of this manual profoundly examines a program of the payments calculation. The user inputs some options of the loan, and the table of the coming payments is to be calculated from these data and displayed on the screen. In the table, every payment is characterized by numerous attributes: the number of the payment, the principal, the interests, the balance and so on.
It is apparent that three qualified specialists, the authors of the manual, have spent years working hard on their program, striving to make a true example to follow. The program has really turned out good enough. It is divided into two modules. One uses the tools of visual programming with infectious energy to deal with the interface. The other successfully takes charge of calculations.
All is OK, as long as there is no necessity to add a new payment attribute or to delete an existing one. All of a sudden it is found out that the implementation of a payment attribute is divided into eight (!) separate fragments spread all over the source code of both modules.
For example, the part of the program, which includes two first locations where the homogeneous components of various payment attributes are gathered together, is the following:
TPayment = CLASS(TObject) { one element in the amortization table } PaymentNum : Integer; PayPrincipal : Real; PayInterest : Real; PrincipalSoFar : Real; InterestSoFar : Real; ExtraPrincipal : Real; Balance : Real; PROCEDURE GetStringForm(VAR StrPayNum, StrPayPrin, StrPayInt, StrPrinSoFar, StrIntSoFar, StrExtraPrin, StrBalance : String15); END;
The first location describes seven present attributes of a payment; and the second one defines seven parameters of the procedure, which transforms each attribute into a text form.
The observation of the straight columns of homogeneous components may make somebody feel delighted, but the opposite emotions are more relevant here. The represented piece of the program resembles a muddleheaded host welcoming his guests (new attributes) at the doorway of his house with a suggestion to leave galoshes in this room, a mackintosh in that, and a hat with gloves in the other one. It can be easily predicted that the arrival and departure of each guest will be complicated by the tedious search for cloakrooms and the clothes. In addition, being spread everywhere, the cloak-rooms inevitably appear near something breakable; and the sorrowful cry of an awkward guest is going to arise, when putting his hat on the due place (that is adding a new program component) he will knock a precious Chinese vase (that is a neighbor debugged statement)...
The #Horizon socket saves the situation once again. The components of any payment should be implemented in the form of project database records and an appropriate #Horizon socket should replace each of the eight locations of homogeneous components. Thereupon there will appear an opportunity to observe any direction of the two-dimensional structure, there will be no need to search for the locations of the components any more, and finally, that is of no less importance, the adding and deleting of new attributes will satisfy the softness criterion of program evolution.
The considered examples have perhaps convinced the well-disposed reader that a homogeneous set is not a rarity in real programs. Nevertheless, it is more difficult to imagine someone, who will take on trust the two following extreme theses.
(1) As a rule, in any real program it is possible to extract one or more homogeneous sets, which cover the greater part of the program.
(2) As a rule, in any real program it is possible to extract one or more homogeneous sets such that the greater part of changes made in the program will reduce to adding and deleting records belonging to these sets.
The rigorous proof of these theses seems to be hopeless, in spite of all reservations like "as a rule" and "the great bulk of". All what can be done is to attempt to provoke some confidence in them by indirect confirmations.
Some examples of large homogeneous sets in well-known programs can be given in support of thesis (1). The considerable part of a translator consists of the homogeneous components realizing various constructs of the source language. The essential part of a text editor consists of the procedures performing various editing actions. The main part of an optimizer is the various methods applying for extremum problem solving. Some other examples of large homogeneous sets are given in the book [4].
The examples given in this article are also quite typical. Frequently enough, the bulk of the program is devoted to the implementation of homogeneous menu items like those described in the first section. In the payment calculating program from the previous section about half of the whole code is really occupied with the work on homogeneous payment attributes. Finally, the reader willing to drill extracting large homogeneous sets is offered to do it in a natural language text, for example to extract the set of two components dealing with theses (1) and (2) from this section.
Let us assume now that thesis (1) is true. At least two corollaries of this are important for practical programming.
Firstly, most programs have considerable reserves of readability. It was just demonstrated that a traditional programming environment has no tools for an adequate representation of homogeneous components. But if the amount of existing homogeneous sets is considerable, their role in the whole structure of a program must be equally large. Consequently, the failure of the environment to present satisfactorily homogeneous sets detracts much from the readability of the source code.
Secondly, the technology of stepwise program development based on the idea of large homogeneous sets certainly merits further attention. This technology is given the name of "broad wise extension". According to it, the minimal number of components in each detected large homogeneous set should be implemented at the initial stage of the development. If thesis (1) is true, then this stage involves a relatively small amount of the work. Nevertheless, the implemented program is sufficiently independent: its debugging needs no stubs required by other popular technologies, such as top-down or bottom-up development. For instance, there can be implemented a translator maintaining a minimal subset of language constructs, or a menu containing only one item "Exit" and so on.
Following stages involve programming and debugging more and more homogeneous components. The use of the #Horizon sockets in the program guarantees the softness of new component addition. No editing of the source code written and debugged at previous stages is needed. And what is more, the developer is given an opportunity to add a new homogeneous component by simply inserting proper data in the fields of the presented table, maybe even without looking at the source code surrounding this data in the program.
Let us proceed to thesis (2) now. There is a well-known observation in its favour. Sooner or later any developer dealing with continual and intensive alterations comes to some regular mechanism of program evolution. Usually such a mechanism is based on the extraction of all (or nearly all) alterable factors affecting the program. A particular homogeneous set should be implemented to reflect each extracted factor, so that the appearing of a new value of this factor is mapped into the extension of the corresponding set by a new component. Of course, all thinkable alterations can't conform to this canon. From time to time revolutions may be inevitable, resulting in profound reformations of the program structure. Still, the dominant part of the alterations can be successfully predicted and accomplished in terms of such a mechanism.
It is not to be supposed that any regular mechanism of program development necessarily needs a sort of #Horizon sockets, with all the records of a homogeneous set from the project database being included in the text of the program. Frequently, a more easily realizable #Variant socket is quite enough, with only one record being selected from the set and inserted into the generated text. The #Variant socket is widely used in programming multivariant tasks (Fig. 2). In these tasks the appearance of a new value of a variable factor means that the old record related to its former value must be replaced with the new one. However, the old record should not be rejected, for the old value of the factor may be wanted again later on. Thus, the homogeneous set of changeable records reflecting the ever used values of a variable factor is accumulated in the project database.
Since only one record is to be inserted into the generated text, there is no more need of the compile-time loop. The fields of the record are to be inserted directly. So the #Variant socket is very simple:
#Variant set.field
where set is the identifier of the homogeneous set of changeable records, it usually represents the mnemonic name of the variable factor; and field is the name of a record field to insert.
In contrast to the #Horizon socket where a simply connected realization is a rather infrequent occurrence, the source code dealing with a variable factor relevant to the #Variant socket can be quite often localized in a single point of the program. In this case the record consists of a single field, and so it is enough to specify only the name of the set in #Variant socket. For instance, if a calculation method, which has a simply connected implementation, is to be periodically varied, then the line
#Variant Method
should be put in its place in the source code. Here Method is the name of the homogeneous set of calculation methods.
In order to build the executable version of the program, changeable records to substitute for the #Variant sockets should be specified. For this purpose, destinations like
set <- record
are used, which make the preprocessor select one particular record from each set and apply this record to building the program.
Despite significant exterior differences, sockets #Horizon and #Variant have the main thing in common; they both provide the softness of program evolution. And so, assuming that thesis (2) is true, the bulk of the work on program evolution can be carried out through alterations satisfying the softness criterion. Certainly, the described socket constructs need some improvement [4] to be fit for practical programming. Still the game is worth the candle, the elimination of such a dangerous procedure as editing the already debugged source code from the process of program evolution brings this process to a new level of reliability.
In one way or another, if there is some truth in thesis (1) or (2), the system support of homogeneous sets is worth supplementing to everyday programming tools. It is the lack of support that hampers a large-scale application of the suggested mechanism. The implementation of the tools supporting homogeneous sets won't take a lot of effort, because all essential prerequisites for this step have presently matured.
The most important prerequisite is that the advancement from an independent translation to a separate one is almost completed, even C++ previously inclined to an independent translation has now a little tendency towards Java. Perhaps, it is a bit too early to consider an ample project database to be something of general use, but it is an actual fact that all the source texts of a program are nowadays handled in interrelationship. In such environments the homogeneous sets in question no longer look like a completely foreign body.
Another necessary tool is the facility of creating and editing a record of homogeneous set. This tool also has a ready prototype, the mentioned above user's tables of attributes in Delphi. They only need to be supplemented with a public mechanism to form the description of a new table, every instance of which is to turn into a record of the corresponding homogeneous set afterwards.
The last essential component of the implementation is a preprocessor able to handle sockets #Horizon and #Variant. Preprocessors are out of fashion now, but there is a large experience of their realization. The main things to do is to equip the preprocessor with the coordinated tools for the browsing of both the source code and output of preprocessor, and to provide a navigation on the source code among #Horizon or #Variant sockets belonging to given homogeneous set.
Thus, the amount of the work on the implementation is not so very huge. And so, hopefully, in the near future at the disposal of the developer there will appear some tools for dealing with homogeneous sets. These tools can add a great deal of so lacking now attractiveness to the process of program evolution.
The work was supported by the Russian Foundation for Basic Research,
Author's personal web page: http://keldysh.ru/persons/english/gorbunov.html