CA2248181A1 - Interactive software development system - Google Patents

Abstract

An interactive program development system which replaces compilers, linkers and debuggers required for conventional software development. The present invention includes a new program compilation system for producing a novel type of translated structure called code object (4), and an Incremental Image (6) which forms the program image from code objects and their respective intermediate language symbols stored in a persistent symbol table (5). The new program compilation system operates on a conventional computer havign a CPU, monitor, memory system and input devices. The present invention obviates the need to halt execution of a program under development or during maintenance update to correct programming errors.

Description

W O97/32250 PCTrUS97/02177 INTERACTIVE SOFrWARE DEVELOPMENT SYSTEM

COPYRIGHT NOTIFICATION
Portions of this patent ~ppli~tiQn contain materials that are subject to copyright ~rol~;lioll. The co~igllL owner has no objection to the f~e~imil~o repr~ r-tion by anyone of the patent doc~ or the' patent disclosure, as it ap~eal~ in the Patent and Tr~ mqrk Office patent file or ~Ol~iS, but otll~ vise re~el~es all copyright rights wl~lso~,.~l.

TECHNICAL FIELD
The present invention relates to co~ aided software çngin~erin~ and more particularly to an illtel~cli~re ~r~gl~ll development system which replacesprior art compilers, linkers, and debuggers with a single proglalll. The presentinvention is particularly applicable to object-oriented pro~li..-..--i-~g. The pro~ l.ing system of the present invention not only operates at speeds greater than prior art compilers, but more\ ull~)o~ ly it precludes the need to halt execution of the program in order to correct pro~li.. -.,-,~tic errors during software development and m~int~n~nre. In other words, the present invention enables interactive "real-time" debugging of a program as it is compiled ("built") without the need for batch recompilation, re-linking, re-loading or the need for the developer to re-create error states.

W O 97/32250 PCTrUS97/02177 BACKGROUND ART
Developing a co~ uLer software product (hereafter, "software") is a complex cyclical process which requires ~ignifit~n~ expenditure of time on the part of the software development team (hereinafter, the "developer"). This development process is often referred to in terms of "edit - compile -link - run", or "edit - compile- test", which refer to some of the major steps of the development process. For reasons of technir~l superiority and programmer productivity-, a si~nifir~nt amount of commercially produced software is currently developed using an object oriented pro~ r""~i"g language (OOL) . By way of example, but not limitation, some languages which support such object oriented progr;lmming are C+ + and Sm~llt~lk Programs developed using these object oriented languages are not written in m~rhin~ code format, and hence must be tr~n~l~ted from the OOL source code in which they are written into object code which is machine-executable. Such m~c~ine code programs operate efficiently and hence rapidly on a given coll~uL~tional platform. This is typically done through the process of compilation or h~ lcLalion. Compilers translate source code into machine-level instructions which are executed by the hardware.
I11Le11J1CIe1~ translate source code into pseudocode (hereinafter PCODE) which is executed using a PCODE hllelyl~elel. While hllel~eL~s enable a programmer to effect changes to a program while it is running, the PCODE h2lel~
introduces an extra level between the programmer and the haldwdre, further reducing p~lrollllallce.
Recallse all but the simplest programs are written in sections, e.g.
modules, each of the source code modules must be linked together to form the image of the complete program (hereinafter, the "image"). This linking process typically includes a "fix-up" which, among other things ensures that each of themodules is colleclly addressed and addressable. Once linked, the image must be loaded and çx~c lted. During execution, and when the developer encounters an error (or "bug") in the image, the program typically halts execution or behaves erratically. Traditionally, this means that program execution halts or must be halted, before a separate program, a debugger, is invoked to assist in determining the error to the faulty executable program. After the developer locates the source of the error using the debugger, he then edits his source code to elimin~te the error, recompiles the code, re-links it, and then re-loads it. The programmer must then re-create the exact error state which caused the failure to verify theefficacy of the fLx.
While prior art compilers l~ltim:-tely produce efficient m~rhin~ language code executable files, use of such files during debugging is both cumbersome andtime con~ullling. This is due to the fact that changes in source code files which are invoked by other source code files have previously re~uired, at a mi,~ l"", the recompilation of each of the affected files and the relinking of all the resl-lting object code files into a program image.
Because each of the prece~ling software development steps must be con-ll)cted by t~lPnf~d co~ uling plofessionals, the cost for each of the steps is high. Accordingly it has long been, and contimlPs to be, a primary aim of the software development industry to develop methods and systems to increase the efficiency of the development cycle. To this end, improvements have been made to many of the development steps.
A series of U.S. Patents to M~Keçm~n et al. (Nos. 5,170,465; 5,182,806;
5,201,050; and 5,325,531 inter alia) teach a con.~uLel a~si.~t~l software engin~çring system called "RCASE". The RCASE system employs a fine grain incremental compiler including an incremental scanner, an inc~ t~l linker, an incremental make facility, and a virtual memory manager to reduce thrashing.
While this methodology may provide econo---ies of operation during the software development cycle, it retains many of the steps of the traditional development cycle while adding a few of its own. RCASE is ~us seen to lC~)~Se.ll an incremental improvement to the traditional development cycle rather than a new development system. McKeeman et al. in U.S. Patent No. 5,193,191 teach the previously di~c--~sed inclel~PI-t~l linker.
Another in~ l compiler is taught by Smith et al. in U.S. Patent 5,204,960. The '960 compiler teaches a system whereby source and object files are org~-i7~d into logical blocks and stored. An intenne~ te file is created which contains information regarding these logical blocks and is likewise stored.
If a change is made to one portion of the source file, only the logical block WO 97/32250 PCTrUS97/02177 involved is recompiled and patched into the object file. Again, Smith representsan incremental improvement to one phase of the traditional software development cycle.
WO 95/00901 and WO 95/00904 to Taligent, Inc. teach an incremental build system and linker system. Taken in sum, these references attempt to replace the tra-1ition~1 batchmode pro~ g systems. In a typical compiler symbol table, Icfel'cllces to other data often make up more than half of the contents of data items. Thus, lefe~cllce translation presents a signifir~nt source of conl~u~Lional overhead. One solution to the translation problem is to use Ids, as taught in the '901 refer~ ce. The incremental builder taught by the '901 reference teach modeling a conl~uL~l- program as project component and a collection of building components, each represellting a single compilable language element. Those components which provide the model for a computer program are stored for access during the build process.
The stored components are ~ccesse~l in sequence, and using a compiler, dependencies associated with a component are c~lr~ te(l to develop client and reference lists. Finally, the components are compiled to build the coll~ e program using the l~lu~clLies of the components along with the compiler dependencies. The '901 Icrelel~ce thus reads on a system in which the program is not defined in files, but in terms of components, addressable by means of Ids.
In this manner, the '901 reference teaches a system which does not have the previously tli~c~ssed translation problem, but replaces it with a number of independently addressable (and reasonably complex) components, the building and lltili7~tion of which are a major source of computational overhead. These contribute to the lack of coll~uL~tional efficiency of this system.
The '904 Icfelcllce teaches an increm~nt~l linker which utilizes the independently addressable components created by the Increm~nt~l Builder of the '901 lcfelellce to link only those components which have been modified during the edit phase of the development cycle. Taken in sum, it will be appreciated that the system taught by the '901 and '904 lcf~lellces again represent an incremental improvement to the steps of the traditional software development cycle. The necessity to ~tlrollll those steps remains llnrh~ngel.

W 097132250 PCTrUS97/02177 s From the preceding discussion it will be appreciated that siEnifir~nt effort has been expended on improving the efficiency of several of the steps of the software development cycle. What has not been re~li7.ec~, up to this time, is the signific~nt increase in programmer efficiency which must ensue by reducing the S number of the steps of the development cycle.
It will be further apl)recial~d that while some of the previously ~ cl~ssed inventions have overcome some of the "batch processing mentality" of the traditional software development cycle, none of them completely elimin~te the need for pelrolll.ing at least some of those steps discretely; i.e., to some extent in batch mode. Accordingly, one underlying condition prevails: once an error is discovered, program execution must be halted, and any actual error discovered during debugging and thereafter corrected during editing will have been removed.In order to verify that the error was actually corrected during editing of the file (i.e., that the fix was effective in removing the bug), the exact state of the program, its error state, must be lecreàl~d. Since program execution has been halted, the developer must re-load the program re-run it, and ensure that each and every variable is set to its previous value when the error occurred to re-createexactly its error state on failure. When a developer is wolkhlg on a complex software system ~ecrealillg the error state thus is a non-trivial task. Further, this step is rendered more problematical in that it is ~liffir~llt to ~lele""i~-~ if the error state has been e~cactly duplicated, further rendering post-fix validation time-Co~ u-,~ g, e~ensi~e, and error prone.
A development system which re-defines the development cycle by ~lcse~ g the developer with fewer steps during each iteration of the cycle wouldpresent signifir~nt economies in time and ~cnse over prior art sc~ Further economies would be realizable if the system were more effiri~nt and presented faster execution speeds, than prior compilation sç~ s. If only some methodology could be found which enables a developer to debug and edit source text interactively, without halting program ~ecuti~n, the methodology would at once obviate the need to re-create the error state and verify the efficacy of the fix. Accordingly, profound and hitherto u~ ble levels of developer productivity would be realized.

W O 97/32250 PCT~US97/02177 DISCLOSURE OF INVENTION
In accordance with one aspect of the invention, there is provided an interactive computer program development system that enables real-time debugging of a program as it is compiled without the need for batch recompilation, re-linking, re-loading or the need for the developer to re-createerror states. In accordance with the principles of the present invention, this interactive system includes a co~ uL~l system having a debugger resident therein, and having a given set of source code files. When a developer invokes the debugger for the first time for a given set of source code files, an incrementalbuilder resident in the system is invoked which translates the source code filesinto code objects and their corresponding intermP.li~te language (IL) symbols.
The code objects and IL symbols are stored in a resident pe~ L~ symbol table.
A resident incremPnt~l imager utilizes the code objects and IL symbols to form run-time code objects. The incremPrlt~l imager then links together run-time code1~ objects forming a program image which is executable. The actions of the program image and the com~-l~r program are cooldhlaled through a probe and a runner. The probe executes in the incremental imager process and the runner executes in the process of the built program. Thus, the image is executed, and the image embodies the program .cimlllt~nPously in two separate threads of execution using memory-mapped data.
In accordance with the principles of the present invention, this is accomplished by means of a new compilation system (l~learler referred to as an Incremental Builder) which produces a novel type of tr~n~l~tç~ structure, the code object; and an I~ nr~ l Imager which forms the prograrn image from code objects and their lespe~;live interm~ te language (IL) symbols stored in a persistent symbol table. After the image is formed, the program is executed in aseparate thread of execution in a separate, pfotected address space. A proprietary debugger interactively assists the developer in debugging the program under development in virtual real time using the formed image and the extoc~lt~ble program. Finally, after a given program has been built and thoroughly debugged, the developer invokes an extractor which converts it into an executable file (e.g., an ".EXE" file).

WO g7132250 PCT/US97102177 "Thread of execution" refers hereil.arlel to a logical structuring of a program - each thread of execlltion has its own sequence of instructions which it will execute. An "address space" is a mapping of memory addresses that are unique to a specific set of threads. Each address space and its associated set of threads define a "process".
A code object is the fully tr~nCl~te~ m~rhin~ language implem~nt~tion of a function definition, or the initial value of a variable, which refers to IL symbols representing other code objects by means of direct memory rerc,ellce as opposed to Ids. Code objects and IL symbols are low level data structures used to createan image, i.e., the "program". Code objects contain the description of those symbols to which the function refers. The IL symbol contains a description of those code objects which refer to the function. The code objects and their respective IL symbols, as well as language symbol table objects are stored in a persistent symbol table.
An overview of the operation of the system of the present invention follows: After clcaLil~g the source code files which define a given ex~c~lt~ble file (e.g., the entire program) or a self-contained portion of a program having no unresolved external lefelellces (e.g., a dynamic link library), the developer invokes the debugger to debug the source files. The debugger then progr~mm~tic~lly invokes the Incrçm~nt~l Builder of the present invention, which first dete~ s if the source code has been previously compiled and saved in a persistent symbol table. Where the source code has not been previously incle~ lly compiled (h~ltil~rlci the term "compiled" will be replaced by the term "built", the latter term being more descli~live of the actual process), theIncrem~nt~l Builder would ~etc . .i.. ~ that no previously built version exists in the persistent symbol table, and would translate the files in toto. Where the sourcecode has been edited, and hence previously increm~t~lly built, the Increm~rlt~l Builder determines that a previously built version exists in the persistent symbol table, and would only translate those portions of the source code whose code objects were affected by the edited portions of the original source code.
The pe~ L symbol table takes the form of an object-oriented database.
Note that in the present context, persistent means that it is capable of being stored to a persistent data storage device, not n~cess~rily that such storage has actually occurred. Of course, when a developer ceases work on a project under development, for in~t:ln~e at the end of the work day, the persisLelll symbol table could well be stored in such a persistent storage device. Prudent Co~ ulillg practice would usually so dictate.
Use of an object-oriented ~l~t~b~e enables the Incremental Builder to build data structures using the exact same structure in the file as is used in system memory by the Incremental Builder while it is running. This results in no time lost in converting between file format and memory format and hence elimin~tes redllntl~nries and signifir~ntly increases execution speed over previous compilers.
The database is not discarded when system operation is termin~ted.
After they are formed by the incremental builder, IL symbols and their corresponding code objects are stored in memory. During image formation code objects are duplicated by the Incremental Imager to form the image as run-time code objects. After the image is formed, the program is executed under a separate thread of execution in a separate, l~lole~;led address space. The probeand the runner cooldindte the actions of the development system and the image (which embodies the program being developed). The probe executes in the Imager process and the runner executes in the process of the built program.
Should an error be detected as the program is executed, program execution under the runner is halted. The operating system L~ data relative to program failures to the debugger, which translates this data into informationrelating to which line or lines of source code caused the failure. When the debugger notifies the developer of the failure, this hlfollllation is passed to him or her.
After the developer edits the source code file to obviate the program failure, he can instruct the system to increm~nt~lly compile and update the modified portions of the program, res~lting in new IL symbols and code objects which implement those modifications. The Builder incrementally compiles only those program units which relate to the edited source code. From the new code objects and IL symbols the hlclclllenl~l imager forms an updated image of the program, and program execution continues. Note that the steps required by the W O 9~/32250 PCTrUS97/02177 prior art of linking and loading source code to first update the .EXE file and then of re-creating the exact error state which caused the failure are eli~ A~d.
After a given program has been built and thoroughly debugged, the programmer converts it into an ex~c~lt~hle file (e.g., an ".EXE" file) by means of S a separate module, the Extractor. This .EXE file is the image used by a traditional development system, and by the underlying operating system to start an application process. .EXE files are very useful for production-level distribution of software: indeed they are still n~cess~ry in order to produce a stand-alone software product. What is no longer nPcess~ry is the need to use .EXE files in the software development process, for which they are not well suited due to their static nature.
Code objects resident in the persistent table are persistent and are not modified as a result of system te~nin~tion, debugging or other prog~ ,~r action. When a function (or data) definition is recompiled, a new code object representing the new definition is stored in the persistent symbol table, and the function's (or data's) IL symbol is ~lp~l~tPd to point to the new code object.
The system of the present invention presents several productivity advantages over prior art translators. A major advantage is that the system of the present invention enables the prograrnmer to change the program while the program process is still "live". This obviates the need to stop the program, debug it, edit the source code, batch recompile, and to lecl~,aLe the error state which caused the "bug" in order to verify the efficacy of the fix. It further completely obviates the need for re-linking and re-loading the program.
A second major advantage of the image formation scheme det~ilPd herein is that a program developer can run a program until it crashes, correct those lines that caused the crash, and continue. Once the source code line which caused a program crash is modified and corrected, the system continues in operation without further developer input until the next system crash. In this manner, once the developer corrects source code errors, the crash can be said to virtually have not occurred.
At least two very hllpoll~ll concepts help define the system of the present invention. First, the IncremPnt~l Builder genel~les code objects as opposed to W 097/32250 PCT~US97/02177 object files. This fact is central to the system of the present invention being able to compile pieces of files, producing code objects only for new declarations or those source code lines which have ch~nged.
A second concept central to the operation of the Incremental Builder of the present is that it derives the structureof what is r~lesel~led in the source file and then divides the source file into program units which can be effectively incrementally recompiled. The division to program units is coarse enough so thatmemory requirements are small enough to be practical for current co~ uler systems and fine enough for very fast response.
With respect to debugging, traditional debuggers usually contain an hllel~leter which, upon request by the developer, evaluates eA~lcssions during program execution. Some special-case expressions, (e.g. function calls) are in fact evaluated in the real application process by e~ec~-ting m~r~line instructions.
In contrast, the debugger of the present invention uses the actual compiler to produce a small code object, which is dyn~mir~lly inserted into the increm~ont~l image, and executed directly. This enables the prog~ "lcl to go beyond mere interactive evaluation of simple eA~.cssions and gives him the ability to amend his prograrn with expression evaluations and st~t~m~-nt executions that are integrated into his program. This is performed at the same speed as any other part of the program. By way of example:
To implement a conditional break point (such as "stop here if this condition is true"), all other known debuggers set a break point at the desired location, execute the program until it gets there and hl~ rel the eA~ ssion (which can't be too complex). If the condition is false, execution again continues until that point in the program is reached again. Then the condition is tested again, etc. While these tests are hidden from the programmer, they nevertheless require large amounts of execution time to perform.
In contrast, the debugger of the present invention uses the compiler to build a code object for the condition, increm~t~lly updates the image, and inserts code at the point in the program where the debugger is supposed to break to evaluate the condition directly in the application process, at full speed. Only when the condition is true does the application actually stop, and involve the W O 97/32250 PCTrUS97/02177 debugger. Nothing is hidden from the user - the debugger isn't doing anything "behind his back". Further, the conditions may be arbitrarily complex.
The switching back and forth between the application and the debugger in the traditional software development cycle requires great deal of time and si~nih~nt1y reduces system pelÇol,llal.ce. Traditionally, condition hllc~e~lion is also much slower than condition ex~cutinn. In contrast, the debugger of the present invention tests the condition as a part of the application, and does notsuffer this severe pelrol,l,~lce penalty. This results in much better turn-around time for conditional breakpoints, which are now generally avoided by pro~ llels due to their poor ptlrolll,allce and the fact that the available conditions are often too trivial to be of any use.
The debugger feature of the present invention is inherent in system design and is not a separate module. Therefore, the compiler and the debugger cannot disagree on the m~nin~ of anything. Furthermore, this debugging schema is much faster than those of prior art debuggers. Another feature of the debugging schema of the present invention is that it does not require termination of progr~mmin~ execution to effect debugging, or r, sLallillg once debugging is completed. The system of the present invention enables debugging to start and stop at any point during program development. For in~t~nre~ if the program ~Uc-llptS to execute an undefined function, the system generates a dialog that gives the developer an oppolLulliLy to define the function. The developer entersthe function and the system compiles it, updates the image, and continues exec~tion by Ic~Li~Lillg only the function call instruction, not the entire program.
From the folcgoillg, it will be appleciated that compiling, linking and debugging are all l)clrol.l-ed by the system of the present invention. It will be further ~ ,ciated that the need, upon the occu,~ lce of a "bug" in the system under development, to stop the compilation process, edit the source code, recompile the source code to object code, re-link the object code to form an cut~hle image, load the exec1~t~hle image, and recreate the error state which caused the bug are all completely obviated unless the program's execution has unrecoverably coll~L~d its memory state. Furthermore, the time required to verify the efficacy of the fix is drastically reduced, because, inter alia, program execution never ceased, which means that the exact state of the program at errorinception is m~int~in~.
Other features of the present invention are disclosed or ap~arent in the section entitled "BEST MODE OF CARRYING OUT THE INVENTION".
s BRII~F DESCRIPTION OF THE DRAVVINGS
For fuller underst~n~ling of the present invention, reference is made to the accompanying drawing taken in conjunction with the following ~let~iled description of the Best Mode of Carrying Out the Invention. In the drawing:
Figure 1 is a diagram illustrating a te,l~oldl comparison of the software development cycle according to the present invention coll,pared to both the traditional view of such a cycle and the improvements thereto obtainable using incremf nt~l technology alone.
Figure 2A is a block diagram of a co,l,~uLer system on which this software development system of the present invention operales.
Figure 2B is a flow diagram of the software development system of the present invention.
Figure 3 is a diagram illustrating the p~ lieS of a generic program object according to the principles of the present invention.
Figure 4 is a diagram illusllaLing the ~rope.lies of a generic program unit.
Figure 5 is a diagram illustrating the ~lope~lies of a declaration program unit.
Pigure 6 is a diagram of the ~ clLies of a source file program unit.
Figure 7 is a tabulation chart of an exempl~ry first source file A.
Figure 8 is a tabulation chart of an exemplary second source file B.
Figure 9 is a tabulation of a modified version of exemplary source file A.
Figure 10 is a diagram illustrating the incremental building of a program unit, program objects and IL symbols responsive to source file A.
Figure 11 is a diagram illu~llaling the incr~me~t~l building of program units, program objects, IL symbols and code objects responsive to source file B
subsequent to the incl~ r~ l building of source file A.

W O 97/32250 PCTrUS97/02177 Figure 12 is a diagram illustrating the increm~nt~l re-building of affected program units, program objects, IL symbols and code objects responsive to modification of source file A.
Figure 13 is a diagram illu~L~aLhlg the use of the code objects and IL
symbols in forming the program image.
Figure 14 is a diag~ tir representation of the sequential formation of code objects, relocation arrays and run-time code objects responsive to an IL
symbol.
Figure 15 is a dia~li....-A~ir ~lesenLation of shared memory allocation using the probe and runner.
Figure 16 is a dia~lA------~tic lc~lesenl~lion of the creation of the run-time image of the program.
Figure 17 is a diagl~ tic rel)lese~ tiQn of the m~ch~ni.cm for ensuring that changes made to a code object are ~iopelly addressable by a dependent runtime code object.
Reference numbers refer to the same or equivalent parts of the invention throughout the several figures of the dlawhlg.

BEST MODE OF CARRYING OUT THE rNVENTION
Definitions In order to provide for a fuller underst~m1ing of the system of the present invention, and to dirr~.e.lliate some of the concepts and constructs thereof from the work of others, the following definitions are provided:
File: As used herein, a file embodies, in the broadest sense, a stream of program text, e.g., source text. It will be imm~ tely apparent to those of oldil~aly skill in the art that this definition includes traditional file-oriented structures, but further comprehends, for in.ct~nre, undirrelt..li~ted data streams.
Program: As used in the description of the system of the present invention, a program consists of a set of files, the declarations contained therein, and a description provided by the developer of the relationships between the program files. This description is referred to as a 'project'.

Declaration: A declaration in C+ +, or any declarative progr~mming language, is a smaller unit of the program description than a whole file. Program declarations typically refer to each other, and form a network of dependencies herein called a dependency graph. It should be noted that this term has an S altogether different mP~nin~ than the term 'depen~l~nre graph', in common use in compiler c ~linli~alion theory.
Declarations are of two filn(1~ment~lly dirrtlcl~l types: those declarations which have only a compile-time representation, and those that have also a run-time represent~tion. The type of a given declaration will depend on the language, but in the C+ + pro~ language, examples of the former are typedefs, enumeration type specifiers and function declarations. In the same language, thelatter are exemplified by variable definitions, function definitions, and, to some extent, class definition types.
Pro~ram Unit: The ~m~ st entity which is compiled at one time. A
program unit is, however, more than a simple declaration. Each program unit contains one or more declarations. Program units ~ sen~ pieces of source files, not independent program entities. A program unit is similar, but not equivalent, to a 'component' in the previously discussed '901 and '904 ~cr~.el1ces. Unlike those "components", a program unit is not an independent article that can be named or referred to by any means other than from other program units. Therefore, program units have far fewer pl~.lies than do components, and are hence more efficient to form and utilize. Further, program units are subject to a strict seq -~nti~l ordering within a file, while components are not.
Another ~i~tinrtion between the '901 components is that program units are not referred to by ID. Tn~te~, program units are referred to by direct memor.,v refercllce. The dirre~ellce is similar to looking up a phone number in a phone book -- whereas the '901 rer~lcllce makes note of the name of the person to callin a given situation, and therefore the phone llulllbel must be looked up every time it needs to call the person, the present invention notes the person's phonenumber, and doesn't care who is called, since it is known that the right number is being called.

W O g7/32250 PCT~US97/02177 Symbol Table: The symbol table is the central and filn~l~m~ntal repository of information about the program that the compiler gathers from the program constructor in terms of relations between files, or ded-lces from the program text itself as it compiles its program units. While all known compilation systems have a symbol table, the system of the present invention utilizes the symbol table to store more i~ "alion therein than in any known prior compiler.
It should be noted that the word 'table' is not being used here in the traditional mP~nin~ that it has, for example, in relational database theory. A table is normally a vector of homogeneous records, each with exactly the same fields as the others. A compiler symbol table, however, is simply a large set of heterogeneous data objects, and for which relationships are described. These relationships are defined by direct memory lef~ ,,lces.
~cl~islence: Primary memory, such as random-access computer memory (e.g., RAM), is normally allocated and made available to a running program in such a fashion that once a first program le.. ;.~A~Ç~~ the memory is allocated to, for i~ re, a second program, and the contents of the memory lost.
Thus, should the first program be l~LalLed, the same memory is no longer available to the program, and must be reallocated. However, the memory will most likely not have the same contents any longer. In fact, most opelali,lg systems take care to make sure that it does not, so that the several programs operating on the co",l,ulel will not mi~t~k~nly rely on memory ret~ining its contents between invocations. Most secollda,y memory devices, however, including mass storage devices such as hard disks or tapes, do not have this ~101)~1 Iy .
Secondary memory devices (e.g.: "mass storage devices") are often referred to as pelsisl~l~l memory, and are con~i~tçntly used by all but the mosttrivial programs to store data from invocation to invocation. The system of the present invention is not such a trivial plugla~l; neither are most compilation systems. Current compilation systems typically store only the end result of compilation in this mass storage in the form of linkable object files and other auxiliary files. The system of the present invention stores not only the end result W O 97132250 PCTrUS97102177 of the compilation process in the form of IL symbols and code objects, but many of the interrn~?i~te data structures as well.
Interm~ te Lan~ua~e: The system of the present invention internally processes the program definition by compiling one program unit at a time, herebyS producing a heterogeneous data structure herein referred to as 'interm~ telanguage' (IL). This language is better suited than source text for internal manipulations such as optimization and m~ in~-code generation.
According to the present invention, there are two major groups of IL data structures: symbols and operators. The latter are used to describe function definitions and initial values of variables. The former are used to describe declarations of both functions and variables.
Code Objects: A code object re~,rest~ a function or data definition as a sequence of bytes in the form required by the CPU architPctl-re of the target processor. A code object includes a set of relocations, each of which specifies that a pointer value within the code object must point to the code object that is the definition of a specific IL symbol in order to load the code object as part of an executable program."
In the system of the present invention, code objects and intermP~ te language symbols are stored in pel~ l memory as a persistent symbol table.
There is one persistent symbol table for the entire program. Code objects refer to IL symbols by direct memory reference as opposed to by name or by Id. This again, has signifir~nt pelfollLIal~ce advantages over both traditional systems and the increm~nt~l technology previously ~i~Cl~sse~ Since the system of the presentinvention utilizes direct memory lerele~ces as the code objects are formed, no linking process is nPcPss~ry. The system of the present invention therefore pe,ro""s lin?cing as a by-product of compilation, thereby elimin~ting the linkerentirely.
Executable Files: An exec~lt~ble file is a complete representation of a tr~n~?~te~ program, with all le~l~nces between code objects resolved, and in a form expected by the underlying CPU architect-lre and ope,dlhlg system. In other words, an executable file is a persistent representation of the program itself, rather than its data. This is typically what a traditional linker or an incremental linker produces. In the system of the present invention, once program editing iscomplete, the developer causes execution of an extractor which produces an executable file. This extractor therefore p.,lrOll,ls some of the same functions as the separate linker taught in the prior art.
- 5 Extractor: A traditional linker collects object files into an internal data structure. Thereafter, the linker traverses the data structure to gel-~lale an exectuable file. In contrast, the e~ldctor of the present invention performs es~enti~lly the same function and operates in essentially the same way as the second phase of a traditional linker, except that the inputs to the extractor are code objects and IL symbols, as opposed to object files.
Pro~ram Loader: A program loader is a special functionality of most operating systems that brings an executable file into primary memory, and startsthe program under development. This functionality is typically provided by the operating system, and is leveraged by the system of the present invention as well lS as other compilation systems.
The system of the present invention embodies the program under development in a proprietary executable format, and not in a file for reasons which will be later explained. To load a program in this proprietary format, thesystem of the present invention further includes a program e~ Pnt referred to asthe ~unner, which operates in conjullclion with the Imager taught herein to perform the loading function. Here, the proprietary executable format and its corresponding loader are used only when a program is being developed. When it is desired to deploy the fini~h~d program, the predefined e~Pc~lt~hle format appropliate to the target alel~ile~ c (e.g. as an ".EXE" file) and the program loader supplied with the col~spollding opel&ling system are used.
Processes: In most modern opel~tillg systems, including all operating systems to which the system of the present invention applies, each program executes in what is commonly referred to as its own 'process.' A
process has three major prope,lies: a program image, which defines the paths of execution of the program; an entry point, which is the location in the program'simage where execution is to begin; and an address space, which is a range of W O 97132250 PCT~US97102177 memory locations that are unique for that process, and not accessible to other processes.
If dirr~ processes, which are running either at the same time or in sequence, need to share some data, this is normally done using files, as fli~C~lsse~
above. Persistent storage files are not n~cess~rily unique to one program. Two sim--lt~n~ous processes may access the same persistent storage at the same time,manipulating the data stored therein to share information.
Shared MemorY: Modern opc.ating systems in~ ding, by way of example but not limitation, Unix and Windows NT / Windows 95 have another means of sharing data by sharing primary memory. This concept is commonly referred to as 'shared memory.' The system of the present invention places the executable image in shared memory rather than in a file. The l)r~lie~ program loader taught herein can not only set itself up to share memory with the compilation system, but can locate the process entry point at a specified location in shared memory instead of in an image file.
Run-time Code Objects: As previously discussed, code objects are stored persistently, and are used to form the program image. Ref.,lellces between code objects are not in a form suitable for execution on ccll~ er haldw~le.
These references must therefore be tr~n~l~t~l again. In order not to have to modify the code objects for the purpose of forming a runnable program image, and then have to restore to the code objects their original contents, this invention makes copies of them in non-persistent memory, where the program image is formed. These non-~rsi~ code objects are termed run-time code objects.
Each run-time code object for a function definition has one instruction more than the corresponding code object, an instruction which is non-essential to the program itself, but very essential to the increment~l build process. At the beginning of such run-time code object, a branch instruction is written therein that initially branches to the next instruction, which is the first instruction of the function itself. Thus, the instruction does not affect the program logic at all.When the run-time code object is updated by compiling a new definition for the corresponding function, the new code object is either copied into the same run-time code object as the first, or, if it does not fit, or if an activation of the first W O 97/32250 PCT~US97/02177 run-time code object exists, copied into another run-time code object. If the latter is the case, the branch instruction in the first run-time code object is modified to branch to the first instruction (also a branch instruction) of the new run-time code object. This ensures that all saved references to the old run-time code object are S still valid, since after br~nr~ing there, the program will imm~ tely branch to the new run-time code object.
Incremental Ima~er: To form the program image in shared memory, the system of the present invention uses som~thing referred to herein as an incremrnt~l imager. This takes code objects, copies them into run-time code objects, which are placed in shared memory, and links them together forming a program image which is ready to execute. Rather than having the program loader be passively waiting for the image to be complete, and then load it, the programloader of the present invention takes an active part in the image building process.
Debug~er: The term 'debugger' traditionally refers to a program that is used to control the execution of another program, in its own process. It derivesits name from the fact that it is used to rid a program of its errors or "bugs." In traditional compilation systems, the compiler tr~n.cl~tes source text into object files, which are linked together into an exec~lt~hle image file by a linker. Thecompiler and the linker are in(lepen~nt programs, and each execute in different processes. The program is then either run directly by the program loader, or by the program loader under the control of a debugger. Once a bug is found, the program is termin~t~d by the debugger, the source text can then be modified or edited by the developer, and the process starts all over. The reason that the program has to lellllinale is that normal program loaders don't allow the image file to be modified while it's in use by a process.
In the system of the present invention the il clemelll~l builder performs the tasks of the compiler and most of the duties of the linker. The incremental builder, the hlclell,c-lt~l imager, and the debugger all execute in the same process, and the program loader of the present invention allows the image, whichis not in a file, to be modified without having to terminate the program.
Thus, the incr~ nt~l builder of the present invention signi~lrantly reduces the time required to fix a program failure by reducing compilation time and -e]imin~ting the time to perform linking, loading (since the image is built in shared memory), and error state recreation time (since the program does not have to be restarted, but can sit waiting in the error state until the fix is in place). Once the fix is effected, program execution continues as though the error had never existed.
A time-line comparison of the advantages presented by the system of the present invention is shown at Pigure 1. Prior art systems require that the executable image be kept in a file that cannot be modified (updated) while the program process is still "live". This requires thererol~ that in order to effect any changes to the system, it must be stopped and restarted to effect program updates. The system of the present invention, on the other hand, does not place the executable image in a file, but in shared, mapped memory. The image is thus made available for simultaneous manipulation (by the Imager) and execution (by the Runner).
As shown in Figure 2A, the system of the present invention op~l~.Les on CPU 50, display monitor 52, memory system 54 and input device 56. CPU 50 preferably is an Intel Pentium; display 52 preferably is a Super-VGA Monitor;
memory 54 preferably in~lu-~es DRAM and m~gn~tic disk storage; and input device 56 includes both a keyboard and a mouse and related interface subsystems.Referring now to Figure 2B, an overview of the program of the present invention is shown. When a developer invokes debugger 1 for the first time for agiven set of source code files 3, the increm~nt~l builder 4 is invoked which translates source code files 3 into code objects and their corresponding IL
symbols. The code objects and IL symbols are stored in a persistent symbol table5. Persistent symbol table S may be implem~nt~d in any type of memory device.
An incre~ n~l imager 6 utilizes the code objects and IL symbols to form run-time code objects. Increm~nt~l imager 6 then links together run-time code objects forming a program image which is executable. The actions of the imager and the program are cooldil~led through a probe 7 and a runner 8. The probe 7 executes in the hl~ melllal imager process and the runner 8 executes in the process of the built program. Thus, the image and the program are ex.ocllted simultaneously in two separate threads of execution using memory-mapped data.

W 097/32250 PCTAJS97/~2177 A principal aim of the h~cre~ l builder of the present invention is to minimi7~ the compilation effort and time required to construct a complete compiled representation of a co~ uler program after a change to the program's source text, without requiring a specialized lepl~se~ ion of program source (e.g.: program source text implem~-nt~d as ~t~h~e objects) or excessive overhead in initial compilation. These shortcomings are evident in the inventions taught for instance by WO 95/00901 and WO 95/00904. The system collectively taught therein requires that program source text be implem~nt~d as rl~t~b~ce objects. Furthermore, while these and other previously developed increm~nt~l compilers may provide advantages in recompilation, they are slow when building an entire program using very complicated int~rn~l structures and they limit the use of a llumbel of desirable tools that operate on source files. In contrast, the increm~nt~l builder of the present invention accomplishes its ~lh~cipal aim by aprocess referred to hereinafter as minim~l recompilation. Rec~n~e the increrne~t~l builder of the present invention does not utilize source text implem~nt~d as database objects, it is capable of much faster initial building than previous increment~l compilers. Furthermore, on rebuilding a program, its use of pointers as opposed to ~l~t~ e object Ids results in faster increm~nt~l rebuilds.
Program source text typically specifies an interface, an implementation, or both. An interface specifies the behavior of one part of a program so that it can be used by another. An implem~ t~tion provides the specified behavior. Most progr~mming languages provide a way of making interfaces accessible to separately-compiled source. The principles of the present invention are applicable to the compilation of any computer progl~ "~ language with a facility for logically including interfaces specified in one compilation into another. This facility is termed named-translation inrhlsion, and is inrhlded in many plo~;ld~"".ing languages. By way of illustration but not limit~tion such languages include Ada, Modula-3, and Java. One feature of the present invention is that it adds named-translation inclusion to the C+ + progr~mming language.
Standard C+ + uses lexical file inclusion. This means that source code to be included in more than one compilation is compiled as a part of each such W O 97/322~0 PCTrUS97/02177 compilation. Clever C~ + imple~ ons can reuse the results of compiling a set of lexically-included files in several compilations, but must completely recompile the entire set if any one lexically-included file changes.
As taught herein, minim~l recompilation may be broadly said to S incorporate the following principles:
(a) Dividing source files into relatively small pieces, termed herein program units. These small file pieces enable simple, fast recompilation.
(b) Keeping track of the dependencies which program units have on even smaller c~ )ol~nl pieces, termed herein program objects. Program units are said to be dependent on program objects to which they refer.
(c) Quickly dclc~ g, after recompilation of a portion of a program, which program objects have logically changed and, thelcrolc, which dependent program units need to be recompiled.
A partial ordering orders groups of entities, allowing any order within a group. A p~fc,lcd embodiment of the present invention employs a partial compilation order for source files--typically, several files can be compiled .cimlllt~n~ously--and so can effectively utilize all available processors in a multiprocessing C~ UIC1.
Selected properties of program objects are shown at Figure 3. Each ProgramObject P has the following properties:
A Dependents set, which contains the ProgramUnits whose compilations depend on the ProgramObject.
An encoding of the m~ning of the ProgramObject, termed herein a Signature. A Signature, in a l!,cfellcd embodiment of the present invention, is a 64-bit h~hing of a string representing the logical properties of the ProgramObject. The possibility of two dirr~"ll strings h~ching to the same Signature is s--ffl~ierltly unlikely as to be ~t~ti~tir~lly in~ignifit~nt A pse~ldoco~le representation of one sign~t lre creation methodology is as follows:
Signature Creation:
Construct a text string ~ ,sGnl---g an encoding of cignifir~nt logical p. u~. Li~S;
Hash text string;
End.

W O 97/32250 PCT~US97102177 For example: consider the program object created for the data member X
declared in source file A according to Figure X declared here:
classC {
~tX;
};
The logical properties of X are that it is named "X", it is a data member of class "C", it is of type signed int, it has an offset of zero, and has a ploleclion ofprivate. One possible encoding of these properties is the string:
"Xpriv~tesign~dintCdata0". A much more compact representation is both possible and desirable. By way of illustration, but not limitation, in a ~lefelled embodiment of the present invention, an encoding of the plup;cllies of this datamember is representable as: "XpSICD0".
By way of illustration, but not limitation, for the C++ progr~mming language, each of the following col~lilulcs a program object:
An entire (lexically-included) source file;
A macro;
A tag, function, typedef, template or variable declared at n~m~space-scope.
An ellulllel~toi, and A tag, function, typedef, or data member of a class.
Each program object is co~ led by a program unit. As previously c~ssed, a program unit is the smallest entity which is compiled at one time by a plcfelled embodiment of the present invention. Dirrtrell~ pro~ ,-"ing languages may impose differing structures which require the formation of program units.
In the general case, the following constitute program units:
Entire files; and Syntactic declarations not contained within other syntactic declarations.
By way of illustration, but not limitation, in the C+ + progr~mming language each of the following con~ les a program unit:
An entire file;

A file scope declaration; or A n~mespare scope declaration.
One of the requirements of the Incremental Builder of the present invention is to recognize all program objects. For each program object, the builder creates an internal representation of the program object called a ProgramObject and associates it with an internal representation of the cont~ining program unit called the ProgramUnit object of the cont~ining program unit.
Selected propcllies of a generic program unit are shown at Figure 4. It should be noted that, in a preferred embodiment of the present invention, Program Units may be further classed as either DeclarationProgramUnits or SourceFileProgramUnits. Each ProgramUnit has the following l~lo~.Lies:
A RequiresCompilation flag, which can have the value true or false.
A DependsOn set, which cont~in~ the ProgramObjects used in the compilation of the ProgramUnit.
Referring now to Figure 5, the additional properties of a Declaration-ProgramUnit include a list of the ProgramObjects contained in the Declaration-ProgramUnit.
Having ler,lellce to Figure 6, the additional properties of a SourceFileProgramUnit include:
A list of the Macros contained in the SourceFileProgramUnit; and A list of the DeclarationProgramUnits which it contains.
Program source text consists of separately-compilable text files (source files) which may be edited by any text editor. A source file that specifies interfaces usable by other source files defines a named translation. A source file that uses interfaces specified by other source files includes the relevant namedtranslations (by name) in its compilation. For each compiled source file, the incremental builder of a plere.led embodiment of the present invention creates aTr~n~l~tiQn. This translation is named or l~nn~m~d as a~propliate.
Tncll~din~ a translation in a compilation recursively includes all tr~n~l~tion~
included in the compilation of the translation. A source file A defining translation AA must be compiled before source file B if the compilation of B includes AA.

This rule defines a partial compilation order for the source files of a program.Any set of source files whose compilations are not ordered by this rule can be - compiled in any order or .~imlllt~n~oll~ly.
Within a source file, the syntactic elern~nt~ are built in the order specified by the progrAmming language definition For the C+ + ~logl,.... l.. l;.~g language, this corresponds to the lexical ordering of the source text.
For the initial building of a source code file every SourceFileProgramUnit requires building. The incrPm~nt~l builder builds the source file represented byeach SourceFileProgramUnit in an order that obeys the partial build order, stopping when no unbuilt source files that don't include unbuilt named Translations remain. For each source file that builds without error, the programbuilder marks the corresponding SourceFileProgramUnit as not requiring building. Every DeclarationProgramUnit in each such SourceFileProgramUnit is marked as not requiring building.
When a source file has been modified, e.g., following editing during the debugging process, the incretnPnt~l builder of a pl~relled embodiment of the present invention rebuilds only those program units whose source text is edited or that depend on one or more program objects whose logical pl~clLies have changed. This is done by ~ -ki~ the SourceFileProgramUnit of each source file whose text has changed (or that lexically includes a source file whose text has changed) as requiring compilation. This is done by setting the Requires Com~ ion flag to true. The increm~nt~l builder then traverses all SourceFileProgramUnits in compilation order. For each SourceFileProgramUnit:
1. If the SourceFileProgramUnit needs building, it is rebuilt in its entirety.

2. Otherwise, the compiler traverses all DeclarationProgramUnits in the SourceFileProgramUnit in lexical order. Each DeclarationProgramUnit that needs compilation is recompiled.
The following steps are p~,.rolllled when rebuilding a Source File Program Unit:

1. For each Declaration Program Unit in the Source File Program Unit SF1, delete the Declaration Program Unit from the Dependents set of each element of its DependsOn set, and then discard its DependsOn set.
2. Compile the text of the source file to produce a new SourceFileProgramUnit SF2 (and a new Translation).

3. Construct a table T cont~inin~ all of the ProgramObjects in all the ProgramUnits in SF2, indexed by Signature.

4. Traverse the Program Objects in SF1. (This requires traversing the list of DeclarationProgramUnits.) For each Program Object O, look up O's Signature in T. If an entry O1 exists, copy each element of O's Dependents set to Ol's Dependents set and delete O1 from T. If an entry does not exist, mark each element of O's Dependents set as n~e~ing rebuilding by setting the RequiresCompilation flag to True.

5. If step 2 did not determine any portion of the declaration to contain malformed source code, mark the Declaration Program Unit as not nloeding building.
The rebuilding of a Declaration Program Unit is accomplished as follows:
1. Delete the Declaration Program Unit from the Dependents set of each element of its Depends On set, and then clear the Depends On set.
2. Compile the text of the Declaration Program Unit to produce a list of Program Objects.
3. Construct a table T co..~ all of the new ~O~lalll Objects, in~ ed by Signature.
4. Traverse the list of the old Program Objects. For each Program Object O, look up O's Signature in T. If an entry O1 exists, copy O's Dependents set to O1 and delete O1 from T. If an entry does not exist, mark each element of O's Dependents set as n~e~ing recompilation.
5. Replace the old list of Program Objects with the new one.

6. If step 2 did not de~ e any portion of the declaration to contain malformed source code, mark the Declaration Program Unit as not n~e-ling building.

W O 97/32250 PCTrUS97/02177 The process for the creation of Program Units, Program Objects, IL Symbols and Code Objects may be represented in pseudocode form as follows:
Parse program text, i~ iryil~g file-scope or ~ F~ e-seope syntactic declarations, and tldlillg an abstract syntax tree (AST) for each such declaration;
Do for each AST:
Create a Program Unit U;
Perform semantic analysis of AST as follows;
Create symbol tab}e nodes, i~ ProgramObjects that ~c~,SC~
entities in U for whieh rec~mrilq~il)n~ r~ .-- if C are IIIF~
RPCO~ P ~ n~ ;~S on existing ProgramObjeets;
Do for each .l~p~ n~d-upon Program Object PO:
Add PO to U's DependsOn set;
End do;
Do for each funetion (or statie-storage datum) FD declared in U:
If FD does not have an IL symbol, create one for it.
If the AST defines FD, generate IL to l~Jlcs~.ltthe definition and then perform code generation on the IL to produce a code object.
End do;
End perform;
End do;
End.
Referring now to Figures 7 through 12, an exemplar demonstrating the principles of the increm~n~l builder of the present invention, as previously ~ cucse~l is shown. These source text examples are written in the C+ +
progrtmming language. The sign~tl-re values that appear in Figures 10, 11 and 12 are arbitrary and not derived using any algo~ n.
Having reference to Figure 7, a first source file A, 1098, declares class C
and declares data member X to be of type integer and declares member functions Fred and Barney to take no arguments and return an integer.
Having reference to Figure 8, a second source file B is shown. Source file B defines the functions Fred and Barney. Function Fred is defined to returnthe value of the X member of the C object for which Fred is invoked. Function Barney is defined to return the result of invoking Fred for the C object for which Barney is invoked.
Figure 9 represents a mo-1ifi~tinn to source file A previously shown at Figure 7, wherein the source file is modified during editing. In this example, the declared type of X, which type was he..,t~fole integer, is changed to floating point. The textual definitions of Fred and Barney remain l~n~ ~nged.

W O 97/322~0 PCTAUS97/02177 Referring now to Figure 10, the compilation of source file A, 1098, is shown. The building of source file A, 1098, generates a first program unit 1100 Program unit 1100 represents the ~m~ st individually compilable element cont~in~d within source file A, 1098. The creation of program unit 1100 causes the creation of program objects 1110, 1120, and 1130. Program object 1110 represents the declaration of data member X and contains a signature, in this example 101, and a dependents set, whose initial state is empty. The declarationfor the function Fred is embodied in program object 1120 also has a sign~ re, inthis example 37, and a dependents set which is initially empty. Because program object 1120 represents a function declaration, it will result in the creation of an IL symbol. Accordingly, the pointer in the IL symbol field in program object 1120 is set to point to IL symbol 1140. IL symbol 1140 contains pointers to an old definition and a new definition both of which are, at this time, set to null.
In similar fashion, the function declaration for the function Barney results in the creation of program object 1130 which, in this example, has a sign~llre value of 483, and a dependents set which is initially empty.
In ~I~llllll~l.y, the building of source file A has resulted in the creation of one program unit which causes the creation of three program objects. The program objects which declare functions result in the creation of IL symbols.
Program objects which are declarations of data members do not result in the formation of IL symbols.
Rcr~ g now to Figure 11, the building of source file B, 1099, results in the creation of two program units 1160 and 1170. Program unit 1160 represents the syntactic definition of the function Fred, previously shown in Figure 8.
Program unit 1170 represents the syntactic definition of the function Barney also shown in source file B.
Having continued lerelcnce to Figure 11, in this example, program units 1160 and 1170 each create one program object, 1180 and 1190 respectively, and the pointers defined in the program objects field of program units 1160 and 1170point to program objects 1180 and 1190. Program object 1180 is the program object representing the definition of the function Fred previously declared by WO 97/32250 PCT~US97/02177 program object 1120, and as such points to IL symbol 1140 which, it will be recalled, was created responsive to the declaration of the function Pred by program object 1120. In similar fashion, program object 1190 le~,lesel,~ the definition of the function Barney previously declared by program object 1130, S and again program objects 1130 and 1190 each contain pointers to the IL symbol 1150 for the function Barney. This is important to note in that this is the m~ch~ni~m whereby the present invention achieves linking during the process of compilation. This feature takes advantage of a function commonly found in most compilation systems, that of creating a unique intermP.li~te language symbol to ~ esenl one function or piece of static data. Rec~llse, however, the present invention is able to track the symbol table across compilation units, it is possible to have one IL symbol for a given function in all compilation units and, therefore, no separate linker is required.
In forming program unit 1160, the incremental builder detects dependencies on program objects 1110 and 1120. Accordingly, the depends-on set of program unit 1160 inrllldes pointers to program objects 1110 and 1120 (asshown along pointer line D), and the dependents sets of 1110 and 1120 are each augmented to include a pointer to program unit 1160. In similar fashion, program unit 1170 depends on program objects 1120 and 1130. Accordingly, the depends-on set of program unit 1170 includes pointers to program objects 1120 and 1130 (as shown along pointer line C), and the dependents sets of 1120 and 1130 are each ~ng~nPl~tPd to include a pointer to program unit 1170."
Summarizing the actions to this point in system execution, data member X has been declared and the functions Fred and Barney have been declared and deflned. Accordingly, the definition fields of IL symbols 1140 and 1150, previously null, are now set to point to code objects 1200 and 1210 respectively.
Code objects 1200 and 1210 are created as the definitions of Fred and Barney, respectively.
At this point the hlcle...P~ l builder of the present invention has built, from source files A and B, the IL symbols and code objects nPcess~ry to form theprogram image of this program. The IL symbols and code objects are stored in the pG~ symbol table.

CA 02248l8l l998-08-28 Referring now to Figure 12, the effects of editing one of the source files of the program is shown. Having again le~Jence to Figure 9, it will be recalled that source file A was Am~nt1e~ therein to change the type of data member X
from integer to floating point. In traditional compiler technology, this change would require the recompilation of all the code in the program, including sourcefiles A and B. However, one of the advantages of the present invention is that of re~uiring only the rebuilding of those program units which have been edited or which depend on program objects that have changed mP~nin~.
In Figure 12 the results of the editing formed in Figure 9 are shown. The change to data member X to be of type floating point requires that the data member declaration contained in Figure 7 be updated as shown in Figure 9. This results in the rebuilding of program unit 1100 and program objects 1110, 1120, and 1130. Accordingly, those elements are rebuilt at this time. It is important to note that program unit 1100 itself, as well as program objects 1120 and 1130 remain llnrh~nged as a result of ch~nging the type of data member X.
Accordingly, while a new program unit 1100 and program objects 1120 are thus formed during the rebuild process, they are i(1enti~l to their previous values.
Program object 1110, however, has changed to reflect the new type of the data member X. Accordingly, program object 1110 ge~ t~s a new sign~h-re now shown to be of value 409. IL symbol 1150 remains l-nrh~nged Because program object 1110 is not rebuilt with an i-lPntic~ sign~hlre, each program unit in its depen~le~t~ set must be rebuilt. Thelefolc, piOgl~ll unit 1160 is rebuilt, which causes program object 1180 to be rebuilt and code object 1220 to be created.
IL symbol 1140 is updated as follows: its old-definition field is set to the prior value of its definition field (i.e. to point to code object 1200) and its definition field is set to point to code object 1220. Program unit 1170 and program object 1190 are not rebuilt. After program object 1180 is rebuilt, IL
symbol 1140 is updated as follows: the definition pointer is changed from the relocations field of code object 1200 to the relocations field of new code object 1220 which contains the llp~ted object code. Similarly, the old definition fieldof IL symbol 1140 is reset from the null value to the relocation value of the W O 97/32250 PCTrUS97/02177 previously defined code object 1200. All program objects, program units, IL
symbols, and code objects are stored in the previously (li~cllsse~l persistent symbol table implem~-nted in a ~t~b~e further implementçd in a memory device.
For reasons of system efficiency, an object oriented f1~t~h~ce is utilized in a S plcfellcd embodiment of the present invention.
With respect to Figures 10-12, the Dependents field of the program objects (e.g., 1110) and the depends on field of the program units (e.g., 1160) are implemented, in a preferred embo-1im~nf, as a binary tree of pointers arranged for efficient sorting. For the sake of clarity in these figures, the actual list of pointers, and the several poil-lel~ referred to therein, have been omitted from the figure elements lcl.fcsellling program units. The concept of a field comprising a binary tree of pOi~ -s is well known to those of ordinary skill in the art.
The net result of the editing process previously ~li.ccu~.~ed is that IL
symbols 1140 and 1150 point to code objects 1220 and 1210 respectively and these IL symbols and code object pairs are now available in the persistent symbol table for use by the inclelllell~l imager of the present invention to form and run the program image thereby defined.
In a traditional compilation system, a linker is invoked to take object code in the form of object files and resolve all the IcÇ~,lences between objects in the object code, and to produce a new, dirrercll~ image using the object files. Thisnew image may Icplesell~ an entire plO~ ll (e.g., an exec~t~ble file) or a self-contained portion of a program. Such a self-contained portion would have no unresolved external refelellces and is often referred to as a dynamic link library, or DLL.
In the system of the present invention the Increm~nt~l Imager pelrolllls some of these tasks normally performed by a linker. The Increment~l Imager performs these tasks not on object files, but on the previously defined Code Objects.
Object code elem~-nt~ are not l-~cçss~-ily placed in the same order or at the same address in the eventual image as they are in the original form. This is intuitively obvious in the case of object files, where addresses of function and WO 97/32250 ~CT~US97/02177 data definitions (ex: code objects) are merely their position relative to the beginning of the file. Thus, there is no chance that a definition will Illtim~tely occupy the same position in the ex~cut~ble file as it has in the object file. This is true even if the linker only needs one object ~lle to produce the execut~ble image, S since the size of file header information is typically very different between the e~ec~lt~hle image and the object file.
In the system of the present invention, code objects are copied into runtime code objects by the Imager, and will most definitely not have the same address as the original Code Objects did. Code objects and ~llntim~ Code Objects coexist in the same address space, and must therefore have different addresses.
There is also a signif1~nt rlictinrtion to be made between the address that a code object has in relation to the beginning of an executable file, and the address that it will have when loaded in memory by the program loader. There is a further distinction in the system taught herein between the address of a runtime code object in the Imager process and that in the Runner process. It is an inherent propclly of shared memory that the same piece of physical memory does not n.ocec.c~rily have the same logical address in all plocesses where it is shared. Some, but not all, op~ g systems do guarantee the s~rnen~ss of addresses of shared memory blocks. This is not generally done, however, out of concern for the application programmer, but because it is a simplistic implem~nt~tion of an operating system.
In summary then, there are two kinds of objects to be dealt with when Ço.nlillg an image:
A. Code Objects, which represent the tr~n.cl~te~ program items in their un-relocated form.
B. Runtime Code Objects, which repleselll the tran.cl~ted program items in their relocated form.
Also, from the precet1ing diccuccion, there are three categories of object addresses which must be accounted for:
a. The addresses of Code Objects in the address space of the Imager/Debugger .

CA 02248l8l l998-08-28 W O97/32250 PCT~US97/02177 b. The address of Runtime Code Objects in the address space of the Imager/ Debugger.
c. The address of Runtime Code Objects in the address space of the Runner.
S In other words, each Code Object T is located at address Ta and its corresponding Runtime Code Object R is located in memory that is found at address R" by the Imager, and at address ~ by the Runner.
Addresses are normally used by software by the means of pointers (the term used in such programming l~n~U~ges as C, Pascal & C+ +), or refclellces (used in such languages as Pascal and C+ +). Both forms simply retain the full, absolute address of an object. While hardware (e.g.: the con~lllel's CPU) also uses addresses, it generally has more complex needs than does software. In orderto simplify the logic of the transistors that implement a given haldwd~e architecture, addresses utilized by haldw~.e are formed by a myriad of differentschemes, each of which depends on the hardware itself. By way of example, but not limitation, these haldwale addresses may be split in two pieces, located in dirr~lcllt instructions, or made relative to the address of the instruction that uses the address.
Accordingly, it is seen that there are three consumers of addresses:
I. The Imager/Debugger which needs to use addresses from categories a and b when writing or reading data in objects (such as Code Objectsand Runtime Code Objects).
II. The Runner which needs to use addresses from category c when writing or reading data in objects (just Runtime Code Objects and their support structures).
III. The CPU, i.e. the haldwale, which needs to use addresses from category c when accessi~-g data, but may often not be able to use them in the same form (pointers) as the Runner software is.
To further complicate this issue, the CPU is not h~ ei.led in the address of the Runtime Code Object itself, but the address of the tr~n~l~ted object codewhich is contained within it at some offset.

3~
The purpose of the process of code object traversal and relocation in this invention is to establish the nf cess~ry set of Runtime Code Objects, establish their addresses Rb and Rc~ and make them available to consumers II and III (the Runnerand the CPU), taking note of the fact that the CPU is really hllc~ d in R~+k, where k is the constant (i.e.: the same for all runtime code objects) offset of the tr:~n.~l~t~1 object code within the runtime code objects.
In the present invention, as in most linker technology, a relocation record is a piece of data that describes what form an address must take at a given point in the program. The form varies not with the referenced address, but with the location where the ~r~lellce occurs. For example, data items that contain addresses of functions usually need to keep absolute pointers to the function, since the address will likely (but not certainly) be used by software.
The CPU might use absolute addresses when using so-called branch-tables, or when calling to an address that was established dyn~mi~lly (at runtime) by the program. This is referred to as an indirect reference, since the address referenced is contained in some piece of data that first has to be referenced. The ha~dw~le is also likely to use direct addresses when lefel~i"g to data, but willnormally use a more complex form when making a direct reference to functions from other functions. An example would be when implem~ntin~ a subroutine call.
From the preceding, it is shown that in the present invention the process of relocation is one of tr~n~l~ting full addresses of ~llntimP Code Objects to whatever form is nPcesC~y at each point of ler~nce.
Since Code Objects are not that which is actually exec lte~ in the system of the present invention as the program image, relocation is performed on Runtime Code O~jects only. However, the relocation records are ~ttaclle-1 to theCode Objects and not the Runtime Code Objects. This is logical, since relocationrecords are used in forming the complete Runtime Code Objects when building the image (either initially or i"cle.,.~ ;.lly).
To perform relocation, the Imager iterates over all Code Objects for which a corresponding Runtime Code Object has been created. For each Code Object T, it processes each relocation record L this way:

1. Note the type (form the address must tal~e when used by the CPU) of relocation L.
2. Note the offset OL within T where the lcr~lellce occurs.
3. Note the addresses Rb and RC of the code object that corresponds to T.
4. Note the address of Sc Of the code object that is referred to at OL.
5. At address Rb place a pointer to SC+k, where k is the constant offset of the tr~n.c!~t~cl object code within the runtime code object.
If this pointer must be relative to the current location, use Re+OL+k as the current location.
The Imager/Debugger not only produces information for the hardware, it also produces information for the Runner. In that process, it need not worry about relocation forms and other complexities, since all the Imager/Debugger needs is to make sure to use addresses from category c whenever it comml-nic~s with the Runner. In some cilc~ es where space is of concern, and it is n~ce~s~ry to avoid duplication of hlrolll,ation, the Runner will in fact accept addresses from category b, and itself translate them to category c.
Figures 13 through 17 illustrate a ~lef~ ,d embodiment of incr~mPnt~l image formation according to the principles of the present invention. Referring now to ~igure 13, the use of the code objects and IL symbols in forming the program image is shown. An IL symbol, previously stored in the ~crsislel1t symbol table, is shownat 2000. Each IL symbol 2000 contains a definition field 2001 which comprises a code object pointer 2004 which points to a corresponding code object 2010 by means of direct memory addressing. Definition field 2001 is defmed by IL
instruction stream 1999. IL symbol 2000 further colllplises a dependents field 2002 and an old definition field 2003. Optionally, IL symbol 2000 further defines an IL name 2005. It is important to note that name 2005 is used only to display h~llllation to the programmer, e.g., in debu~ mode, and is defined during image creation. Each IL symbol 2000 iS formed by the incremental builder of the present invention from part of the IL stream 1999.

W O 97/32250 PCTrUS97/02177 Code object pointer 2004 points to a code object 2010 corresponding to IL
symbol 2000. Each code object 2010 contains a size field 2011, a relocations field 2012 and a code section, 2013. Code sections 2013 contain the previously ~i.cc~lssed fully tr~ncl~tpd m~hinP-language implem~nt~tions of function definitions and the initial values of variables. Relocations field 2012 defines a relocations pointer 2014, which points to a relocations array 2020. Relocations array 2020 comprises a vector of relocation offsets and IL symbol references.
The IL symbol to which relocations pointer 2014 points may be the same IL symbol which invoked the formation of code object 2010, as in the case of a recursive process, or in the more typical case, may point to a dirr.,lell~ IL symbol altogether.
The update table is m~int~inPd by the Builder for the Imager, so that following editing or other changes to the source file, the Imager can locate thecode objects which have changed. This ensures that Imager forms the image according to the newest code objects when updating the image.
Referring now to Figure 14, the formation of run-time code objects is ~liccllcse~ Each IL symbol 1000 causes the formation of a run-time code object 1010. As the IncremP-rlt~l Imager 200 of the present invention continues to analyze code objects 1010, a-i-lition~l IL symbols 1000 are added to the image.
Each of these code objects contains a pointer to a relocations table 1020 as previously ~liccllcsed, which points in turn to an IL symbol 1000, typically of f~ner granularity, which in turn points to its code object 1010 and so forth. At the point where no more IL symbols 1000 can be added, all the code objects 1020 which define the program have been formed. When all the code objects in the image have been formed, a state of Llallsilive closure is said to exist.
Run time code objects are copies of code objects having additional scratchpad memory inrl~ldPd in their structure, and with references changed to m~hine dependent forms during relocation. Before copies of the code objects are made, it is nPcess~ry to start the probe and the runner. The runtime loader starts prior to copying the code objects to the run-time code objects. This is because the utilization of shared memory requires that the two processes, i.e. the probe and the runner, must agree on memory to be allocated between the two processes. Each run-time code object contains refclellces to ~wo addresses, one for each process.
Referring now to Figure 15, for each code object 1020 having a persistent address, e.g.: 100, the incre-mP-nt~l imager 200 directs probe 300 to allocate memory. Probe 300 then directs runner 400 to allocate memory. It is important to recall that probe 300 and runner 400 are operating under two separate threadsof execution in two distinct processes, and are logically separated by a processboundary 399. Responsive to the comm~n-1 to allocate memory, runner 400 allocates a m~chin~ address, e.g. 550, and returns this address across process boundary 399 to probe 300. Probe 300 passes the address to incremental imager 200 which associates it with the run-t~me code object.
The probe, in a pler~lled embodiment of the present invention, is implemented according to the following pseudocode representation:
while (true) do accept ~ ' from Imager = >
Stream data into communication pipe send data to Runner Runner = >
Gather streamed data Runner send data to Imager end while The runner in a plefelled embodiment of the present invention, is implem~n~ecl according to the following pseudocode r~les~ ;.til)n:
Startup Thread:
start C.. -.:cationThread start ApplicationThread wait for ProgramT. ,~ . . on Application Thread:
wait for StartAlnpliention find program entry point execute from program entry point send ProgramTennination to StartupThread terminate A,nplicntianThread C~ io~- Thread:
terminate - false while (not terrninate) loop accepr ~ I from Probe = >
SUICIDE = >
termin~l e = true DEFINE ENTRY = ~
set program entry point ALLOCATE = >
0 allocate a block of sh~red memory DEALLOCATE = >
de-allocate a block of slwred mernory REAL~P = >
map already n11~cnt~rl block into memo~y again ACIIVATE = ~
performfixups and initialize static data in program send Srart ~rplicnti(7n to ApplicationThread READ MEMORY = >
send data from ~unner memory to ~ L,~ger WRITE ME~MORY
write data from debugger into runner memory end accept send answer data to Probe 30 en~ while tenninate C~ Ationlhread To create the run-time image of the program this process is iterated for each code object, as shown in Figure 16. Having reference to that figure, to create the program image, the developer directs the debugger 600 to start at step 1. Debugger 600 orders imager 200, at step 2, to build as previously di.ccl?ssecl.
For each code object (not shown), the imager 200 directs the probe 300 to allocate memory at step 4. Probe 300 orders runner 400 to allocate memory at step 5. Runner 400 allocates m~hinP memory, and returns the address to probe 300. Probe 300 returns the address to the imager 200. This causes a request for the next code object to be issued. Steps 2 through 5 are repeated for each code object in the file. When the runner has completed allocating machine memory for the last code object, it informs probe 300 that it has fini~h~d allocating memory at 6. Responsively, probe 300 informs imager 200 at 7 that all memory has been allocated. The imager, at 8 informs the debugger at 8 that the build is W O 97/32250 PCTrUS97/02177 complete. At this point, the image is completely formed in memory, and the program may be executed.
Having continued r~ cnce to Figure 16, responsive to the receipt by debugger 600 at step 8 that the image is built, debugger 600 orders probe 300 S directly, at 9, to start the application. This order is tr~n~mitted in turn to runner 400 at 10. This results in the image being ex~cllte~l the image embodying the program ~imlllt~n~ously in two separate threads of execution using memory-mapped data.
During this process, should the application fail, the computer's operating system reports the failure to the debugger, along with the m~r1tin~ address of the failure and, optionally depending on the operating system, other information relating to the nature of the failure. The debugger tr~n~l~te~ the m~chin~ address to the code address, along with the other failure infu~mation, and informs the developer of the failure, its code address and the failure data.
The developer uses this i.lrollllation to edit the source code file. Once the source file is edited, the developer instructs the debugger to resume program execution.
The debugger directs the imager 200, at step 11, to update the code objects affected by the editing as shown in Figure 17. Having reference to that figure, IL symbol 901 caused the creation of code object 902 which in turn invoked the creation of runtime code object 903. For the purposes of illustration, in this example, runtime code object 903 is referred to by a second runtime codeobject 904. After the developer edits that portion of the source code which caused program failure, e.g. an illll,rol)elly defined function 901, the pointerfrom IL symbol 901 which represents the function is reset from old code object 902 to the new code object 905 which embodies the cull~;ct~d function. New code object invokes the creation of a new code object 906. To effect this functionality, the pointer 910 of old runtime code object 903 is reset from line 0 of its contained code to line 0 of the contained code of new runtime code object 906.
The present invention may be implemented on any data storage or transfer mPdillm known to those of ordil~y skill in the art. By way of illustration, but not limitation, such media include but are not nPcess~rily limited to: m~gn.oti~

W O 97132250 PCT~US97/02177 media including m~gn~tic disks, tapes, hard drives and the like; electronic memory devices including RAM, flash memory and the like; mech~nir~l data storage materials inr.lutling punch tape and Hollerith cards; and optical media including CD-ROM, "floptical" drives and the like. Furthermore, the principles of the present invention specifi~ ~lly collt~mrlate the llal~Ç~l thereof over any media including telephone lines or other electronic networks to include local and wide area nets, including but not limited to the Tnt~rn~t The principles of the present invention further specifically contemplate such implem~nt~tion for the storage, use, l~ r~L, sale, lease, rental or any and all other usages thereof.
The present invention has been particularly shown and described with respect to certain ~l~fel-~d embodi,llelll~ of features thereof. However, it should be readily a~pal~,lll to those of ordinary skill in the art that various changes and modifications in form and detail may be made without departing from the spirit and scope of the invention as set forth in the appended claims. The invention disclosed herein may be practiced without any element which is not specifically disclosed herein.

Claims (13)

1. 41What is claimed is:
   An interactive computer program development system that enables real-time debugging of a program as it is compiled without the need for batch recompilation, re-linking, re-loading or the need for the developer to re-createerror states, said system comprising:
   a computer system having a given set of source code files, and having first and second internal processes;
   a debugger resident in said computer system for debugging said given set of source code files;
   an incremental builder resident in said computer system which when invoked translates said source code files into code objects and their corresponding intermediate language (IL) symbols, said incremental builder beingassociated with said first internal process for forming said code objects and program units into a built program which is executable;
   a persistent symbol table resident in said computer system for storing said code objects and IL symbols;
   an incremental imager resident in said computer system that utilizes said code objects and IL symbols to form runtime code objects, said incremental imager being associated with said first internal process for combining together said runtime code objects to form a program image which is executable, said incremental imager introducing a branch instruction at the beginning of each of said runtime code objects;
   a runner resident in said computer system that executes said built program in said second internal process, said runner finalizing said program image and executing said program image in said second internal process; and a probe module resident in said computer system and in said first internal process for coordinating actions of said incremental imager and said runner;
   whereby said program image defining said program, utilizing shared memory data, is simultaneously executed and incrementally manipulated to reflect changes in said source code defining said program.
2. 2. An interactive computer program development system as in Claim 1 wherein said code objects and said IL symbols replace object code files, and said runtime code objects replace executable files.
3. 3. An interactive computer program development system as in Claim 2 wherein said runner operates in conjunction with said imager to perform a loading function.
4. 4. A method of building code in memory for retention after execution of the resulting program without producing any object files or any executable files.
5. 5. A method as in Claim 4 wherein said code is code objects and said memory is persistent.
6. 6. A method as in Claim 5 further including the step of storing at least code objects and intermediate language symbols in a persistent symbol table.
7. 7. A method as in Claim 6 further including the step of storing program units, program objects and other data structures in said persistent symbol table.
8. 8. A method as in Claim 7 wherein said executing program may be simultaneously manipulated for the purpose of updating, wherein said program is not terminated or restarted during said manipulation.
9. 9. A method as in Claim 8 wherein said step of simultaneously manipulating uses the jump instruction at the beginning of the runtime code objects to facilitate image updating.
10. 10. An interactive computer program development system as in Claim 1, wherein the same internal data structures are used to produce executable files as are used for incremental imaging.
11. 11. An interactive computer program development system as in Claim 10 furthers including an extractor for producing an executable file from said code objects and intermediate language symbols whereby the need for returning to the source files is eliminated.
12. 12. A method as in Claim 6 further including the steps of:
    translating an expression or a statement to a code object;
    updating an existing image with a corresponding runtime code object; and executing the runtime code object for the purposes of interactive expression evaluation or statement execution while debugging.
13. 13. A method as in Claim 12 further including the steps of translating, updating and executing used for conditional breakpoints.