WO2005038630A2 - Automated safe secure techniques for eliminating undefined behavior in computer software - Google Patents

Automated safe secure techniques for eliminating undefined behavior in computer software Download PDF

Info

Publication number
WO2005038630A2
WO2005038630A2 PCT/US2004/034422 US2004034422W WO2005038630A2 WO 2005038630 A2 WO2005038630 A2 WO 2005038630A2 US 2004034422 W US2004034422 W US 2004034422W WO 2005038630 A2 WO2005038630 A2 WO 2005038630A2
Authority
WO
WIPO (PCT)
Prior art keywords
pointer
safe secure
safe
requirements
bounds
Prior art date
Application number
PCT/US2004/034422
Other languages
French (fr)
Other versions
WO2005038630A3 (en
Inventor
Thomas S. Plum
Original Assignee
Plum Thomas S
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from PCT/US2004/030029 external-priority patent/WO2005029241A2/en
Application filed by Plum Thomas S filed Critical Plum Thomas S
Publication of WO2005038630A2 publication Critical patent/WO2005038630A2/en
Publication of WO2005038630A3 publication Critical patent/WO2005038630A3/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/36Preventing errors by testing or debugging software
    • G06F11/362Software debugging
    • G06F11/3624Software debugging by performing operations on the source code, e.g. via a compiler
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00Arrangements for software engineering
    • G06F8/40Transformation of program code
    • G06F8/41Compilation
    • G06F8/43Checking; Contextual analysis
    • G06F8/436Semantic checking

Definitions

  • the technology herein relates to the design and construction of compilers which implement control flow analysis, data flow analysis, optimizations (including the type- based aliasing requirements), definite-assignment analysis, and the ANSI/ISO standards for the subject languages, and to software testing and verification.
  • the technology herein relates to safe secure software compiling techniques, and more particularly to a "blueprint" for methods, techniques and systems that can be used to create a particular exemplary implementation for use with the C and C++ programming languages which satisfies specified criteria for safety and security.
  • permissible undefined behavior ranges from ignoring the situation completely with unpredictable results, to behaving during translation or program execution in a documented manner characteristic of the environment (with or without the issuance of a diagnostic message), to terminating a translation or execution (with the issuance of a diagnostic message). Many erroneous program constructs do not engender undefined behavior; they are required to be diagnosed.]
  • Every data value can be fetched safely; in that sense, there are no "trap representations".
  • a "trap” can result if fetch or store of an invalid pointer is attempted, but not upon calculation or comparison of addresses. Therefore, uninitialized memory can be fetched safely.
  • An incompletely-read buffer after a read error (such as in Standard C subclauses 7.19.7.2, 7.19.7.7, 7.24.3.2, etc) still contains data bytes which will not cause traps upon fetch.
  • a request to the allocation functions malloc and calloc to allocate zero bytes can cause the allocation of the smallest non-zero allocation.
  • If the number-of-elements argument is zero, string and wide-string and sorting and searching functions can do-nothing gracefully.
  • the sorting and searching functions can be limimted to more than an implementation- defined maximum number of iterations.
  • the algorithms for converting between wide characters and (narrow) characters can produce deterministic results for all inputs, in either direction. Therefore, when a stream was written wide-oriented and read byte-oriented, the behavior can be implementation-defined and not undefined, and similarly for a stream written byte- ' oriented and read wide-oriented.
  • the wcstok function can be implemented so that, if it is invoked with a null pointer, then the pointer argument need not be equal to the pointer argument of the previous, but can require only that the "saved" pointer must designate some non-const array of characters, null-terminated. i.
  • the wcstok and strtok functions can be implemented so that, if the first invocation passes a null pointer, the function can ignore it and return a null pointer; alternatively, the function can invoke a safe termination such as ss_unwind (see below).
  • Text streams and character represe/ttati ns fcodej [0010]
  • An exemplary implementation can use a specific choice among the Unix/POSLX/Linux encoding of text files (with LF line terminators), the Macintosh encoding of text files (with CR line terminators), or the Microsoft Windows encoding of text files (with CR LF line terminators). All mbstate_t conversions can produce implementation-defined results, even after changing the LC_CTYPE category.
  • An implementation can make truncated-result behavior well-defined in strxfrm, strftime, wcsxfrm, or wcsftime.
  • the multibyte functions can behave gracefully when given a sequence not in the initial shift state, or when given any mbstate_t object.
  • the wide-character classifying and conversion functions can be well-defined for any wint_t input and for any LC_CTYPE setting.
  • the longjmp function (and any other functions which "unwind" the stack), can check whether execution of atexit-registered functions has started. If so, one of the following implementation-defined actions can be performed: cause a return from the function that invoked the unwind or longjmp function, invoke an "extreme exit” cleanup function; or invoke the abort function.
  • a system sanity check can be performed before continuing or re-starting.
  • the exit function can check whether execution of the exit function has started. If so, one of the following implementation-defined actions can be performed: cause a return from the function that invoked the longjmp function, invoke an "extreme exit” cleanup function; or invoke the abort function.
  • a rithmetic exceptions fexc [0020] If at compile-time the right operand of division or remainder is zero, a fatal diagnostic message can be produced. In Debug mode, if at run-time the right operand of division or remainder is zero, an "unwind" (such as ss_unwind) can be invoked, and the implementation may throw an exception of an implementation-defined type. In non-Debug mode, if at run-time the right operand of division or remainder is zero, the result can be the maximum value of the result type, which for a floating-point type may be an infinity.
  • the left operand of division or remainder is the maximum negative value of its type and the right operand is -1
  • a fatal diagnostic message can be produced.
  • Debug mode if at run-time the left operand of division or remainder is the maximum negative value of its type and the right operand is -1, an "unwind" (such as ss_unwind) can be invoked, and the implementation may throw an exception of an implementation-defined type.
  • non-Debug mode if at run-time the left operand of division or remainder is the maximum negative value of its type and the right operand is -1, the result can be the maximum value of the result type.
  • garbage collection either conservative (see, e.g., Hans-J Boehm, "A Garbage Collector for C and C++", . (http://www.hpl.hp.com/personal/Hans Boehm/gc/)) or accurate (see e.g., Fergus Henderson, "Accurate Garbage Collection in an Uncooperative Environment", ISMM'02, June 2021, 2002, Berlin, Germany, ACM 1581135394/02/0006), supplemented with the following special treatment of pointers to terminated stack frames. Directly assigning the address of the current function's stack frame to a longer-life pointer can be prohibited.
  • a pointer- retainer function as a function which stores a pointer argument in heap or static storage. Passing a pointer to stack to a pointer-retainer function can be prohibited. (Whatever data resides in the stack can be copied to heap or to static, to avoid the prohibition.)
  • the not_ptrs attribute can be removed from any storage by assigning zero to the bytes of the storage; a byte-oriented alias is mandatory (char, or unsigned char, or a library functions such as memset which modifies the bytes of memory).
  • SSC Safe Secure Compiler
  • SSICL Safe Secure Inputs Check-List
  • SLBDF Safe Secure Bounds Data File
  • SSL Safe Secure Linker
  • SSSA Safe Secure Semantic Analyzer
  • SSDG Safe Secure Diagnostic Generator
  • SSCG Safe Secure Code Generator
  • SSPAH Safe Secure Pointer Attribute Hierarchy
  • SSPAPT Safe Secure Pointer Attribute Predicate Table
  • SBDT Safe Secure Bounds Data Table
  • SSLIT Safe Secure Interface Inference Table
  • SSBDST Safe Secure Bounds Data Symbol Table
  • SSLTA Safe Secure Link-Time Analyzer
  • SSLDG Safe Secure Link Diagnostic Generator
  • Figure 1 shows an example safe secure software development system
  • Figure 2 shows a flow diagram of an exemplary illustrative non-limiting implementation of a program translation system
  • Figure 3 shows a block diagram of an exemplary illustrative non-limiting compiler implementation
  • Figure 4 shows exemplary illustrative annotation of pointer-null-state to a non- limiting illustrative flow-chart
  • Figure 5 shows a different exemplary illustrative representation of the same flow-chart;
  • Figure 6 shows a further exemplary illustrative flow-chart, including some
  • Figure 7 shows an exemplary illustrative non-limiting Safe Secure Pointer
  • Figure 8 shows an exemplary illustrative non-limiting Safe Secure Pointer
  • Figure 9 shows exemplary illustrative non-limiting components of a Safe
  • FIG. 10 shows an exemplary illustrative non-Umiting Safe Secure Interface
  • Figure 11 shows exemplary illustrative non-limiting components of the Safe
  • FIG. 1 shows an example illustrative non-limiting automated safe secure computer software development system 100.
  • an exemplary non-limiting system 100 may comprise a central processing unit (CPU) 11 coupled to a mass storage device 12 such as a magnetic or other disk.
  • the CPU 11 may execute program instructions including for example a safe secure translator 13 the executable code for which is stored on the mass storage device and executed by CPU 11.
  • the translator 13 in simplified terms, takes a source code input 14 and converts it to an executable output 15 that can be executed by the same or different computer.
  • the computer system 100 may further include input/output devices 16 such as for example keyboards, pointing devices, display devices and the like so that human software developers can interact with the computer.
  • the techniques described herein can be used to enhance a conventional compiler for the C and C++ programming language (as will be described below) to eliminate undefined behavior.
  • Lexical Analyzer 31, Preprocessor 32, and Parser 33 perform as commonly understood in the art. Then, Semantic Analyzer 34 verifies that the integer 0 is an acceptable value to return from the function "main" which is defined to return "int”. After that, the Code Generator 38 will produce an Object File 25, the ultimate output from Safe Secure Compiler 23. Finally, the Safe Secure Linker 28 will combine the Object File 25 with various system-dependent library code, producing Executable Program 29.
  • One well-known state in modern programming languages is the "definitely assigned" state. Definite-assignment analysis is a central feature of the Java and C# programming languages.
  • the present exemplary illustrative implementation of a Safe Secure method does not use the definitely-assigned state in quite the same way, but does make extensive use of state analysis.
  • One example of a state used in the Safe Secure method is the pointer-null-state, which has three alternative state values. (The pointer-null-state in this example is one dimension of a more complicated pointer-range-state, shown later.) a. Nul - pointer is null. b. Nnul - pointer is non-null. c. Qnul - pointer might, or might not, be null; i.e. "questionably-null".
  • Semantic Analyzer 34 determines that pointer p is initialized on its declaration and determines that the initial value is the null pointer value.
  • control flow analysis determines that there is no transfer of control in or out of the control-flow arc from the point of initialization to the point of usage (which attempts to fetch a value from the target of pointer p) and data-flow analysis determines that there are no alterations of the value of pointer p within that arc.
  • Control-flow analysis and data-flow analysis are commonly used to support optimization, but they also provide input data for the Diagnostic Generator 36.
  • the Semantic Analyzer 34 will commonly provide to the Diagnostic Generator 36 data indicating that upon all possible arcs the program would encounter the indirecting of a null pointer, and the Diagnostic Generator 36 will commonly produce a fatal diagnostic message and will commonly prevent the Code Generator 38 from producing any object code. (Each use of the term "fatal diagnostic" implies without exception that object code is never produced for any program which causes a fatal diagnostic.)
  • Figure 4 shows the flowchart which corresponds to the control flow in this example.
  • the two arcs re-join into the same flow- of -control (program-counter) state, but the pointer-null-state of p is different after the branch.
  • SSSA 35 verifies that the pointer-null-state of pointer p pre-determines the outcomes of the second "if statement, so there are only two actual flow-of-control paths in the flowchart of Figure 6.
  • SSSA 35 the flow chart of each function is factored into separate continuations of each arc at every place in the function, except that all arcs re-join at the return from the function (and the epilog implied by that return).
  • the compile-time attributes as used in the SSSA 35 are sometimes capable of modeling every detail of the run-time state analysis; in these cases the compile-time attributes are all that is needed in the analysis. In other cases, the compile-time attributes are capable of modeling many, but not all, details of the run-time state analysis; in these cases, the compile-time attributes permit compile-time diagnosis of many but not all errors. Therefore, these cases will require some bits in the run-time storage to record run-time state information.
  • SSPAH Safe Secure Pointer Attribute Hierarchy
  • Lo, Hi, End, Nth, and Ntl attributes can be applied either to a pointer or to a subscript (if the subscript's array context is clear).
  • Guarantees are not essential, and can be chosen to meet any other requirements.
  • the attribute name "Nullterm” could be abbreviated as “Nullt”
  • the attribute name “Indirectable” could be abbreviated as "Indir”, provided that those names are unambiguous in the context in which they are used.
  • I Indirectable
  • null pointer compares low to all other pointer values.
  • Stream objects can in one exemplary implementation contain an "is-open" run-time state which is reflected in a compile-time attribute (called e.g. "is_open”).
  • the representation of the is-open state can in one exemplary implementation be non-zero.
  • Safe pointers ⁇ ufj [0071] In the present method, all pointers are implemented as "compile-time-safe pointers", subject to a maximal verification at compile time.
  • the target-size is defined as the number of bytes in the object or array designated by the pointer.
  • the Bound is defined as the Tsize divided by the size of the type T to which the pointer is pointing.
  • the Tsize-adjusted is the Tsize minus the size of the type T, i.e. the number of bytes in the array between the address of the first object of type T and the last object of type T. If there is only one element in the array, the Tsize-adjusted is zero.
  • the SSSA 35 maintains this bounds information for every pointer used to fetch-or-store; therefore, the analysis presented in this Safe Secure method will incorporate the syntax and semantics of the restrict keyword (from C99). Whenever this keyword appears, it reflects in syntax a restriction that otherwise must be stated in (non-formalized) semantics, i.e. a restriction that the target object designated by the restrict-qualified pointer does not overlap any other object that is accessible through external pointers or other argument pointers.
  • This restriction can in one exemplary implementation be verified at compile-time to the extent that the bounds information is available. The remaining checks can in one exemplary implementation be verified at run-time by examination of the bounds data of the fore-mentioned set of pointers.
  • the restrict keyword is not part of the (1998 or 2002) C++ standard, but the no-overlap restriction is part of the (non-formalized) semantics of the C library contained in that C++ standard, and therefore is part of the requirements for the elimination of undefined behavior in C++ as well as C.
  • a pointer value that designates an object in an array has a compile-time relationship with the bounding addresses .of the array.
  • the target pointer object receives all the attributes pertaining to the original pointer. All pointer arithmetic (ptr+n, ptr-n, ptr-ptr, ptr[n]) checks the result (at compile- time, link-time, or run-time) against the Lo-Hi bounds; the End ("one-too-far") value is permissible as a result from pointer arithmetic.
  • a compile-time-safe pointer uses the same (one-word) representation as the ordinary C or C++ implementation. In all contexts that require bounds information, that information is made known to the SSSA 35 through a variety of mechanisms. In compile- time-safe code, the SSSA 35 ensures that each non-scalar pointer must have, within the same region (global static, block static, block local, parameters, or heap) bounds information (or bounds information stored in the global static region), stored in locations made known to the Safe Secure Compiler 23, sufficient to verify the safety of any fetches or stores expressed in the program.
  • the SSSA 35 stores Guarantees and Requirements for pointer bounds in the
  • Safe Secure Bounds-Data Table (“SSBDT”) 61.
  • SSBDF Safe Secure Bounds-Data File
  • all bounds data entries which provide information about the Requirements and Guarantees of references to, and definitions of, external arrays, external pointers, and the parameters and return type of externally-callable functions, i.e. function prototype entries.
  • All function prototype entries in the SSBDT 61 are written to the SSBDF 26 contained in a persistent storage ("file storage").
  • the format and representation of the SSBDT 61 can be optimized to suit the requirements of each SSSA 35.
  • the Guarantees and Requirements are attached as needed to expressions, sub-expressions, declarations, identifiers, and function prototypes, using whatever indexing or coordinates as are needed to identify the relevant location in the internal representation of the Parser 33.
  • all entries in the SSBDT 61 will be represented in the narrative text or in the style of macro invocations inserted into the Source Program 107. In the present method, this representation does not presuppose that any such invocations of actual macros are present in the actual Source Program 107.
  • the Safe Secure method requires that one or more of the following are provided:
  • the Parser 33 recognizes some notation identical to or equivalent to the macro notation shown herein; or,
  • the SSSA 35 provides a method whereby the human programmer can add annotations to the SSBDT 61 and SSBDF 26. If the Compiler 23 is a component in an Interactive Design Environment (IDE), then the mechanism for adding annotations to the SSBDT 61 and SSBDF 26 can be integrated into the IDE.
  • IDE Interactive Design Environment
  • the initial value of each pointer in each scope is initially the Lo bound.
  • the initial Lo bound plus the initial Bound (or Length or Tsize) value determines the Hi bound.
  • the default for a pointer without any bounds annotation is Not-Indirectable.
  • the SSSA 35 can infer from the source code of a function that a parameter is being used as the Hi, or Bound, or Length, or Tsize, of another parameter.
  • the itemized set of rules for the inference is detailed in Figure 10.
  • the SSBDT 61 also uses a special notation for NULLTERM, for char arrays which are guaranteed to be null-terminated: int main(int argc, NULLTERM char* argv[] BOUND_IS(argc) ); [0090] In the SSBDT 61, void* generic pointers are annotated just the same as char* pointers.
  • one parameter such as n in this example
  • a pointer such as si in this example
  • that pointer is understood to be Indirectable.
  • one parameter provides bounds data but a pointer is explicitly Maybe-Indirectable, then either the pointer is null or it is indirectable and has bounds data.
  • Tsize is the Tsize of one element
  • Bound is the number of elements.
  • standard library function calloc is represented in the SSBDT 61 as follows:
  • Tsize is understood as the largest object that will be accessed through this pointer.
  • strncpy is represented in the SSBDT 61 as follows:
  • the attributes for "indirectable” and “null- terminated” represent Requirements-and-Guarantees for arguments and Guarantees for returned value. These attributes can also be used for Guarantees (post-conditions) upon an argument when there is no corresponding Requirement (pre-condition) upon entry to the function.
  • An additional marker (“POST") is used to indicate post-conditions (Guarantees) in the notation of the SSBDT 61:
  • attribute information might be redundantly attached to several different objects. If one of these objects is subsequently modified, another object may retain the attribute as before.
  • attribute information might be redundantly attached to several different objects. If one of these objects is subsequently modified, another object may retain the attribute as before.
  • another object may retain the attribute as before.
  • the SSSA 34 embodies a rule that can be summarized informally as "half a loaf is better than none": if there is only one copy of the LO, HI, END, BOUND, TSIZE, TSIZEA, or LENGTH, and if that copy is incremented or decremented, then that copy retains the appropriate attribute.
  • a rule that can be summarized informally as "half a loaf is better than none": if there is only one copy of the LO, HI, END, BOUND, TSIZE, TSIZEA, or LENGTH, and if that copy is incremented or decremented, then that copy retains the appropriate attribute.
  • half a loaf is better than none if there is only one copy of the LO, HI, END, BOUND, TSIZE, TSIZEA, or LENGTH, and if that copy is incremented or decremented, then that copy retains the appropriate attribute.
  • n is the Tsize of s (a Requirement upon entry to fgets, and a Guarantee upon return from fgets), and stream is Indirectable (a Requirement upon entry to fgets, and a Guarantee upon return from fgets).
  • the bounds information for a pointer is made available to the Safe Secure Compiler 23 through a variety of mechanisms depending upon the context: explicit attribute macros in a calhng sequence (or equivalent programmer notations through IDE or other means), attributes inferred from usage in the called function, a visible allocation expression such as malloc or new, information made known to the linker by the SSBDF 26, or from direct visibility of an array declaration. If the programmer has provided explicit notations of bounds information, the Safe Secure Compiler 23 checks them against the inferred bounds; otherwise, the inferred bounds will be used during the compilation. It is likely that professional programmers providing library code for consumption by others will be urged to provide explicit notations.
  • the SSCG 39 Even if the argument to malloc has been modified, when the SSSA 35 determines that the ptr is unmodified from the original returned value, the SSCG 39 generates code to implement all the bounds macros (LO_OF, HJLOF, END_OF, BOUNDJDF, TSIZE_OF, TSIZEA_OF) by accessing the heap- allocation data of the pointer ptr.
  • the function passes ptr as an argument to an external function, the function must pass the bounds information through an argument.
  • the SSBDF 26 does not provide notation to distinguish dynamically-allocated pointers from ordinary pointers.
  • Analysis of pointer-range-state starts by identifying all fetch-or-store expressions that involve pointers or subscripts, and all objects and variables (i.e. named objects) that are part of those expressions. ⁇ Next, identify all expressions and statements that involve those objects and variables. Then identify certain control-flow patterns which guarantee the "counted-loop" ("counted-plus” and "counted-minus") semantics.
  • m Since m is defined in this translation unit, its bounds are known to the SSSA 35; specifically, its Bound is 2, its Lo is the address of m[0], its Hi is the address of m[l], and its End is the address of m[2].
  • the Tsize of m is 2 times sizeof(int), and its Tsize-adjusted is sizeof(int).
  • the variables involved in this expression are m and i. As the Practitioner knows, the semantics of C and C++ requires that the subscript expression m[i] must be within the range of Lo to Hi; we designate this as the Requirement of the expression.
  • a scalar pointer is one which, in its scope and Hfetime, is never incremented/decremented, never participates in address arithmetic, and is never subscripted (except using p[0]). It does not require bounds information.
  • loop limited by null terminator the loop increments an indirectable pointer until its target is null
  • the SSCG 39 provides semantics for several attributes of pointers and/or subscripts.
  • the "_Sat” (“saturated") semantics generates machine-dependent instructions or software code sequences such that the pointer or subscript never increments (or adds) higher than its Hi value, and never decrements (or subtracts) lower than its Lo value. (In other words, it is like a guage that "pegs" at its minimum and maximum values.)
  • the "_Modwrap” semantics generates machine-dependent instructions or software code sequences such that pointer and/or subscript is wrapped modulo the End value. Therefore, as the pointer or subscript is successively incremented, it runs from the Lo value to the Hi value, then starts again at the Lo value.
  • the "_Zerobound" semantics generates machine-dependent instructions or software code sequences such that any fetch from outside the Lo-Hi limits (including the null-pointer value) produces a zero value (or a default-initialized value for a non-scalar), and any store outside the Lo-Hi limits (including the null-pointer value) produces a no-op. All three of these semantics can be implemented in silicon.
  • the SSCG 39 inserts executable bounds-check code.
  • the Safe Secure method there are two modes of bounds-check code.
  • the SSCG 39 implements a choice among the following options: (a) use Debug semantics; or (b) every attempt to fetch or store out of bounds produces a well defined result (either _Sat semantics, or _Modwrap semantics, or _Zerobound semantics) whichever produces the best fit to the current optimization profile (which might be "minimum code size” or "minimum CPU time” or other). A warning should be generated to inform the programmer.
  • every attempt to create a pointer value out of the full range of values produces an algebraically well-defined pointer value, even though that value would not be usable for indirection or bounds-testing.
  • Debug mode is conditional upon a switch set either at compile-time, at Unk-time (using an external flag), at load time (using an environment variable), or during run-time (using an interactive debugger)
  • the SSDG 37 In both Debug and non-Debug mode, the SSDG 37 generates a warning message to inform the programmer that run-time bounds-checking has been inserted by the Safe Secure Compiler 23.
  • the SSSA 35 provides a mechanism by which the programmer can request that, if the programmer believes that the control flow does in fact guarantee adequate bounds checks, the original control flow is communicated to the organization which distributes the SSSA .
  • said organization will have the opportunity to increase the set of counted-bounds control-flow contexts and Requirement-Guarantee patterns.
  • said client feedback is subject to the same third-party processes, since the assurances of safety and security provided by the Safe Secure method are dependent upon the correctness of the patterns that are accepted by the Safe Secure method.
  • the full set of bounds-enforcement mechanisms may produce a number of inserted bounds-check code segments.
  • the Safe Secure method provides a further tool by which these bounds-checks (and the associated warnings) can be eliminated.
  • an environmental interaction as a function invocation which produces a returned value or other state change which depends upon the external environment (such as command-line argument, input function, or getenv invocation).
  • the Safe Secure method ensures that for every environmental interaction there is a guarantee that returned values and modified argument strings are either bounded in size and value, or null-terminated. If unbounded but null-terminated, the Safe Secure method requires that some bounding construct be imposed before use of the input value.
  • the environmental interactions can be i simulated, using the minimum and maximum values for the input, and/or using the complete range of values for the input, and the pure code which follows those interactions can be executed in the environment of the full range of values for the input.
  • the set of input values must include any values which affect the control flow within the pure-code segment, and should also include the values one-less-than and one-greater-than these special values.
  • the SSSA 35 must interact with the optimization analysis performed in Semantic Analyzer 34.
  • the analysis necessary to keep a pointer or an integer in a register (“aliasing logic") may be required to determine that that pointer or that integer retains its bounds-related attributes during specific arcs of the flow graph.
  • the methods described above for enforcing bounds safety have relied upon insertion by the SSCG 39 of bounds-check code segments when state analysis is unable to determine the validity of fetch-or-store accesses.
  • the SSCG 39 inserts invocations of macros or inline functions in an intermediate representation of the original program.
  • macros-or-functions can be used:
  • a string or wide string utility function is instructed to access an array buf C7.24.4 beyond the end of an object
  • Each operation upon pointers, arrays, and/or subscripts may impose one or more Requirements upon the preceding operations, declarations, and/or initializations.
  • Specific rules for inferring Requirements are shown in the Safe Secure Interface Inference Table ("SSITT") 71.
  • SSLT Safe Secure Interface Inference Table
  • Each operation, declaration, and/or initialization may produce one or more
  • Guarantees for subsequent operations Each Guarantee persists across subsequent operations and statements until some subsequent event terminates the Guarantee; such events include the end of the lifetime of any objects in the Guarantee.
  • the steps itemized above can be performed in any order during the semantic analysis. Each time that these steps are applied to the entire translation unit the SSSA 35 must determine whether progress was made, i.e. whether any new Guarantee was determined, or whether any Requirement (not previously satisfied on previous iterations) is now satisfied. If an iteration covers the entire translation unit without making progress, then the SSSA 35 makes a list of all unsatisfied Requirements. For each such Requirement, if it is capable of link-time resolution, then entries are created in the SSBDF 26 so that it can be resolved or diagnosed at link-time.
  • the SSCG 39 generates one of the appropriate code sequences as described above. If none of these link-time or run-time resolutions are available then the SSDG 37 issues a diagnostic. If the code whose Requirement cannot be satisfied occurs in a code region which can never be executed (i.e. "dead code") or is not executed in this application (i.e. "sleeper code”), then the implementation is permitted to treat the diagnostic as a warning. In any other context, the "unsatisfied Requirement" diagnostic must prevent production of object code.
  • the SSSA 35 makes notations in the SSBDF 26 to indicate that the code cannot be safely invoked, and the SSCG 39 generates a stub which satisfies the linkage requirements and produces safe behavior in Debug or non- Debug mode.
  • the Requirement-Guarantee patterns are itemized in the lists below.
  • the first list itemizes the patterns that were defined in the Preliminary Application.
  • the Guarantee may be provided by semantics of that function (such as "TSIZE_OF(ptr) is n") as specified in its entry in the SSBDF 26.
  • a run-time Guarantee can be provided that uses bounds data from the argument(s) to the heap allocation.
  • the bounds Guarantee is immediately available to the SSSA 35 from the defining declaration.
  • the SSSA 35 has identified a Requirement that the Tsize of pointer p is at least N (in other words, the bytes from *p to *(p+N-l) are all within one object), and if p loops from a upwards to b, then the Requirement is satisfied for all p iff it is satisfied for the maximum value, b.
  • Requirements and Guarantees can proceed in any order. For a non-limiting example, start with those Requirements and Guarantees that require no flow analysis.
  • the static declarations indicate that the Bound of a is N, and that p is initialized to the same bounds as a.
  • the expression a+N designates the End of a. On the false ("else") arc, lim has Nth or End attribute, as determined by the SSPAPT 51, and ret has whatever attributes p had on entry to my_alloc. Then flow analysis determines that p is never decremented, that no external code can modify it (because of the file-scope static), and that it is incremented only by an unsigned value which is guaranteed not to "wrap-around" the address space.
  • the SSSA 35 needs only to determine the attributes of the returned value. On one arc the returned value is null. On the "else" arc the returned value is at least Indirectable, because p is always strictly less than a value (lim) which is Nth or End. Then the SSSA 35 determines the bounds of the returned value.
  • the returned value is a pointer into array a, so the End of p is the End of a, but the attributes of array a are not visible outside the source file of my_alloc and therefore cannot be used to define the bounds.
  • the only externally-visible object involved in my_alloc is the parameter n.
  • the SSSA 35 can guarantee that the Tsize of the returned value is at least n, because p+n is at most equal to the End of a.
  • the Tsize cannot be greater than n, because p+n can be equal to the End of a. Therefore the SSSA 35 enters into the SSBDT 61 an entry for my_alloc as follows:
  • the SSSA 35 determines that the Requirement upon p is at least an Indirectable requirement. Because *p is never modified (consistent with the "const" qualifier), and no de-allocation is performed upon p, the Indirectable Requirement is also a Guarantee. The pattern of the loop determines the stronger Nullterm Requirement, which is also a Guarantee. The relationship between the parameter and the returned value is also dictated by the pattern of the loop. The inference of Requirements and Guarantees produces the following entry into the SSBDT 61: strlen(str NULLTERM LENGTH_IS(return) )
  • the Safe Secure Linker 28 reads Object File(s) 25, creating a Symbol Table 81 of symbols provided and symbols needed.
  • the Link-Time Symbol Resolver 83 determines the addresses of the symbols which will be loaded in the executable image. If any symbols are needed but not provided (or if typesafe linkage fails in C++), then the Link Diagnostic Generator 85 produces a message, as is customary in the common art.
  • the Safe Secure Linker 28 provides further methods beyond the common art.
  • Safe Secure Compiler 23 produced a SSBDF 26 containing Requirements and Guarantees.
  • the Safe Secure Linker 28 reads the SSBDF(s) 26 which accompany the Object File(s) 25 being linked.
  • Each "undefined symbol” entry in the Symbol Table 81 is augmented in the Safe Secure Bounds Data Symbol Table ("SSBDST”) 82 with the Requirements and Guarantees provided by the various code contexts that require the symbol.
  • Each "defined symbol” entry in the Symbol Table 81 is augmented augmented in the Safe Secure Bounds Data Symbol Table (“SSBDST”) 82 with the Requirements and Guarantees provided by the defining instance of that symbol.
  • the Safe Secure Link-Time Analyzer (“SSLTA”) 84 performs these tests:
  • each Requirement of the called function is satisfied by a Guarantee in the calling function.
  • each Requirement in one object file is satisfied by a Guarantee in the other object file.
  • SSLDG Link Diagnostic Generator
  • SPEC benchmarks We chose to test the Safe Secure method with a hand-simulation on one complete application.
  • the smallest is "256.bzip2", which contains about 4600 lines of code in two C source files (spec.c and bzip2.c).
  • the smaller file (spec.c) contains about 36 fetch-or-store instances and the larger file (bzip2.c) contains about 250 fetch-or-store instances.
  • the example below shows the first 8 instances of fetch-and-store operations in spec.c.
  • TSIZE_OF(spec_fd[fd].buf+i) is RANDOM_CHUNK_SIZE
  • Requirement 7b TSIZE_OF(random ext[(int)(ran()*RANDOM_CFiUNKS)]) is RANDOM_CHUNK_SIZE
  • the Tsize of the first argument is the number of bytes from spec_fd[fd].buf+i to the last byte of spec_fd[fd].buf. This number of bytes reaches its minimum (its tightest constraint) when spec_fd[bufj+i reaches its maximum value.
  • the maximum value of spec_fd[fd].buf+i is reached when i reaches its maximum value, namely spec_fd[fd]. limit.
  • the relationship between spec Jfd[fd], buf and spec_fd[fd]. limit is established in the function specjnit: TSIZE_OF(spec_fd[i].buf) is spec_fd[i].limit+FUDGE_BUF [00153] Therefore Requirement 7a is equivalent to
  • Requirement 7a cannot be satisfied without changes to the source code.
  • FUDGEJBUF could be increased to the same value as RANDOM_CHUNK_SIZE.
  • This unsatisfiable Requirement occurs inside "sleeper code", i.e. code which is not executed in this application.
  • a compiler which is being tested by SPEC would in all likelihood not give fatal diagnostics for dead code and sleeper code, and would compile and execute the application despite the unsatisfiable Requirement 7a in the sleeper-code function spec_random Joad.
  • the resulting executable program is still certifiably free from buffer overflows.
  • Requirement 8 is that (int)(ran()*RANDOM_CHUNKS) must be usable as a subscript for randornjext; i.e.
  • Bitwise manipulation of pointers fbitpj [00160] If the low-order bits of a pointer are masked OFF or ON by a compile-time constant mask, and if that mask specifies a value smaller than the alignment requirement of the pointed-to type, then the pointer-bound-state of the pointer is not altered thereby. In all other cases, the pointer-bound-state of the pointer is indeterminate (Unk) after manipulation. Reading a pointer from external store (via %p, fread, or other method) produces the indeterminate pointer-bound-state
  • the range of unsigned integer values can in one exemplary implementation be sufficient to represent the difference of any two pointers, as well as sufficient to represent unambiguously the value of any pointer.
  • a compiler 23 configured for the Safe Secure method can in one exemplary illustrative non-limiting implementation produce special assistance for invocation of functions declared with ellipsis ("variable-argument functions"). At each invocation, the Safe Secure Compiler 23 is aware of the promoted types of the actual arguments, and can in one exemplary implementation produce a null-terminated string containing the name- mangled representation of the promoted arguments.
  • the Safe Secure Compiler 23 can in one exemplary implementation provide special handling of the va_arg macro as defined in header ⁇ stdarg.h> as follows: If the type argument is a scalar type which produces a one-byte encoding in the mangled name string (e.g. double, which produces the single character 'd' in typical name-mangling, then an invocation such as
  • the enhanced _va_arg3 macro can in one exemplary implementation test that the next byte in the argument mangled-name string is the character 'd', incrementing the pointer after the test. If the test is false, an associated "match" function is invoked, such as
  • Type-secure linkage fts/J [00166]
  • C functions are type-checked using an implementation mechanism similar to that used for C++. This does not introduce overloading in C; the underlying function-name identifier must be unique for each C function.
  • the rules for matching the type of C functions also permit linkage of functions whose signature differs from the expected type by the rules of "same representation and alignment" (see [R13] 6.2.5).
  • the representation of the long double type is actually the same representation as the double type, i.e., a 64-bit floating point representation.
  • the test for "same representation and alignment" extends to pointers, both for the pointer value itself and for the type pointed-to.
  • Pointer-to-character (signed, unsigned, or unspecified) is compatible with pointer-to-void.
  • the reason for using the relaxed rules of "same representation and alignment” rather than the strict “same type” rules is to avoid negative responses from today's C and C++ programmers, who are currently accustomed to "practical” rather than “pedantic” standards for proper behavior of variable-argument functions. Portability is often important, but portabihty cannot over-ride the requirements of safety and security. (An alternate name for "type-secure linkage" is "type-compatible linkage”.)
  • Mode strings for fop en /fop en J [00169] If at compile-time the string pointed to by the mode argument in a call to the fopen function does not exactly match one of the expected character sequences a fatal diagnostic message can be produced. If the error occurs at run-time in Debug mode, ssjunwind can be invoked; the implementation may throw an exception of an implementation-defined type. If the error occurs at run-time in non-Debug mode, the implementation can ignore all unrecognized portions of the mode argument.
  • the Safe Secure Compiler 23 produces the Safe Secure. Inputs Check-List
  • SSICL file-names (or equivalent designation) of all the source components which were read by the compiler during the compilation of one or more outputs.
  • the SSICL 24 also contains one or more checksums for the contents of the corresponding component.
  • check-list is used here to designate a list of names and checksums.
  • At least one of the checksums embodies the following method of forming the checksum:
  • Text files represented in ASCII, ISO 646, ISO 8859-1 ("Latin-1"), or the UTF-8 encoding of Unicode or ISO 10646 are checksummed using the values of that encoding. Otherwise, each text file is converted to the UTF- 8 form of ISO 10646 before checksumming.
  • the SSBDF 26 contains a similar check-list listing each output file produced by the compiler (object file, executable file, or dynamic link-library), together with one or more checksums. At least one of the checksums embodies the method given above, using the actual 8-bit value of each byte as an unsigned character value (from 0 to 255).
  • This check-list includes one entry designating the SSICL 24, including the one-or-more checksums formed in the same way as specified above for the entries contained in the SSICL 24.
  • Compilation Signature formed as follows. At the N m occurrence of a compiler operation specified by the Safe Secure method (e.g., the inference of an attribute, or the satisfying of a Requirement by a Guarantee), the SSC 23 forms a checkum of the operand(s) of that operation. This checksum is stored as the first entry on the Compilation Signature list. Repeat this process at the 2*N ⁇ operation, the 4*N ⁇ operation, the 8*N ⁇ operation, etc. The resulting list of checksums in the Compilation Signature is of a size roughly proportional to the base-2 logarithm of the program size.
  • This Compilation Signature provides a record of the compilation of a specific program by a specific Safe Secure Compiler 23.
  • boolean (non-zero is success): setjmp, the character classification functions (7.1.4), feof, ferror, mblen(with null s), mbtowc(with null s), wctomb(with null s), mbsinit, wide character classification functions (7.25.2.1) b.
  • null is failure: setlocale, tmpfile, tmpnam, fopen, freopen, fgets, gets, calloc, malloc, realloc, getenv, bsearch, memchr, strchr, strpbrk, strrchr, strstr, strtok, gmtime, localtime, fgetws, wcschr, wcspbrk, wcsrchr, wcsstr, wcstok, wmemchr c.
  • SIG_ERR is failure: signal d.
  • EOF failure: fflush, fgetc, fputc, fputs, getc, getchar, putc, putchar, puts, ungetc, fputws, wctob f.
  • negative failure: fprintf, printf, sprintf, vfprintf, vprintf, vsprintf, fwprintf, swprintfvfwprintf, vswprintf, vwprintf, wprintf g.
  • fscanf fscanf, scanf, sscanf, vfscanf, vscanf, vsscanf, fread, fwrite, fwscanf, swscanf, vfwscanf, vswscanf, vwscanf, wscanf h. nonnegative and less than n: snprintf, vsnprintf i.
  • negative one is failure: ftell, mblen(with non-null s), mbtowc(with non-null s), wctomb(with non-null s), mbstowcs, wcstombs, clock, mktime, time, wcrtomb, mbsrtowcs, wcsrtombs j. bigh-low-equal: memcmp, strcmp, strcoll, strncmp, fwide, wcscmp, wcscoll, wcsncmp, wmemcmp k. greater than n is failure: strxfrm, wcsxfrm 1.
  • the comparison value is zero.
  • the comparison value is SIGJERR-
  • the comparison value is EOF.
  • the comparison value is a value unambiguously determined by the list of arguments.
  • the comparison value is WEOF.
  • the comparison value can be -2, or -1, or a value unambiguously determined by the list of arguments.
  • the C standard defines a diagnostic message as follows: "3.10 diagnostic message: message belonging to an implementation-defined subset of the implementation's message output".
  • the big-safe-pointer mechanism is achieved by transforming the C or C++ source program into a C++ program in which each big-safe pointer is implemented as a template struct: template ⁇ class T> struct bsafe_ptr ⁇ T* ptr; T* lo; T* hi; ⁇ ;
  • every big-safe pointer will include low and high bounds information.
  • the high-bound information is represented as a Tsize-adjusted field.
  • the Bound is defined as the Tsize divided by the size of the type T to which the pointer is pointing.
  • the Tsize-adjusted is the Tsize minus the size of the type T, i.e. the number of bytes in the array between the address of the first object of type T and the last object of type T. If there is only one element in the array, the Tsize-adjusted is zero.
  • END_OF(p) is a bsafe pointer with the same Lo and Hi values as p and containing the "one-too-far" address as its current ptr value.
  • LO OF(p) is a bsafe pointer with the same Lo and Hi values as p and containing p.lo as its current ptr value.
  • the BOUND_OF(p) is an integer designating the number of elements in the array that extends from LO_OF(p) to HI_OF(p).
  • the TSIZE _OF(p) ("target size of p") is the number of bytes in the storage accessed through p.
  • Lo-Hi information or Lo-Hi bounds, as the information embodied in the lo and hi fields.
  • a pointer value that designates an object in an array includes the bounding addresses of the array. Assigning a pointer value to a pointer object preserves the lo. hi, and ptr members. Creating a pointer via the big-safe version of malloc(N) produces a bsafe_ptr containing the ptr address of the new storage, a lo equal to ptr, and a hi of N/ sizeof(T). Casting a bsafe_ptr, for example from char* to T*, converts ptr and lo to T* values; if ptr or lo are not properly aligned, an ss_unwind is raised.
  • Pointer arithmetic ptr+n, ptr-n, ptr-ptr, ptr[n]
  • fetch-and-store indirection operators (explicitly via "star” or implicitly in the lvalue use of subscript) are checked and diagnosed according to the rules as described in section "Safe pointers [buf]", with the obvious inclusion of run-time checking against the values store in the big-safe-pointer.
  • the first member of a bsafe_ptr is the pointer value ptr. This is intentional and required; any non-BSAFE code which refers to a bsafe_ptr as an ordinary pointer will properly address the pointer value.
  • the Safe Secure method has defined four modes of code generation: Debug BSAFE, Debug CSAFE, non-Debug BSAFE, and non-Debug CSAFE.
  • a further alternative is to reduce the modes to two: Debug BSAFE and non-Debug CSAFE.
  • One non-limiting method for implementing the SSBDF 26 makes use of the name-mangling facilities of C++, employing appropriate encodings in the object-file representation to convey the bounds information for each fetch-and-store variable in each function's calling sequence.
  • the parameter type can be encoded into the apparent classname of the pointer argument:
  • the object code for memcpy would embody encodings such as these: csafe_ptr_Tal ⁇ void> memcpy( csafe_ptr_Tal ⁇ void> si, csafe_ptr_Tal ⁇ void> s2, size_t n);
  • the augmented signature of the function f is as follows: void f( csafe_n_Iela ⁇ int>) [00208] This presents a Requirement for the invocation of f in File 2, namely, that the argument must be suitable for use as a subscript on array a, and informs the main program that the variable j is being used for fetch-or-store access to a. The main program must determine the range of values for j and verify that j is suitable as a subscript on array a.
  • the returned value is indicated as "argument number zero", with digit zero.
  • any pointer argument can have the "not-indirectable" attribute.
  • This attribute is the default attribute for pointer parameters. This attribute is explicitly documented via the template type csafe_ptr_NI ⁇ T> ("csafe pointer, not- indirectable"), or more simply as an ordinary T* pointer.

Abstract

Automated (e.g., compiler implemented) techniques provide safe secure software development addressing undefined behaviors in C and C++ programming languages used source input files. A secure compiler (23) automatically identifies and/or eliminates all undefined behaviors of the input files (21) by maintaining a bounds data file as records (26) of requirements for the behavior of the programm constructs at link time. The compilor also uses a parse tree and cheksum information in order to determine the above bounds requirements and guarantees the appropriate behavior for the symbols of the object file (25) to be produced. Such link time (28) determination includes generating of fatal diagnostic messages (22) in situations that undefined behaviors would result.

Description

TITLE
AUTOMATED SAFE SECURE TECHNIQUES FOR ELIMINATING UNDEFINED BEHAVIOR IN COMPUTER SOFTWARE
CROSS-REFERENCES TO RELATED APPLICATIONS
[0001] This application claims the benefit of Provisional Application No. 60/502,620, filed September 15, 2003 and Provisional Application No. 60/511,649, filed October 17, 2003. The entire contents of these provisional applications are incorporated by reference
FIELD
[0002] The technology herein relates to the design and construction of compilers which implement control flow analysis, data flow analysis, optimizations (including the type- based aliasing requirements), definite-assignment analysis, and the ANSI/ISO standards for the subject languages, and to software testing and verification. In more detail, the technology herein relates to safe secure software compiling techniques, and more particularly to a "blueprint" for methods, techniques and systems that can be used to create a particular exemplary implementation for use with the C and C++ programming languages which satisfies specified criteria for safety and security. i BACKGROUND AND SUMMARY
[0003] Most computer users have experienced times when their computer seemingly has "lost its mind" and starts behaving in seemingly unexplainable ways. For example, sometimes we command the computer to do something — but instead of doing what we ask, the computer "stops responding" and needs to be "rebooted" (e.g., tamed off and back on again). This process can waste significant time while the computer restarts. Work product is sometimes lost - frustrating users to no end.
[0004] Ultimately, most such problems are caused by programming errors (sometimes called "bugs"). As computer programs become increasingly complex, it is more difficult for the people writing the computer code to take into account every possible condition that the computer program may encounter. Unfortunately, a computer program will "break" if the code encounters an undefined condition it does not "know" how to handle. This can cause serious problems. Consider for example if the software controls an airplane autopilot, a missile guidance system or a hospital life support system.
[0005] Another range of problems relates to attackers taking advantage of undefined computer program behavior to do harm. Several of the undefined behaviors of C and C++ have received much attention in the popular press as well as technical journals, because their effects have inflicted billions of dollars of damage in the USA and worldwide. In particular, the "buffer overflow" (also known as "buffer overrun") and "null pointer indirection" behaviors have created vulnerabilities in widely-used software from many different vendors. This problem of buffer overflows is no longer an obscure technical topic. This is the vulnerability through which most worms and viruses attack. The worldwide total costs due to malicious hacker attacks during 2002 have been estimated to be between 40 and 50 billion USD; costs for 2003 were estimated between 120 and 150 billion USD. See e.g., David Berlind. "Ex-cybersecurity czar Clarke issues gloomy report card" (ZDNet TechUpdate October 22, 2003).
[0006] An international standard has been developed for the programming language
C, which is designated ISO/EEC 9899:2002(E). Similarly, an international standard has been developed for the programming language C++, which is designated ISO/IEC 14882:2003(E). Each of these standards defines certain situations using the category of "undefined behavior". The C Standard contains the following definition: "3.4.3 undefined behavior: behavior, upon use of a nonportable or erroneous program construct or of erroneous data, for which this International Standard imposes no requirements. NOTE Possible undefined behavior ranges from ignoring the situation completely with unpredictable results, to behaving during translation or program execution in a documented manner characteristic of the environment (with or without the issuance of a diagnostic message), to terminating a translation or execution (with the issuance of a diagnostic message)." The C++ Standard contains a similar definition: "1.3.12 undefined behavior: behavior, such as might arise upon use of an erroneous program construct or erroneous data, for which this International Standard imposes no requirements. Undefined behavior may also be expected when this International Standard omits the description of any explicit definition of behavior. [Note: permissible undefined behavior ranges from ignoring the situation completely with unpredictable results, to behaving during translation or program execution in a documented manner characteristic of the environment (with or without the issuance of a diagnostic message), to terminating a translation or execution (with the issuance of a diagnostic message). Many erroneous program constructs do not engender undefined behavior; they are required to be diagnosed.]"
[0007] Some undefined behaviors can be eliminated by using techniques already known in the current art. The next sections will describe some exemplary such techniques.
Design-time choices fdtj
[0008] Several undefined behaviors can be addressed by design choices; these undefined behaviors are marked with "dt" in column one of the table below. In general, the guiding principle behind these design choices is that non-portable behavior is generally not as bad as undefined (unsafe) behavior. For example; byte-ordering affects the numeric value of results, but so long as address bounds are not exceeded, byte-ordered integer values produce something well-defined on each hardware platform. a. The representation of a null pointer can be all-bits-zero. b. The representation of pointers can be binary two's-complement with non-signaling wraparound. c. Every possible binary value can be interpreted as a valid data element. Every data value can be fetched safely; in that sense, there are no "trap representations". A "trap" can result if fetch or store of an invalid pointer is attempted, but not upon calculation or comparison of addresses. Therefore, uninitialized memory can be fetched safely. An incompletely-read buffer after a read error (such as in Standard C subclauses 7.19.7.2, 7.19.7.7, 7.24.3.2, etc) still contains data bytes which will not cause traps upon fetch. d. A request to the allocation functions malloc and calloc to allocate zero bytes can cause the allocation of the smallest non-zero allocation. e. If the number-of-elements argument is zero, string and wide-string and sorting and searching functions can do-nothing gracefully. f. The sorting and searching functions can be limimted to more than an implementation- defined maximum number of iterations. g. The algorithms for converting between wide characters and (narrow) characters can produce deterministic results for all inputs, in either direction. Therefore, when a stream was written wide-oriented and read byte-oriented, the behavior can be implementation-defined and not undefined, and similarly for a stream written byte- ' oriented and read wide-oriented. h. The wcstok function can be implemented so that, if it is invoked with a null pointer, then the pointer argument need not be equal to the pointer argument of the previous, but can require only that the "saved" pointer must designate some non-const array of characters, null-terminated. i. The wcstok and strtok functions can be implemented so that, if the first invocation passes a null pointer, the function can ignore it and return a null pointer; alternatively, the function can invoke a safe termination such as ss_unwind (see below).
[0009] The methods shown in this section can be used to eliminate the following undefined behaviors:
Figure imgf000006_0001
Figure imgf000007_0001
Figure imgf000008_0001
Text streams and character represe/ttati ns fcodej [0010] An exemplary implementation can use a specific choice among the Unix/POSLX/Linux encoding of text files (with LF line terminators), the Macintosh encoding of text files (with CR line terminators), or the Microsoft Windows encoding of text files (with CR LF line terminators). All mbstate_t conversions can produce implementation-defined results, even after changing the LC_CTYPE category. [0011] An implementation can make truncated-result behavior well-defined in strxfrm, strftime, wcsxfrm, or wcsftime.
[0012] The multibyte functions can behave gracefully when given a sequence not in the initial shift state, or when given any mbstate_t object.
[0013] The wide-character classifying and conversion functions can be well-defined for any wint_t input and for any LC_CTYPE setting.
[0014] The methods shown in this section can be used to eliminate the following undefined behaviors:
Figure imgf000009_0001
Secure library fs/ib/ The secure library enhancements proposed to ISO/IEC JTC 1 SC22 WG14 will eliminate many opportunities for undefined behavior. Furthermore, if a formatted I/O function produces more than INTJV1AX chars of output, then it can return INT_MAX. [0015] The methods shown in this section can be used to eliminate the following undefined behaviors: '
Figure imgf000009_0002
Ss_un wind f long; J [0016] The longjmp function (and any other functions which "unwind" the stack), can check whether execution of atexit-registered functions has started. If so, one of the following implementation-defined actions can be performed: cause a return from the function that invoked the unwind or longjmp function, invoke an "extreme exit" cleanup function; or invoke the abort function. Optionally, at the point of catching the ss_unwind, a system sanity check can be performed before continuing or re-starting. [0017] The methods shown in this section can be used to eliminate the following undefined behavior:
Figure imgf000010_0001
Specia/ behavior of atexit functions fatexj [0018] The exit function can check whether execution of the exit function has started. If so, one of the following implementation-defined actions can be performed: cause a return from the function that invoked the longjmp function, invoke an "extreme exit" cleanup function; or invoke the abort function.
[0019] The methods shown in this section can be used to eliminate the following undefined behavior:
A rithmetic exceptions fexc] [0020] If at compile-time the right operand of division or remainder is zero, a fatal diagnostic message can be produced. In Debug mode, if at run-time the right operand of division or remainder is zero, an "unwind" (such as ss_unwind) can be invoked, and the implementation may throw an exception of an implementation-defined type. In non-Debug mode, if at run-time the right operand of division or remainder is zero, the result can be the maximum value of the result type, which for a floating-point type may be an infinity.
[0021] If at compile-time the left operand of division or remainder is the maximum negative value of its type and the right operand is -1, a fatal diagnostic message can be produced. In Debug mode, if at run-time the left operand of division or remainder is the maximum negative value of its type and the right operand is -1, an "unwind" (such as ss_unwind) can be invoked, and the implementation may throw an exception of an implementation-defined type. In non-Debug mode, if at run-time the left operand of division or remainder is the maximum negative value of its type and the right operand is -1, the result can be the maximum value of the result type. [0022] If at compile-time the result of an integral arithmetic operation is too large for its type, a fatal diagnostic message can be produced. In Debug mode, if at run-time the result of an integral arithmetic operation is too large for its type, an "unwind" (such as ss_unwind) can be invoked, and the implementation may throw an exception of an implementation- defined type. In non-Debug mode, if at run-time the result of an integral arithmetic operation is too large for its type, the result can be the value of the twos-complement operation with wrap-around.
[0023] The methods shown in this section can be used to eliminate the following undefined behaviors:
Figure imgf000011_0001
Control of dang/ing pointers fdangj [0024] One category of undefined behavior arises from accessing freed storage. Furthermore, each freed pointer must previously have been allocated.
[0025] These undefined behaviors can be eliminated by use of garbage collection, either conservative (see, e.g., Hans-J Boehm, "A Garbage Collector for C and C++", . (http://www.hpl.hp.com/personal/Hans Boehm/gc/)) or accurate (see e.g., Fergus Henderson, "Accurate Garbage Collection in an Uncooperative Environment", ISMM'02, June 2021, 2002, Berlin, Germany, ACM 1581135394/02/0006), supplemented with the following special treatment of pointers to terminated stack frames. Directly assigning the address of the current function's stack frame to a longer-life pointer can be prohibited. Define a pointer- retainer function as a function which stores a pointer argument in heap or static storage. Passing a pointer to stack to a pointer-retainer function can be prohibited. (Whatever data resides in the stack can be copied to heap or to static, to avoid the prohibition.)
[0026] Memory that could contain pointers can be initialized to zeroes. Therefore, (as in Boehm conservative garbage-collection) malloc allocates space that might have pointers in it, so the space is zero-filled. There can be a new attribute to describe a state named e.g "not_ptrs" for any storage which is guaranteed not to contain pointers, and a different version of malloc can be used for such storage (equivalent to GC_malloc_atomic in the Boehm library): void * malloc_not_ptrs(size_t n); [0027] If storage with the not_ptrs attribute is cast to pointer-to-anything, then a fatal diagnostic message can be produced. The not_ptrs attribute can be removed from any storage by assigning zero to the bytes of the storage; a byte-oriented alias is mandatory (char, or unsigned char, or a library functions such as memset which modifies the bytes of memory).
[0028] An alternative method for prevention of dangling pointers is known (see e.g., Todd M. Austin et al., Efficient Detection of All Pointer and Array Access Errors. Proceedings of the ACM SIGPLAN '94 Conference on Programming Language Design and Implementation, June 1994), which is a feasible solution for an implementation which operates entirely in BSAFE mode (see below).
[0029] The methods shown in this section can be used to eliminate the following undefined behaviors:
Figure imgf000012_0001
Exclusion of C 1999 extensions [c99] [0030] The exemplary implementation described herein does not specifically address those extensions added in the 1999 revision of C which are not features of C++. Further refinements can be produced to address the undefined behaviors related to those extensions:
Figure imgf000013_0001
[0031] It would be desirable to eliminate, with commercially acceptable efficiency, further undefined behaviors in the execution of programs in the "intersection" of C and C++; that is, in C programs which use only the features described in the C++ standard, and of C++ programs which use only the features described in the C standard. [0032] It would furthermore be desirable to automated (e.g., through compiler design) techniques provide safe secure development of software, including but not limited to techniques for addressing undefined behavior in the C and C++ programming languages.
[0033] Advantageous features provided by exemplary illustrative non-limiting implementations of the technology herein include:
• A Safe Secure Compiler ("SSC") which produces Safe Secure Object Files or fatal diagnostic messages.
• A Safe Secure Inputs Check-List ("SSICL") which records checksum information for the inputs to the execution of a Safe Secure Compiler.
• A Safe Secure Bounds Data File ("SSBDF") which records Requirements and Guarantees for the defined and undefined symbols in one or more corresponding object files, as well as checksum information.
• A Safe Secure Linker ("SSL") which combines object files and the corresponding Safe Secure Bounds Data Files , producing either fatal link-time diagnostics or a Safe Secure Executable Program.
• A Safe Secure Semantic Analyzer ("SSSA") which uses the parse tree to determine Requirements and Guarantees.
• A Safe Secure Diagnostic Generator ("SSDG") which generates fatal diagnostic messages in situations where undefined behavior would result and generates various warning messages to call the programmer's attention to various other situations.
• A Safe Secure Code Generator ("SSCG") which generates object code which is free from the designated sets of undefined behaviors (including "buffer overflow" and "null pointer indirection").
• A Safe Secure Pointer Attribute Hierarchy ("SSPAH") which controls the inference of attributes based upon other attributes.
• A Safe Secure Pointer Attribute Predicate Table ("SSPAPT") which controls the determination of attributes resulting from predicate expressions. • A Safe Secure Bounds Data Table ("SSBDT") which tabulates the Guarantees and Requirements for expressions, sub-expressions, declarations, identifiers, and function prototypes.
• A Safe Secure Interface Inference Table ("SSLIT") which controls the inference of Requirements on the interface of each externally-callable function.
• A Safe Secure Bounds Data Symbol Table ("SSBDST") which tabulates the Requirements and Guarantees for defined and undefined symbols during the Safe Secure Linking process.
• A Safe Secure Link-Time Analyzer ("SSLTA") which matches Requirements to Guarantees for function-call, external array, and external pointer linkage contexts. • A Safe Secure Link Diagnostic Generator ("SSLDG") which generates a fatal diagnostic at link-time if any Requirement is unsatisfied; this prevents the production of any executable program.
BRIEF DESCRIPTION OF THE DRAWINGS
[0034] These and other features and advantages will he better and more completely understood by referring to the following detailed description of exemplary non-limiting illustrative embodiments in conjunction with the drawings of which:
[0035] Figure 1 shows an example safe secure software development system;
[0036] Figure 2 shows a flow diagram of an exemplary illustrative non-limiting implementation of a program translation system;
[0037] Figure 3 shows a block diagram of an exemplary illustrative non-limiting compiler implementation;
[0038] Figure 4 shows exemplary illustrative annotation of pointer-null-state to a non- limiting illustrative flow-chart;
[0039] Figure 5 shows a different exemplary illustrative representation of the same flow-chart; [0040] Figure 6 shows a further exemplary illustrative flow-chart, including some
"can't-happen" outcomes;
[0041] Figure 7 shows an exemplary illustrative non-limiting Safe Secure Pointer
Attribute Hierarchy (SSPAH 41);
[0042] Figure 8 shows an exemplary illustrative non-limiting Safe Secure Pointer
Attribute Predicate Table (SSPAPT 51);
[0043] Figure 9 shows exemplary illustrative non-limiting components of a Safe
Secure Semantic Analyzer (SSSA 35):
[0044} Figure 10 shows an exemplary illustrative non-Umiting Safe Secure Interface
Inference Table (SSHT 71); and
[0045] Figure 11 shows exemplary illustrative non-limiting components of the Safe
Secure Linker (SSL 28).
DETAILED DESCRIPTION
[0046] Figure 1 shows an example illustrative non-limiting automated safe secure computer software development system 100. For example, an exemplary non-limiting system 100 may comprise a central processing unit (CPU) 11 coupled to a mass storage device 12 such as a magnetic or other disk. The CPU 11 may execute program instructions including for example a safe secure translator 13 the executable code for which is stored on the mass storage device and executed by CPU 11. When executed by CPU 11, the translator 13, in simplified terms, takes a source code input 14 and converts it to an executable output 15 that can be executed by the same or different computer. The computer system 100 may further include input/output devices 16 such as for example keyboards, pointing devices, display devices and the like so that human software developers can interact with the computer. In one exemplary arrangement, the techniques described herein can be used to enhance a conventional compiler for the C and C++ programming language (as will be described below) to eliminate undefined behavior.
[0047] Referring to Figure 2. consider the following source program serving as a non- limiting illustrative example of Source Program 21 providing input to Safe Secure Compiler
23: int main() { return 0; }
[0048] Exemplary components of Safe Secure Translator 13 are shown in Figure 3.
In compiling this example, Lexical Analyzer 31, Preprocessor 32, and Parser 33 perform as commonly understood in the art. Then, Semantic Analyzer 34 verifies that the integer 0 is an acceptable value to return from the function "main" which is defined to return "int". After that, the Code Generator 38 will produce an Object File 25, the ultimate output from Safe Secure Compiler 23. Finally, the Safe Secure Linker 28 will combine the Object File 25 with various system-dependent library code, producing Executable Program 29.
States of objects fstat]
[0049] The standards for C and C++ make use of modifiers upon types, known as
"qualifiers" in C and "cv-qualifiers" in C++. Once declared, an identifier that designates an object (region of memory) retains its type and cv-qualifiers throughout its scope. A related concept, but distinctly different, is the state of an object.
[0050] One well-known state in modern programming languages is the "definitely assigned" state. Definite-assignment analysis is a central feature of the Java and C# programming languages. The present exemplary illustrative implementation of a Safe Secure method does not use the definitely-assigned state in quite the same way, but does make extensive use of state analysis. One example of a state used in the Safe Secure method is the pointer-null-state, which has three alternative state values. (The pointer-null-state in this example is one dimension of a more complicated pointer-range-state, shown later.) a. Nul - pointer is null. b. Nnul - pointer is non-null. c. Qnul - pointer might, or might not, be null; i.e. "questionably-null".
[0051] Analysis of program state proceeds using flow analysis. Consider the following source program serving as another non-limiting illustrative example of Source Program 21 providing input to Safe Secure Compiler23: int main() { int *p = (int*)0; return *p; }
[0052] As described previously, Lexical Analyzer 31 , Preprocessor 32, and Parser 33 perform as commonly understood in the art. Semantic Analyzer 34 determines that pointer p is initialized on its declaration and determines that the initial value is the null pointer value. Within Semantic Analyzer 34, control flow analysis determines that there is no transfer of control in or out of the control-flow arc from the point of initialization to the point of usage (which attempts to fetch a value from the target of pointer p) and data-flow analysis determines that there are no alterations of the value of pointer p within that arc. Control-flow analysis and data-flow analysis are commonly used to support optimization, but they also provide input data for the Diagnostic Generator 36. In the present example, the Semantic Analyzer 34 will commonly provide to the Diagnostic Generator 36 data indicating that upon all possible arcs the program would encounter the indirecting of a null pointer, and the Diagnostic Generator 36 will commonly produce a fatal diagnostic message and will commonly prevent the Code Generator 38 from producing any object code. (Each use of the term "fatal diagnostic" implies without exception that object code is never produced for any program which causes a fatal diagnostic.)
[0053] Consider this portion of another Source Program 21 : if (p!=0) ++n;
[0054] Figure 4 shows the flowchart which corresponds to the control flow in this example. In this conventional flowchart presentation, the two arcs re-join into the same flow- of -control (program-counter) state, but the pointer-null-state of p is different after the branch. An alternative presentation provides numbers (or labels) to each flow-of-control state, but records the continuation of each arc as a distinctly different state, as shown in Figure 5. The difference is made clearer by adding another instruction to the example: if (p!=0) ++n; if (p=Q) ++m;
[0055] The resulting flowchart is shown in Figure 6. The Safe Secure Semantic
Analyzer ("SSSA") 35 verifies that the pointer-null-state of pointer p pre-determines the outcomes of the second "if statement, so there are only two actual flow-of-control paths in the flowchart of Figure 6. In the SSSA 35 the flow chart of each function is factored into separate continuations of each arc at every place in the function, except that all arcs re-join at the return from the function (and the epilog implied by that return).
[0056] The concept of "state" applies to objects during run-time. Herein, the term "attribute" is used for the compile-time designation of a run-time state of an identifier or expression.
[0057] As will be explained below, the compile-time attributes as used in the SSSA 35 are sometimes capable of modeling every detail of the run-time state analysis; in these cases the compile-time attributes are all that is needed in the analysis. In other cases, the compile-time attributes are capable of modeling many, but not all, details of the run-time state analysis; in these cases, the compile-time attributes permit compile-time diagnosis of many but not all errors. Therefore, these cases will require some bits in the run-time storage to record run-time state information.
[0058] With this background, we can describe the system of pointer-range states and attributes that is employed in the SSSA 35. We distinguish the following attributes of pointers, noting that the attributes are not mutually exclusive:
Figure imgf000019_0001
[0059] Some attributes imply other attributes; see the Safe Secure Pointer Attribute Hierarchy ("SSPAH") 41 in Figure 7 for details.
[0060] When the source program contains a test (i.e., predicate) to a pointer (or subscript) value, that pointer's attribute will be set, upon the true and false outcome arcs from that predicate; see the Safe Secure Pointer Attribute Predicate Table ("SSPAPT") 51 in Figure 8 for details.
[0061] The Lo, Hi, End, Nth, and Ntl attributes can be applied either to a pointer or to a subscript (if the subscript's array context is clear).
[0062] The spelling and/or capitalization of the names of the attributes, values,
Requirements, and Guarantees are not essential, and can be chosen to meet any other requirements. For a non-limiting example, the attribute name "Nullterm" could be abbreviated as "Nullt", and the attribute name "Indirectable" could be abbreviated as "Indir", provided that those names are unambiguous in the context in which they are used.
[0063] To determine the state transition for the Nullterm attribute, the SSSA 35 first determines the outcome Indirectable ("I") attribute; if the outcome attribute is I then it is also Nullterm; if it is not I then it is not Nullterm either. For example if a pointer p is Nullterm, and comparison <=Hi is True, the result attribute is I and is also Nullterm. But if the comparison is False, the result attribute is Ni and not Nullterm.
[0064] The present method does not require that the null pointer compares low to all other pointer values. In an alternative method, this requirement ("unsigned pointer comparison") is added to the pointer semantics; "nul >= Lo" becomes a "can't happen" condition that produces the "Ntl" attribute, and "nul > Hi" becomes a "can't happen" condition that produces the "Ni" attribute.
[0065] The Safe Secure method relies heavily upon analysis of pointer-bounds states, but there are some other significant states of storage.
[0066] In C and C++, a union can contain objects of different types at different times.
In the present Safe Secure method there are no trap representations; any sequence of bytes can be fetched without causing any exceptions, and those bytes determine a value within the range of values for the type of the lvalue. Therefore, accessing a union object through an lvalue of an "incorrect" type may produce an incorrect value but will not cause an exception or any undefined behavior. Assigning a value to a union member determines the current effective type of the union, which is an important aspect of the run-time system of states. In the Safe Secure method, assigning a pointer (address) value to a union object establishes the state of all union members which are pointers to types with the same representation and alignment. Consider the following example:
union { char *p; unsigned char *q; int n; } u; char a[3] = "ab"; u.p = a; // u.p and u.q are in Indirectable (i) state, and equal to Lo of a u.n = 0; // u.p and u.q are in Unknown (unk) state
[0067] In the terminology of the Safe Secure method, all the attributes shown so far appear as Guarantees, i.e. post-conditions, established by various predicates on arcs of control flow. In subsequent sections, attributes will be used in Requirements, i.e. pre-conditions,
[0068] Stream objects can in one exemplary implementation contain an "is-open" run-time state which is reflected in a compile-time attribute (called e.g. "is_open"). The representation of the is-open state can in one exemplary implementation be non-zero.
[0069] After the standard C and C++ library function setvbuf is invoked, the calling program should make no further use of the array provided to the buf argument. In the Safe Secure Bounds Data File 26 which contains data for the setvbuf function, a post-condition specifies that the buf argument is Not-Indirectable after invocation, using methods defined in the next section.
[0070] The methods shown in this section will eliminate the following undefined behaviors:
Figure imgf000022_0001
Safe pointers βufj [0071] In the present method, all pointers are implemented as "compile-time-safe pointers", subject to a maximal verification at compile time.
[0072] The target-size, or Tsize, is defined as the number of bytes in the object or array designated by the pointer. The Bound is defined as the Tsize divided by the size of the type T to which the pointer is pointing. The Tsize-adjusted is the Tsize minus the size of the type T, i.e. the number of bytes in the array between the address of the first object of type T and the last object of type T. If there is only one element in the array, the Tsize-adjusted is zero.
[0073] The SSSA 35 maintains this bounds information for every pointer used to fetch-or-store; therefore, the analysis presented in this Safe Secure method will incorporate the syntax and semantics of the restrict keyword (from C99). Whenever this keyword appears, it reflects in syntax a restriction that otherwise must be stated in (non-formalized) semantics, i.e. a restriction that the target object designated by the restrict-qualified pointer does not overlap any other object that is accessible through external pointers or other argument pointers. This restriction can in one exemplary implementation be verified at compile-time to the extent that the bounds information is available. The remaining checks can in one exemplary implementation be verified at run-time by examination of the bounds data of the fore-mentioned set of pointers. Note that the restrict keyword is not part of the (1998 or 2002) C++ standard, but the no-overlap restriction is part of the (non-formalized) semantics of the C library contained in that C++ standard, and therefore is part of the requirements for the elimination of undefined behavior in C++ as well as C.
[0074] Components of the SSSA 35 are as shown in Figure 9.
[0075] A pointer value that designates an object in an array has a compile-time relationship with the bounding addresses .of the array. When a pointer value is assigned to a pointer object, the target pointer object receives all the attributes pertaining to the original pointer. All pointer arithmetic (ptr+n, ptr-n, ptr-ptr, ptr[n]) checks the result (at compile- time, link-time, or run-time) against the Lo-Hi bounds; the End ("one-too-far") value is permissible as a result from pointer arithmetic. Fetching or storing via indirection operators (explicitly via "star" or implicitly in the lvalue use of subscript) causes the SSSA 35 to check the pointer or subscript against the Lo-Hi bounds; the "one-too-far" value is not permissible for fetching or storing. If at run-time, any bounds are exceeded, in Debug mode ss_unwind is invoked (see Ss_unwind [longj]); in non-Debug mode the semantics of _Sat, _Modwrap or _Zerobound (defined later in this section) are produced by the SSCG 39, based upon considerations of best performance. In the alternative, any algorithm of alternating between semantics is allowable. The underlying intent of this alternative is that programmers will not be given an expectation that any specific semantics is predictable or deterministic.
[0076] A compile-time-safe pointer uses the same (one-word) representation as the ordinary C or C++ implementation. In all contexts that require bounds information, that information is made known to the SSSA 35 through a variety of mechanisms. In compile- time-safe code, the SSSA 35 ensures that each non-scalar pointer must have, within the same region (global static, block static, block local, parameters, or heap) bounds information (or bounds information stored in the global static region), stored in locations made known to the Safe Secure Compiler 23, sufficient to verify the safety of any fetches or stores expressed in the program. [0077] Here is an itemization of the contexts that require or use bounds information for a pointer: arithmetic (p+n, p-n, p-p), subscripting (p[n] and n[p]), pointer argument passing, returning a pointer, taking the address-of, pointer copying and assignment. For each pointer variable or expression appearing in a context which requires bounds information, that bounds information is inferred by the SSSA 35, using one of several alternative methods.
[0078] The SSSA 35 stores Guarantees and Requirements for pointer bounds in the
Safe Secure Bounds-Data Table ("SSBDT") 61. A subset of this information is also stored in the Safe Secure Bounds-Data File ("SSBDF") 26, namely all bounds data entries which provide information about the Requirements and Guarantees of references to, and definitions of, external arrays, external pointers, and the parameters and return type of externally-callable functions, i.e. function prototype entries. At or before termination of execution of the SSSA 35, all function prototype entries in the SSBDT 61 are written to the SSBDF 26 contained in a persistent storage ("file storage").
[0079] The format and representation of the SSBDT 61 can be optimized to suit the requirements of each SSSA 35. In the SSBDT 61, the Guarantees and Requirements are attached as needed to expressions, sub-expressions, declarations, identifiers, and function prototypes, using whatever indexing or coordinates as are needed to identify the relevant location in the internal representation of the Parser 33. For purposes of presentation in the present Safe Secure method, all entries in the SSBDT 61 will be represented in the narrative text or in the style of macro invocations inserted into the Source Program 107. In the present method, this representation does not presuppose that any such invocations of actual macros are present in the actual Source Program 107. The Safe Secure method requires that one or more of the following are provided:
1. The Parser 33 recognizes some notation identical to or equivalent to the macro notation shown herein; or,
2. The SSSA 35 provides a method whereby the human programmer can add annotations to the SSBDT 61 and SSBDF 26. If the Compiler 23 is a component in an Interactive Design Environment (IDE), then the mechanism for adding annotations to the SSBDT 61 and SSBDF 26 can be integrated into the IDE. [0080] The concepts of pre-condition and post-condition are well-known. Note that "Requirement" as used herein is synonymous with "pre-condition" and that "Guarantee" is synonymous with "post-condition", with the following contextual difference: in some technology communities the terms "pre-condition" and "post-condition" are primarily used for annotations made by the human programmer. The choice herein of different terminology is intended to avoid this connotation; the huge number of Requirements and Guarantees required by the Safe Secure methods would be an unreasonable burden if performed only by human annotation.
[0081] The entries in the SSBDT 61 are represented as follows:
Figure imgf000025_0001
[0082] For convenience and flexibility, alternative annotations are provided, which can be attached to the declaration of the integer or pointer which is providing the bounds information:
Figure imgf000025_0002
[0083] As indicated by the last examples, all the IS_*_OF annotations accept a variable number of names, enclosed within a second level of parentheses.
[0084] If the Lo bound is not explicitly specified, then the initial value of each pointer in each scope is initially the Lo bound. The initial Lo bound plus the initial Bound (or Length or Tsize) value determines the Hi bound.
[0085] The default for a pointer without any bounds annotation is Not-Indirectable. [0086] The SSSA 35 can infer from the source code of a function that a parameter is being used as the Hi, or Bound, or Length, or Tsize, of another parameter. The itemized set of rules for the inference is detailed in Figure 10.
[0087] There are several special cases for the bounds annotations in the SSBDT 61 :
Figure imgf000026_0002
[0088] Note that any attributes provided for the function' s returned value are always Guarantees and never Requirements, because the returned value comes into existence only upon function return. Attributes provided for the function's parameters are both Requirements and Guarantees (unless specifically indicated otherwise), since the attribute is presumed to hold before and after execution of the function.
[0089] The SSBDT 61 also uses a special notation for NULLTERM, for char arrays which are guaranteed to be null-terminated: int main(int argc, NULLTERM char* argv[] BOUND_IS(argc) ); [0090] In the SSBDT 61, void* generic pointers are annotated just the same as char* pointers.
[0091] When one parameter (such as n in this example) provides bounds data for a pointer (such as si in this example), that pointer is understood to be Indirectable. However, if one parameter provides bounds data but a pointer is explicitly Maybe-Indirectable, then either the pointer is null or it is indirectable and has bounds data.
[0092] If both Tsize and Bound are specified for the same object, then the Tsize is the Tsize of one element, and the Bound is the number of elements. For a non-limiting example, the standard library function calloc is represented in the SSBDT 61 as follows:
Figure imgf000026_0001
[0093] If bothTsize (or other bounds data) and Nullterm are specified for the same pointer, then the Tsize is understood as the largest object that will be accessed through this pointer. For example, the standard library function strncpy is represented in the SSBDT 61 as follows:
char * strncpy( char *sl, const char *s2 NULLTERM, size_t n IS_TSIZE_OF((sl,s2))
[0094] In the notation described so far, the attributes for "indirectable" and "null- terminated" represent Requirements-and-Guarantees for arguments and Guarantees for returned value. These attributes can also be used for Guarantees (post-conditions) upon an argument when there is no corresponding Requirement (pre-condition) upon entry to the function. An additional marker ("POST") is used to indicate post-conditions (Guarantees) in the notation of the SSBDT 61:
int swprintf(wchar_t * restrict s POST(NULLTERM) LENGTH_IS(return), size_t n IS_BOUND_OF(s), const wcharj: * restrict format NULLTERM, ...);
[0095] This notation indicates that there is no Requirement for null-termination of s upon entry to the swprintf function, but there is a Guarantee of null-termination upon return from the function.
[0096] When a pointer or subscript which has a bounds attribute is assigned to another pointer or integer variable, both the source and the target acquire that specific bounds attribute. When one of those variables is modified, the other variable retains the bounds attribute. For a non-limiting example, if a pointer equals the Lo bound, and then that pointer is incremented, then any other Lo for that pointer retains its Lo attribute. Here is a non- limiting example:
void f(int *p) { int *q = p; // p and q are the LO of p and the LO of q ++p; // now only q is the LO of p; it is also the LO of q
[0097] At any particular point in the control flow of the Source Program 107, attribute information might be redundantly attached to several different objects. If one of these objects is subsequently modified, another object may retain the attribute as before. Here is a non-limiting example:
void f(int n, int *p BOUNDJS(n) ) { int *q = p + n; // at this point, q is the END of p — n; // because q holds the END value, n is no longer BOUND_OF(p)
[0098] The SSSA 34 embodies a rule that can be summarized informally as "half a loaf is better than none": if there is only one copy of the LO, HI, END, BOUND, TSIZE, TSIZEA, or LENGTH, and if that copy is incremented or decremented, then that copy retains the appropriate attribute. Here is an illustrative non-limiting example:
void f (char *p) { int a[N] = {0}; // at this point, n is the BOUND of p int *p = a; // now, p is the LO of a and the LO of p int *q = p + n; // at this point, q is the END of p
-n; // because we saved the END value, n is no longer BOUND_OF(p)
[0099] Here is a similar non-limiting example:
int n = N; int a[N] = (0 } ; // at this point, n is the BOUND of p int *p = a; // now, p is the LO of a and the LO of p int *q = p + n; // at this point, q is the END of p
-n; // because we saved the END value, n is no longer BOUND. OF(p)
[00100] In the SSBDT 61 there is no need to re-state the type information that was present in the function prototypes; the table needs only to contain the bounds-data annotations. Some of the subsequent non-limiting examples of SSBDT 61 notation will omit extraneous type information to permit more concise presentation. However, complicated declarators such as pointer-to-pointer will still require some way to show which level of pointer or array the annotation pertains to; these methods are indifferent to the specific scheme of notation. The present method covers the functional behavior of the use of the bounds data without limitation to one specific notation.
[00101] In some cases, separate bounds-data entries are required for the separate annotation of SUCCEED and FAIL cases (and see Function invocation patterns [invoc]). For a non-limiting example, consider the two separate annotations which the SSSA 35 would make in the SSBDT 61 after analyzing the standard C and C++ library function fgets:
SUCCEED fgets(s POST(NULLT), n IS_TSIZE_OF(s), stream INDIR) FAIL fgets(s, n IS_TSIZE_OF(s), stream INDIR)
[00102] This notation means that when fgets returns successfully, the string pointed to by s is null-terminated; when fgets returns unsuccessfully, there is no Guarantee about null- termination. In either case, n is the Tsize of s (a Requirement upon entry to fgets, and a Guarantee upon return from fgets), and stream is Indirectable (a Requirement upon entry to fgets, and a Guarantee upon return from fgets).
[00103] When the Requirements and Guarantees within one function are different in different blocks within the function, and when one set of such blocks is controlled by an if- else test upon a parameter value, that function is analyzed as two separate functions which depend upon an attribute of an argument. For a non-Umiting example, consider the two separate annotations which the SSSA 35 would make in the SSBDT 61 after analyzing the standard C and C++ library function mbtowc:
mbtowc(pwc, s IS_NULL, n) mbtowc(pwc INDIR, s, n IS_TSIZE_OF(s) )
[00104] For further non-limiting examples of the annotation in the SSBDT 61 (and the various special cases mentioned above), consider the following bounds-data entries for various functions from the C and C++ standard library.
setbuf(stream INDIR, buf INDIR) setvbuf(stream INDIR, buf IS_NULL, mode, size) setvbuf(stream INDIR, buf POST NOTjNDIR, mode, size IS_TSIZE_OF(buf) ) vfprintf(stream INDIR, fmt NULLT, arg) fgetpos(stream INDIR, pos INDIR) fclose(stream INDIR) fsetρos(stream INDIR, pos INDIR) fseek(stream INDIR, offset, whence) wcstombs(s POST NULLT, pwcs NULLT, n IS_BOUND_OF(s) )
SUCCEED fgets(s POST(NULLT), n IS_TSIZE_OF(s), stream INDIR)
FAIL fgets(s, n IS_TSIZE_OF(s), stream INDIR) vsnprintf(s POST NULLT, n IS_BOUND_OF(s), fmt NULLT, arg) wctomb(s BOUND_IS(MB_CUR_MAX), wc) fopen(fιlename NULLT, mode NULLT) freopen(filename NULLT, mode NULLT, stream INDIR) vprintf(fmt NULLT, arg) strtod(rφtr NULLT, endptr IS_NULL) strtod(nptr NULLT, char NULLT ** endptr INDIR) strtoKnptr NULLT, endptr IS_NULL, base) strtod(nptr NULLT, char NULLT ** endptr INDIR, base) fputs(s NULLT, stream INDIR) remove(filename NULLT) rename(old NULLT, new NULLT) strchr(s NULLT, c) mblen(s IS_NULL, n) mblen(s NULLT, n IS_BOUND_OF(s) ) strncmp(sl NULLT, s2 NULLT, n IS_BOUND_OF((sl,s2)) ) fwrite(ptr, size IS_TSIZE_OF(ptr), nmemb IS_BOUND_OF(s), stream INDIR) bsearch(key, base, n IS_BOUND_OF(base), size IS_TSIZE_OF((key,base)), cmp INDIR) memchr(s NULLT, c, n IS_TSIZE_OF(s) ) memcmp(sl, s2, n IS_TSIZE_OF((sl,s2)) ) fputc(int c, stream INDER) fread(ptr, size IS_TSIZE_OF(ptr), nmemb IS_BOUND_OF(ptι), stream INDIR) memcpy(sl, s2, n IS_TSIZE_OF((sl,s2)) ) qsort(base, nmemb IS_BOUND_OF(base), size IS_TSIZE_OF(base), cmp INDIR) memset(s, c, n IS_TSIZE_OF(s)) ) memmove(sl, s2, n IS_TSIZE_OF((sl,s2,return)) ) mbtowc(pwc, s IS_NULL, n) mbtowc(ρwc INDIR, s, n IS_TSIZE_OF(s) ) mbstowcs(pwcs, s, n IS_BOUND_OF(pwcs) )
[00105] As has been shown, the bounds information for a pointer is made available to the Safe Secure Compiler 23 through a variety of mechanisms depending upon the context: explicit attribute macros in a calhng sequence (or equivalent programmer notations through IDE or other means), attributes inferred from usage in the called function, a visible allocation expression such as malloc or new, information made known to the linker by the SSBDF 26, or from direct visibility of an array declaration. If the programmer has provided explicit notations of bounds information, the Safe Secure Compiler 23 checks them against the inferred bounds; otherwise, the inferred bounds will be used during the compilation. It is likely that professional programmers providing library code for consumption by others will be urged to provide explicit notations.
[00106] When the Safe Secure Compiler 23 can see the assignment ptr=malloc(n) (or other standardized allocation), the specific bounds macros (LO_IS, HI_IS, END_IS, BOUNDJS, TSIZEJS, TSIZEAJS) are allowed but not required, because the Safe Secure Compiler 23 can infer that variable n IS_TSIZE_OF(ptr). So long as the variable n is not subsequently modified, it provides bounds information for ptr. Even if the argument to malloc has been modified, when the SSSA 35 determines that the ptr is unmodified from the original returned value, the SSCG 39 generates code to implement all the bounds macros (LO_OF, HJLOF, END_OF, BOUNDJDF, TSIZE_OF, TSIZEA_OF) by accessing the heap- allocation data of the pointer ptr. However, if the function passes ptr as an argument to an external function, the function must pass the bounds information through an argument. The SSBDF 26 does not provide notation to distinguish dynamically-allocated pointers from ordinary pointers.
[00107] Analysis of pointer-range-state starts by identifying all fetch-or-store expressions that involve pointers or subscripts, and all objects and variables (i.e. named objects) that are part of those expressions. Next, identify all expressions and statements that involve those objects and variables. Then identify certain control-flow patterns which guarantee the "counted-loop" ("counted-plus" and "counted-minus") semantics.
[00108] The process can be illustrated with a small non-limiting example:
Figure imgf000031_0001
[00109] Since m is defined in this translation unit, its bounds are known to the SSSA 35; specifically, its Bound is 2, its Lo is the address of m[0], its Hi is the address of m[l], and its End is the address of m[2]. The Tsize of m is 2 times sizeof(int), and its Tsize-adjusted is sizeof(int). There is only one fetch-or-store expression involving pointers or subscripts; it is the expression m[i]. The variables involved in this expression are m and i. As the Practitioner knows, the semantics of C and C++ requires that the subscript expression m[i] must be within the range of Lo to Hi; we designate this as the Requirement of the expression. The expressions and statements that involve those variables are contained in the one for-loop. The for-loop matches one of the "counted-plus" patterns, therefore the subscripting is valid. In our terminology, we refer to the for-loop as the Guarantee that satisfies the Requirement given above. Another way of stating this Requirement is that "i is a Subscript-For the array m". (For a compact notation, we can write "i SUB4(m)". The same Requirement can be applied to pointers; "p SUB4(m)" means that p is bounded between the Lo and Hi of m.)
[00110] When recognizing the patterns for a counted-loop (counted-plus and counted- minus), subscripting expressions can be converted into pointer expressions and vice-versa. Some patterns may be expressed more easily with subscripts and others with pointers. The previous example can be converted into this equivalent example:
Figure imgf000032_0001
[00111] A scalar pointer is one which, in its scope and Hfetime, is never incremented/decremented, never participates in address arithmetic, and is never subscripted (except using p[0]). It does not require bounds information.
[00112] The rules for a counted-loop (counted-plus and or counted-minus) are the same whether a pointer is incremented/decremented or an integer subscript is incremented/decremented. These are the patterns that identify a counted-plus and/or counted- minus: Loop limited by Bound: the loop performs N repetitions where N is less than or equal to the Bound of p, and the pointer designates a sequence of objects p through p+N-1, or p through p-N+1
Loop hmited by Hi: the loop increments an indirectable pointer until it is equal to Hi
Loop limited by Lo: the loop decrements an indirectable pointer until it is equal to Lo
Loop limited by null terminator: the loop increments an indirectable pointer until its target is null
[00113] The SSCG 39 provides semantics for several attributes of pointers and/or subscripts. The "_Sat" ("saturated") semantics generates machine-dependent instructions or software code sequences such that the pointer or subscript never increments (or adds) higher than its Hi value, and never decrements (or subtracts) lower than its Lo value. (In other words, it is like a guage that "pegs" at its minimum and maximum values.) The "_Modwrap" semantics generates machine-dependent instructions or software code sequences such that pointer and/or subscript is wrapped modulo the End value. Therefore, as the pointer or subscript is successively incremented, it runs from the Lo value to the Hi value, then starts again at the Lo value. The "_Zerobound" semantics generates machine-dependent instructions or software code sequences such that any fetch from outside the Lo-Hi limits (including the null-pointer value) produces a zero value (or a default-initialized value for a non-scalar), and any store outside the Lo-Hi limits (including the null-pointer value) produces a no-op. All three of these semantics can be implemented in silicon.
[00114] If some control flow is too complicated and doesn't fit a simple counted-plus or counted-minus pattern, and pointer-range attributes don't guarantee safe fetch-or-store access and all other compile-time bounds-checks fail, then the SSCG 39 inserts executable bounds-check code. In the Safe Secure method there are two modes of bounds-check code. When the SSCG 39 is compiling in Debug mode, every attempt to fetch or store out of bounds, or to create a pointer value out of the full range of values (which includes the one- too-far address), or to fetch-or-store through a null pointer, causes an invocation of ss_unwind (which in an interactive debugging environment causes an interactive breakpoint). In non-Debug mode, the SSCG 39 implements a choice among the following options: (a) use Debug semantics; or (b) every attempt to fetch or store out of bounds produces a well defined result (either _Sat semantics, or _Modwrap semantics, or _Zerobound semantics) whichever produces the best fit to the current optimization profile (which might be "minimum code size" or "minimum CPU time" or other). A warning should be generated to inform the programmer. In non-Debug mode, every attempt to create a pointer value out of the full range of values produces an algebraically well-defined pointer value, even though that value would not be usable for indirection or bounds-testing. Note that if a pointer has the Nth (not- too-high) attribute, and is being dynamically compared against the Hi or End value, the _Modwrap semantics will not be as efficient as the _Sat or JZerobound semantics, because the latter semantics require only the Hi (or End) value to complete the bounds-checking. The same is true for a pointer with the Ntl (not-too-low) attribute and a Lo limit.
[00115] The selection between Debug mode and non-Debug mode is conditional upon a switch set either at compile-time, at Unk-time (using an external flag), at load time (using an environment variable), or during run-time (using an interactive debugger)
[00116] In both Debug and non-Debug mode, the SSDG 37 generates a warning message to inform the programmer that run-time bounds-checking has been inserted by the Safe Secure Compiler 23.
[00117] The SSSA 35 provides a mechanism by which the programmer can request that, if the programmer believes that the control flow does in fact guarantee adequate bounds checks, the original control flow is communicated to the organization which distributes the SSSA . By incorporating this automated client feedback mechanism, said organization will have the opportunity to increase the set of counted-bounds control-flow contexts and Requirement-Guarantee patterns. To the extent that third-party organizations perform certification, validation, and/or branding of technology herein, said client feedback is subject to the same third-party processes, since the assurances of safety and security provided by the Safe Secure method are dependent upon the correctness of the patterns that are accepted by the Safe Secure method.
[00118] The full set of bounds-enforcement mechanisms may produce a number of inserted bounds-check code segments. As an optional Alternative, the Safe Secure method provides a further tool by which these bounds-checks (and the associated warnings) can be eliminated. Define an environmental interaction as a function invocation which produces a returned value or other state change which depends upon the external environment (such as command-line argument, input function, or getenv invocation). The Safe Secure method ensures that for every environmental interaction there is a guarantee that returned values and modified argument strings are either bounded in size and value, or null-terminated. If unbounded but null-terminated, the Safe Secure method requires that some bounding construct be imposed before use of the input value. (For example, a potentially very long command line argument can be converted into a numeric value by one of the string-to- numeric library functions, at which point the resulting value is bounded by the min and max of its type.) Define all the segments of code in between environmental interactions as "pure code". Use the incremental code-coverage method (Plum U.S. Patent No. 5,758,061) to identify environmental interactions which precede code blocks which contain inserted bounds-check code segments. Let the Safe Secure Compiler 23 insert checkpoint/restart code segments at those environmental interactions. Now the environmental interactions can be i simulated, using the minimum and maximum values for the input, and/or using the complete range of values for the input, and the pure code which follows those interactions can be executed in the environment of the full range of values for the input. Note that the set of input values must include any values which affect the control flow within the pure-code segment, and should also include the values one-less-than and one-greater-than these special values.
[00119] The static analysis necessary to determine bounds-checking need not be as aggressive as whole-program-optimization. The Safe Secure method imposes upon programmers the requirement that each function contains within its calling sequence and its control flow the guarantees necessary to assure that all fetches and stores are within bounds. Therefore, no entity larger than the function need be analyzed. Furthermore, good software design suggests that "proving" that each fetch-or-store is valid should not require examination of hundreds of lines of code. If the validity is not already obvious, then the programmer should add such constructs as are required to make it obvious.
[00120] Note that the SSSA 35 must interact with the optimization analysis performed in Semantic Analyzer 34. For a non-limiting example, the analysis necessary to keep a pointer or an integer in a register ("aliasing logic") may be required to determine that that pointer or that integer retains its bounds-related attributes during specific arcs of the flow graph.
[00121] The methods described above for enforcing bounds safety have relied upon insertion by the SSCG 39 of bounds-check code segments when state analysis is unable to determine the validity of fetch-or-store accesses. In an alternative method, the SSCG 39 inserts invocations of macros or inline functions in an intermediate representation of the original program. For a non-limiting example, the following macros-or-functions can be used:
Figure imgf000036_0001
[00122] Alternative names can be used, avoiding conflict with other implementation - defined names while reducing the length of the name. Efficient implementation of these macros (or inline functions) will depend upon characteristics of the target implementation. The SSSA 35 ensures that the pointer arguments in the macro invocations are named variables (which may be named temporaries introduced by the SSSA 35), so that no side- effects (i.e. no increment, decrement, assignment, or function-call) are present in the macro argument expressions. In the event that p is not in-bounds, the macro behaves in accordance with the requirements specified above for Debug and non-Debug mode.
[00123] The methods shown in this section will eliminate the following undefined behaviors:
Figure imgf000036_0002
C7.21.1 , A string or wide string utility function is instructed to access an array buf C7.24.4 beyond the end of an object
Iterative method for determining requirements and guarantees fiterj
[00124] The exemplary methods described above are implemented in the SSSA 35, which follows an interative method to be described in this section. (This section spells out in greater detail the steps described in section "Safe pointers [buf]" above.)
[00125] Each operation upon pointers, arrays, and/or subscripts may impose one or more Requirements upon the preceding operations, declarations, and/or initializations. Specific rules for inferring Requirements are shown in the Safe Secure Interface Inference Table ("SSITT") 71. Each Requirement causes the initiation of an upward search for the Guarantee that satisfies the Requirement.
[00126] Each operation, declaration, and/or initialization may produce one or more
Guarantees for subsequent operations. Each Guarantee persists across subsequent operations and statements until some subsequent event terminates the Guarantee; such events include the end of the lifetime of any objects in the Guarantee.
[00127] The steps itemized above can be performed in any order during the semantic analysis. Each time that these steps are applied to the entire translation unit the SSSA 35 must determine whether progress was made, i.e. whether any new Guarantee was determined, or whether any Requirement (not previously satisfied on previous iterations) is now satisfied. If an iteration covers the entire translation unit without making progress, then the SSSA 35 makes a list of all unsatisfied Requirements. For each such Requirement, if it is capable of link-time resolution, then entries are created in the SSBDF 26 so that it can be resolved or diagnosed at link-time. Otherwise, if the Requirement is capable of run-time prevention (using _Sat or _Modwrap or _Zerobound logic), then the SSCG 39 generates one of the appropriate code sequences as described above. If none of these link-time or run-time resolutions are available then the SSDG 37 issues a diagnostic. If the code whose Requirement cannot be satisfied occurs in a code region which can never be executed (i.e. "dead code") or is not executed in this application (i.e. "sleeper code"), then the implementation is permitted to treat the diagnostic as a warning. In any other context, the "unsatisfied Requirement" diagnostic must prevent production of object code. Furthermore, if the unsatisfied Requirement occurs in sleeper code, then the SSSA 35 makes notations in the SSBDF 26 to indicate that the code cannot be safely invoked, and the SSCG 39 generates a stub which satisfies the linkage requirements and produces safe behavior in Debug or non- Debug mode.
[00128] Any Requirement of "at least enough" or "no more than" can be satisfied by a
Guarantee that is more strict than the Requirement.
[00129] These Requirement-Guarantee patterns are intended to reflect the basics, as understood by all Practitioners, of objects in machine memory, memory addressing, and aliasing rules. The patterns illustrated below are non-limiting examples.
[00130] Acknowledging the risk of redundancy, the Requirement-Guarantee patterns are itemized in the lists below. The first list itemizes the patterns that were defined in the Preliminary Application.
1. If the SSSA 35 has identified a Requirement that a pointer p is non-null, and if the current control flow proceeds from a non-null arc of a pointer test, then the non-null arc provides the necessary Guarantee.
2. If the SSSA 35 has identified a Requirement that a pointer, array, and/or subscript has attributes Qi, I, Nth, Ntl, End, Lo, or Hi, then various relevant Guarantees are found in Safe Secure Pointer Attribute Hierarchy 41 (see Figure 7) .
3. If the SSSA 35 has identified a Requirement that a pointer or array has the Nullterm attribute, then various relevant Guarantees are found in the paragraph "To determine the state transition for the Nullterm attribute" in section "States of objects [stat]".
4. If the SSSA 35 has identified a Requirement that an integer is usable as a subscript for array A, and if BOUND_OF(B) <= BOUND_OF(A), and if the integer is usable as a subscript for array B, then this provides the needed Guarantee.
5. If the SSSA 35 has identified any bounds Requirement after invocation of a C and C++ standard library function (such as "ptr = malloc(n)"), then the Guarantee may be provided by semantics of that function (such as "TSIZE_OF(ptr) is n") as specified in its entry in the SSBDF 26.
6. If the SSSA 35 has identified a bounds Requirement for a pointer which is visible to the SSSA 35 as a heap-allocated pointer, then a run-time Guarantee can be provided that uses bounds data from the argument(s) to the heap allocation.
7. If the SSSA 35 has identified any bounds Requirement for an array defined in the current translation unit, the bounds Guarantee is immediately available to the SSSA 35 from the defining declaration.
8. If the SSSA 35 has identified any bounds Requirement involving a subscript or pointer within the body of a counted-loop (counted-plus or counted-minus loop), then the bounds Guarantee is given by that counted- plus or counted-minus loop.
[00131] This second list itemizes additional patterns:
1. If the SSSA 35 has identified a Requirement that, for some limit LIM, 0 <= i < LLM, and we have a bounding loop for (i = 0; M * i < N; ++i), then the loop provides a Guarantee that LLM = N M.
2. If the SSSA 35 has identified a Requirement that the Tsize of pointer p is at least N (in other words, the bytes from *p to *(p+N-l) are all within one object), and if p loops from a upwards to b, then the Requirement is satisfied for all p iff it is satisfied for the maximum value, b.
3. If the SSSA 35 has identified a Requirement that some property is true when N+c reaches its maximum value (where N is independent of c), then the Requirement is satisfied when c reaches its maximum value.
[00132] For a non-limiting example consider the following source program for a rudimentary storage allocator:
static char a[N]; static char *p = a; void * my_alloc(size_t n) { char *ret = p; char *lim; if (n = 0) n = l; lim = p + n; if (n > N || lim > a+N) return (void*)0; else { p = lim; return (void*)ret; } }
[00133] As per the definition of the iterative method, the determination of
Requirements and Guarantees can proceed in any order. For a non-limiting example, start with those Requirements and Guarantees that require no flow analysis. The static declarations indicate that the Bound of a is N, and that p is initialized to the same bounds as a. The expression a+N designates the End of a. On the false ("else") arc, lim has Nth or End attribute, as determined by the SSPAPT 51, and ret has whatever attributes p had on entry to my_alloc. Then flow analysis determines that p is never decremented, that no external code can modify it (because of the file-scope static), and that it is incremented only by an unsigned value which is guaranteed not to "wrap-around" the address space. Therefore, p is never less than its initial value, the Lo of a. The function performs no fetch-or-store operations using pointers or subscripts. Therefore, the SSSA 35 needs only to determine the attributes of the returned value. On one arc the returned value is null. On the "else" arc the returned value is at least Indirectable, because p is always strictly less than a value (lim) which is Nth or End. Then the SSSA 35 determines the bounds of the returned value. The returned value is a pointer into array a, so the End of p is the End of a, but the attributes of array a are not visible outside the source file of my_alloc and therefore cannot be used to define the bounds. The only externally-visible object involved in my_alloc is the parameter n. The SSSA 35 can guarantee that the Tsize of the returned value is at least n, because p+n is at most equal to the End of a. The Tsize cannot be greater than n, because p+n can be equal to the End of a. Therefore the SSSA 35 enters into the SSBDT 61 an entry for my_alloc as follows:
QI TSIZE S(n) my_alloc(n)
[00134] Note that if the array a had been declared extern instead of static, the entry for my_alloc would instead specify "QI SUB4(a)". The programmer can use one of the alternative methods as defined above to indicate an intention to restrict the return attributes to "QI TSIZE_IS(n)" (or change the "extern" back to "static", of course).
[00135] As a further example consider the source code for an implementation of the standard C and C++ library function strlen:
Figure imgf000041_0001
[00136] Because it contains a fetch using p, the SSSA 35 determines that the Requirement upon p is at least an Indirectable requirement. Because *p is never modified (consistent with the "const" qualifier), and no de-allocation is performed upon p, the Indirectable Requirement is also a Guarantee. The pattern of the loop determines the stronger Nullterm Requirement, which is also a Guarantee. The relationship between the parameter and the returned value is also dictated by the pattern of the loop. The inference of Requirements and Guarantees produces the following entry into the SSBDT 61: strlen(str NULLTERM LENGTH_IS(return) )
[00137] Consider a situation in which the programmer had explicitly indicated conflicting annotations; for example NULL, or QI, or TSIZE JS (return). It is a requirement of the Safe Secure method that a diagnostic message shall be produced by the SSDG 37 to call the conflict to the attention of the programmer.
[00138] The components of the Safe Secure Linker are shown in Figure 11. As is common in the art, the Safe Secure Linker ("SSL") 28 reads Object File(s) 25, creating a Symbol Table 81 of symbols provided and symbols needed. The Link-Time Symbol Resolver 83 determines the addresses of the symbols which will be loaded in the executable image. If any symbols are needed but not provided (or if typesafe linkage fails in C++), then the Link Diagnostic Generator 85 produces a message, as is customary in the common art. The Safe Secure Linker 28 provides further methods beyond the common art. Along with each Object File 25 and/or Dynamic Link-Library 27, Safe Secure Compiler 23 produced a SSBDF 26 containing Requirements and Guarantees. The Safe Secure Linker 28 reads the SSBDF(s) 26 which accompany the Object File(s) 25 being linked. Each "undefined symbol" entry in the Symbol Table 81 is augmented in the Safe Secure Bounds Data Symbol Table ("SSBDST") 82 with the Requirements and Guarantees provided by the various code contexts that require the symbol. Each "defined symbol" entry in the Symbol Table 81 is augmented augmented in the Safe Secure Bounds Data Symbol Table ("SSBDST") 82 with the Requirements and Guarantees provided by the defining instance of that symbol. The Safe Secure Link-Time Analyzer ("SSLTA") 84 performs these tests:
1. For each function-call linkage context, each Requirement of the called function is satisfied by a Guarantee in the calling function. 2. For each external array or pointer linkage context, each Requirement in one object file is satisfied by a Guarantee in the other object file.
[00139] Any unsatisfied Requirement causes a fatal diagnostic from the Safe Secure
Link Diagnostic Generator ("SSLDG") 86; this prevents the production of any executable program.
[00140] As a further example we present part of the analysis of one program from the
SPEC benchmarks. We chose to test the Safe Secure method with a hand-simulation on one complete application. Of the 12 different apps in SPEC CTNT2000, the smallest is "256.bzip2", which contains about 4600 lines of code in two C source files (spec.c and bzip2.c). The smaller file (spec.c) contains about 36 fetch-or-store instances and the larger file (bzip2.c) contains about 250 fetch-or-store instances. The example below shows the first 8 instances of fetch-and-store operations in spec.c.
int spec_init ( ) { int i , j ,- debug (3, "spec_init\n" ) ; /* Clear the spec_fd structure */ /* Allocate some large chunks of memory, we can tune this later */ for (i = 0; i < MAX_SPEC_FD; i++) { int limit = spec_fd[i] /*1*/ . limit, memse (&spec_fd[i] , 0, sizeof (*spec_fd/*2*/) ) ; spec_fd[i] .limit = limit; spec_fd[i] .buf = (unsigned char *)malloc(limit+FUDGE_BUF) ; if (spec_fd[i] .buf == NULL) { printf ("spec_init: Error allocing memory ! \n" ) ; exi (1) ; } for (j = 0; j < limit; j+=1024) { spec_fd[i] -buf [j]/*3*/ = 0 ; } } return 0 ;
} int spec_random_load (int fd) { /* Now fill up the first chunk with random data, if this data is truly random then we will not get much of a boost out of it */ #define RANDOM_CHUNK_SIZE (128*1024) #define RANDOM_CHUNKS (32) /* First get some "chunks" of random data, because the gzip algorithms do not look past 32K */ int i , j ; char random_text[RANDOM_CHUN S] [RAND0M_CH1JNK_SIZE] ; debug (4, "Creating Chunks\n" ) ; for (i = 0; i < RANDOM_CHUNKS ; i++) { debugK 5, "Creating Chunk %d\n" , i) ; for (j = 0; j < RAND0M_CHUNK_SIZE; j++) { random_text[i]/*4*/[j]/*5*/ = (int) (ran( ) *256) ; } } debug (4, "Filling input file\n"); /* Now populate the input "file" with random chunks */ for (i = 0 ; i < spec_fd[fd]/*6*/. limit; i+= RANDOM_CHUNK_SIZE) { memcpy (spec_fd[fd] .buf + i, 1 *1*1 random_text [ (int) (ran ( ) *RANDO _CHUNKS) ] /*8*/ , RANDOM_CHUNK_SIZE) ; } /* TODO-REMOVE: Pretend we only did 1M */ spec_fd[fd] .len = 1024*1024; return 0; [00141] Eight instances of fetch-or-store operations are marked in the code sample above. Here is the analysis of Requirements and Guarantees for these eight marked instances:
1. spec_fd[i] | Requires: i IS_SUBSCRTPT_FOR(spec_fd)
[00142] Searching for the Guarantee to meet this Requirement, we search up to the for- loop. The for-loop is a counted-plus which guarantees that i is less than MAX_SPEC_FD. MAX_SPEC_FD is 3, so the maximum value of i is 2. which is the highest subscript for the specjd array. Therefore, i IS_SUBSCRIPT_FOR(spec_fd). (We can abbreviate this as "i SUB4(spec_fd)".) So Requirement 1 is satisfied by the counted-plus loop.
[00143] Note a series of occurrences of the same expression (spec_fd[i]) which are satisfied by the same Guarantee, so this by-hand analysis won't re-state the previous process.
Figure imgf000044_0001
[00144] This expression (*spec_fd) looks like an indirection but it isn't; it's just syntax for the size of one element of specjd. Because sizeof doesn't evaluate its argument, there is no fetch-or-store. Therefore, there is really no Requirement here.
Figure imgf000044_0002
[00145] Search up to the for-loop. The loop is monotone increasing, so the largest value for j is limit- 1. In how many places is buf initialized? Only one, 6 lines earlier. The malloc initialization guarantees that TSIZE_OF(spec_fd[i].buf) is greater than limit. So Requirement 3 is satisfied.
Figure imgf000044_0003
[00146] This one is trivial. The "for i" loop runs i to RANDOM_CHUNKS-l, which is the Hi of random_text. So Requirement 4 is satisfied. 5: random_text[ι][]] Requires: j SUB4(randorn_text[i])
[00147] Also trivial; see the "for j" loop to the Hi of random_text[i]. Therefore Requirement 5 is satisfied.
Figure imgf000045_0001
[00148] Searching upward in the spec_randomJoad function, nothing provides the needed Guarantee, so the Requirement becomes a Requirement of the function. Using the "bounds-data file" notation, the requirement is spec_randornJoad(fd SUB4(spec_fd) )
[00149] It appears that this spec_randornJoad function is never called in this application, so this Requirement never propagates.
7: memcpy(-, -, RANDOM_CHUNK_SIZE) Requires: see below
[00150] The "bounds-data" file entry for memcpy will look like this: memcpy(s, p, n IS_TSIZE_OF((s,p)) )
[00151] Therefore, there are two parts to this Requirement 7:
Requirement 7a: TSIZE_OF(spec_fd[fd].buf+i) is RANDOM_CHUNK_SIZE
Requirement 7b: TSIZE_OF(random ext[(int)(ran()*RANDOM_CFiUNKS)]) is RANDOM_CHUNK_SIZE
[00152] The Tsize of the first argument is the number of bytes from spec_fd[fd].buf+i to the last byte of spec_fd[fd].buf. This number of bytes reaches its minimum (its tightest constraint) when spec_fd[bufj+i reaches its maximum value. The maximum value of spec_fd[fd].buf+i is reached when i reaches its maximum value, namely spec_fd[fd]. limit. The relationship between spec Jfd[fd], buf and spec_fd[fd]. limit is established in the function specjnit: TSIZE_OF(spec_fd[i].buf) is spec_fd[i].limit+FUDGE_BUF [00153] Therefore Requirement 7a is equivalent to
spec d[i]. limit + RANDOM_CHUNK_SIZE <= specjdfi] .limit +FUDGE_BUF
which simplifies to
RANDOM_CHUNK_SIZE <= FUDGE_BUF
[00154] But RANDOM_CHUNK_SIZE is 128*1024, and FUDGE_BUF is 100*1024..
Therefore Requirement 7a cannot be satisfied without changes to the source code. (For example, FUDGEJBUF could be increased to the same value as RANDOM_CHUNK_SIZE.) Note that this unsatisfiable Requirement occurs inside "sleeper code", i.e. code which is not executed in this application. A compiler which is being tested by SPEC would in all likelihood not give fatal diagnostics for dead code and sleeper code, and would compile and execute the application despite the unsatisfiable Requirement 7a in the sleeper-code function spec_random Joad. The resulting executable program is still certifiably free from buffer overflows.
[00155] The Tsize of each row of the random_text array is
RANDOM_CHUNK_SrZE, as given by the declaration of randornjext; therefore Requirement 7b is trivially satisfied.
8: random ext[(int)(ran()*RANDOM_CHUNKS)] Requirement: see below
[00156] Requirement 8 is that (int)(ran()*RANDOM_CHUNKS) must be usable as a subscript for randornjext; i.e.
(int)(ran()*RANDOM_CHUNKS SUB4(random_text).
[00157] From the declaration of randornjext, its Bound is RANDOM_CHUNKS .
Therefore Requirement 8 requires
0 <= (int)(ran()*RANDOM_CHUNKS) < RANDOM_CHUNKS or 0 <= ran() < 1.0
[00158] This imposes a Requirement upon the ran() function (not shown in this example). Determining at compile-time that ran() actually produces a non-negative result smaller than 1.0 is beyond the scope of the Safe Secure Requirement-Guarantee patterns at compile time. Therefore the Safe Secure Compiler 23 must analyze the options for run-time checking, namely _Sat, _Modwrap, and _Zerobound. Let TI designate the temporary result of calculating the subscript (int)(ran()*RANDOM_CHUNKS). Since RANDOM_CHUNKS is a constant (32), the code generation for the run-time checking is as follows:
Figure imgf000047_0001
[00159] Since this particular example arises in a SPEC benchmark, the compiler will probably favor optimization for speed. Because the modulus is a constant power of two, the compiler will in all likelihood recognize a special-case pattern in the JVlodwrap alternative and generate the platform-dependent optimized version, which typically looks like random_text[Tl & OxlF].
Bitwise manipulation of pointers fbitpj [00160] If the low-order bits of a pointer are masked OFF or ON by a compile-time constant mask, and if that mask specifies a value smaller than the alignment requirement of the pointed-to type, then the pointer-bound-state of the pointer is not altered thereby. In all other cases, the pointer-bound-state of the pointer is indeterminate (Unk) after manipulation. Reading a pointer from external store (via %p, fread, or other method) produces the indeterminate pointer-bound-state
[00161] The range of unsigned integer values can in one exemplary implementation be sufficient to represent the difference of any two pointers, as well as sufficient to represent unambiguously the value of any pointer.
Figure imgf000048_0001
Variable argument list checking [v arg J [00162] A compiler 23 configured for the Safe Secure method can in one exemplary illustrative non-limiting implementation produce special assistance for invocation of functions declared with ellipsis ("variable-argument functions"). At each invocation, the Safe Secure Compiler 23 is aware of the promoted types of the actual arguments, and can in one exemplary implementation produce a null-terminated string containing the name- mangled representation of the promoted arguments. (For a description of a typical name- mangling algorithm, see e.g., Mark Mitchell et al., "Itanium C++ ABI" • (http://www.codesomcery.com/cxx-abi/abi.html)) The Safe Secure Compiler 23 can in one exemplary implementation provide special handling of the va_arg macro as defined in header <stdarg.h> as follows: If the type argument is a scalar type which produces a one-byte encoding in the mangled name string (e.g. double, which produces the single character 'd' in typical name-mangling, then an invocation such as
P = va_arg(ap, double); produces a translated invocation such as p = _va_arg3(ap, double, d'): [00163] The enhanced _va_arg3 macro can in one exemplary implementation test that the next byte in the argument mangled-name string is the character 'd', incrementing the pointer after the test. If the test is false, an associated "match" function is invoked, such as
_va_arg_double_match(&ap); [00164] The rules for type-matching for variable-argument functions permit more matches than the strict same-type rules of C++; refer to section [tsl]. If the argument mangled-name character fails the type-matching rules described there, then a run-time error is reported by using the ss_unwind function. [00165] The methods shown in this section will eliminate the following undefined behaviors:
Figure imgf000049_0001
Type-secure linkage fts/J [00166] In the Safe Secure method, C functions are type-checked using an implementation mechanism similar to that used for C++. This does not introduce overloading in C; the underlying function-name identifier must be unique for each C function. The rules for matching the type of C functions also permit linkage of functions whose signature differs from the expected type by the rules of "same representation and alignment" (see [R13] 6.2.5). For example, on many implementations, the representation of the long double type is actually the same representation as the double type, i.e., a 64-bit floating point representation. The test for "same representation and alignment" extends to pointers, both for the pointer value itself and for the type pointed-to. Pointer-to-character (signed, unsigned, or unspecified) is compatible with pointer-to-void. The reason for using the relaxed rules of "same representation and alignment" rather than the strict "same type" rules is to avoid negative responses from today's C and C++ programmers, who are currently accustomed to "practical" rather than "pedantic" standards for proper behavior of variable-argument functions. Portability is often important, but portabihty cannot over-ride the requirements of safety and security. (An alternate name for "type-secure linkage" is "type-compatible linkage".)
[00167] The type-secure linkage rules apply to data as well as to functions.
[00168] The methods shown in this section will eliminate the following undefined behaviors:
Figure imgf000050_0001
Mode strings for fop en /fop en J [00169] If at compile-time the string pointed to by the mode argument in a call to the fopen function does not exactly match one of the expected character sequences a fatal diagnostic message can be produced. If the error occurs at run-time in Debug mode, ssjunwind can be invoked; the implementation may throw an exception of an implementation-defined type. If the error occurs at run-time in non-Debug mode, the implementation can ignore all unrecognized portions of the mode argument.
[00170] The methods shown in this section will eliminate the following undefined behaviors:
Figure imgf000050_0002
Digital signatures for all components fdigsigj [00171] Safety and security are compromised when compiler components (such as system header files) do not contain the approved un-modified sources as released and installed. The Safe Secure method requires that the Safe Secure Compiler 23 provides one menu choice, or one command-line invocation, which tells the user whether the components of the Safe Secure Compiler 23 (including executable programs, library object files, and system headers) have been altered (whether deliberately or inadvertently) from their initial state as delivered. The specific mechanism for such verification is left to the vendor's choice, and several mechanisms (such as checksums and digital signatures) are well-known in the current art. Furthermore, during each compilation, the first time each .header (or other component) is opened for reading, the same checking of that header takes place. The same checking is also performed for all library components referenced at link-time.
[00172] The Safe Secure Compiler 23 produces the Safe Secure. Inputs Check-List
("SSICL") 24, a list of the file-names (or equivalent designation) of all the source components which were read by the compiler during the compilation of one or more outputs. In addition to each file-name, the SSICL 24 also contains one or more checksums for the contents of the corresponding component. (The term "check-list" is used here to designate a list of names and checksums.) At least one of the checksums embodies the following method of forming the checksum:
1. Text files represented in ASCII, ISO 646, ISO 8859-1 ("Latin-1"), or the UTF-8 encoding of Unicode or ISO 10646, are checksummed using the values of that encoding. Otherwise, each text file is converted to the UTF- 8 form of ISO 10646 before checksumming.
2. Each sequence of whitespace (except in literals) is replaced by one Space character before checksumming.
3. On each instance of the word "return" (except in double-quoted string literals) followed by one or more occurrences of the Space character or the Open Parenthesis character, any parentheses around the returned expression are removed before forming the checksum.
4. No other transformations are applied to the source characters.
[00173] The SSBDF 26 contains a similar check-list listing each output file produced by the compiler (object file, executable file, or dynamic link-library), together with one or more checksums. At least one of the checksums embodies the method given above, using the actual 8-bit value of each byte as an unsigned character value (from 0 to 255). This check-list includes one entry designating the SSICL 24, including the one-or-more checksums formed in the same way as specified above for the entries contained in the SSICL 24.
[00174] Also contained in the SSBDF 26 is a list of checksums known as the
Compilation Signature, formed as follows. At the Nm occurrence of a compiler operation specified by the Safe Secure method (e.g., the inference of an attribute, or the satisfying of a Requirement by a Guarantee), the SSC 23 forms a checkum of the operand(s) of that operation. This checksum is stored as the first entry on the Compilation Signature list. Repeat this process at the 2*NΛ operation, the 4*Nώ operation, the 8*Nώ operation, etc. The resulting list of checksums in the Compilation Signature is of a size roughly proportional to the base-2 logarithm of the program size. This Compilation Signature provides a record of the compilation of a specific program by a specific Safe Secure Compiler 23.
Format strings fo printf scanf strftime functions fmtj
[00175] If at compile-time the string pointed to by the format argument in a call to any of the printf/scanf functions does not match the number and types of the actual arguments, a fatal diagnostic message can be produced (as is well-known in current art). If the format string is a run-time character string (i.e., not a compile-time string), no additional argument- checking is required beyond the type-checking performed for variable argument lists (see section "Variable argument list checking [varg]").
[00176] If at compile-time the string pointed to by the format argument in a call to the strftime function does not exactly match one of the expected character sequences a fatal diagnostic message can be produced. If the error occurs at run-time, ss_unwind can be invoked; the implementation may throw an exception of an implementation-defined type.
[00177] The methods shown in this section will eliminate the following undefined behaviors:
SSM# C-Std# Description C7.19.6.1. C7.19.6.2, An invalid conversion specification is found in the format for C7.23.3.5, C7.24.2.1 , one of the formatted input, coutput functions, or the strftime fmt C7.24.2.2, C7.24.5.1 or wcsftime function An n conversion specification for one of the formatted input, c7.19.6.1 , c7.19.6.2, coutput functions includes any flags, an assignment- fmt C7.24.2.1. C7.24.2.2 suppressing character, a field width, or a precision fmt c7.19.6.1 , c7.19.6.2, There are insufficient arguments for the format in a call to
Figure imgf000053_0001
Function invocation patterns [invoc] [00178] ) Some functions in the standard C and C++ library return nothing (the "void" functions). Other functions always return a value with no success-or-fail status information. Some functions return a value which provides status information. The C library exhibits a rather inconsistent set of status-returning patterns. Here is a list of the various patterns of status-returns for standard C and C++ functions: a. boolean (non-zero is success): setjmp, the character classification functions (7.1.4), feof, ferror, mblen(with null s), mbtowc(with null s), wctomb(with null s), mbsinit, wide character classification functions (7.25.2.1) b. null is failure: setlocale, tmpfile, tmpnam, fopen, freopen, fgets, gets, calloc, malloc, realloc, getenv, bsearch, memchr, strchr, strpbrk, strrchr, strstr, strtok, gmtime, localtime, fgetws, wcschr, wcspbrk, wcsrchr, wcsstr, wcstok, wmemchr c. SIG_ERR is failure: signal d. zero is success: raise, rename, fclose, setvbuf, fgetpos, fseek, fsetpos, atexit e. EOF is failure: fflush, fgetc, fputc, fputs, getc, getchar, putc, putchar, puts, ungetc, fputws, wctob f. negative is failure: fprintf, printf, sprintf, vfprintf, vprintf, vsprintf, fwprintf, swprintfvfwprintf, vswprintf, vwprintf, wprintf g. number of items transmitted: fscanf, scanf, sscanf, vfscanf, vscanf, vsscanf, fread, fwrite, fwscanf, swscanf, vfwscanf, vswscanf, vwscanf, wscanf h. nonnegative and less than n: snprintf, vsnprintf i. negative one is failure: ftell, mblen(with non-null s), mbtowc(with non-null s), wctomb(with non-null s), mbstowcs, wcstombs, clock, mktime, time, wcrtomb, mbsrtowcs, wcsrtombs j. bigh-low-equal: memcmp, strcmp, strcoll, strncmp, fwide, wcscmp, wcscoll, wcsncmp, wmemcmp k. greater than n is failure: strxfrm, wcsxfrm 1. [reserved - no L] m. zero is failure: strftime, wcsftime n. WEOF is failure: fgetwc, fputwc, getwc, getwchar, putwc, putwchar, ungetwc, btowc o. minus two through n: mbrlen, mbrtowc
[00179] In each of the calling contexts listed above, there is a special "comparison value" against which the returned value should be tested. For cases 1, 2, 4, 6, 10, and 12, the comparison value is zero. For case 3, the comparison value is SIGJERR- For case 5, the comparison value is EOF. For cases 7 and 11, the comparison value is a value unambiguously determined by the list of arguments. For case 8, the comparison value can be either zero or a value unambiguously determined by the list of arguments. For case 13, the comparison value is WEOF. For case 14, the comparison value can be -2, or -1, or a value unambiguously determined by the list of arguments.
[00180] There are several calling contexts which will meet the requirements listed above: a. A conditional control-flow test (if, while, for, switch, or logical operator) which explicitly or implicitly tests the returned value against the appropriate comparison value. b. Assignment of the returned value to some object, after which the first conditional control-flow test is a test that matches the preceding context (#1). [00181] This determination of proper calling context can in one exemplary implementation also be made available for user-written functions by providing to user programs a macro such as CALLED_LIKE(uame).
[00182] If the function is called properly, then error returns are handled normally, without incurring additional run-time overhead in the calling or called function. In Debug mode, if the function was not called properly, extra code is generated in the calling function such that a failure return causes an ss_unwind with a descriptive string argument. In non- Debug mode, in all cases where execution can continue without causing fetch-or-store errors, then execution continues without interruption; otherwise a fatal diagnostic is produced at compile-time. The cases where a fatal diagnostic is produced in non-Debug mode are specified below: a. boolean (non-zero is success): none b. null is failure: getenv. (Note that FTTH* pointer parameters are required to have the "indirectable" attribute in CSAFE mode, so passing "maybe-indirectable" FILE* variables will be diagnosed at compile time. Similarly, pointer returns from allocation and searching functions do not demand immediate null-testing, because the "maybe- indirectable" attribute will require null-testing before later indirecting.) c. SIGJERR is failure: none d. zero is success: atexit (Note: most I/O failures may produce erroneous results but do not cause fetch-or-store errors.) e. EOF is failure: none (Note: most I/O failures may produce erroneous results but do not cause fetch-or-store errors.) f. negative is failure: none (Note: most I/O failures may produce erroneous results but do not cause fetch-or-store errors.) g. number of items transmitted: snprintf, sprintf, vsprintf (Note: most I/O failures may produce erroneous results but do not cause fetch-or-store errors.) h. nonnegative and less than n: none (Note: most I/O failures may produce erroneous results but do not cause fetch-or-store errors.) i. negative one is failure: mktime j. high-low-equal: none k. greater than n is failure: strxfrm, wcsxfrm 1. [reserved - no L] m. zero is failure: strftime, wcsftime n. WEOF is failure: none o. minus two through n: : none
Compile-time diagnosis fctj
[00183] Many of the situations labeled as "undefined behavior" are capable of diagnosis by the compiler. In the methods presented here, the cases identified by the code "ct" in column one result in the production of a fatal diagnostic message by the Safe Secure Compiler 23. Furthermore, production of a fatal diagnostic message prevents the Safe Secure Compiler 23 from producing a translated file.
[00184] The C standard defines a diagnostic message as follows: "3.10 diagnostic message: message belonging to an implementation-defined subset of the implementation's message output".
[00185] The methods shown in this section will eliminate the following undefined behaviors: )
Figure imgf000057_0001
Conditionally -defined ' ehaviors fcdefj
[00186] Many of the situations defined as undefined behavior could be more precisely delineated by permitting a reduced range of the alternatives (as has been described in various places in the current art). The method described here requires the Safe Secure Compiler 23 to implement a choice for each behavior: either produce a fatal diagnostic message, or produce a specified implementation-defined behavior, for each of the situations coded with "cdef ' in column one of the following table.
[00187] The methods shown in this section will eliminate the following undefined behaviors:
Figure imgf000059_0001
Figure imgf000060_0001
Figure imgf000061_0001
Big Safe Pointers /bsafe/ [00188] This section presents an alternative to the compile-time-safe ("CSAFE") pointers defined in the sections "States of objects [stat]" and "Safe pointers [buf]". In this alternative, all pointers are implemented as "big-safe pointers" (or "BSAFE" pointers) using a struct containing three ordinary pointers (each represented as a pointer to an underlying type T). The methods shown in this section provide an alternative mechanism to address the undefined behaviors that are listed above in the section "Safe pointers [buf]".
[00189] In one specific implementation, the big-safe-pointer mechanism is achieved by transforming the C or C++ source program into a C++ program in which each big-safe pointer is implemented as a template struct: template <class T> struct bsafe_ptr {T* ptr; T* lo; T* hi;};
[00190] Thus, every big-safe pointer will include low and high bounds information.
[00191] In an alternative embodiment, the high-bound information is represented as a Tsize-adjusted field. The Bound is defined as the Tsize divided by the size of the type T to which the pointer is pointing. The Tsize-adjusted is the Tsize minus the size of the type T, i.e. the number of bytes in the array between the address of the first object of type T and the last object of type T. If there is only one element in the array, the Tsize-adjusted is zero.
[00192] In order to make a copy of a bsafe_ptr, we provide a clone(T* p) member: bsa£e_ptr<T> clone (T*p) { bsafe_ptr<T>* ret = new bsaf e_ptr<T> (p , lo ,hi) ; if (Debug && p > hi+1 ) { delete ret ; ss_unwind ( "bsafe_ptr out of range : clone \n" ) ; } ret->ptr=p; ret->lo=lo ; ret->hi=hi ; return *ret ; }
[00193] Since the initial target language is the intersection of C and C++, we want to map the semantics of big-safe-pointers into a macro-compatible syntax:
fdefine LO_QF (p) ( (p) . clone ( (p) . lo) )
#define HI_OF (p) ( (p) , clone ( (p) .hi) tdefine END_OF (p) ( (p) ■ clone ( (p) +BQTJND_OF (p) )
#define BOOKD_OF(p) (TS1ZE_0F (p) /sizeof ( (*p) ) )
#define TSIZE_OF(p) ( (size_t) ( (p) . s) + sizeof (* (p) ) )
[00194] In other words, END_OF(p) is a bsafe pointer with the same Lo and Hi values as p and containing the "one-too-far" address as its current ptr value. Similarly, LO OF(p) is a bsafe pointer with the same Lo and Hi values as p and containing p.lo as its current ptr value. The BOUND_OF(p) is an integer designating the number of elements in the array that extends from LO_OF(p) to HI_OF(p). The TSIZE _OF(p) ("target size of p") is the number of bytes in the storage accessed through p.
[00195] We define the Lo-Hi information, or Lo-Hi bounds, as the information embodied in the lo and hi fields.
[00196] A pointer value that designates an object in an array includes the bounding addresses of the array. Assigning a pointer value to a pointer object preserves the lo. hi, and ptr members. Creating a pointer via the big-safe version of malloc(N) produces a bsafe_ptr containing the ptr address of the new storage, a lo equal to ptr, and a hi of N/ sizeof(T). Casting a bsafe_ptr, for example from char* to T*, converts ptr and lo to T* values; if ptr or lo are not properly aligned, an ss_unwind is raised.
[00197] Pointer arithmetic (ptr+n, ptr-n, ptr-ptr, ptr[n]) and fetch-and-store indirection operators (explicitly via "star" or implicitly in the lvalue use of subscript) are checked and diagnosed according to the rules as described in section "Safe pointers [buf]", with the obvious inclusion of run-time checking against the values store in the big-safe-pointer.
[00198] Note that the first member of a bsafe_ptr is the pointer value ptr. This is intentional and required; any non-BSAFE code which refers to a bsafe_ptr as an ordinary pointer will properly address the pointer value.
[00199] In the big-safe-pointer alternative, all external pointers have bsafe_ptr representation. In particular, any system-defined external pointers (such as environ) have bsafe_ptr representation. (The cost is only two extra words per external pointer.)
[00200] There are a few application niches which might conceivably use only big-safe pointers, such as the training of programmers, the creation of user-provided callback functions, the production of specialized "one-off prototypes, etc. All are areas that might be willing to trade some time and space for greatly increased reliability. However, most application domains are using C and C++ because of superior intrinsic efficiency, and the overheads of big-safe pointers would be commercially unsupportable; this favors the compile-time-safe pointer methods as described earlier.
[00201] Note that the printf/scanf functions use pointer arguments for the %s and %p formats. Therefore, although the old "unsafe" versions can be called via a "wrapper" in the CSAFE mode, the wrapper must indicate by a state flag that p input produces a "not- indirectable" pointer in the CSAFE mode. The fully bounds-checked versions can be provided in BSAFE mode
[00202] Including the big-safe-pointer alternative, the Safe Secure method has defined four modes of code generation: Debug BSAFE, Debug CSAFE, non-Debug BSAFE, and non-Debug CSAFE. A further alternative is to reduce the modes to two: Debug BSAFE and non-Debug CSAFE.
Implementing the SSBDF using name-mangling fmanglj
[00203] One non-limiting method for implementing the SSBDF 26 makes use of the name-mangling facilities of C++, employing appropriate encodings in the object-file representation to convey the bounds information for each fetch-and-store variable in each function's calling sequence. The parameter type can be encoded into the apparent classname of the pointer argument:
Figure imgf000064_0001
[00204] Using this system, the object code for memcpy would embody encodings such as these: csafe_ptr_Tal<void> memcpy( csafe_ptr_Tal<void> si, csafe_ptr_Tal<void> s2, size_t n);
[00205] Parameters which are not pointers may be involved in fetch-or-store expressions in the called function, so a similar encoding is provided for non-pointers:
Figure imgf000064_0002
[00206] Sometimes within a Source Program 107 an integer parameter is used within the called function to specify the bounds, or subscript, for several different external arrays; the SSSA 35 knows which arrays those are from examination of the code in the called function. Rather than encoding all those array names into the parameter's augmented-type, the array with the smallest bound should be chosen; if two or more have this same bound, choose the shortest name; if two ore more names have the same length, choose the lexicographically smallest name. Here is an example that illustrates the intended usage:
Figure imgf000065_0001
File 2: int a[64] = {0}; int main() { intj; for (j = 0;j < 64; ++j) f(i); 1 [00207] The augmented signature of the function f is as follows: void f( csafe_n_Iela<int>) [00208] This presents a Requirement for the invocation of f in File 2, namely, that the argument must be suitable for use as a subscript on array a, and informs the main program that the variable j is being used for fetch-or-store access to a. The main program must determine the range of values for j and verify that j is suitable as a subscript on array a.
Since the names of these "helper" classes are intended only for consumption by the Safe Secure Compiler 23 itself, there is no need for human-readable names in this context. In the preferred implementation, an alternative encoding is produced by reducing the length of the names of the " csafe" classes while avoiding collision with implementation-defined names in the implementer' s namespace. The parsing of the class names as part of a mangled name requires no lookahead or added terminators. When integers are permitted in the name, a single-digit integer is encoded as-is, and an N-digit integer is prefixed by N-l underscore characters. When an identifier is encoded, it is prefixed by the N-digit integer designating the length of the identifier (without any leading underscore characters). [00209] Notatmg an attribute as a post-condition (shown as POST earlier) is achieved by adding a marker such as 'P' in the augmented name-mangling method:
int swprintf( csafe_ptr_NTBalPLaO<wcharj;> restrict s, size_t n , csafe_ptτ_NT<const wchar_t> restrict format. ...);
[00210] Note that in the Guarantees, the returned value is indicated as "argument number zero", with digit zero.
[00211] In CSAFE mode, any pointer argument can have the "not-indirectable" attribute. This attribute is the default attribute for pointer parameters. This attribute is explicitly documented via the template type csafe_ptr_NI<T> ("csafe pointer, not- indirectable"), or more simply as an ordinary T* pointer.
Φ Φ ^ % Φ *
[00212] All documents referred to above are incorporated herein by reference.
[00213] While the technology herein has been described in connection with exemplary illustrative non-limiting embodiments, the invention is not to be limited by the disclosure. For example, while specific examples above relate to specific programming languages, the techniques herein could be used for any of a variety of other programming languages and other machine-performed operations. Also, while examples above are presented in the context of software code generation (e.g., using compilers, cross-compilers, linkers, etc.), the techniques herein can be applied in various other machine implementation and other contexts including but not limited to software test and/or verification; generation and/or test and/or verification of hardware-based and/or software/hardware hybrid implementations; code interpreters; instruction disassemblers and other analyzers; and other contexts The invention is intended to be defined by the claims and to cover all corresponding and equivalent arrangements whether or not specifically disclosed herein.

Claims

I CLAM:
1. A software development method comprising: receiving a C or C++ input source file; compiling said input source file into an executable object file, wherein said compiling automatically identifies and/or ehminates, with commercially acceptable efficiency, all undefined behaviors of said input source file.
2. The method of claim 1 wherein said compiling provides object files or fatal diagnostic messages.
3. The method of claim 1 further including processing an inputs check-list which records checksum information for the inputs to the execution of said compiling.
4. The method of claim 1 further including maintaining a bounds data file which records requirements and guarantees for the defined and undefined symbols in one or more corresponding object files, as well as checksum information.
5. The method of claim 4 wherein said compiling includes linking or combining object files and corresponding bounds data files, producing fatal link-time diagnostics and/or an executable program.
6. The method of claim 1 wherein said compiling includes using a parse tree to determine requirements and guarantees.
7. The method of claim 1 wherein said compiling includes generating fatal diagnostic messages in situations where undefined behavior would result.
8. The method of claim 1 wherein said compiling includes generating warning messages to call attention to risky situations.
9. The method of claim 1 wherein said compiling includes generating object code which is free from the designated sets of undefined behaviors including buffer overflow and null pointer indirection.
10. The method of claim 1 further including using a pointer attribute hierarchy to control the inference of attributes based upon other attributes.
11. The method of claim 1 further including using a pointer attribute predicate data structure to control the determination of attributes resulting from predicate expressions.
12. The method of claim 1 further including generating a bounds data data structure which tabulates guarantees and requirements for expressions, sub-expressions, declarations, identifiers, and function prototypes.
13. The method of claim 1 further includes generating an interface inference data structure which controls the inference of requirements on the interface of each externally- callable function.
14. The method of claim 1 further including developing a bounds data symbol data structure which tabulates the requirements and guarantees for defined and undefined symbols during the Unking process.
15. The method of claim 1 further including performing link-time analysis which matches requirements to guarantees for function-call, external array, and external pointer linkage contexts.
16. The method of claim 1 further including performing link diagnosis to generate a fatal diagnostic at link-time if any requirement is unsatisfied, preventing the production of any executable program.
17. A safe secure software development system comprising: a storage device storing a C or C++ input source file; and a processor coupled to said storage device, said processor performing safe secure compilation to convert said input source file, with commercially acceptable efficiency, into an executable object file, wherein said safe secure compilation automatically identifies and/or eliminates all undefined behaviors within said input source file.
18. The system of claim 17 wherein processor-performed compilation provides safe secure object files or fatal diagnostic messages.
19. The system of claim 17 wherein said processor processes a safe secure inputs check-list which records checksum information for the inputs to the execution of said safe secure compilation.
20. The system of claim 17 wherein said processor maintains a safe secure bounds data file which records requirements and guarantees for the defined and undefined symbols in one or more corresponding object files, as well as checksum information.
21. The system of claim 20 wherein said safe secure compilation includes safe secure linking which combines object files and corresponding safe secure bounds data files, producing either fatal link-time diagnostics or a safe secure executable program.
22. The system of claim 17 wherein said safe secure compilation includes safe secure semantic analysis which uses the parse tree to determine requirements and guarantees.
23. The system of claim 17 wherein said safe secure compilation includes safe secure diagnostic generation which generates fatal diagnostic messages in situations where undefined behavior would result and generates various warning messages to call a programmer's attention to various other situations.
24. The system of claim 17 wherein said safe secure compilation includes safe secure code generation of object code which is free from the designated sets of undefined behaviors including "buffer overflow" and "null pointer indirection".
25. The system of claim 17 wherein said processor maintains a safe secure pointer attribute hierarchy which controls the inference of attributes based upon other attributes.
26. The system of claim 17 wherein said processor maintains and updates a safe secure pointer attribute predicate table which controls the determination of attributes resulting from predicate expressions.
27. The system of claim 17 wherein said processor develops a safe secure bounds data table which tabulates the guarantees and requirements for expressions, sub-expressions, declarations, identifiers, and function prototypes.
28. The system of claim 17 wherein said processor develops a safe secure interface inference table which controls the inference of requirements on the interface of each externally-callable function.
29. The system of claim 17 wherein said processor develops a safe secure bounds data symbol table which tabulates the requirements and guarantees for defined and undefined symbols during the safe secure linking process.
30. The system of claim 17 wherein said processor performs safe secure link-time analysis which matches requirements to guarantees for function-call, external array, and external pointer linkage contexts.
31. The system of claim 17 wherein said processor performs safe secure link diagnosis to generate a fatal diagnostic at Unk-time if any requirement is unsatisfied, preventing the production of any executable program.
32. A method for developing operations to be automatically performed by a machine, comprising: (a) checking whether fetch and store operations fall outside boundaries associated with an object; and (b) generating fetch and store operations such that any fetch'from outside said Umits produces a zero value and any store outside said Umits produces a no-operation (NOP).
33. A storage device storing: a C or C++ input source file; and a translator that, when excuted, automatically identifies and/or ehminates substantially all undefined behaviors of said input source file.
PCT/US2004/034422 2003-10-17 2004-10-18 Automated safe secure techniques for eliminating undefined behavior in computer software WO2005038630A2 (en)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US51164903P 2003-10-17 2003-10-17
US60/511,649 2003-10-17
PCT/US2004/030029 WO2005029241A2 (en) 2003-09-15 2004-09-15 Automated safe secure techniques for eliminating
USPCT/US/04/30029 2004-09-15
US10/964,830 2004-10-15
US10/964,830 US7584461B2 (en) 2003-09-15 2004-10-15 Automated safe secure techniques for eliminating undefined behavior in computer software

Publications (2)

Publication Number Publication Date
WO2005038630A2 true WO2005038630A2 (en) 2005-04-28
WO2005038630A3 WO2005038630A3 (en) 2006-05-11

Family

ID=34468372

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2004/034422 WO2005038630A2 (en) 2003-10-17 2004-10-18 Automated safe secure techniques for eliminating undefined behavior in computer software

Country Status (1)

Country Link
WO (1) WO2005038630A2 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6149318A (en) * 1997-04-15 2000-11-21 Samuel C. Kendall Link-time and run-time error detection, and program instrumentation
US6542990B1 (en) * 2000-05-04 2003-04-01 Sun Microsystems, Inc. Array access boundary check by executing BNDCHK instruction with comparison specifiers
US6634023B1 (en) * 1998-06-18 2003-10-14 International Business Machines Corporation Compile method, exception handling method and computer

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6149318A (en) * 1997-04-15 2000-11-21 Samuel C. Kendall Link-time and run-time error detection, and program instrumentation
US6634023B1 (en) * 1998-06-18 2003-10-14 International Business Machines Corporation Compile method, exception handling method and computer
US6542990B1 (en) * 2000-05-04 2003-04-01 Sun Microsystems, Inc. Array access boundary check by executing BNDCHK instruction with comparison specifiers

Also Published As

Publication number Publication date
WO2005038630A3 (en) 2006-05-11

Similar Documents

Publication Publication Date Title
US7584461B2 (en) Automated safe secure techniques for eliminating undefined behavior in computer software
US7856624B2 (en) Automated safe secure techniques for eliminating undefined behavior in computer software
US7810080B2 (en) Automated safe secure techniques for eliminating undefined behavior in computer software
US7818729B1 (en) Automated safe secure techniques for eliminating undefined behavior in computer software
US6993751B2 (en) Placing exception throwing instructions in compiled code
US5907709A (en) Development system with methods for detecting invalid use and management of resources and memory at runtime
US5909580A (en) Development system and methods with direct compiler support for detecting invalid use and management of resources and memory at runtime
Ramanathan et al. Static specification inference using predicate mining
Bodden Efficient hybrid typestate analysis by determining continuation-equivalent states
Stallman Using the gnu compiler collection
US7584458B2 (en) Source code annotation language
Reps et al. Intermediate-representation recovery from low-level code
Lee et al. Jinn: synthesizing dynamic bug detectors for foreign language interfaces
US10423397B2 (en) Systems and/or methods for type inference from machine code
Kell et al. The missing link: explaining ELF static linking, semantically
Burrows et al. Run-time type checking for binary programs
Jeong et al. Utopia: Automatic generation of fuzz driver using unit tests
Chen et al. Hopper: Interpretative fuzzing for libraries
Spinellis et al. A framework for the static verification of API calls
WO2005038630A2 (en) Automated safe secure techniques for eliminating undefined behavior in computer software
Gibson Waddle: A proven interpreter and test framework for a subset of the Go semantics
Li et al. Detecting undefined behaviors in CUDA C
Kwon et al. Translation Validation for JIT Compiler in the V8 JavaScript Engine
Pflanzer et al. Automatic test case reduction of randomly generated OpenCL kernels
Lissy et al. Verifications around the Linux kernel

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): BW GH GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
122 Ep: pct application non-entry in european phase