US20090172664A1 - Adding a profiling agent to a virtual machine to permit performance and memory consumption analysis within unit tests - Google Patents
Adding a profiling agent to a virtual machine to permit performance and memory consumption analysis within unit tests Download PDFInfo
- Publication number
- US20090172664A1 US20090172664A1 US11/968,351 US96835108A US2009172664A1 US 20090172664 A1 US20090172664 A1 US 20090172664A1 US 96835108 A US96835108 A US 96835108A US 2009172664 A1 US2009172664 A1 US 2009172664A1
- Authority
- US
- United States
- Prior art keywords
- virtual machine
- memory
- performance
- agent
- framework
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3089—Monitoring arrangements determined by the means or processing involved in sensing the monitored data, e.g. interfaces, connectors, sensors, probes, agents
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/301—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system is a virtual computing platform, e.g. logically partitioned systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3037—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a memory, e.g. virtual memory, cache
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3409—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3409—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
- G06F11/3419—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment by assessing time
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3438—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment monitoring of user actions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3476—Data logging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3604—Software analysis for verifying properties of programs
- G06F11/3612—Software analysis for verifying properties of programs by runtime analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45504—Abstract machines for programme code execution, e.g. Java virtual machine [JVM], interpreters, emulators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
- G06F2201/815—Virtual
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
- G06F2201/865—Monitoring of software
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
- G06F2201/88—Monitoring involving counting
Definitions
- the present invention relates to the field of unit testing of virtual machine components and, more particularly, to adding a profiling agent to a virtual machine to permit performance and memory consumption analysis within unit tests.
- a number of languages like JUNIT, exist expressly for conducting unit tests of software units configured to execute within a virtual machine environment, such as a JAVA VIRTUAL MACHINE (JVM).
- Unit testing can be valuable as it permits one or more software units to be tested throughout a software development cycle by simply executing a unit test program.
- JVM JAVA VIRTUAL MACHINE
- Black and white box testing of units using unit tests generally don't include tests for memory usage and/or performance. This is expressed in the JUNIT language by a lack of memory and/or performance related asserts.
- unit tests include memory and/or performance checks, results are often less than perfect and writing the testing code can be challenging.
- Executing unit tests for memory and performance related information is difficult because needed information from a virtual machine is not currently exposed. For example, in the JUNIT API, there is no API access to the heap, there is no access to object size, there is no memory specific asserts, and there is no control over a garbage collector. Additionally, no information is available to access thread specific timers, to query a method invocation count, and no performance related asserts exist.
- the present invention discloses a solution for adding a profiling agent to a virtual machine, which permits unit test programs to access information relating to memory and performance from runtime data areas of the virtual machine (VM).
- Operations permitted by the agent can include starting/stopping VM monitor, getting objects from the heap, retrieving an invocation count, determining a size of an instantiated object, getting a current thread time, and the like.
- Memory and performance information gathered during a test can be stored for later analysis under a test version identifier.
- a comparison engine can create reports that use the stored performance and memory data, which can help developers compare memory/performance deltas among different unit versions.
- the VM can be a JAVA VIRTUAL MACHINE (JVM) and the unit test programs can be JUNIT programs. Further, JUNIT can be extended to include one or more asserts related to memory and/or performance information exposes through the agent.
- JVM JAVA VIRTUAL MACHINE
- JUNIT can be extended to include one or more asserts related to memory and
- one aspect of the present invention can include a system for performing unit testing that includes a virtual machine, a profiling agent, and a unit testing framework.
- the virtual machine can be configured to execute software units.
- the unit testing framework can be configured to test software units executing within the virtual machine.
- the profiling agent can be an agent of the virtual machine that exposes memory and performance information from runtime data areas of the virtual machine to the unit testing framework, which would not otherwise be exposed.
- the unit testing framework can execute a set of methods relating to memory and performance of the runtime data areas, where the set of methods interact with the profiling agent.
- Another aspect of the present invention can include an extension to a unit testing framework that includes a set of published methods available for unit testing purposes relating to memory and/or performance aspects of an application virtual machine. These memory and performance aspects can be gathered from runtime data areas of the application virtual machine by a profiling agent.
- the published methods can include any or all of the following: a get heap method, a get invocation count method, a get size of object method, a get current thread time method, and a get data model method.
- Still another aspect of the present invention can include a method for exposing runtime data areas of a virtual machine to a unit testing framework.
- the method can include a step of placing a profiling agent within an application virtual machine that has access to runtime data areas of the application virtual machine.
- An interface class can be established for exchanging information with the profiling agent.
- the interface class can include a set of public methods available to a unit testing framework. The public methods can provide memory and performance related information concerning objects in a heap of the application virtual machine and threads executing within the application virtual machine.
- various aspects of the invention can be implemented as a program for controlling computing equipment to implement the functions described herein, or a program for enabling computing equipment to perform processes corresponding to the steps disclosed herein.
- This program may be provided by storing the program in a magnetic disk, an optical disk, a semiconductor memory, any other recording medium, or can also be provided as a digitally encoded signal conveyed via a carrier wave.
- the described program can be a single program or can be implemented as multiple subprograms, each of which interact within a single computing device or interact in a distributed fashion across a network space.
- FIG. 1 is a schematic diagram illustrating a system that uses a profiling agent to expose memory and performance information of a virtual machine (VM) to a test unit framework in accordance with an embodiment of the inventive arrangements disclosed herein.
- VM virtual machine
- FIG. 2 is a schematic diagram of an implementation for an agent that exposes virtual machine information to a unit testing framework.
- FIG. 3 is a flow chart showing a method in which a profiling agent is able to expose memory and performance information concerning a VM to unit test programs in accordance with an embodiment of the inventive arrangements disclosed herein.
- FIG. 4 is a schematic for a sample model class diagram, such as a model conforming to an ECLIPSE MODEL FRAMEWORK (EMF) data model as described in FIG. 2 .
- EMF ECLIPSE MODEL FRAMEWORK
- FIG. 5 shows JUNIT extensions for memory and performance in accordance with an embodiment of the inventive arrangements disclosed herein.
- FIG. 6 shows a sample differences report able to be generated by comparing differences between different unit tests of a common software object executing in a VM in accordance with an embodiment of the inventive arrangements disclosed herein.
- FIG. 1 is a schematic diagram illustrating a system 100 that uses a profiling agent 112 to expose memory and performance information of a virtual machine (VM) 110 to a test unit framework in accordance with an embodiment of the inventive arrangements disclosed herein.
- the profiling agent 112 can access information from runtime data areas 120 of the VM 110 , such as a heap 122 and objects 124 contained therein.
- the agent 112 can either trigger a garbage collector 126 to perform collection actions or can prevent normally occurring actions from executing.
- the agent 112 can access thread information 129 from stacks and/or registers 128 of the VM 100 . All of this information is made available to an agent front end 140 , which can be an interface class in an object oriented implementation of system 100 .
- the agent front-end 140 can include a plurality of public methods for obtaining information from the runtime data areas 120 and for initiating actions involving the runtime data areas 120 . These methods can include, but are not limited to, start/stop methods 142 , a get heap method 143 , a get invocation count 144 method, a get size of object 145 method, a get current thread time 146 method, a get data model 147 method, and the like. Each of the methods 142 - 147 can be public methods able to be accessed from within xUnit 160 framework.
- the start/stop 142 methods can start/stop a monitoring of memory and/or performance involving agent 112 .
- Get heap 143 can force the garbage collector 126 to start and can then give access to all objects 124 in the heap 122 .
- Get invocation count 144 can return how many times a specific method has been called during a monitoring period.
- One implementation of get size of object 145 can return a shallow size of an object 124 . This result can depend upon the virtual machine 110 and the operating system.
- a different implementation of get size of object 145 can return a retained size of a specified object 124 .
- Get current thread time 146 can return a current thread CPU time.
- Get data model 147 can return a set of optionally processed run data 132 from the data store 130 .
- a compare engine 152 can be a software engine that compares data collected across different runs 132 .
- a report engine 154 can generate customizable reports based upon results of the compare engine 152 .
- a data model can be established that represents a set of collected data for a run as well as deltas between that run and one or more previous runs for past versions of the tested unit.
- the xUnit 160 framework can be extended to include one or more asserts 162 - 165 that are related to memory and/or performance of the virtual machine 110 . These asserts 162 - 165 can depend upon actions performed by and data provided by the agent 112 .
- the VM 110 can be a process VM or an application VM, which runs as a normal application inside an operating system and that supports a single process.
- the VM 110 can provide a platform independent programming environment that abstracts away details of the underlying hardware/OS and that permits programs written for the VM 110 to execute in the same way on any platform.
- the VM 110 can include a JAVA VIRTUAL MACHINE, a COMMON LANGUAGE RUNTIME from the .NET FRAMEWORK, and the like.
- the xUnit 160 is a code-driven testing framework, which can include test fixtures, test suites, and test execution.
- xUnit 160 includes JUNIT as well as other sUnit based language ports, such as PHPUnit, NUnit, PyUnit, fUnit, Test::Class, Test::Unit, CPPUnit, and the like.
- FIG. 2 is a schematic diagram 200 of an implementation for an agent that exposes virtual machine information to a unit testing framework. Specifically, diagram 200 presents an implementation for JUNIT based testing of software units within a JAVA VIRTUAL MACHINE 232 . Diagram 200 represents one contemplated implementation of system 100 .
- the profiling agent 230 can be written using a C++ programming language, which can include builds for various operating system platforms, such as WIN32, WIN64, LINUX, etc.
- the agent 230 can be a JVM profiling interface (JVMPI) agent, a JVM tooling interface (JVMTI) agent, and the like.
- JVMPI JVM profiling interface
- JVMTI JVM tooling interface
- the agent 230 can communicate with the JVM 232 , which exposes some of its internal functionality specific to memory and performance analysis.
- the JVM Helper 222 can be a JAVA interface for the C++ agent 230 that enables data to be exchanged between the JAVA and the C++ interfaces.
- API's for the JVM Helper 222 can include the methods analogous to methods 142 - 147 detailed in system 100 .
- Helper can use a Java Native Interface (JNI) 234 to communicate with agent 230 .
- JNI Java Native Interface
- An Eclipse Modeling Framework (EMF) 224 can communicate with JUNIT 220 and the JVM helper 222 .
- THE EMF 224 can include a data model that represents collected data model and the delta model to support comparing data collected across different runs.
- the EMF 224 model can include a compare engine, which accepts data roots and products a delta model that represents the differences.
- FIG. 3 is a flow chart showing a method 300 in which a profiling agent is able to expose memory and performance information concerning a VM to unit test programs in accordance with an embodiment of the inventive arrangements disclosed herein.
- the method 300 can be performed in the context of a system 100 or 200 .
- Method 300 can begin in step 305 , where a profiling agent for a VM can be established.
- memory/performance specific interfaces can be established using an interface front end linked to the profiling agent.
- the front-end can, for example, be an interface class that exposes memory and performance related methods to a test framework.
- a unit test program can execute while the agent is monitoring performance/memory aspects of the VM.
- the unit test program can include one or more method calls to a memory/performance related method, as shown by step 320 .
- the agent can access a runtime data area of the VM and can convey results to the unit test program and/or perform programmatic actions initiated by the unit test program.
- results of the unit test can be presented.
- step 330 a determination can be made as to whether a data capture feature for the test run was enabled. If so, the profiling agent can store all captured memory/performance data related to the test, as shown by step 335 .
- step 340 a determination can be made as to whether another run of the test program is to be executed. When so, the method can progress from step 340 to step 315 where the unit test program can again execute. This execution can be for a different or for the same version of unit software.
- step 345 an option can be provided to compare one or more runs, which can include a newly executed run. If no comparison is opted for, the method can end.
- step 345 the method can progress from step 345 to step 350 , where two or more runs can be identified and retrieved from a storage area, such as the storage area where data from step 335 was stored.
- step 355 a run comparison report based upon the retrieved data can be generated and presented to a requester.
- FIG. 4 is a schematic for a sample model class diagram 400 , such as a model conforming to an ECLIPSE MODEL FRAMEWORK (EMF) data model as described in system 200 .
- the supported deltas 408 shown in model 400 include an add delta 412 , a delete delta 414 , and a change delta 410 .
- Model specifics are variable by implementation and diagram 400 is presented to illustrate a concept.
- the scope of the disclosed invention is not to be limited by specifics shown in diagram 400 in any manner since the disclosed invention is able to utilize any model and not just that detailed in diagram 400 .
- FIG. 5 shows JUNIT extensions for memory 510 and performance 520 in accordance with an embodiment of the inventive arrangements disclosed herein.
- FIG. 6 shows a sample differences report 610 able to be generated by comparing differences between different unit tests of a common software object executing in a VM in accordance with an embodiment of the inventive arrangements disclosed herein.
- the report 610 can represent an HTML report that has been generated from a delta data model (i.e., one of the EMF data models expressed in diagram 400 )
- the present invention may be realized in hardware, software, or a combination of hardware and software.
- the present invention may be realized in a centralized fashion in one computer system or in a distributed fashion where different elements are spread across several interconnected computer systems. Any kind of computer system or other apparatus adapted for carrying out the methods described herein is suited.
- a typical combination of hardware and software may be a general purpose computer system with a computer program that, when being loaded and executed, controls the computer system such that it carries out the methods described herein.
- the present invention also may be embedded in a computer program product, which comprises all the features enabling the implementation of the methods described herein, and which when loaded in a computer system is able to carry out these methods.
- Computer program in the present context means any expression, in any language, code or notation, of a set of instructions intended to cause a system having an information processing capability to perform a particular function either directly or after either or both of the following: a) conversion to another language, code or notation; b) reproduction in a different material form.
Abstract
Description
- 1. Field of the Invention
- The present invention relates to the field of unit testing of virtual machine components and, more particularly, to adding a profiling agent to a virtual machine to permit performance and memory consumption analysis within unit tests.
- 2. Description of the Related Art
- A number of languages, like JUNIT, exist expressly for conducting unit tests of software units configured to execute within a virtual machine environment, such as a JAVA VIRTUAL MACHINE (JVM). Unit testing can be valuable as it permits one or more software units to be tested throughout a software development cycle by simply executing a unit test program. When a software unit requires iterative testing during its lifecycle, an initial cost of constructing a unit test program can be considerably outweighed with long term savings gained throughout the software development cycle.
- Black and white box testing of units using unit tests generally don't include tests for memory usage and/or performance. This is expressed in the JUNIT language by a lack of memory and/or performance related asserts. When unit tests include memory and/or performance checks, results are often less than perfect and writing the testing code can be challenging. Executing unit tests for memory and performance related information is difficult because needed information from a virtual machine is not currently exposed. For example, in the JUNIT API, there is no API access to the heap, there is no access to object size, there is no memory specific asserts, and there is no control over a garbage collector. Additionally, no information is available to access thread specific timers, to query a method invocation count, and no performance related asserts exist.
- The present invention discloses a solution for adding a profiling agent to a virtual machine, which permits unit test programs to access information relating to memory and performance from runtime data areas of the virtual machine (VM). Operations permitted by the agent can include starting/stopping VM monitor, getting objects from the heap, retrieving an invocation count, determining a size of an instantiated object, getting a current thread time, and the like. Memory and performance information gathered during a test can be stored for later analysis under a test version identifier. A comparison engine can create reports that use the stored performance and memory data, which can help developers compare memory/performance deltas among different unit versions. In one embodiment, the VM can be a JAVA VIRTUAL MACHINE (JVM) and the unit test programs can be JUNIT programs. Further, JUNIT can be extended to include one or more asserts related to memory and/or performance information exposes through the agent.
- The present invention can be implemented in accordance with numerous aspects consistent with material presented herein. For example, one aspect of the present invention can include a system for performing unit testing that includes a virtual machine, a profiling agent, and a unit testing framework. The virtual machine can be configured to execute software units. The unit testing framework can be configured to test software units executing within the virtual machine. The profiling agent can be an agent of the virtual machine that exposes memory and performance information from runtime data areas of the virtual machine to the unit testing framework, which would not otherwise be exposed. The unit testing framework can execute a set of methods relating to memory and performance of the runtime data areas, where the set of methods interact with the profiling agent.
- Another aspect of the present invention can include an extension to a unit testing framework that includes a set of published methods available for unit testing purposes relating to memory and/or performance aspects of an application virtual machine. These memory and performance aspects can be gathered from runtime data areas of the application virtual machine by a profiling agent. The published methods can include any or all of the following: a get heap method, a get invocation count method, a get size of object method, a get current thread time method, and a get data model method.
- Still another aspect of the present invention can include a method for exposing runtime data areas of a virtual machine to a unit testing framework. The method can include a step of placing a profiling agent within an application virtual machine that has access to runtime data areas of the application virtual machine. An interface class can be established for exchanging information with the profiling agent. The interface class can include a set of public methods available to a unit testing framework. The public methods can provide memory and performance related information concerning objects in a heap of the application virtual machine and threads executing within the application virtual machine.
- It should be noted that various aspects of the invention can be implemented as a program for controlling computing equipment to implement the functions described herein, or a program for enabling computing equipment to perform processes corresponding to the steps disclosed herein. This program may be provided by storing the program in a magnetic disk, an optical disk, a semiconductor memory, any other recording medium, or can also be provided as a digitally encoded signal conveyed via a carrier wave. The described program can be a single program or can be implemented as multiple subprograms, each of which interact within a single computing device or interact in a distributed fashion across a network space.
- There are shown in the drawings, embodiments which are presently preferred, it being understood, however, that the invention is not limited to the precise arrangements and instrumentalities shown.
-
FIG. 1 is a schematic diagram illustrating a system that uses a profiling agent to expose memory and performance information of a virtual machine (VM) to a test unit framework in accordance with an embodiment of the inventive arrangements disclosed herein. -
FIG. 2 is a schematic diagram of an implementation for an agent that exposes virtual machine information to a unit testing framework. -
FIG. 3 is a flow chart showing a method in which a profiling agent is able to expose memory and performance information concerning a VM to unit test programs in accordance with an embodiment of the inventive arrangements disclosed herein. -
FIG. 4 is a schematic for a sample model class diagram, such as a model conforming to an ECLIPSE MODEL FRAMEWORK (EMF) data model as described inFIG. 2 . -
FIG. 5 shows JUNIT extensions for memory and performance in accordance with an embodiment of the inventive arrangements disclosed herein. -
FIG. 6 shows a sample differences report able to be generated by comparing differences between different unit tests of a common software object executing in a VM in accordance with an embodiment of the inventive arrangements disclosed herein. -
FIG. 1 is a schematic diagram illustrating asystem 100 that uses aprofiling agent 112 to expose memory and performance information of a virtual machine (VM) 110 to a test unit framework in accordance with an embodiment of the inventive arrangements disclosed herein. Theprofiling agent 112 can access information fromruntime data areas 120 of theVM 110, such as aheap 122 andobjects 124 contained therein. Theagent 112 can either trigger agarbage collector 126 to perform collection actions or can prevent normally occurring actions from executing. Additionally, theagent 112 can accessthread information 129 from stacks and/or registers 128 of theVM 100. All of this information is made available to anagent front end 140, which can be an interface class in an object oriented implementation ofsystem 100. - The agent front-
end 140 can include a plurality of public methods for obtaining information from theruntime data areas 120 and for initiating actions involving theruntime data areas 120. These methods can include, but are not limited to, start/stop methods 142, a getheap method 143, aget invocation count 144 method, a get size of object 145 method, a get current thread time 146 method, a get data model 147 method, and the like. Each of the methods 142-147 can be public methods able to be accessed from withinxUnit 160 framework. - The start/stop 142 methods can start/stop a monitoring of memory and/or
performance involving agent 112. Get heap 143 can force thegarbage collector 126 to start and can then give access to allobjects 124 in theheap 122. Getinvocation count 144 can return how many times a specific method has been called during a monitoring period. One implementation of get size of object 145 can return a shallow size of anobject 124. This result can depend upon thevirtual machine 110 and the operating system. A different implementation of get size of object 145 can return a retained size of aspecified object 124. Get current thread time 146 can return a current thread CPU time. Get data model 147 can return a set of optionally processedrun data 132 from thedata store 130. - Each time a unit test program is run, the
agent 112 can monitor memory and/or performance. This information can be stored withindata store 130 as rundata 132 along with a run version identifier. A compareengine 152 can be a software engine that compares data collected across different runs 132. Areport engine 154 can generate customizable reports based upon results of the compareengine 152. In one embodiment, a data model can be established that represents a set of collected data for a run as well as deltas between that run and one or more previous runs for past versions of the tested unit. - In addition to being able to execute the exposed methods 142-147, the
xUnit 160 framework can be extended to include one or more asserts 162-165 that are related to memory and/or performance of thevirtual machine 110. These asserts 162-165 can depend upon actions performed by and data provided by theagent 112. - In
system 100, theVM 110 can be a process VM or an application VM, which runs as a normal application inside an operating system and that supports a single process. TheVM 110 can provide a platform independent programming environment that abstracts away details of the underlying hardware/OS and that permits programs written for theVM 110 to execute in the same way on any platform. TheVM 110 can include a JAVA VIRTUAL MACHINE, a COMMON LANGUAGE RUNTIME from the .NET FRAMEWORK, and the like. - The
xUnit 160 is a code-driven testing framework, which can include test fixtures, test suites, and test execution.xUnit 160 includes JUNIT as well as other sUnit based language ports, such as PHPUnit, NUnit, PyUnit, fUnit, Test::Class, Test::Unit, CPPUnit, and the like. -
FIG. 2 is a schematic diagram 200 of an implementation for an agent that exposes virtual machine information to a unit testing framework. Specifically, diagram 200 presents an implementation for JUNIT based testing of software units within a JAVAVIRTUAL MACHINE 232. Diagram 200 represents one contemplated implementation ofsystem 100. - In diagram 200, the
profiling agent 230 can be written using a C++ programming language, which can include builds for various operating system platforms, such as WIN32, WIN64, LINUX, etc. Theagent 230 can be a JVM profiling interface (JVMPI) agent, a JVM tooling interface (JVMTI) agent, and the like. Theagent 230 can communicate with theJVM 232, which exposes some of its internal functionality specific to memory and performance analysis. - The
JVM Helper 222 can be a JAVA interface for theC++ agent 230 that enables data to be exchanged between the JAVA and the C++ interfaces. API's for theJVM Helper 222 can include the methods analogous to methods 142-147 detailed insystem 100. Helper can use a Java Native Interface (JNI) 234 to communicate withagent 230. - An Eclipse Modeling Framework (EMF) 224 can communicate with
JUNIT 220 and theJVM helper 222. THEEMF 224 can include a data model that represents collected data model and the delta model to support comparing data collected across different runs. TheEMF 224 model can include a compare engine, which accepts data roots and products a delta model that represents the differences. -
FIG. 3 is a flow chart showing amethod 300 in which a profiling agent is able to expose memory and performance information concerning a VM to unit test programs in accordance with an embodiment of the inventive arrangements disclosed herein. Themethod 300 can be performed in the context of asystem -
Method 300 can begin instep 305, where a profiling agent for a VM can be established. Instep 310, memory/performance specific interfaces can be established using an interface front end linked to the profiling agent. The front-end can, for example, be an interface class that exposes memory and performance related methods to a test framework. Instep 315, a unit test program can execute while the agent is monitoring performance/memory aspects of the VM. The unit test program can include one or more method calls to a memory/performance related method, as shown bystep 320. The agent can access a runtime data area of the VM and can convey results to the unit test program and/or perform programmatic actions initiated by the unit test program. Instep 325, results of the unit test can be presented. - In
step 330, a determination can be made as to whether a data capture feature for the test run was enabled. If so, the profiling agent can store all captured memory/performance data related to the test, as shown bystep 335. Instep 340, a determination can be made as to whether another run of the test program is to be executed. When so, the method can progress fromstep 340 to step 315 where the unit test program can again execute. This execution can be for a different or for the same version of unit software. Instep 345, an option can be provided to compare one or more runs, which can include a newly executed run. If no comparison is opted for, the method can end. Otherwise, the method can progress fromstep 345 to step 350, where two or more runs can be identified and retrieved from a storage area, such as the storage area where data fromstep 335 was stored. Instep 355, a run comparison report based upon the retrieved data can be generated and presented to a requester. -
FIG. 4 is a schematic for a sample model class diagram 400, such as a model conforming to an ECLIPSE MODEL FRAMEWORK (EMF) data model as described insystem 200. The supporteddeltas 408 shown inmodel 400 include anadd delta 412, adelete delta 414, and achange delta 410. - Model specifics are variable by implementation and diagram 400 is presented to illustrate a concept. The scope of the disclosed invention is not to be limited by specifics shown in diagram 400 in any manner since the disclosed invention is able to utilize any model and not just that detailed in diagram 400.
-
FIG. 5 shows JUNIT extensions formemory 510 andperformance 520 in accordance with an embodiment of the inventive arrangements disclosed herein. -
FIG. 6 shows a sample differences report 610 able to be generated by comparing differences between different unit tests of a common software object executing in a VM in accordance with an embodiment of the inventive arrangements disclosed herein. In one embodiment, thereport 610 can represent an HTML report that has been generated from a delta data model (i.e., one of the EMF data models expressed in diagram 400) - The present invention may be realized in hardware, software, or a combination of hardware and software. The present invention may be realized in a centralized fashion in one computer system or in a distributed fashion where different elements are spread across several interconnected computer systems. Any kind of computer system or other apparatus adapted for carrying out the methods described herein is suited. A typical combination of hardware and software may be a general purpose computer system with a computer program that, when being loaded and executed, controls the computer system such that it carries out the methods described herein.
- The present invention also may be embedded in a computer program product, which comprises all the features enabling the implementation of the methods described herein, and which when loaded in a computer system is able to carry out these methods. Computer program in the present context means any expression, in any language, code or notation, of a set of instructions intended to cause a system having an information processing capability to perform a particular function either directly or after either or both of the following: a) conversion to another language, code or notation; b) reproduction in a different material form.
- This invention may be embodied in other forms without departing from the spirit or essential attributes thereof. Accordingly, reference should be made to the following claims, rather than to the foregoing specification, as indicating the scope of the invention.
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/968,351 US9727436B2 (en) | 2008-01-02 | 2008-01-02 | Adding a profiling agent to a virtual machine to permit performance and memory consumption analysis within unit tests |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/968,351 US9727436B2 (en) | 2008-01-02 | 2008-01-02 | Adding a profiling agent to a virtual machine to permit performance and memory consumption analysis within unit tests |
Publications (2)
Publication Number | Publication Date |
---|---|
US20090172664A1 true US20090172664A1 (en) | 2009-07-02 |
US9727436B2 US9727436B2 (en) | 2017-08-08 |
Family
ID=40800297
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/968,351 Expired - Fee Related US9727436B2 (en) | 2008-01-02 | 2008-01-02 | Adding a profiling agent to a virtual machine to permit performance and memory consumption analysis within unit tests |
Country Status (1)
Country | Link |
---|---|
US (1) | US9727436B2 (en) |
Cited By (30)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20100131803A1 (en) * | 2008-11-25 | 2010-05-27 | Express Logic | Computation of stack usage in embedded computing systems |
US20100153924A1 (en) * | 2008-12-16 | 2010-06-17 | Cadence Design Systems, Inc. | Method and System for Performing Software Verification |
US20120110573A1 (en) * | 2010-11-03 | 2012-05-03 | International Business Machines Corporation | Transferring objects between virtual machines |
US20130007720A1 (en) * | 2011-06-30 | 2013-01-03 | International Business Machines Corporation | Dynamic Isolated Debugging via Class Instrumentation |
CN102929693A (en) * | 2012-09-18 | 2013-02-13 | 杭州华为数字技术有限公司 | Performance evaluation method and device for servers of whole equipment cabinet |
US20130110578A1 (en) * | 2011-05-31 | 2013-05-02 | Software Ag | Systems and/or methods for identifying service candidates based on service identification indicators and associated algorithms |
CN103377124A (en) * | 2012-04-25 | 2013-10-30 | 腾讯科技(深圳)有限公司 | Testing method and device |
US20140237550A1 (en) * | 2009-11-25 | 2014-08-21 | Novell, Inc. | System and method for intelligent workload management |
WO2014165092A1 (en) * | 2013-03-12 | 2014-10-09 | Netflix, Inc. | Using canary instances for software analysis |
US20150324227A1 (en) * | 2014-05-12 | 2015-11-12 | Netapp, Inc. | Techniques for virtual machine migration |
US20160026496A1 (en) * | 2010-07-02 | 2016-01-28 | Code Systems Corporation | Method and system for profiling virtual application resource utilization patterns |
CN105808421A (en) * | 2014-12-31 | 2016-07-27 | 中兴通讯股份有限公司 | White box test realization method and test control end |
US9471237B1 (en) | 2015-02-04 | 2016-10-18 | Amazon Technologies, Inc. | Memory consumption tracking |
US9569286B2 (en) | 2010-01-29 | 2017-02-14 | Code Systems Corporation | Method and system for improving startup performance and interoperability of a virtual application |
US9582219B2 (en) | 2013-03-12 | 2017-02-28 | Netapp, Inc. | Technique for rapidly converting between storage representations in a virtualized computing environment |
US9626237B2 (en) | 2010-04-17 | 2017-04-18 | Code Systems Corporation | Method of hosting a first application in a second application |
CN107092541A (en) * | 2017-04-27 | 2017-08-25 | 郑州云海信息技术有限公司 | A kind of method that utilization virtual machine carries out storage performance test |
US9749393B2 (en) | 2010-01-27 | 2017-08-29 | Code Systems Corporation | System for downloading and executing a virtual application |
US9747425B2 (en) | 2010-10-29 | 2017-08-29 | Code Systems Corporation | Method and system for restricting execution of virtual application to a managed process environment |
US9773017B2 (en) | 2010-01-11 | 2017-09-26 | Code Systems Corporation | Method of configuring a virtual application |
US9779111B2 (en) | 2008-08-07 | 2017-10-03 | Code Systems Corporation | Method and system for configuration of virtualized software applications |
US9817592B1 (en) | 2016-04-27 | 2017-11-14 | Netapp, Inc. | Using an intermediate virtual disk format for virtual disk conversion |
US9864600B2 (en) | 2008-08-07 | 2018-01-09 | Code Systems Corporation | Method and system for virtualization of software applications |
CN107577519A (en) * | 2017-09-18 | 2018-01-12 | 郑州云海信息技术有限公司 | A kind of virtual machine pressure method and device |
US10110663B2 (en) | 2010-10-18 | 2018-10-23 | Code Systems Corporation | Method and system for publishing virtual applications to a web server |
US10176068B2 (en) * | 2016-11-29 | 2019-01-08 | Keysight Technologies Singapore (Holdings) Pte. Ltd. | Methods, systems, and computer readable media for token based message capture |
US10216531B2 (en) | 2014-05-12 | 2019-02-26 | Netapp, Inc. | Techniques for virtual machine shifting |
US10409627B2 (en) | 2010-01-27 | 2019-09-10 | Code Systems Corporation | System for downloading and executing virtualized application files identified by unique file identifiers |
US10515216B2 (en) * | 2017-06-30 | 2019-12-24 | Paypal, Inc. | Memory layout based monitoring |
US11080603B2 (en) * | 2018-05-18 | 2021-08-03 | Google Llc | Systems and methods for debugging neural networks with coverage guided fuzzing |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11256590B1 (en) | 2018-10-05 | 2022-02-22 | Cisco Technology, Inc. | Agent profiler to monitor activities and performance of software agents |
US20230351022A1 (en) * | 2022-05-02 | 2023-11-02 | Cisco Technology, Inc. | Systems and Methods for Merging Performance and Security into a Unit Testing Environment |
Citations (25)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030105777A1 (en) * | 2001-11-21 | 2003-06-05 | Seidl Matthew L. | Method and apparatus to facilitate testing of garbage collection implementations |
US20040133895A1 (en) * | 2002-12-20 | 2004-07-08 | Bea Systems, Inc. | System and method for memory leak detection in a virtual machine environment |
US20050114844A1 (en) * | 2003-11-24 | 2005-05-26 | International Business Machines Corporation | Method and apparatus for generating data for use in memory leak detection |
US20050204342A1 (en) * | 2004-03-11 | 2005-09-15 | International Business Machines Corp. | Method, system and article for detecting memory leaks in Java software |
US20050223362A1 (en) * | 2004-04-02 | 2005-10-06 | Gemstone Systems, Inc. | Methods and systems for performing unit testing across multiple virtual machines |
US20050240641A1 (en) * | 2003-05-09 | 2005-10-27 | Fujitsu Limited | Method for predicting and avoiding danger in execution environment |
US20060085156A1 (en) * | 2004-10-18 | 2006-04-20 | Kolawa Adam K | Method and system for predicting memory leaks from unit testing |
US7086064B1 (en) * | 2000-05-27 | 2006-08-01 | International Business Machines Corporation | Performance profiling tool |
US20060195823A1 (en) * | 2005-02-28 | 2006-08-31 | Sap Ag. | Memory debugging tool |
US20060206885A1 (en) * | 2005-03-10 | 2006-09-14 | Seidman David I | Identifying memory leaks in computer systems |
US20060247907A1 (en) * | 2005-04-29 | 2006-11-02 | Microsoft Corporation | Deciding assertions in programs with references |
US20060277440A1 (en) * | 2005-06-02 | 2006-12-07 | International Business Machines Corporation | Method, system, and computer program product for light weight memory leak detection |
US7185367B2 (en) * | 1999-05-11 | 2007-02-27 | Cylant, Inc. | Method and system for establishing normal software system behavior and departures from normal behavior |
US20070074170A1 (en) * | 2005-09-09 | 2007-03-29 | Rossmann Paul A | Application monitoring using profile points |
US20070083854A1 (en) * | 2005-10-11 | 2007-04-12 | Dietrich Mayer-Ullmann | Testing usability of a software program |
US20070185984A1 (en) * | 2005-09-27 | 2007-08-09 | Bea Systems, Inc. | System and method for issue severity handling for performance test |
US7266808B2 (en) * | 2001-08-10 | 2007-09-04 | Parasoft Corporation | Method and system for dynamically invoking and/or checking conditions of a computer test program |
US7293260B1 (en) * | 2003-09-26 | 2007-11-06 | Sun Microsystems, Inc. | Configuring methods that are likely to be executed for instrument-based profiling at application run-time |
US20080244546A1 (en) * | 2007-03-30 | 2008-10-02 | Sap Ag | Method and system for providing on-demand profiling infrastructure for profiling at virtual machines |
US20080244525A1 (en) * | 2007-03-29 | 2008-10-02 | Microsoft Corporation | Test Automation Using Virtual Machines |
US20090125891A1 (en) * | 2007-11-13 | 2009-05-14 | International Business Machines Corporation | Method and system for monitoring code change impact on software performance |
US7620856B2 (en) * | 2006-12-28 | 2009-11-17 | Sap Portals Israel Ltd. | Framework for automated testing of enterprise computer systems |
US7721154B1 (en) * | 2006-09-05 | 2010-05-18 | Parasoft Corporation | System and method for software run-time testing |
US7793269B2 (en) * | 2005-02-15 | 2010-09-07 | Ebay Inc. | Parallel software testing based on a normalized configuration |
US7908590B1 (en) * | 2006-03-02 | 2011-03-15 | Parasoft Corporation | System and method for automatically creating test cases through a remote client |
-
2008
- 2008-01-02 US US11/968,351 patent/US9727436B2/en not_active Expired - Fee Related
Patent Citations (25)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7185367B2 (en) * | 1999-05-11 | 2007-02-27 | Cylant, Inc. | Method and system for establishing normal software system behavior and departures from normal behavior |
US7086064B1 (en) * | 2000-05-27 | 2006-08-01 | International Business Machines Corporation | Performance profiling tool |
US7266808B2 (en) * | 2001-08-10 | 2007-09-04 | Parasoft Corporation | Method and system for dynamically invoking and/or checking conditions of a computer test program |
US20030105777A1 (en) * | 2001-11-21 | 2003-06-05 | Seidl Matthew L. | Method and apparatus to facilitate testing of garbage collection implementations |
US20040133895A1 (en) * | 2002-12-20 | 2004-07-08 | Bea Systems, Inc. | System and method for memory leak detection in a virtual machine environment |
US20050240641A1 (en) * | 2003-05-09 | 2005-10-27 | Fujitsu Limited | Method for predicting and avoiding danger in execution environment |
US7293260B1 (en) * | 2003-09-26 | 2007-11-06 | Sun Microsystems, Inc. | Configuring methods that are likely to be executed for instrument-based profiling at application run-time |
US20050114844A1 (en) * | 2003-11-24 | 2005-05-26 | International Business Machines Corporation | Method and apparatus for generating data for use in memory leak detection |
US20050204342A1 (en) * | 2004-03-11 | 2005-09-15 | International Business Machines Corp. | Method, system and article for detecting memory leaks in Java software |
US20050223362A1 (en) * | 2004-04-02 | 2005-10-06 | Gemstone Systems, Inc. | Methods and systems for performing unit testing across multiple virtual machines |
US20060085156A1 (en) * | 2004-10-18 | 2006-04-20 | Kolawa Adam K | Method and system for predicting memory leaks from unit testing |
US7793269B2 (en) * | 2005-02-15 | 2010-09-07 | Ebay Inc. | Parallel software testing based on a normalized configuration |
US20060195823A1 (en) * | 2005-02-28 | 2006-08-31 | Sap Ag. | Memory debugging tool |
US20060206885A1 (en) * | 2005-03-10 | 2006-09-14 | Seidman David I | Identifying memory leaks in computer systems |
US20060247907A1 (en) * | 2005-04-29 | 2006-11-02 | Microsoft Corporation | Deciding assertions in programs with references |
US20060277440A1 (en) * | 2005-06-02 | 2006-12-07 | International Business Machines Corporation | Method, system, and computer program product for light weight memory leak detection |
US20070074170A1 (en) * | 2005-09-09 | 2007-03-29 | Rossmann Paul A | Application monitoring using profile points |
US20070185984A1 (en) * | 2005-09-27 | 2007-08-09 | Bea Systems, Inc. | System and method for issue severity handling for performance test |
US20070083854A1 (en) * | 2005-10-11 | 2007-04-12 | Dietrich Mayer-Ullmann | Testing usability of a software program |
US7908590B1 (en) * | 2006-03-02 | 2011-03-15 | Parasoft Corporation | System and method for automatically creating test cases through a remote client |
US7721154B1 (en) * | 2006-09-05 | 2010-05-18 | Parasoft Corporation | System and method for software run-time testing |
US7620856B2 (en) * | 2006-12-28 | 2009-11-17 | Sap Portals Israel Ltd. | Framework for automated testing of enterprise computer systems |
US20080244525A1 (en) * | 2007-03-29 | 2008-10-02 | Microsoft Corporation | Test Automation Using Virtual Machines |
US20080244546A1 (en) * | 2007-03-30 | 2008-10-02 | Sap Ag | Method and system for providing on-demand profiling infrastructure for profiling at virtual machines |
US20090125891A1 (en) * | 2007-11-13 | 2009-05-14 | International Business Machines Corporation | Method and system for monitoring code change impact on software performance |
Non-Patent Citations (2)
Title |
---|
Gamma et al., "JUnit: A Cook's Tour", 2002, http://www.computing.dcu.ie/~renaat/ca421/cookstour.htm * |
Viswanathan et al., "Java Virtual Machine Profiler Interface", 2000, IBM Systems Journal Vol. 39, No. 1 * |
Cited By (48)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9864600B2 (en) | 2008-08-07 | 2018-01-09 | Code Systems Corporation | Method and system for virtualization of software applications |
US9779111B2 (en) | 2008-08-07 | 2017-10-03 | Code Systems Corporation | Method and system for configuration of virtualized software applications |
US20100131803A1 (en) * | 2008-11-25 | 2010-05-27 | Express Logic | Computation of stack usage in embedded computing systems |
US8930912B2 (en) * | 2008-12-16 | 2015-01-06 | Cadence Design Systems, Inc. | Method and system for performing software verification |
US20100153924A1 (en) * | 2008-12-16 | 2010-06-17 | Cadence Design Systems, Inc. | Method and System for Performing Software Verification |
US9432350B2 (en) * | 2009-11-25 | 2016-08-30 | Novell, Inc. | System and method for intelligent workload management |
US20140237550A1 (en) * | 2009-11-25 | 2014-08-21 | Novell, Inc. | System and method for intelligent workload management |
US9773017B2 (en) | 2010-01-11 | 2017-09-26 | Code Systems Corporation | Method of configuring a virtual application |
US9749393B2 (en) | 2010-01-27 | 2017-08-29 | Code Systems Corporation | System for downloading and executing a virtual application |
US10409627B2 (en) | 2010-01-27 | 2019-09-10 | Code Systems Corporation | System for downloading and executing virtualized application files identified by unique file identifiers |
US11196805B2 (en) | 2010-01-29 | 2021-12-07 | Code Systems Corporation | Method and system for permutation encoding of digital data |
US9569286B2 (en) | 2010-01-29 | 2017-02-14 | Code Systems Corporation | Method and system for improving startup performance and interoperability of a virtual application |
US11321148B2 (en) | 2010-01-29 | 2022-05-03 | Code Systems Corporation | Method and system for improving startup performance and interoperability of a virtual application |
US10402239B2 (en) | 2010-04-17 | 2019-09-03 | Code Systems Corporation | Method of hosting a first application in a second application |
US9626237B2 (en) | 2010-04-17 | 2017-04-18 | Code Systems Corporation | Method of hosting a first application in a second application |
US9639387B2 (en) | 2010-07-02 | 2017-05-02 | Code Systems Corporation | Method and system for prediction of software data consumption patterns |
US10114855B2 (en) | 2010-07-02 | 2018-10-30 | Code Systems Corporation | Method and system for building and distributing application profiles via the internet |
US9483296B2 (en) | 2010-07-02 | 2016-11-01 | Code Systems Corporation | Method and system for building and distributing application profiles via the internet |
US10108660B2 (en) | 2010-07-02 | 2018-10-23 | Code Systems Corporation | Method and system for building a streaming model |
US20160026496A1 (en) * | 2010-07-02 | 2016-01-28 | Code Systems Corporation | Method and system for profiling virtual application resource utilization patterns |
US9984113B2 (en) | 2010-07-02 | 2018-05-29 | Code Systems Corporation | Method and system for building a streaming model |
US10158707B2 (en) * | 2010-07-02 | 2018-12-18 | Code Systems Corporation | Method and system for profiling file access by an executing virtual application |
US10110663B2 (en) | 2010-10-18 | 2018-10-23 | Code Systems Corporation | Method and system for publishing virtual applications to a web server |
US9747425B2 (en) | 2010-10-29 | 2017-08-29 | Code Systems Corporation | Method and system for restricting execution of virtual application to a managed process environment |
US20120110573A1 (en) * | 2010-11-03 | 2012-05-03 | International Business Machines Corporation | Transferring objects between virtual machines |
US8495630B2 (en) * | 2010-11-03 | 2013-07-23 | International Business Machines Corporation | Transferring objects between virtual machines |
US9135039B2 (en) | 2010-11-03 | 2015-09-15 | International Business Machines Corporation | Transferring objects between virtual machines |
US20130110578A1 (en) * | 2011-05-31 | 2013-05-02 | Software Ag | Systems and/or methods for identifying service candidates based on service identification indicators and associated algorithms |
US20130007720A1 (en) * | 2011-06-30 | 2013-01-03 | International Business Machines Corporation | Dynamic Isolated Debugging via Class Instrumentation |
CN103377124A (en) * | 2012-04-25 | 2013-10-30 | 腾讯科技(深圳)有限公司 | Testing method and device |
CN102929693A (en) * | 2012-09-18 | 2013-02-13 | 杭州华为数字技术有限公司 | Performance evaluation method and device for servers of whole equipment cabinet |
WO2014165092A1 (en) * | 2013-03-12 | 2014-10-09 | Netflix, Inc. | Using canary instances for software analysis |
US10318399B2 (en) | 2013-03-12 | 2019-06-11 | Netflix, Inc. | Using canary instances for software analysis |
US9582219B2 (en) | 2013-03-12 | 2017-02-28 | Netapp, Inc. | Technique for rapidly converting between storage representations in a virtualized computing environment |
US20150324227A1 (en) * | 2014-05-12 | 2015-11-12 | Netapp, Inc. | Techniques for virtual machine migration |
US10216531B2 (en) | 2014-05-12 | 2019-02-26 | Netapp, Inc. | Techniques for virtual machine shifting |
US9841991B2 (en) * | 2014-05-12 | 2017-12-12 | Netapp, Inc. | Techniques for virtual machine migration |
CN105808421A (en) * | 2014-12-31 | 2016-07-27 | 中兴通讯股份有限公司 | White box test realization method and test control end |
US9471237B1 (en) | 2015-02-04 | 2016-10-18 | Amazon Technologies, Inc. | Memory consumption tracking |
US9817592B1 (en) | 2016-04-27 | 2017-11-14 | Netapp, Inc. | Using an intermediate virtual disk format for virtual disk conversion |
US10176068B2 (en) * | 2016-11-29 | 2019-01-08 | Keysight Technologies Singapore (Holdings) Pte. Ltd. | Methods, systems, and computer readable media for token based message capture |
CN107092541A (en) * | 2017-04-27 | 2017-08-25 | 郑州云海信息技术有限公司 | A kind of method that utilization virtual machine carries out storage performance test |
US10515216B2 (en) * | 2017-06-30 | 2019-12-24 | Paypal, Inc. | Memory layout based monitoring |
US11314864B2 (en) * | 2017-06-30 | 2022-04-26 | Paypal, Inc. | Memory layout based monitoring |
US20220222344A1 (en) * | 2017-06-30 | 2022-07-14 | Paypal, Inc. | Memory layout based monitoring |
US11893114B2 (en) * | 2017-06-30 | 2024-02-06 | Paypal, Inc. | Memory layout based monitoring |
CN107577519A (en) * | 2017-09-18 | 2018-01-12 | 郑州云海信息技术有限公司 | A kind of virtual machine pressure method and device |
US11080603B2 (en) * | 2018-05-18 | 2021-08-03 | Google Llc | Systems and methods for debugging neural networks with coverage guided fuzzing |
Also Published As
Publication number | Publication date |
---|---|
US9727436B2 (en) | 2017-08-08 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9727436B2 (en) | Adding a profiling agent to a virtual machine to permit performance and memory consumption analysis within unit tests | |
US8930912B2 (en) | Method and system for performing software verification | |
US9274923B2 (en) | System and method for stack crawl testing and caching | |
US8776014B2 (en) | Software build analysis | |
US8887141B2 (en) | Automatically modifying a native code module accessed from virtual machine bytecode to determine execution information | |
US8578339B2 (en) | Automatically adding bytecode to a software application to determine database access information | |
US8938729B2 (en) | Two pass automated application instrumentation | |
EP2359247B1 (en) | Transforming user script code for debugging | |
US7644394B2 (en) | Object-oriented creation breakpoints | |
CN111756575B (en) | Performance analysis method and device of storage server and electronic equipment | |
US7945591B2 (en) | Identifying problems, usage patterns, and performance in a database interface using aspect-oriented programming | |
CN101645119A (en) | Method and system for automatically analyzing malicious codes based on virtual hardware environment | |
US9183114B2 (en) | Error detection on the stack | |
EP3387536B1 (en) | Method and device for non-intrusively collecting function trace data | |
CN115935631A (en) | Test system and method for vehicle equipment and storage medium | |
Seo et al. | A profiling method by PCB hooking and its application for memory fault detection in embedded system operational test | |
US11599342B2 (en) | Pathname independent probing of binaries | |
Watson | Writing high-performance. NET code | |
CN113778838A (en) | Binary program dynamic taint analysis method and device | |
EP3635561B1 (en) | Asynchronous operation query | |
Waddington et al. | Dynamic analysis and profiling of multithreaded systems | |
CN112905474B (en) | Hardware-based advanced program dynamic control flow tracking method and device | |
CN110096888A (en) | A kind of method and system for accelerating verifying and analysis SMM security risk | |
CN106897051B (en) | Process cleaning method and device | |
Senthilkumaran et al. | Insight: A framework for application diagnosis using virtual machine record and replay |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MOSTAFA, MOHAMMED;REEL/FRAME:020306/0813 Effective date: 20071218 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FEPP | Fee payment procedure |
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
LAPS | Lapse for failure to pay maintenance fees |
Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20210808 |