US20030233445A1 - Determining client latencies over a network - Google Patents

Determining client latencies over a network Download PDF

Info

Publication number
US20030233445A1
US20030233445A1 US10/170,460 US17046002A US2003233445A1 US 20030233445 A1 US20030233445 A1 US 20030233445A1 US 17046002 A US17046002 A US 17046002A US 2003233445 A1 US2003233445 A1 US 2003233445A1
Authority
US
United States
Prior art keywords
client
data
time
server
latency
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
US10/170,460
Other versions
US7747729B2 (en
Inventor
Hanoch Levy
Mark Marshak
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zarbana Digital Fund LLC
Original Assignee
Ramot at Tel Aviv University Ltd
Research and Ind Dev Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ramot at Tel Aviv University Ltd, Research and Ind Dev Ltd filed Critical Ramot at Tel Aviv University Ltd
Priority to US10/170,460 priority Critical patent/US7747729B2/en
Assigned to RAMOT UNIVERSITY AUTHORITY FOR APPLIED RESEARCH & INDUSTRIAL DEVELOPMENT LTD. reassignment RAMOT UNIVERSITY AUTHORITY FOR APPLIED RESEARCH & INDUSTRIAL DEVELOPMENT LTD. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: LEVY, HANOCH, MARSHAK, MARIK
Publication of US20030233445A1 publication Critical patent/US20030233445A1/en
Assigned to KHORSABAD LOCKDOWN LLC reassignment KHORSABAD LOCKDOWN LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: RAMOT AT TEL AVIV UNIVERSITY LIMITED
Priority to US11/514,378 priority patent/US7676570B2/en
Assigned to RAMOT AT TEL-AVIV UNIVERSITY LTD. reassignment RAMOT AT TEL-AVIV UNIVERSITY LTD. CHANGE OF NAME (SEE DOCUMENT FOR DETAILS). Assignors: RAMOT UNIVERSITY AUTHORITY FOR APPLIED RESEARCH & INDUSTRIAL DEVELOPMENT LTD.
Application granted granted Critical
Publication of US7747729B2 publication Critical patent/US7747729B2/en
Assigned to ZARBAÑA DIGITAL FUND LLC reassignment ZARBAÑA DIGITAL FUND LLC MERGER (SEE DOCUMENT FOR DETAILS). Assignors: KHORSABAD LOCKDOWN LLC
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/08Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0852Delays
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/08Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0852Delays
    • H04L43/0864Round trip delays
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/08Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0876Network utilisation, e.g. volume of load or congestion level
    • H04L43/0888Throughput
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/16Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/16Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
    • H04L69/163In-band adaptation of TCP data exchange; In-band control procedures
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/28Timers or timing mechanisms used in protocols
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/10Active monitoring, e.g. heartbeat, ping or trace-route

Definitions

  • the present invention relates to determining client latencies over networks, particularly during the course of downloading procedures, and more particularly but not exclusively to determining client latencies in downloads of web pages over the Internet.
  • Client latency may be defined as the time a client has to wait, between requesting data from a server, and finishing to download the requested data, and all data associated with the request.
  • the main way of conducting client latency measurements is to use agents external to the site.
  • the agents are deployed at a limited number of locations around a network, be it an intranet, or on the Internet, and the agents fetch specific web pages from a web server, specifically to measure the latency from the limited number of locations where those agents reside.
  • the disadvantages of this method are:
  • the server is dependent upon an external body for conducting the measurements
  • Client latencies will vary based on what version of HTTP a client is using.
  • HTTP 1.0 a client would establish a new TCP connection for each HTTP request.
  • Using a new TCP connection for each request led to incurring connection-establishment latencies on each request. Connection establishment times will be described in greater detail below.
  • HTTP 1.1 clients establish a persistent TCP connection with the server, and those connections are re-used for multiple HTTP requests, and responses.
  • a network latency estimation apparatus for estimating a client's latency in a network communication between a server and said client, the apparatus comprising:
  • an event observer for observing occurrences of pre-selected events associated with said communication occurring at said server
  • a logging device associated with said event observer for logging into a data store the occurrence of said events together with respective time information
  • a latency estimator associated with said logging device for using said logged occurrences with said respective time information to arrive at an estimation of a client's latency for said communication.
  • said client's latency comprises the difference between a first time at which a client sends request data and a second time at which said client completes receipt of the requested data.
  • the latency estimator further comprises a round-trip-time estimator, capable of estimating a round trip time for said client, said estimation being based on receipt at the server of a request for data from the client in consequence of data previously sent to the client by the server.
  • a round-trip-time estimator capable of estimating a round trip time for said client, said estimation being based on receipt at the server of a request for data from the client in consequence of data previously sent to the client by the server.
  • said round-trip-time-estimator is operable to estimate the round trip time by determining the shortest duration over a plurality of durations, measured between when a server sends data to a client, and when the server receives a subsequent request from the client, said subsequent request being in consequence of said data sent to the client by the server.
  • the apparatus comprises a pinger associated with said event observer.
  • said events comprise sending out a ping, and receiving a response.
  • the round-trip-time estimator is operable to estimate the round trip time based on the shorter of:
  • said data store contains at least some of:
  • said latency estimator is operable to form an initial estimate of the client's latency by adding an estimated round trip time, to a delay between when the server receives an initial request for data from said client, and when the server receives a subsequent request for data from said client
  • the apparatus comprises a receipt indicator, associated with said event observer, operable to add to the end of data being sent an automatic end of data receipt indicator, for obtaining an acknowledgment of receipt of the end of the data by said client.
  • the apparatus comprises a client-data-reception-time estimator operable to estimate a time when the client received said data sent as the time when the server receives the acknowledgment of the receipt of said data.
  • said latency estimator is further operable to estimate the client's latency by using the earliest of the times determined from:
  • said latency estimator is further operable to estimate the client's latency by using the earliest of the times determined by when the server received the acknowledgment of receipt of the data sent, and when the server received any other request from the client wherein said other request is in consequence of said data sent.
  • the apparatus comprises a queuing latency probe, associated with said logging device, capable of sending a queue probing request to the server.
  • said pre-selected events further comprise:
  • said request is an HTTP request.
  • the apparatus comprises an adder for adding an elapsed time between when said queuing latency probe request is sent to the server, and when said request is accepted for handling by the server, to said initial latency as measured by the client-latency estimator.
  • the apparatus comprises a transmission-data-rate estimator, connected to the latency estimator, said data rate estimator comprising a divider and a subtractor, said subtractor being connected to receive as inputs the estimated round trip time, and the initial estimate of the client's latency, and to produce an output being the difference therebetween, said divider being connected to receive as a first input an amount of data sent by the server in the course of the transmission, and as a second input said subtractor output, and to produce an estimate of a transmission data rate by dividing therebetween.
  • said data rate estimator comprising a divider and a subtractor, said subtractor being connected to receive as inputs the estimated round trip time, and the initial estimate of the client's latency, and to produce an output being the difference therebetween
  • said divider being connected to receive as a first input an amount of data sent by the server in the course of the transmission, and as a second input said subtractor output, and to produce an estimate of a transmission data rate by dividing therebetween.
  • the apparatus comprises omprising an overall-client-data-rate estimator comprising an averager for averaging together a plurality of successively estimated transmission data rates for the given client connection thereby to estimate an overall data rate for said given client.
  • the apparatus comprises a client-data-reception-time estimator, comprising a multiplier and an adder, said adder being functional to add a time at which the server dispatched an end part of the requested data to the channel, to an output of the multiplier, wherein the multiplier is operable to multiply an amount of data left to send, by said overall data rate, the client-data reception time estimator thereby obtaining an estimate of client-data-reception-time.
  • a client-data-reception-time estimator comprising a multiplier and an adder, said adder being functional to add a time at which the server dispatched an end part of the requested data to the channel, to an output of the multiplier, wherein the multiplier is operable to multiply an amount of data left to send, by said overall data rate, the client-data reception time estimator thereby obtaining an estimate of client-data-reception-time.
  • the apparatus comprises a subtractor and an adder, said subtractor operable to subtract the time at which the server received an initial request for data from a client from said time the client received the last of the requested data, said adder operable to add one and a half times the estimated round trip time, and further add the latency as estimated by the queuing latency probe, to the output of the subtractor.
  • the apparatus comprises a client-connection-data-rate estimator comprising an averager for averaging together all the transmission data rates for a given client connection wherein the corresponding transmissions meet a specific size criteria.
  • the apparatus comprises a client-data-reception-time estimator comprising an adder and a multiplier, said adder for adding the time the server last dispatched data to the client, to the output of the multiplier, wherein said multiplier is operable to multiply an amount of data left to send by said data rate for a given client connection, thereby to produce an estimate of a time of receipt of said last data to the client.
  • a client-data-reception-time estimator comprising an adder and a multiplier, said adder for adding the time the server last dispatched data to the client, to the output of the multiplier, wherein said multiplier is operable to multiply an amount of data left to send by said data rate for a given client connection, thereby to produce an estimate of a time of receipt of said last data to the client.
  • the apparatus comprises a client latency estimator for estimating a respective client latency's, the latency estimator having a subtractor and an adder, said subtractor for subtracting the time the server received an initial request for data from a client from said estimate of a time of receipt of said last data to the client, said adder for adding together one and a half times the estimated round trip time, the latency as estimated by the queuing latency probe, and the output of the subtractor, thereby to form an estimate of said respective client's latency.
  • the latency estimator having a subtractor and an adder, said subtractor for subtracting the time the server received an initial request for data from a client from said estimate of a time of receipt of said last data to the client, said adder for adding together one and a half times the estimated round trip time, the latency as estimated by the queuing latency probe, and the output of the subtractor, thereby to form an estimate of said respective client's latency.
  • the apparatus comprises a global-transmission-rate estimator comprising an averager, said averager for averaging together successive data transmission rates, thereby to estimate a global transmission rate.
  • the apparatus comprises a client-data-reception-time estimator comprising an adder and a multiplier, said multiplier operable to multiply an amount of data left to send by said global transmission data rate and said adder operable to add the time the server dispatched the last data to the client, to the output of the multiplier, thereby to provide an estimate of a client data reception time.
  • a client-data-reception-time estimator comprising an adder and a multiplier, said multiplier operable to multiply an amount of data left to send by said global transmission data rate and said adder operable to add the time the server dispatched the last data to the client, to the output of the multiplier, thereby to provide an estimate of a client data reception time.
  • the apparatus comprises a subtractor and an adder, said subtractor for subtracting the time the server received an initial request for data from a client from said time of receipt of said last data at said client, said adder connected to add one and a half times the estimated round trip time, to the latency as estimated by the queuing latency probe, thereby to provide a revision of the estimate of the client's latency.
  • the apparatus comprises a global-transmission-rate estimator comprising an averager operable to estimate a global transmission data rate, said averager operable to average together a series of successively obtained transmission data rates wherein respective transmissions meet a pre-determined size criteria.
  • the apparatus comprises a client-data-reception-time estimator operable to estimate the time the client received a last part of the requested data, the estimator comprising a multiplier and an adder, the multiplier being connected to multiply the amount of data left to send by said global transmission data rate, and to output a result to provide a first input to said adder, said adder being arranged to add said input to said time at which the server dispatched said last data part to the client, thereby to provide an estimate of said client data reception time.
  • the estimator comprising a multiplier and an adder, the multiplier being connected to multiply the amount of data left to send by said global transmission data rate, and to output a result to provide a first input to said adder, said adder being arranged to add said input to said time at which the server dispatched said last data part to the client, thereby to provide an estimate of said client data reception time.
  • said subtractor subtracts the time the server received an initial request for data from a client from said time the client received the last part of said requested data, said adder further adding, to the output of the subtractor, one and a half times the estimated round trip time, and the time duration between a first time when the queuing latency probe sends the request to the server, and a second time when the server accepts the request sent by the probe.
  • a plurality of data transmission rates are measured for a given channel, the apparatus further comprising a channel-data-rate estimator comprising an averager for averaging together all the data rates for each transmission on said given channel.
  • the apparatus comprises an overall-client-data-rate estimator comprising an averager, said averager being operable to average together a plurality of successively measured data rates for a given client's channels.
  • the apparatus comprises ng a global-data-rate estimator for estimating a global data rate, the estimator comprising an averager for averaging together a plurality of successively measured data rates.
  • the apparatus comprises a transmission-packet-rate estimator operable to estimate a packet rate for a transmission, the estimator comprising a divider and a subtractor, wherein said subtractor is operable to subtract the round trip time from the initial estimate of the client's latency for the transmission to produce a subtractor output, said divider being connected to divide the number of packets the server sends in the course of the transmission, by said subtractor output.
  • the apparatus comprises an overall-client-packet-rate estimator operable to estimate the overall packet rate for a given client connection, the estimator comprising an averager, for averaging together a plurality of successively measured transmission packet rates for the given client connection.
  • the apparatus comprises a client-data-reception-time estimator operable to estimate a time the client received the last of the data, the estimator comprising a multiplier and an adder, said multiplier operable to multiply a number of packets remaining to be sent by said overall packet rate, thereby to produce a multiplier output, and said adder being connected to add the time the server wrote the end of the data to the client, to said multiplier output.
  • the estimator comprising a multiplier and an adder, said multiplier operable to multiply a number of packets remaining to be sent by said overall packet rate, thereby to produce a multiplier output, and said adder being connected to add the time the server wrote the end of the data to the client, to said multiplier output.
  • the latency estimator estimates the client latency, said subtractor further subtracting the time the server received an initial request for data from a client from said time the client received the last of the data, thereby to produce a subtractor output, said adder further adding to the result of the subtractor one and a half times the estimated round trip time, and further adding thereto the latency as estimated by the queuing latency probe.
  • a client-connection-packet-rate estimator estimates the packet rate for a given client connection, the estimator comprising an averager operable to average together ones of a succession of packet rates for given transmissions of a client connection wherein respective transmissions meet a pre-selected size criteria.
  • the apparatus comprises a client-data-reception-time estimator operable to estimate a time the client received the last of the data, the estimator comprising an adder and a multiplier, said multiplier operable to multiply the number of packets left to send by said packet rate for a given client connection, to produce a multiplier output, said adder adding the time the server wrote the last data to the client, to said multiplier output.
  • the estimator comprising an adder and a multiplier, said multiplier operable to multiply the number of packets left to send by said packet rate for a given client connection, to produce a multiplier output, said adder adding the time the server wrote the last data to the client, to said multiplier output.
  • the apparatus estimates the client latency, said subtractor further operable to subtract the time the server received an initial request for data from a client from said estimated time the client received the last of the data, said adder being further operable to add to the output of the subtractor one and a half times the estimated round trip time, and said adder further operable to add thereto a queuing latency duration between a queuing latency probing request sending time when the queuing latency probe sends said probing request to the server, and a second, queuing latency probe request receipt time, when the server accepts said probing request.
  • the apparatus comprises a global-transmission-rate estimator, comprising an averager operable to estimate a global transmission packet rate, said averager operable to average together all the transmission packet rates of all the connections to the server.
  • the apparatus comprises a client-data-reception-time estimator operable to estimate the time the client received the last of the data, the estimator comprising an adder and a multiplier, said multiplier operable to multiply the number of packets left to send by said global transmission packet rate, to form a multiplier output, said adder for adding the time the server wrote the last data to the client, to said multiplier output.
  • the estimator comprising an adder and a multiplier, said multiplier operable to multiply the number of packets left to send by said global transmission packet rate, to form a multiplier output, said adder for adding the time the server wrote the last data to the client, to said multiplier output.
  • the apparatus estimates the client latency.
  • it comprises a further adder and a further subtractor, said further subtractor operable to subtract the time the server received an initial request for data from a client from said time the client received the last of the data thereby to form a subtractor output, said adder operable to add to the output of the subtractor one and a half times the estimated round trip time, and the duration between the a first time when the queuing latency probe sends the request to the server, and a second time when the server accepts the request sent by the probe.
  • the apparatus comprises a global-transmission-rate estimator operable to estimate a global transmission packet rate, said averager being further operable to average together the transmission rates from all transmissions meeting a predetermined size criteria.
  • the apparatus comprises a client-data-reception-time estimator operable to estimate the time the client received the last of the data, the estimator comprising an adder and a multiplier, said multiplier multiplying the number of packets left to send by said global transmission packet rate, said adder operable to add the time the server wrote the last data to the client, to the output of the multiplier.
  • the estimator comprising an adder and a multiplier, said multiplier multiplying the number of packets left to send by said global transmission packet rate, said adder operable to add the time the server wrote the last data to the client, to the output of the multiplier.
  • the apparatus comprises a subtractor and a further adder, said subtractor being operable to subtract the time the server received an initial request for data from a client from said time the client received the last of the data, said adder being operable to add to the output of the subtractor one and a half times the estimated round trip time, and the time duration between the a first time when the queuing latency probe sends the latency test request to the server, and a second time when the server accepts the latency test request.
  • the apparatus comprises a channel-packet-rate estimator operable to estimate the packet rate for a given channel, the estimator comprising a subtractor, and a divider, wherein said subtractor is operable to subtract the round trip time from the latency measured for said client for said channel, thereby to produce a subtractor output, said divider being connected to divide a number of packets the server sends on the channel, by the output of the subtractor.
  • each client connects via a plurality of channels
  • the apparatus further comprising an overall-client-packet-rate estimator operable to estimate an overall packet rate for a given client, the estimator comprising an averager for averaging together a plurality of successively measured packet rates for each of said plurality of channels.
  • the apparatus comprises a global-packet-rate estimator operable to estimate the global packet rate, the estimator comprising an averager operable to average together a plurality of successively measured packet rates.
  • the apparatus is operable to log HTTP transmissions.
  • a method for estimating the latency of a user client on a network communication, using measurements made in association with a server with which said client is in communication comprising the steps of:
  • said user client latency is the latency between a first time when a client sends a request for data to a server and a second time at which said client receives a last datum of the requested data.
  • the method comprises a round-trip-time estimation step, comprising estimating a round trip time, said estimation being based on when a request for data from the client is received by the server in consequence of data previously sent to the client by the server.
  • the method comprises said round-trip-time-estimation step comprises estimating the round trip time by finding a shortest duration between a third time when the server sends data to said client, and a fourth time when the server receives a subsequent request from said client, said subsequent request being made by said client in consequence of data sent to the client by the server.
  • the method comprises a step of pinging the client.
  • the method comprises recording time information about a duration between sending said ping and receiving a response from a respective client.
  • the method comprises estimating a data round trip time based on the shorter of:
  • the logging step logs a selection from the group consisting of:
  • the method comprises a client-latency estimation step, of:
  • client-latency estimation step comprises providing a first client latency estimate as an earliest of:
  • the method comprises a queuing latency probing step, comprising
  • said request is an HTTP request.
  • said client-latency estimation step further comprises adding said recorded queuing latency to said initial client latency estimate, thereby to provide a revised client latency estimation.
  • the method comprises a transmission-data-rate estimation step comprising dividing an amount of data the server sends in the course of a given transmission, by said client latency estimate for said given transmission, and subtracting the round trip time, thereby to estimate a transmission data rate.
  • the method comprises an overall-client-data-rate estimation step of averaging together a plurality of successively measured transmission data rates for the given client connection.
  • the method comprises a client-data-reception-time estimation step comprising:
  • the user client latency estimation step comprises:
  • the method comprises a client-connection-data-rate estimation step comprising:
  • the method comprises a client-data-reception-time estimation step comprising:
  • the method comprises:
  • the method comprises a global-transmission-rate estimation step, comprising estimating a global transmission data rate, by averaging together transmission rates.
  • the method comprises a client-data-reception-time estimation step of:
  • the method comprises:
  • the method comprises a global-transmission-rate estimation step, comprising
  • the method comprises a client-data-reception-time estimation step, comprising:
  • the method comprises:
  • each said client communicates via at least one channel, the method further comprising a channel-data-rate estimation step of:
  • each client communicates via a plurality of channels, the method further comprising an overall-client-data-rate estimation step of:
  • the method comprises a global-data-rate estimation step comprising:
  • the method comprises a transmission-packet-rate estimation step of:
  • the method comprises an overall-client-packet-rate estimation step of:
  • the method comprises a client-data-reception-time estimation step of:
  • the client-latency estimation step comprises:
  • the method comprises a client-connection-packet-rate estimation step of
  • the method comprises a client-data-reception-time estimation step, of
  • the method comprises:
  • the method comprises a global-transmission-rate estimation step of:
  • the method comprises a client-data-reception-time estimation step of:
  • the method comprises
  • the method comprises a global-transmission-rate estimation step of:
  • the method comprises a client-data-reception-time estimation step of:
  • the method comprises:
  • the method comprises a channel-packet-rate estimation step, wherein said client communicates using at least one channel, the method including:
  • the client has at least one additional channel, in which case the method may further comprise:
  • the method comprises a global-packet-rate estimation step of:
  • the method comprises logging HTTP transmissions.
  • a data carrier carrying data usable in combination with a general purpose computer to provide functionality capable of estimating the latency between when a client sends a request for data to a server, and when that client receives the data, using measurements made at a server, the data being usable to provide:
  • a latency estimator associated with said logging device for using said logged occurrences with said respective time information to arrive at an estimation of latency in said communication.
  • FIG. 1 is a simplified schematic of the structure of the Internet, or other like network
  • FIG. 2 is a simplified diagram showing some of the elements of the latency a client may experience in downloading content from a server
  • FIG. 3 is a simplified schematic diagram of a client establishing a connection with a server, and downloading a web page from that server, using multiple (in this case, two) persistent connections (channels),
  • FIG. 4 is a simplified schematic diagram of a single channel between a client and a server, showing a relationship between the times the client requests different components of a web-page on the channel,
  • FIG. 5A is a simplified diagram showing some of the log files that may be kept by a typical server, while FIG. 5B shows additional logs that a server associated with an apparatus according to a first preferred embodiment of the present invention, may keep,
  • FIG. 6 is a simplified block diagram showing an apparatus according to a first preferred embodiment of the present invention.
  • FIG. 7 is a simplified flow chart showing how an apparatus, according to the embodiment of FIG. 6, may estimate the round trip time for a given client
  • FIG. 8 is a simplified flow chart showing how an apparatus, according to the embodiment of FIG. 6, may estimate the transmission rate for a given transmission, and which describes estimations of other, more general transmission rates,
  • FIG. 9 is a simplified flow chart showing how an apparatus, according to the embodiment of FIG. 6, may estimate, from a server, the time when a client receives the HTML portion of a web page,
  • FIG. 10 is a simplified flow chart showing how an apparatus, according to the embodiment of FIG. 6, may estimate the client latency for a given download of the initial HTML text of a page,
  • FIG. 11 is a simplified flow chart showing how an apparatus, according to the embodiment of FIG. 6, may estimate the time when a client receives an entire web page
  • FIG. 12 is a simplified flow chart showing how an apparatus, according to the embodiment of FIG. 6, may estimate the client latency for a given download of a complete web page
  • FIG. 13 is a simplified schematic of an architecture used to test a preferred embodiment of the invention.
  • FIG. 14 is a graph of accuracies for measurements of the main page latency, and web page latency
  • FIGS. 15A 15 B 15 C and 15 D, 16 A and 16 B are cumulative distributions of latency estimation error, and RTT error, for typical web pages where the server is under a normal load, and,
  • FIGS. 17A and 17B are cumulative distributions of latency estimation error, for typical HTML, where the server is overloaded.
  • the present invention allows a client's latency, that is, the amount of time between a first time, when a client makes an initial request for data, and a second time when the client receives the requested data, to be estimated in association with a server serving data to the requesting client.
  • the client's latency may be estimated using calculations, based on measurements made in association with the server.
  • FIG. 1 is a simplified schematic diagram of the connection between a client and a server using the Internet, or other like network.
  • a content server 62 is connected to at least one Internet Service Provider (an ISP), router, switch or hub or a plurality thereof 64 .
  • the ISP, router, switch, or hub is connected to the Internet or main network 66 , which may consist of a series of other routers, hubs, switches, other network connection, and various nodes with information about the network topologies, as well as other connection devices, including various data pipes, such as Ethernet cable, fiber-optic cable, telephone lines, and satellite connections.
  • Connected to the Internet, or other network is a client's ISP, router, switch, or hub 68 .
  • the client's ISP, router, switch or hub may or may not be the same as the content server's ISP router, switch or hub, but often will not be.
  • the client 70 is in turn connected to the client's ISP, router, switch or hub.
  • the data may pass through the server's ISP, router, switch or hub 64 , through the Internet, or various routers switches and hubs 66 , through the client's ISP, router, switch or hub 68 , and finally arrive at the client 70 .
  • the server's ISP, router, switch or hub 64 may pass through the server's ISP, router, switch or hub 64 , through the Internet, or various routers switches and hubs 66 , through the client's ISP, router, switch or hub 68 , and finally arrive at the client 70 .
  • the connection between the client and the server may have limited bandwidth, and only a certain amount of data may be able to travel from a client to a server in a given period of time.
  • a server and a client there may be a limited bandwidth between a server and a client, which may or may not be the same as the bandwidth from the client to the server (the client's upload bandwidth, and download bandwidth, respectively).
  • FIG. 2 is a simplified flow chart showing a typical web-page download procedure, and illustrating sources of latency for a download.
  • a client sends data to a server it is referred to as a request
  • the server sends data to a client it is referred to as a response.
  • the sum total of all exchanges that occur between a client and a server, in the client's downloading of a single web page is referred to as a web-page transaction.
  • a client may make a DNS lookup in order to contact a server 72 .
  • a client sends a request to a known server (a DNS server) requesting the address of a target server (the server the client is trying to initiate a connection with).
  • the DNS server either has the address of the server the client is looking for, or has to look up the address with another server. If the other server does not have the address it may consult a different server, which may in turn have to consult a further server, until finally a server finds the requested address, or determines that there is no server with such an address.
  • DNS lookup is a small part of the web page transaction. Assuming that the DNS server finds the address, the DNS server eventually responds to the client with the address of the target server.
  • the client may establish a connection with the target server.
  • the client may establish persistent connections, referred to as a channels, with the target server 74 .
  • a channel is a connection that is used for multiple requests and responses, as opposed to opening a new connection for each request.
  • the establishment of a connection takes a certain amount of time, referred to as connection latency.
  • a client may open multiple channels with a server. Each of these channels may be opened at a different time, or may be opened simultaneously. When channels are opened simultaneously, the connection latencies, for each channel, may overlap.
  • TCP/IP establishing a connection is initiated by the client sending a request to the server.
  • the server places the request on an uncompleted connection queue and sends an acknowledgment of the request back to the client.
  • the client Upon receiving the server's acknowledgment of the initial request, the client sends a request to the server acknowledging the server's acknowledgment of the initial request.
  • the server receives the client's acknowledgment of the server's acknowledgment the server moves the request onto a completed connection queue 76 .
  • the queuing latency The time the connection remains in the server's completed connection queue is referred to as the queuing latency.
  • the connection queue has a maximum length, and if that length is reached, then the client connection is dropped, and the channel is not established. Assuming the server does not drop the connection, the server accepts the connection.
  • the client sends one or more requests to the server for an initial HTML page 78 , called the main page. Note that this request may have been sent while the server accepts the connection, that is to say, the first request on the channel may be sent as soon as the client receives the server's initial acknowledgment of the client's connection request.
  • the server may read the request sent by the client, for the main page. There may be delay between when the request for the main page was sent, until the server receives the request for the main page 80 .
  • the server Upon receiving the request for the main page, the server processes the request, which may consist of disk time such as finding static HTML, processing time, such as generating dynamic content, and possibly a reverse DNS lookup 82 (the server looking up the client's address). It takes time for the server to write the response to the request for the main page to the channel 84 , and then it takes time for the client to receive the main page (the time it takes the response to travel from the server to the client) 86 .
  • disk time such as finding static HTML
  • processing time such as generating dynamic content
  • a reverse DNS lookup 82 the server looking up the client's address
  • the main page may indicate that there are other components of the page for the client to download, in which case the client may send one or more additional requests to the server for those components 88 .
  • the client may send requests for additional components before the client has finished downloading the main page.
  • the client may find a reference to an additional component of the web page, and upon recognizing the reference, the client may send a request for that component, before the client has finished downloading the entire main page.
  • the client may have to wait until the server receives requests 90 for those components, following which, the server processes those requests 92 , and writes one or more responses to the additional requests 94 .
  • the client then waits for the additional responses to travel from the server to the client 96 .
  • the client may request additional content 88 . Once the client has received all the responses, the web page transaction is complete, and the client has the entire web page.
  • FIG. 3 is a simplified schematic timing diagram showing delays involved when a client establishes a connection with a server, and downloads a web page from that server, using multiple (in this case, two) channels.
  • the client sends the server requests, to open a first 240 , and a second 264 , channel. Theses two requests may be sent simultaneously, or one closely following the other.
  • the server receives the requests from the client 242 , and 266 , and places the requests in the uncompleted connection queue.
  • the server sends out acknowledgments of the client's connection requests.
  • the client receives the acknowledgments 244 , and 268 , and sends acknowledgments of the acknowledgements to the server, 244 , and 268 .
  • the client sends the server a request for the HTML, namely the main page, on the first channel 244 .
  • the server receives the acknowledgments of the server's acknowledgment 246 , and 270 and places the connections on the completed connection queue 246 , and 270 .
  • the establishment of a connection comprises a 3-way handshake (1.client request, 2.server acknowledgement, 3. client acknowledgment of server acknowledgment), and the server's queuing time. The time it takes to establish a connection is referred to as the connection latency. It is noted that each channel may have connection latency, and that the connection latencies of each channel may overlap.
  • the server accepts the connections 248 , and 272 , and reads the request on the first channel 248 .
  • the server processes the request, and writes a response to the client 250 .
  • the client starts receiving the response (the main page) on the first channel.
  • the client detects 274 a reference to a first component of the main page, such as an image, a JavaScript, a java applet, a flash object, or any other object referenced by the main page, even if the client has not finished receiving the entire HTML portion of the web page, the client may send the server a request for the referenced object on the second channel 276 .
  • the server receives the request for the component 278 , and may start processing the request.
  • the server processes the request for the first component, and starts sending the client the first component 280 on the second channel.
  • the client may have finished receiving the entire HTML (the client may have received the main page) 252 , may have parsed the entire HTML of the main page 254 , and may send a request, for a second component referenced by the HTML, on the first channel 256 .
  • the server receives the request for the second component on the first channel 258 .
  • the server processes the request for the second component and sends a response to the request for the second component 260 on the first channel.
  • the client receives the first component on the second channel 282 , and the client receives the second component, on the first channel 262 . If there is only the HTML and two components referenced by the HTML, the client will have finished downloading the page. Otherwise the client will continue sending requests to the server on as many channels as the client has, until the client has all the components of the page referenced by the HTML.
  • the round trip time is defined as the time it takes for a theoretically infinitely small piece of data to go from a server to a client and return to the server.
  • An estimation of the main page latency may be provided by: ⁇ Main ⁇ ⁇ page ⁇ ⁇ latency ⁇ T DNS ⁇ ⁇ Lookup + 1.5 ⁇ RTT + T Queuing + T Server ⁇ ⁇ processing ⁇ ⁇ time + 0.5 ⁇ RTT + HTML Size Bandwidth ( eq . ⁇ 1 )
  • T DNS Lookup is the time the client takes for a DNS lookup
  • RTT is the round trip time
  • T Queueuing is the queuing latency
  • T Server processing time is the time it takes the server to process the request
  • HTML size is the size of the HTML being sent.
  • Bandwidth is the bandwidth from the server to the client.
  • An estimation of the web page latency may be provided by: Web ⁇ ⁇ page ⁇ ⁇ latency ⁇ T DNS ⁇ ⁇ Lookup + 1.5 ⁇ RTT + T Queuing + max 1 ⁇ i ⁇ N ⁇ ⁇ E i + 0.5 ⁇ RTT + Response ⁇ ⁇ Size i bandwidth - S 1 ⁇ ( eq . ⁇ 2 )
  • T DNS Lookup is the time the client takes for a DNS lookup
  • RTT is the round trip time
  • T Queueuing is the queuing latency
  • N is the number of HTTP responses the server writes to send the client the web page
  • E 1 is the time at which a particular response, response i is written to the channel.
  • S 1 is the time the server received the first HTTP request from the client.
  • Response Size i is the size of the response whose transmission bandwidth is being determined
  • t server — recv i+1 is the time the server received a first request consequent to the respone of Response Size i ,
  • t serv — send is the time the server sent the response of Response Size i .
  • Rtt is the round trip time.
  • Equation 3 depends on an accurate knowledge of the round-trip-time, for the given transmission. Merely using an average round trip time the above estimation could estimate, for certain transmissions an almost infinite bandwidth, or a negative bandwidth.
  • FIG. 4 is a simplified schematic timing diagram of a single channel between a client and a server, showing a relationship between the times the client requests different web-page components on a channel.
  • FIG. 4 illustrates some of the dependence between a server's response, and a client's sending a request, subsequent to the server's response, on the channel.
  • the client sends a request, request i for a component i, of a page, for instance, a web page, to the server 290 , at time t client — send — i .
  • Request i arrives at the server 292 , at time, t server — recv — i , which is after t client — send — i .
  • the server processes request i and sends a response, response i 294 to request i , at time t server — send — i .
  • Response i is received at the client 296 at time t client — recv — i .
  • the client processes response i and then may send a subsequent request, 298 , request i+1 for another component, i+1, at time t client — send — i+1 .
  • the server receives request i+1 at time t server — recv — i+1 , processes request i+1 , and sends a response, responses i+1 302 to request i+1 , at time t server — send — i+1 .
  • the client then receives response i+1 304 , at time t client — recv — i+1 . It is observed that there is a dependency between t server — send — i and t server — recv — i+1 .
  • the client may send a request on B for an additional component, while the client is downloading a component on A, but the client will not send a request, on channel A, until that client has finished downloading a component on channel A.
  • the time between the server sending a response to a request (responses) and the server receiving a subsequent request (request i+1 ) on the same channel is called the inter-request-time.
  • the inter-request time may help determine the bandwidth of a client connection, as well as helping to determine the client's round trip time, as will be described below.
  • a server may typically record, in a first log 98 , the IP addresses, names or other identification information of clients connecting thereto.
  • a server may also record a second log 100 of a first line of the client's request, including the HTTP method and the URL requested.
  • the server may record a third log 102 of a date/time stamp, which, depending on the server, can hold a representation of the time the server starts processing the request or the time when the server started writing a response.
  • the server may record a fourth log 104 of the HTTP response status code, and a fifth log 106 , of the number of bytes in the response, not including headers.
  • Servers may optionally record a sixth log 108 of information about cookies stored on the client, and a seventh log 110 , of the server's processing time.
  • FIG. 5B shows additional logs that may be recorded by a server using a server side latency estimator, according to preferred embodiments of the present invention.
  • the logs may be recorded by the logging mechanism already on the server, or may be recorded independently.
  • the server records a first additional log, for each channel, of the time the server accepts the channel connection 116 .
  • the server For each request the client sends on each channel, the server, preferably, additionally records:
  • a first request log 112 which comprises a flag which indicates if the request is the first request on a given channel
  • a second request log 114 comprising the client's port number
  • a third request log 118 comprising the time the server starts processing the request
  • a fourth request log 120 comprising the time the server completes writing a response to the channel
  • a fifth request log 122 comprising the number of bytes left to send when the server logs the request, that is, the number of bytes left to send when the server completes writing the response to the channel, and records the fifth request log.
  • the server For each web transaction, the server also logs a client's ping time 119 . Preferably, at pre-defined intervals, the server additionally logs a queuing latency, 121 , as recorded by a queuing latency probe, which is described below. Preferably, the log includes a timestamp, showing the time the latency was recorded.
  • FIG. 6 is a simplified block diagram of a server side latency estimator, operative in accordance with a first preferred embodiment of the present invention.
  • the server side latency estimator is associated with a server, for example a web server.
  • the server side latency estimator comprises an event observer 123 , which is operable to observe various pre-defined events happening at, or about, the server the device is associated with.
  • the event observer is connected to a logging device 134 , which is operable to log events, and time information relating to those events, such as when events occur, or a time duration between related events.
  • the server side latency estimator may use the server's logging device, or the server side latency estimator may have its own, separate, logging device.
  • the server side latency estimator may record logs, with the logging device, as described in FIGS. 5A and 5B.
  • the server side latency estimator also comprises the queuing latency probe 130 , referred to above, which may be connected to the logging device 134 .
  • the queuing latency probe preferably sends a small request to the server using a new connection, such as a new TCP connection, at pre-specified intervals.
  • the queuing latency probe measures the time it takes for a server to accept the new connection, that time corresponding to the so-called queuing latency.
  • the time for the 3-way handshake will be minimal, and the queuing time will be dominant, as the connection between the probe and the server is likely to have high bandwidth, and a small round trip time.
  • the queuing latencies are approximately the same for all requests, as the queuing time is primarily dependent on the server's conditions, namely how many requests the server is handling at a given time.
  • the queuing latency probe measures the time it takes for the server to accept new requests, and the logging device may keep a log of time information regarding the server's responses as described above.
  • the queuing latency estimator may also alert a server's administrator if the queuing latency estimator finds that its requests are being dropped.
  • An external pinger 132 is connected to the logging device, and preferably sends out a ping, or ping like communication, to each client connecting to the server. Sending out a ping to each client may be accomplished in a number of ways.
  • the external pinger runs in parallel to the web server and at low rate reads the access log. For each new client IP address appearing in the access log the external pinger sends a single ping and waits for the response. It is noted that because the pinger uses a single ping measurement and that the ping measurement may take place some time after the actual web page transaction, the measurement may not be completely accurate, but the ping measurement still serves as a good approximation of the actual round trip time.
  • the external pinger has minimal processing and communication overhead.
  • the external pinger may also be used to identify routing problems, by pinging a list of frequent hosts/subnets, at low rate, and reporting any problems to a server's administrator.
  • connection procedure is preferably modified such that on each connection establishment an interrupt is sent to the external pinger, which then pings the client. This has the advantage of giving a more accurate ping time. Also the external pinger does not then have to read the log.
  • variation increases system overhead.
  • the logging device 134 preferably records a log of all the measured ping times for each client, that is, the log records time information relating to the duration between the external pinger pinging the client, and receiving the client's response.
  • a sentry inserter 136 a device that may modify data being sent to include a prompt for the client to make an additional request at the end of the client's reception of the data, the prompt for the additional request hereafter referred to as a sentry 136 .
  • the sentry preferably causes the client to make an additional request at the end of data receipt, the additional request thus serving as an indicator that all of the data has been received.
  • the sentry simply comprises an additional image tag, located at the end of the HTML, for an image that has no contents and may be one pixel. The request prompted by the sentry determines a latest time when the client finishes receiving the main page.
  • the sentry may be used to provide an initial estimation for the main page latency, and, in certain cases, for example, when the web page has a large amount of HTML, and few, small images, the web page latency.
  • the sentry may generate two sequentially dependent requests which enable the rate estimator to make an estimation of the bandwidth and the round trip time, as will be described below. It is noted that the sentry has a negligible overhead on the server, the client, and the communication network.
  • the client When the client receives the sentry, the client preferably makes the additional request to the server. As mentioned, the image is only one pixel at the end of the page, and therefore the client is preferably able to render the page.
  • the request generated by the client as a result of the sentry may thus serve to help determine when the client receives the last of the data, that is when the client finishes downloading the web page.
  • the purpose of the ssentry is to provide a better main page latency estimator, but it provides better web-page latency estimation. If the client is using pipelining and the last image sent [not including the sentry] is large, it will finish after the client receives the sentry. Hence, in such a case, the time the client receives the sentry does not mean that he received all the web page!
  • the server side latency estimator preferably has a round trip time estimator 129 associated with the logging device 134 , for estimating round trip times.
  • the server side latency estimator preferably has a data rate estimator 124 , connected to the logging device, which is composed of:
  • connection perceived packet rate estimator 126 connected to the connection perceived packet rate estimator 126 ,
  • a perceived packet line rate estimator 128 connected to the perceived packet rate estimator 125 .
  • the perceived packet rate estimator 125 may be used to estimate a perceived packet rate, Rate inter-request , for each transmission.
  • the connection perceived packet rate estimator 126 may estimate a connection perceived packet rate, Rate Conn , which is a packet rate for each client connection, preferably based on individual channels to individual clients. Each new download of a new web page is treated separately, so that for each download the client preferably receives one ping.
  • the packet rate is preferably based on all the estimated packet rates per transmission for each respective client.
  • the web transaction perceived packet rate estimator 127 may estimate a web transaction perceived packet rate, Rate Web-Trans , which is taken to be the average packet rate of all connection rates Rate conn . to a particular client.
  • the perceived packet line rate estimator 128 may estimate a perceived packet line rate, Rate line , which is an average packet rate per client, based on all the estimated packet rates per transmission estimated for each respective client, wherein those packet rates fall within specific size criteria as discussed below.
  • FIG. 7 is a simplified flow chart showing how the round trip time estimator 129 may estimate a round trip time for a given client.
  • the round trip time estimator accesses the log of ping times 140 , and finds a respective ping time, as determined by the external pinger, for the client of a current web transaction 142 .
  • the determined ping time called T1.
  • T1 serves as an initial estimate of the round trip time.
  • the round trip time estimator examines the logs for the current client connection, and determines the smallest inter-request time for the current web-transaction 144 , the smallest inter-request time providing a second estimated round trip time T2.
  • the round trip time estimator checks if T1 is smaller than T2 146 . If T1 is smaller, the round trip time estimator estimates the round trip time to be T1 148 . Otherwise, the round trip time estimator estimates the round trip time to be T2 150 . It will be noted that the estimated round trip time is not the real round trip time, rather, it is larger then the real round trip time.
  • the estimated round trip time is biased due to the size of the transmission, and that applies to a ping, even though it is a single packet.
  • the accuracy of the estimated round trip time depends on the bandwidth, as the data being sent and received is not infinitely small. The estimation will improve as the bandwidth increases.
  • the estimated round trip time may be smaller than the actual, average round trip time.
  • Different packets may take different routes, and as the more packets go from the server to the client there is a tendency towards a best route.
  • Eventually most of the packets use the smallest round trip time, which is from the best route, and this may in fact be better than the average round trip time.
  • the RTT as estimated is just an estimator of the real one, and and is close to the average RTT but not exactly the same.
  • FIG. 8 is a simplified flow chart showing how the data rate estimator 124 of FIG. 6 may estimate various transmission rates.
  • a transmission rate is a measure of how much data is sent over a given time period. In general transmission rates may be estimated by dividing an estimated transmission time by the amount of data transmitted.
  • the Internet is a packet based network, so in the present embodiment the data rate estimator estimates a packet rate, a rate at which packets are sent, i.e. the number of packets sent in a given period of time. Before the server side latency estimator may start processing the logs, it may split the logs into web page transactions.
  • Spitting the log into web-page transactions may be done by sorting the server's logs according to the timestamps of the events logged, and then splitting the logs according to the client IP addresses. In this manner the server side latency estimator may deal with only a small part of the log, for each client latency estimation.
  • Response Size i is the size of the present response
  • t server — recv i+1 is the time the server receives a request subsequent to the present request, i,
  • t serv — send is the time the server starts sending the response, i.
  • RTT is the minimum round trip time, in accordance with the embodiment of FIG. 7.
  • the perceived packet rate estimator divides 162 the response size (the number of bytes in the response, or any other applicable measurement of size), by the size of the packets (or other transmission unit) being sent across the connection, giving the approximate number of packets (or other transmission units) the server may send for the response.
  • the data rate estimator determines a time, T subsequentReq , when a subsequent request is received on the channel the response is sent on, provided that such a subsequent request exists.
  • the data rate estimator preferably determines an initial transmission time T TransTime by subtracting the time when the server finishes writing the response with the T subsequentReq request, from T subsequentReq .
  • the initial T TransTime estimation includes the time it takes for the T subsequentReq request to get from the client to the server.
  • the time it takes for the T subsequentReq request to get to the server is not part of transmitting the response from the server to the client, so the data rate estimator subtracts half the estimated round trip time 164 estimated for the respective client determined by the round trip time estimator 129 .
  • the actual transmission time cannot be less than half the round trip time, so the data rate estimator estimates the transmission time to be the larger of half the round trip time, and T TransTime minus half the round trip time 166 .
  • the perceived packet rate namely the Rate inter-request
  • the data rate estimator estimates a connection perceived packet rate, namely Rate conn , which is the average Rate inter-request for a given connection 169 to a given channel.
  • the data rate estimator estimates a web transaction perceived packet rate, namely Rate Web-Trans , which is the average Rate conn for all connections 170 .
  • the data rate estimator checks if there is a sufficient number of properly sized Rate inter-request 's, that is, Rate inter-request 's whose transmission times are within a certain time range 171 .
  • the data rate estimator marks the Rate line as invalid 172 . Otherwise the data rate estimator estimates 174 a perceived packet line rate, namely Rate line , as the average of all properly sized Rate inter-request .
  • FIG. 9 is a simplified flow chart showing how a server side latency estimator may estimate the time when a client completes the receipt of the HTML portion of a web page.
  • t server — send HTML is the time the server sends the client the HTML, t server_recv next_inter ⁇ _request
  • RTT is the round trip time
  • the server side latency estimator may estimate the time the client receives the HTML portion of the page based on equation 5.
  • the server side latency estimator estimates three different times the client may have received the main page, and sets the minimum (earliest) of these estimates as an approximation of the time the client receives the HTML potion of the web page.
  • the server side latency estimator checks if the Rate Conn . is valid 182 . If the Rate conn is not valid the server side latency does not make or use an estimation dependent on the Rate conn , but instead marks the first preliminary estimated time the client receives the HTML portion of the web page, T1, as invalid 184 .
  • T1 is set to a time the server finishes writing the last of the HTML potion of the web page to the present channel, plus the estimated time it takes for remaining data to be sent, once the server finishes writing the HTML portion of the web page to the present channel.
  • the time needed for sending the remaining data is estimated by dividing the size of the data to be sent by the packet size, giving a total number of packets to be sent.
  • the number of packets to send is then divided by the Rate conn , giving an estimation of the time needed to send the remaining data.
  • the server side latency estimator checks if there have been more than two HTTP requests 188 on the channel the server sent the HTML portion on.
  • the server side latency estimator does not make or use the second estimated time of receipt of the HTML, and marks T2 invalid 190 . Otherwise, the server side latency estimator estimates T2 using the time the server received the second HTTP request on the present channel, minus half the round, trip time 192 . Assuming the server side latency estimator is using a sentry (FIG. 6, 136) the server side latency estimator makes a third estimate of the time the client receives the HTML, T3, using the time the server received the request generated by the sentry ( 136 ) minus half the round trip time 194 . The server side latency estimator then estimates the time the client receives the HTML as the minimum (earliest) of the valid estimations of T1, T2, and T3 196 .
  • FIG. 10 is a simplified flow chart showing how a server side latency estimator may estimate the user perceived latency for a given download of the HTML portion, namely the main page, of a page.
  • the client's main page latency may be estimated by the equation: Main ⁇ ⁇ Page ⁇ ⁇ Latency ⁇ T Queuing + 1.5 ⁇ RTT + t client_recv HTML - t serv_recv HTML ( eq . ⁇ 6 )
  • T Queueing is the queuing latency
  • RTT is the round trip time
  • t client — recv HTML is the time the client receives the HTML portion of the web page
  • t server — recvHTML is the time the server receives the request for the HTML from the client.
  • the server side latency estimator estimates the time at which the client receives the HTML 200 , as described in the embodiment of FIG. 9
  • the server side latency estimator subtracts the time the server accepts the connection from the client 202 , as recorded in the logging arrangement of FIG. 5B, which is used as the approximate time the server received the request for the HTML 116 .
  • the server side latency estimator adds the queuing latency as determined by the queuing latency probe ( 130 ) 204 , as an estimation of T Queueing and adds one and a half the estimated round trip time 206 , to account for the three-way handshake time 74 (FIG. 2), necessary to establish the connection.
  • FIG. 11 is a simplified flow chart showing how a server side latency estimator may estimate the time when a client receives an entire web page.
  • a given channel may carry one or more transmissions, and a given client may use more than one channel.
  • [0338] is the time the client receives the last of the server's responses, namely the time the client receives the web page, t client_recv last , j
  • [0339] is the time the client receives the last response on a given channel, j, and
  • N conn is the number of connections, namely the number of channels that the client uses, in its communication with the server.
  • t client — recv last,j is the time the client receives the last response on a given channel, j,
  • t client — recv html is the time the client receives the HTML
  • N HTML is the number of requests on the channel of the HTML request
  • j is a current connection, the present channel
  • C HTML is the channel the HTML is sent on
  • t serv — send last,j is the time the server finishes writing the last HTTP request into the channel j
  • Response size last,j is the remaining part of the last response when the server finishes writing to connection j, i.e. the number of bytes left to send, and
  • Packet Size is the size of the packets being sent.
  • the server side latency estimator may perform the following analysis on all the channels a client has opened with the server 210 to determine when the client received the last data on a given channel 212 .
  • the server side latency estimator checks 214 if, on the current channel:
  • the server side latency estimator may determine the time at which the client received the last data on the current channel to be the time the client receives the HTML, that is to say the result of FIG. 9 step 196 . If the above conditions are not both true, the server side latency estimator checks if Rate line is valid (FIG. 8, 172, 174 ). If Rate line is valid the server side latency estimator estimates the time the client receives the last data on the current channel by taking the time the server finishes writing the last data to the channel, and adding an estimate, using Rate line , of the time it takes for the data remaining to get to the client.
  • the time it takes for the data remaining, to get to the client is estimated by determining the number of packets left once the server has finished writing to the channel, and dividing the number of packets left by the Rate line .
  • the number of packets left is determined by dividing the size of the data remaining, when the server has finished writing the last of the data to the channel (recorded in the embodiment of FIG. 5B, 122), by the size of the packets being sent.
  • the number of packets left is divided by the Rate line 220 , and the result is added to the time the server finishes writing the response to the channel, to give an estimated time at which the last data from the current channel is received by the client 224 .
  • the server side latency estimator estimates the time the client receives the last data on the current channel by taking the time the server finishes writing the last data to the channel, and adding an estimate, using Rate Web-Trans , of the time it takes for the data remaining, to get to the client.
  • the time it takes for the data remaining, to get to the client is estimated by determining the number of packets left, once the server has finished writing to the channel, and dividing the number of packets left by the Rate Web-Trans .
  • the number of packets left is determined by dividing the size of the data remaining to be sent, once the server has finished writing the last of the data to the channel (recorded in the embodiment of FIG. 5B, 122), by the size of the packets being sent.
  • the number of packets left is divided by the Rate Web-Trans 222 , the result being added to the time the server finishes writing the response to the channel, to give an estimated time at which the last data form the current channel is received by the client 224 .
  • FIG. 12 is a simplified flow chart showing how a server side latency estimator may estimate a client's latency for a given download of a complete web page.
  • FIG. 11 shows a means to estimate the time when the client receives the last of the data sent. To estimate the client's latency, it is necessary to know both when the client receives the final data, and when the client sends the initial request.
  • the client's latency may be estimated by the equation: Web ⁇ ⁇ Page ⁇ ⁇ Latency ⁇ T Queuing + 1.5 ⁇ RTT + t client_recv Last_Response - t serv_recv HTML ⁇ where ⁇ : ( eq . ⁇ 9 )
  • T Queueing is the queuing latency, t client_recv Last_Response
  • [0360] is the time the server receives the request for the HTML from the client.
  • the server side latency estimator adds the queuing time 230 , to one and a half times the round trip time 232 , and then to the time when the client receives the entire document (FIG. 11, 226) 234 .
  • the server side latency estimator subtracts the time the server receives the initial request for the first HTML 236 which is to say the time the server accepts the connection on which the HTML was sent, FIG. 5B, 116, from the previous result, thereby obtaining a total web page latency.
  • DNS lookup time is largely a function of the various clients ISP's and is not largely affected by a web-server's behavior, provided that the web server is listed with the root DNS servers.
  • DNS lookup times may vary widely, but are generally only a small part of the total page loading time, and thus do not affect the estimation algorithm very much.
  • a server may, however, verify that it does not have DNS problems.
  • the world is divided into several zones, wherein each zone is under the responsibility of a well known root DNS server. Using a program like nslookup to query each one of the root DNS servers with a server's domain name at predetermined intervals a server may report to an administrators about DNS problems in any zone.
  • clients pipeline all requests. Pipelining involves the client sending requests for subsequent page components on a channel before completely receiving previous components on that channel. Referring to FIG. 4, t client — send — i+1 298 , may precede t client — receive — i 296 .
  • t client — send — i+1 298 may precede t client — receive — i 296 .
  • the client may send a request for a subsequent component, before the client has finished downloading a given component, measuring the round trip time, based on inter-request times, may become much less accurate. However, the ping time remains a valid estimation of the round trip time. Thus in the embodiment described in FIG. 7, the round trip time is estimated to be Ti (step 142 ).
  • the server may write to non-empty buffers, and subsequent measuring of the various transmission rates may use the decrease in the buffer size over time, to determine the rate of data delivery, instead of the size of the entire buffer, and the inter-request time.
  • step 162 the response size may be determined to be the decrease in the buffer size over time, as opposed to the total buffer size, over the time when the client sends a subsequent request. It is noted that it is possible to simply determine a data rate, instead of the various packet rates, as the reduction in buffer size/time and this can be achieved without the need to log any further information. No new entries are needed since, at the time of finishing writing the images it is possible to make a determination of the current buffer size. At that determination it is possible to know how much of these belong to a current image and how much belong to the previous images.
  • the client may be served data by multiple servers.
  • Multiple servers reduce the bottle-neck effect of a single server, and are often referred to as load balancing servers, as they balance the load from the many requests for web pages, between multiple different servers.
  • the server side latency estimator merges the logs of the various servers. With the logs merged together the server side latency estimator proceeds in the same manner as described for a single server log.
  • the client accesses the server through web-proxies. Multiple clients may use the same proxy to access a server, and each of those clients may have different connection rates, and connection times. The logs would make it appear that multiple clients accessing the server through the same proxy are the same client.
  • the server side latency estimator may determine which client each request is from by assigning each client a cookie, and then using cookie information incorporated into the requests to determine which request is from which client.
  • servers which, when contacted for data, refer the client to other servers, so that a single request to a single server may lead to downloads that come from numerous servers.
  • a web site may have an advertisement, and the advertisement may be served by a server, which keeps records about the advertisements.
  • client browsers find references to other servers, the browser may open additional connections (channels) to those servers to download material from them.
  • These new connections (channels) may slow down the rate of the connections (channels) to the original server, if the client has a limited bandwidth.
  • the new requests on the already established connections (channels) to the original server may suffer from decrease in rate, which may be noticeable from the access log, which may be taken into account by the server side latency estimator.
  • the server side latency estimator may accurately estimate the latency of the HTML document, in the above cases.
  • the server side latency estimator may accurately estimate the download time of the images located in the original server, which, for most cases is the major portion of the images.
  • FIG. 13 is a simplified schematic of an architecture used to test a preferred embodiment of the invention.
  • a web-server 310 which is connected to a LAN 326 .
  • the LAN has four computers, 312 , 314 , 316 , 318 , which serve to send requests to the web-server. These requests may simulate various loads on the web server.
  • the LAN is connected to a WAN, and the Internet 324 , which is connected to two web-client simulators, 320 , and 322 , located at various distances from the server.
  • the server side latency estimator was implemented on an Apache web-server 310 , running version 1.3.9, a public domain HTTP/1.1 server. Measurements were conducted at three locations, namely at the web server, referred to as TAU, 310 , and at both clients 320 , and 322 .
  • TAU web server
  • MTA One web client
  • UCI UCI
  • 322 client was located relatively far from the server.
  • the web server 310 includes three elements, the modified Apache Web server, the queuing latency probe and the external pinger.
  • the server 310 runs on a dedicated PC running the Linux operating system version 2.2.14.
  • the hardware platform is an Intel Pentium III 550 MHz system, with 512M of memory and 512K cache.
  • the disk size is 6 Gigabyte.
  • the server 310 has a standard 100 Mbit/sec Ethernet card, which is connected to a 1 Gbit/sec LAN.
  • web clients, 320 , 322 that simulate a web browser, and perform measurements of the main page latency, the web page latency and other parameters like RTT, are needed.
  • a multi-thread web browser program utilizing a BSD-socket interface was implemented to perform these functions.
  • the web-browser program supported HTTP/1.1 persistent connections (channels) and simulated the dependency between the request for the HTML document and the requests for the embedded images.
  • the program fetched whole web pages (HTML document and embedded images) using a predefined number of persistent TCP connections.
  • a computer ran ten web loader threads.
  • several computers 312 , 314 , 316 , 318 , running the web loader program were located on the same LAN, 236 , that the server, TAU, 310 was located on.
  • FIG. 13 shows four such computers used with the web loader program to generate the appropriate load on the server. During testing, between zero and nine computers are used with the web-loader program, as will be explained in greater detail below.
  • the PingER project at Stanford Accelerator Center (SLAC) (L. Cottrell, W. Matthews, and C. Logg. Tutorial on internet monitoring pinger at SLAC. Available from http://www.slac.stanford.edu/comp/net/wanmon/tutorial.html/, 1999) conducts continuous network monitoring to measure network latency and packet loss over the Internet.
  • the PingER measurements show average loss rate of 0.5% and RTT of 30-60 milliseconds in the U.S., Asia and Europe, while between the continents the average loss rate is of 1.4% and RTT varies between 200 to 600 milliseconds.
  • the server side latency estimator experiments were conducted at various hours of the day, over a one week span. During the week of experiments, the clients' network characteristics, in terms of RTT, bandwidth and loss rate were measured. UCI's, 322 , RTT was 600-620 milliseconds, bandwidth 30-40 KB/s and loss rate of 0%. MTA's, 320 , RTT was 30-100 milliseconds, bandwidth 5-17 KB/s and loss rate of 2%. UCI's, 322 , RTT reflect longer network RTT to the US from the server (located in Tel-Aviv). UCI's, 322 , RTT, bandwidth, and loss rate showed a minor disparity. MTA, 320 , showed large disparity in the RTT and in the bandwidth. The two sites chosen thus show good representation of real-world web traffic characteristics, as they cover large ranges of RTT, bandwidth and loss rate.
  • FIG. 14 is a graph of the accuracies for the main page latency, and web page latency
  • FIG. 14 depicts the accuracy of the server side latency estimator's main page latency, and web page latency, estimations, for all the tests runs for both of the clients under the various server loads.
  • the graph of FIG. 14 also shows the server side latency estimator's accuracy for the RTT (RTTmin from the inter request times, or the estimated RTT from the external pinger).
  • RTTmin from the inter request times, or the estimated RTT from the external pinger.
  • the external pinger is an integral part of a preferred embodiment of the server side latency estimator.
  • the performance of the server side latency estimator is preferably evaluated using the pinger for estimating the RTT More precisely, what may be used is a minimum of a ping time and the minimum inter-request time.
  • the table below summarizes the median and average of the estimation error.
  • the table below shows the median value of the estimation error as the average values are shifted by the few high errors in the tests runs using an overloaded server.
  • the average latency estimation error of a preferred embodiment of the server side latency estimator, for the various tested web pages, is 4% under normal server loads, and 10% when the server is overloaded.
  • the server side latency estimator uses the external pinger for accurate RTT estimations.
  • FIGS. 15A, 15B, 15 C and 15 D are cumulative distributions of latency estimation error, and RTT error, for typical web-pages.
  • FIG. 15A is a graph of the error of a page using 15 embedded images, from a relatively close client, MTA, 320 .
  • FIG. 15B is a graph of the error of a page using 50 embedded images, from a relatively distant client, UCI, 322 .
  • FIG. 15C is a graph of the error of a page using 30 embedded images, from a relatively close client, MTA, 320 .
  • FIG. 15D is a graph of the error of a page using large embedded images, from a relatively distant client, UCI, 322 .
  • FIGS. 15A, 15B, 15 C, and 15 D depict the accuracy of the server side latency estimator's, main page latency, web page latency, and RTT estimations as a function of the number of the embedded images, for both of the tested clients.
  • the runs of client downloads of web pages with the same number of embedded images, under various server loads are aggregated to form graphs.
  • the latency estimation errors for MTA and UCI's are similar so MTA's errors are presented for web pages with 15 and 30 embedded images and UCI's errors are presented for web pages with 50 embedded images.
  • the below table summarizes the average estimation errors.
  • the average estimation error for MTA is larger than the error estimated for UCI because: 1) UCI has high bandwidth, 2) MTA has high packet loss. It is concluded that the average latency estimation error is 4% for a typical web page,
  • FIG. 16A is a graph of the cumulative distribution of latency estimation error for HTML documents, with five inline images, for the client simulator at MTA.
  • FIG. 16B is a graph of the cumulative distribution of latency estimation error for HTML documents, with five inline images, for the client simulator at UCI.
  • FIGS. 16A and B depict the accuracy of main page latency and web page latency as estimated by the server side latency estimator for web pages with few embedded (inline) images. The latency estimation error for web pages with two and five embedded images behave similarly so the error for web pages with five embedded images is presented. Each figure also depicts the effect of the method for estimating the RTT.
  • the below table summarizes the average errors for UCI and MTA.
  • FIGS. 17A and 17B are graphs of the cumulative distribution of latency estimation error for web pages with various numbers of embedded (inlined) images, where the estimations were made using data collected on a server that was overloaded.
  • FIG. 17A is a graph of the cumulative distribution of latency estimation error for web pages with few embedded (inlined) images, with the estimations made using data collected on a server that was overloaded
  • FIG. 17B is a cumulative distribution of latency estimation error for web pages with an intermediate number of embedded (inlined) images, with the estimations made using data collected on a server that was overloaded.
  • FIGS. 17A and 17B depict only the runs with 5 and 15 embedded images as other tests showed similar behavior, and these are representative. UCI's results are not shown, as they exhibit similar behavior.
  • the runs of web pages with the same number of embedded images are aggregated.
  • Each figure depicts the accuracy of an embodiment of the server side latency estimator for estimating the RTT.
  • the below table summarizes the median and average of the latency estimation errors for the serve side latency estimator at MTA.
  • the behavior of the latency estimation error is similar for the tests run under normal server load and overload server load as may be seen in FIGS. 15A, 15B, 15 C, 15 D, 16 A, 16 B and 17 A and 17 B, with the exception of the long tail in the overloaded cases FIGS. 17A and 17B.
  • the queuing latency probe of the present embodiment samples the queuing latency at a low frequency. If the queuing latency probe were to sample the queue at high frequency, the estimation accuracy of server side estimator, for an overloaded server, may go up dramatically. There is need for the external pinger only for web pages with few embedded images. The average latency estimation error is 10%, in contrast the median latency estimation error is only 4%. Meaning RTT Average Median Estimation RTTmin RTTmin Method (1) Pinger (1) Pinger No.
  • the server CPU overhead due to the additional fields logged is 0.15%.
  • the queuing latency probe samples the server every four seconds, which adds an average 0.025% CPU overhead.
  • the external pinger runs once every 30 seconds, which adds, on average 0.04% CPU overhead.
  • the total average server CPU overhead due to our measurements is less the 0.2%.

Abstract

A network latency estimation apparatus for estimating latency in network communications between a server and a client. The apparatus comprises an event observer for observing occurrences of pre-selected events. The events associated with the communication occurring at the server. A logging device associated with the event observer for logging into a data store the occurrence of the events together with respective time information. A latency estimator associated with the logging device for using the logged occurrences with the respective time information to arrive at an estimation of a client's latency for the communication.

Description

    FIELD OF THE INVENTION
  • The present invention relates to determining client latencies over networks, particularly during the course of downloading procedures, and more particularly but not exclusively to determining client latencies in downloads of web pages over the Internet. [0001]
  • BACKGROUND OF THE INVENTION
  • A central performance problem in the World Wide Web, in recent years, has been client latency. Client latency may be defined as the time a client has to wait, between requesting data from a server, and finishing to download the requested data, and all data associated with the request. [0002]
  • Impatience with poor performance is the most common reason a user's visit to a web sites is abandoned. For e-commerce sites, these abandonments translate into lost revenue. Measurements of client latencies are critical in order to minimize client latencies, so that users are satisfied with their experience, and do not abandon the download. Once the client latency is understood, a site can reduce its latency in a number of ways, for example: [0003]
  • (a) deploying a mirror site, [0004]
  • (b) buying wider connectivity to the Internet, [0005]
  • (c) deploying a more powerful web server [0006]
  • (d) load balancing [0007]
  • (e) altering the objects on the site, and [0008]
  • (f) altering the placement of the objects on the site. [0009]
  • As of today, the main way of conducting client latency measurements is to use agents external to the site. The agents are deployed at a limited number of locations around a network, be it an intranet, or on the Internet, and the agents fetch specific web pages from a web server, specifically to measure the latency from the limited number of locations where those agents reside. The disadvantages of this method are: [0010]
  • (1) The server is dependent upon an external body for conducting the measurements, [0011]
  • (2) The agent's measurements do not necessarily reflect actual client experience, [0012]
  • (3) The perceived latency measured by the agents does not have a breakdown of the various latency components, and [0013]
  • (4) The agents' DNS lookup time is effected by prior DNS lookup queries. [0014]
  • Client latencies will vary based on what version of HTTP a client is using. In HTTP 1.0 a client would establish a new TCP connection for each HTTP request. Using a new TCP connection for each request led to incurring connection-establishment latencies on each request. Connection establishment times will be described in greater detail below. With HTTP 1.1, clients establish a persistent TCP connection with the server, and those connections are re-used for multiple HTTP requests, and responses. [0015]
  • What is needed is a solution that does not require agents to be placed at external locations on the network. Preferably, such a solution should analyze, for each client of the server, what latencies are experienced by the individual client. It would be additionally beneficial if such a measuring device required no additional hardware, and could be optionally run during actual server operation, or during off peak times. [0016]
  • An attempt at a description of such a solution was made by Balachander Krishnamurthy, and Jennifer Rexford in their paper “En Passant: Predicting HTTP/1.1 traffic. Proc. Global Internet Symposium, 1999.” Their means required accurate clock synchronization, between a client and a server, and low lever TCP/IP traces. [0017]
  • SUMMARY OF THE INVENTION
  • According to a first aspect of the present invention there is thus provided a network latency estimation apparatus for estimating a client's latency in a network communication between a server and said client, the apparatus comprising: [0018]
  • an event observer for observing occurrences of pre-selected events associated with said communication occurring at said server, [0019]
  • a logging device, associated with said event observer for logging into a data store the occurrence of said events together with respective time information, and [0020]
  • a latency estimator associated with said logging device for using said logged occurrences with said respective time information to arrive at an estimation of a client's latency for said communication. [0021]
  • Preferably, said client's latency comprises the difference between a first time at which a client sends request data and a second time at which said client completes receipt of the requested data. [0022]
  • Preferably, the latency estimator further comprises a round-trip-time estimator, capable of estimating a round trip time for said client, said estimation being based on receipt at the server of a request for data from the client in consequence of data previously sent to the client by the server. [0023]
  • Preferably, said round-trip-time-estimator is operable to estimate the round trip time by determining the shortest duration over a plurality of durations, measured between when a server sends data to a client, and when the server receives a subsequent request from the client, said subsequent request being in consequence of said data sent to the client by the server. [0024]
  • Preferably, the apparatus comprises a pinger associated with said event observer. [0025]
  • Preferably, said events comprise sending out a ping, and receiving a response. [0026]
  • Preferably, the round-trip-time estimator is operable to estimate the round trip time based on the shorter of: [0027]
  • (a) a duration between a first time when the server sends data to a client, and a second time when the server receives a subsequent request from the client, and [0028]
  • (b) the logged times of said ping event. [0029]
  • Preferably, said data store contains at least some of: [0030]
  • (a) an IP address of the client, [0031]
  • (b) a number of bytes in a server's response, [0032]
  • (c) a server's processing time, [0033]
  • (d) a flag indicating whether the current request is the first request on a current channel, [0034]
  • (e) a time the server accepts a connection from the client, [0035]
  • (g) a time the server starts processing a client's request, [0036]
  • (h) a time the server completes writing the response to the channel, and [0037]
  • (i) a number of bytes left to be sent when the server completes writing requests to the channel. [0038]
  • Preferably, said latency estimator is operable to form an initial estimate of the client's latency by adding an estimated round trip time, to a delay between when the server receives an initial request for data from said client, and when the server receives a subsequent request for data from said client [0039]
  • Preferably, the apparatus comprises a receipt indicator, associated with said event observer, operable to add to the end of data being sent an automatic end of data receipt indicator, for obtaining an acknowledgment of receipt of the end of the data by said client. [0040]
  • Preferably, the apparatus comprises a client-data-reception-time estimator operable to estimate a time when the client received said data sent as the time when the server receives the acknowledgment of the receipt of said data. [0041]
  • Preferably, said latency estimator is further operable to estimate the client's latency by using the earliest of the times determined from: [0042]
  • when the server receives the acknowledgment of receipt of the data, and [0043]
  • when the server receives any other request from the client. [0044]
  • Preferably, said latency estimator is further operable to estimate the client's latency by using the earliest of the times determined by when the server received the acknowledgment of receipt of the data sent, and when the server received any other request from the client wherein said other request is in consequence of said data sent. [0045]
  • Preferably, the apparatus comprises a queuing latency probe, associated with said logging device, capable of sending a queue probing request to the server. [0046]
  • Preferably, said pre-selected events further comprise: [0047]
  • sending, by the queuing latency probe, of said queue probing request, and [0048]
  • the server accepting said queue probing request. [0049]
  • Preferably, said request is an HTTP request. [0050]
  • Preferably, the apparatus comprises an adder for adding an elapsed time between when said queuing latency probe request is sent to the server, and when said request is accepted for handling by the server, to said initial latency as measured by the client-latency estimator. [0051]
  • Preferably, the apparatus comprises a transmission-data-rate estimator, connected to the latency estimator, said data rate estimator comprising a divider and a subtractor, said subtractor being connected to receive as inputs the estimated round trip time, and the initial estimate of the client's latency, and to produce an output being the difference therebetween, said divider being connected to receive as a first input an amount of data sent by the server in the course of the transmission, and as a second input said subtractor output, and to produce an estimate of a transmission data rate by dividing therebetween. [0052]
  • Preferably, the apparatus comprises omprising an overall-client-data-rate estimator comprising an averager for averaging together a plurality of successively estimated transmission data rates for the given client connection thereby to estimate an overall data rate for said given client. [0053]
  • Preferably, the apparatus comprises a client-data-reception-time estimator, comprising a multiplier and an adder, said adder being functional to add a time at which the server dispatched an end part of the requested data to the channel, to an output of the multiplier, wherein the multiplier is operable to multiply an amount of data left to send, by said overall data rate, the client-data reception time estimator thereby obtaining an estimate of client-data-reception-time. [0054]
  • Preferably, the apparatus comprises a subtractor and an adder, said subtractor operable to subtract the time at which the server received an initial request for data from a client from said time the client received the last of the requested data, said adder operable to add one and a half times the estimated round trip time, and further add the latency as estimated by the queuing latency probe, to the output of the subtractor. [0055]
  • Preferably, the apparatus comprises a client-connection-data-rate estimator comprising an averager for averaging together all the transmission data rates for a given client connection wherein the corresponding transmissions meet a specific size criteria. [0056]
  • Preferably, the apparatus comprises a client-data-reception-time estimator comprising an adder and a multiplier, said adder for adding the time the server last dispatched data to the client, to the output of the multiplier, wherein said multiplier is operable to multiply an amount of data left to send by said data rate for a given client connection, thereby to produce an estimate of a time of receipt of said last data to the client. [0057]
  • Preferably, the apparatus comprises a client latency estimator for estimating a respective client latency's, the latency estimator having a subtractor and an adder, said subtractor for subtracting the time the server received an initial request for data from a client from said estimate of a time of receipt of said last data to the client, said adder for adding together one and a half times the estimated round trip time, the latency as estimated by the queuing latency probe, and the output of the subtractor, thereby to form an estimate of said respective client's latency. [0058]
  • Preferably, the apparatus comprises a global-transmission-rate estimator comprising an averager, said averager for averaging together successive data transmission rates, thereby to estimate a global transmission rate. [0059]
  • Preferably, the apparatus comprises a client-data-reception-time estimator comprising an adder and a multiplier, said multiplier operable to multiply an amount of data left to send by said global transmission data rate and said adder operable to add the time the server dispatched the last data to the client, to the output of the multiplier, thereby to provide an estimate of a client data reception time. [0060]
  • Preferably, the apparatus comprises a subtractor and an adder, said subtractor for subtracting the time the server received an initial request for data from a client from said time of receipt of said last data at said client, said adder connected to add one and a half times the estimated round trip time, to the latency as estimated by the queuing latency probe, thereby to provide a revision of the estimate of the client's latency. [0061]
  • Preferably, the apparatus comprises a global-transmission-rate estimator comprising an averager operable to estimate a global transmission data rate, said averager operable to average together a series of successively obtained transmission data rates wherein respective transmissions meet a pre-determined size criteria. [0062]
  • Preferably, the apparatus comprises a client-data-reception-time estimator operable to estimate the time the client received a last part of the requested data, the estimator comprising a multiplier and an adder, the multiplier being connected to multiply the amount of data left to send by said global transmission data rate, and to output a result to provide a first input to said adder, said adder being arranged to add said input to said time at which the server dispatched said last data part to the client, thereby to provide an estimate of said client data reception time. [0063]
  • Preferably, said subtractor subtracts the time the server received an initial request for data from a client from said time the client received the last part of said requested data, said adder further adding, to the output of the subtractor, one and a half times the estimated round trip time, and the time duration between a first time when the queuing latency probe sends the request to the server, and a second time when the server accepts the request sent by the probe. [0064]
  • Preferably, a plurality of data transmission rates are measured for a given channel, the apparatus further comprising a channel-data-rate estimator comprising an averager for averaging together all the data rates for each transmission on said given channel. [0065]
  • Preferably, the apparatus comprises an overall-client-data-rate estimator comprising an averager, said averager being operable to average together a plurality of successively measured data rates for a given client's channels. [0066]
  • Preferably, the apparatus comprises ng a global-data-rate estimator for estimating a global data rate, the estimator comprising an averager for averaging together a plurality of successively measured data rates. [0067]
  • Preferably, the apparatus comprises a transmission-packet-rate estimator operable to estimate a packet rate for a transmission, the estimator comprising a divider and a subtractor, wherein said subtractor is operable to subtract the round trip time from the initial estimate of the client's latency for the transmission to produce a subtractor output, said divider being connected to divide the number of packets the server sends in the course of the transmission, by said subtractor output. [0068]
  • Preferably, the apparatus comprises an overall-client-packet-rate estimator operable to estimate the overall packet rate for a given client connection, the estimator comprising an averager, for averaging together a plurality of successively measured transmission packet rates for the given client connection. [0069]
  • Preferably, the apparatus comprises a client-data-reception-time estimator operable to estimate a time the client received the last of the data, the estimator comprising a multiplier and an adder, said multiplier operable to multiply a number of packets remaining to be sent by said overall packet rate, thereby to produce a multiplier output, and said adder being connected to add the time the server wrote the end of the data to the client, to said multiplier output. [0070]
  • Preferably, the latency estimator estimates the client latency, said subtractor further subtracting the time the server received an initial request for data from a client from said time the client received the last of the data, thereby to produce a subtractor output, said adder further adding to the result of the subtractor one and a half times the estimated round trip time, and further adding thereto the latency as estimated by the queuing latency probe. [0071]
  • Preferably, a client-connection-packet-rate estimator estimates the packet rate for a given client connection, the estimator comprising an averager operable to average together ones of a succession of packet rates for given transmissions of a client connection wherein respective transmissions meet a pre-selected size criteria. [0072]
  • Preferably, the apparatus comprises a client-data-reception-time estimator operable to estimate a time the client received the last of the data, the estimator comprising an adder and a multiplier, said multiplier operable to multiply the number of packets left to send by said packet rate for a given client connection, to produce a multiplier output, said adder adding the time the server wrote the last data to the client, to said multiplier output. [0073]
  • Preferably, the apparatus estimates the client latency, said subtractor further operable to subtract the time the server received an initial request for data from a client from said estimated time the client received the last of the data, said adder being further operable to add to the output of the subtractor one and a half times the estimated round trip time, and said adder further operable to add thereto a queuing latency duration between a queuing latency probing request sending time when the queuing latency probe sends said probing request to the server, and a second, queuing latency probe request receipt time, when the server accepts said probing request. [0074]
  • Preferably, the apparatus comprises a global-transmission-rate estimator, comprising an averager operable to estimate a global transmission packet rate, said averager operable to average together all the transmission packet rates of all the connections to the server. [0075]
  • Preferably, the apparatus comprises a client-data-reception-time estimator operable to estimate the time the client received the last of the data, the estimator comprising an adder and a multiplier, said multiplier operable to multiply the number of packets left to send by said global transmission packet rate, to form a multiplier output, said adder for adding the time the server wrote the last data to the client, to said multiplier output. [0076]
  • Preferably, the apparatus estimates the client latency. For this purpose it comprises a further adder and a further subtractor, said further subtractor operable to subtract the time the server received an initial request for data from a client from said time the client received the last of the data thereby to form a subtractor output, said adder operable to add to the output of the subtractor one and a half times the estimated round trip time, and the duration between the a first time when the queuing latency probe sends the request to the server, and a second time when the server accepts the request sent by the probe. [0077]
  • Preferably, the apparatus comprises a global-transmission-rate estimator operable to estimate a global transmission packet rate, said averager being further operable to average together the transmission rates from all transmissions meeting a predetermined size criteria. [0078]
  • Preferably, the apparatus comprises a client-data-reception-time estimator operable to estimate the time the client received the last of the data, the estimator comprising an adder and a multiplier, said multiplier multiplying the number of packets left to send by said global transmission packet rate, said adder operable to add the time the server wrote the last data to the client, to the output of the multiplier. [0079]
  • Preferably, the apparatus comprises a subtractor and a further adder, said subtractor being operable to subtract the time the server received an initial request for data from a client from said time the client received the last of the data, said adder being operable to add to the output of the subtractor one and a half times the estimated round trip time, and the time duration between the a first time when the queuing latency probe sends the latency test request to the server, and a second time when the server accepts the latency test request. [0080]
  • Preferably, the apparatus comprises a channel-packet-rate estimator operable to estimate the packet rate for a given channel, the estimator comprising a subtractor, and a divider, wherein said subtractor is operable to subtract the round trip time from the latency measured for said client for said channel, thereby to produce a subtractor output, said divider being connected to divide a number of packets the server sends on the channel, by the output of the subtractor. [0081]
  • Preferably, each client connects via a plurality of channels, the apparatus further comprising an overall-client-packet-rate estimator operable to estimate an overall packet rate for a given client, the estimator comprising an averager for averaging together a plurality of successively measured packet rates for each of said plurality of channels. [0082]
  • Preferably, the apparatus comprises a global-packet-rate estimator operable to estimate the global packet rate, the estimator comprising an averager operable to average together a plurality of successively measured packet rates. [0083]
  • Preferably, the apparatus is operable to log HTTP transmissions. [0084]
  • According to a second aspect of the present invention there is provided a method for estimating the latency of a user client on a network communication, using measurements made in association with a server with which said client is in communication, the method comprising the steps of: [0085]
  • (a) logging in association with a server, events, and the times of occurrence of said events, and [0086]
  • (b) processing said logged times to estimate said latency. [0087]
  • Preferably, said user client latency is the latency between a first time when a client sends a request for data to a server and a second time at which said client receives a last datum of the requested data. [0088]
  • Preferably, the method comprises a round-trip-time estimation step, comprising estimating a round trip time, said estimation being based on when a request for data from the client is received by the server in consequence of data previously sent to the client by the server. [0089]
  • Preferably, the method comprises said round-trip-time-estimation step comprises estimating the round trip time by finding a shortest duration between a third time when the server sends data to said client, and a fourth time when the server receives a subsequent request from said client, said subsequent request being made by said client in consequence of data sent to the client by the server. [0090]
  • Preferably, the method comprises a step of pinging the client. [0091]
  • Preferably, the method comprises recording time information about a duration between sending said ping and receiving a response from a respective client. [0092]
  • Preferably, the method comprises estimating a data round trip time based on the shorter of: [0093]
  • (a) a shortest duration between when the server sends data to a client, and when the server receives a corresponding subsequent request from the client, and [0094]
  • (b) a time duration recorded by said pinging. [0095]
  • Preferably, the logging step logs a selection from the group consisting of: [0096]
  • (a) an IP address of the client, [0097]
  • (b) a number of bytes in a response, [0098]
  • (c) a server's processing time, [0099]
  • (d) a flag indicating if a current request is a first request on a current channel, [0100]
  • (e) a time the server accepts a connection from the client, [0101]
  • (g) a time the server starts processing a request, [0102]
  • (h) a time the server completes writing a request to a channel, and [0103]
  • (i) a number of bytes left to be sent when the server completes writing requests to a channel. [0104]
  • Preferably, the method comprises a client-latency estimation step, of: [0105]
  • determining a duration between when the server receives an initial request for data from a client, and when the server receives a subsequent request for data from that client, as a response request duration, and [0106]
  • adding thereto an estimated round trip time, thereby to form an approximation of said client latency. [0107]
  • 60. The method of claim 59, comprising adding, at the end of the data being sent, an indication to the client to send an additional request to the server. [0108]
  • 61. The method of claim 60, further comprising a client-data-reception-time estimation step comprising approximating a time when the client receives data sent as a time when the server receives a request resulting from said indication. [0109]
  • 62. The method of claim 61, wherein said client-latency estimation step comprises providing a first client latency estimate as an earliest of: [0110]
  • a time when the server receives the request in response to said indicator, and [0111]
  • a time when the server receives any other request from the client wherein said other request is in consequence of said data sent. [0112]
  • Preferably, the method comprises a queuing latency probing step, comprising [0113]
  • sending a request to the server, [0114]
  • measuring a first time at which said request is sent to the server, [0115]
  • measuring a second time at which said request is handled by the server, and [0116]
  • recording a queuing latency as a duration between said first time and said second time. [0117]
  • Preferably, said request is an HTTP request. [0118]
  • Preferably, said client-latency estimation step further comprises adding said recorded queuing latency to said initial client latency estimate, thereby to provide a revised client latency estimation. [0119]
  • Preferably, the method comprises a transmission-data-rate estimation step comprising dividing an amount of data the server sends in the course of a given transmission, by said client latency estimate for said given transmission, and subtracting the round trip time, thereby to estimate a transmission data rate. [0120]
  • Preferably, the method comprises an overall-client-data-rate estimation step of averaging together a plurality of successively measured transmission data rates for the given client connection. [0121]
  • Preferably, the method comprises a client-data-reception-time estimation step comprising: [0122]
  • multiplying an amount of data left to send by said overall data rate, and [0123]
  • adding thereto a time at which the server dispatches an end of data indication to the client, [0124]
  • thereby to provide an approximation of a time at which the client receives a last part of the data being sent. [0125]
  • Preferably, the user client latency estimation step comprises: [0126]
  • subtracting the time the server received an initial request for data from the client from said time the client received the last of the data, [0127]
  • adding one and a half times the estimated round trip time, and [0128]
  • adding the recorded queuing latency, thereby to estimate the client latency. [0129]
  • Preferably, the method comprises a client-connection-data-rate estimation step comprising: [0130]
  • identifying ones of transmissions for a respective client connection, being larger than a predetermined size, [0131]
  • averaging together respective transmission data rates of said identified transmissions, thereby to provide an approximation of the data rate for a given client connection. [0132]
  • Preferably, the method comprises a client-data-reception-time estimation step comprising: [0133]
  • multiplying the amount of data remaining to send by said data rate for a given client connection, and [0134]
  • adding thereto a time at which the server dispatched the last data part of the requested data to the client, [0135]
  • thereby to provide an approximation of a time the client received the last of the requested data. [0136]
  • Preferably, the method comprises: [0137]
  • subtracting the time at which the server received an initial request for data from a client from said time at which the client received the last of the requested data, [0138]
  • adding thereto one and a half times the estimated round trip time, and [0139]
  • adding the queuing latency thereto, [0140]
  • thereby to provide an approximation of the client latency. [0141]
  • Preferably, the method comprises a global-transmission-rate estimation step, comprising estimating a global transmission data rate, by averaging together transmission rates. [0142]
  • Preferably, the method comprises a client-data-reception-time estimation step of: [0143]
  • multiplying the amount of data left to send by said global transmission data rate, [0144]
  • adding thereto the time of server dispatch of the last data to the client, thereby to provide an approximation of a time at which the client received the last of the requested data. [0145]
  • Preferably, the method comprises: [0146]
  • subtracting the time the server received an initial request for data from a client from said time the client received the last of the requested data, [0147]
  • adding thereto one and a half times the estimated round trip time, and [0148]
  • adding thereto the queuing latency, [0149]
  • thereby to provide an estimation of the client latency. [0150]
  • Preferably, the method comprises a global-transmission-rate estimation step, comprising [0151]
  • identifying ones of a plurality of transmissions exceeding a predetermined threshold size, and [0152]
  • averaging together transmission rates of said identified transmissions, thereby to estimates a global transmission data rate. [0153]
  • Preferably, the method comprises a client-data-reception-time estimation step, comprising: [0154]
  • multiplying the amount of data left to send by said global transmission data rate, [0155]
  • adding thereto the time the server wrote the last data to the client, thereby to estimate the time the client received the last of the requested data. [0156]
  • Preferably, the method comprises: [0157]
  • subtracting the time the server received an initial request for data from a client from said time the client received the last of the requested data, [0158]
  • adding one and a half times the estimated round trip time, and [0159]
  • adding thereto the recorded queuing latency, [0160]
  • thereby to estimate the client latency. [0161]
  • Preferably, each said client communicates via at least one channel, the method further comprising a channel-data-rate estimation step of: [0162]
  • dividing an amount of data the server sends, by an approximation of the client latency measured for said channel, and [0163]
  • subtracting therefrom the round trip time, thereby to provide an approximation of the data rate for a given channel. [0164]
  • Preferably, each client communicates via a plurality of channels, the method further comprising an overall-client-data-rate estimation step of: [0165]
  • providing data rates for each one of said plurality of channels, [0166]
  • averaging together said plurality of data rates from said plurality of channels, thereby to estimate an approximation of a data rate for a given client. [0167]
  • Preferably, the method comprises a global-data-rate estimation step comprising: [0168]
  • providing data rates for a plurality of clients each communicating over a plurality of channels, and [0169]
  • averaging together a plurality of successively measured data rates thereby to estimate the global data rate. [0170]
  • Preferably, the method comprises a transmission-packet-rate estimation step of: [0171]
  • obtaining a number of packets being sent by a server in the course of a transmission, [0172]
  • dividing said number of packets by said first client latency estimate, [0173]
  • subtracting therefrom the estimated round trip time, thereby to estimate a transmission packet rate for said transmission. [0174]
  • Preferably, the method comprises an overall-client-packet-rate estimation step of: [0175]
  • obtaining a plurality of said transmission packet rates over a plurality of transmissions for a given client connection, [0176]
  • averaging together said plurality of transmission packet rates, thereby to provide an estimate of an overall client connection packet rate. [0177]
  • Preferably, the method comprises a client-data-reception-time estimation step of: [0178]
  • multiplying the number of packets left to send by said overall packet rate, [0179]
  • adding thereto the time the server wrote the end of the data to the client, thereby to provide an approximation of a time at which the client received the last of the requested data. [0180]
  • Preferably, the client-latency estimation step comprises: [0181]
  • subtracting the time the server received an initial request for data from said client from said time the client received the last of the requested data, [0182]
  • adding thereto one and a half times the estimated round trip time, and [0183]
  • adding thereto the recorded queuing latency, thereby to provide an approximation of the client latency. [0184]
  • Preferably, the method comprises a client-connection-packet-rate estimation step of [0185]
  • obtaining a plurality of transmission packet rates for a given connection, [0186]
  • identifying ones of said plurality of transmission packet rates whose respective transmissions exceed a predetermined size threshold, and [0187]
  • averaging together said identified transmission packet rates, thereby to provide a client connection packet rate approximation. [0188]
  • Preferably, the method comprises a client-data-reception-time estimation step, of [0189]
  • multiplying the number of packets left to send by said client connection packet rate approximation, and [0190]
  • adding thereto the time the server dispatched the last of the requested data to the client, thereby to provide an estimate of a time the client received the last of the requested data. [0191]
  • Preferably, the method comprises: [0192]
  • subtracting the time the server received an initial request for data from a client from said time the client received the last of the requested data, [0193]
  • adding thereto one and a half times the estimated round trip time, and [0194]
  • adding thereto the recorded queuing latency, thereby to estimate the client latency. [0195]
  • Preferably, the method comprises a global-transmission-rate estimation step of: [0196]
  • obtaining a plurality of transmission packet rates, and [0197]
  • averaging together said plurality of transmission packet rates, thereby to estimate a global transmission packet rate. [0198]
  • Preferably, the method comprises a client-data-reception-time estimation step of: [0199]
  • multiplying the number of packets left to send by said global transmission packet rate, [0200]
  • adding thereto the time the server wrote the last data to the channel, thereby to provide an approximation of a time at which the client received the last of the requested data. [0201]
  • Preferably, the method comprises [0202]
  • subtracting the time the server received an initial request for data from a client from said approximation of the time at which the client received the last of the requested data, [0203]
  • adding thereto one and a half times the estimated round trip time, and [0204]
  • adding the queuing latency, [0205]
  • thereby to provide an approximation of the client latency. [0206]
  • Preferably, the method comprises a global-transmission-rate estimation step of: [0207]
  • obtaining a plurality of transmission rates for each of a plurality of data transmissions, [0208]
  • identifying ones of said plurality of transmission rates whose respective transmissions exceed a predetermined size threshold, and [0209]
  • averaging together said identified transmission rates to form an approximation of a global transmission packet rate. [0210]
  • Preferably, the method comprises a client-data-reception-time estimation step of: [0211]
  • multiplying the number of packets left to send by said approximation of said global transmission packet rate, and [0212]
  • adding thereto the time the server wrote the last data to the channel, thereby to provide an estimate of the time the client received the last of the requested data. [0213]
  • Preferably, the method comprises: [0214]
  • subtracting the time the server received an initial request for data from a client from said estimate of the time the client received the last of the requested data, [0215]
  • adding thereto one and a half times the estimated round trip time, and [0216]
  • adding thereto the recorded queuing latency, thereby to provide an approximation of said client latency. [0217]
  • Preferably, the method comprises a channel-packet-rate estimation step, wherein said client communicates using at least one channel, the method including: [0218]
  • obtaining a client latency approximation for said channel, [0219]
  • dividing the number of packets the server is sending over the channel by the client latency approximation for said channel, [0220]
  • subtracting therefrom the round trip time, thereby to provide an approximation of a packet rate for a given channel. [0221]
  • Preferably, the client has at least one additional channel, in which case the method may further comprise: [0222]
  • obtaining an approximation of a packet rate for each of said channels, [0223]
  • averaging together said packet rates, thereby to estimate a packet rate for said client. [0224]
  • Preferably, the method comprises a global-packet-rate estimation step of: [0225]
  • obtaining a plurality of packet rates for each one of a plurality of clients, and [0226]
  • averaging together said packet rates to provide an approximation of a global packet rate. [0227]
  • Preferably, the method comprises logging HTTP transmissions. [0228]
  • According to a third aspect of the present invention there is provided a data carrier carrying data usable in combination with a general purpose computer to provide functionality capable of estimating the latency between when a client sends a request for data to a server, and when that client receives the data, using measurements made at a server, the data being usable to provide: [0229]
  • an event observer for observing pre-selected events associated with said communication occurring at said server, [0230]
  • a logging device associated with said event observer for logging into a data store the occurrence of said events together with respective time information, and [0231]
  • a latency estimator associated with said logging device for using said logged occurrences with said respective time information to arrive at an estimation of latency in said communication. [0232]
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • For a better understanding of the invention and to show how the same may be carried into effect, reference will now be made, purely by way of example, to the accompanying drawings. [0233]
  • With specific reference now to the drawings in detail, it is stressed that the particulars shown are by way of example and for purposes of illustrative discussion of the preferred embodiments of the present invention only, and are presented in the cause of providing what is believed to be the most useful and readily understood description of the principles and conceptual aspects of the invention. In this regard, no attempt is made to show structural details of the invention in more detail than is necessary for a fundamental understanding of the invention, the description taken with the drawings making apparent to those skilled in the art how the several forms of the invention may be embodied in practice. In the accompanying drawings: [0234]
  • FIG. 1 is a simplified schematic of the structure of the Internet, or other like network, [0235]
  • FIG. 2 is a simplified diagram showing some of the elements of the latency a client may experience in downloading content from a server, [0236]
  • FIG. 3 is a simplified schematic diagram of a client establishing a connection with a server, and downloading a web page from that server, using multiple (in this case, two) persistent connections (channels), [0237]
  • FIG. 4 is a simplified schematic diagram of a single channel between a client and a server, showing a relationship between the times the client requests different components of a web-page on the channel, [0238]
  • FIG. 5A is a simplified diagram showing some of the log files that may be kept by a typical server, while FIG. 5B shows additional logs that a server associated with an apparatus according to a first preferred embodiment of the present invention, may keep, [0239]
  • FIG. 6 is a simplified block diagram showing an apparatus according to a first preferred embodiment of the present invention, [0240]
  • FIG. 7 is a simplified flow chart showing how an apparatus, according to the embodiment of FIG. 6, may estimate the round trip time for a given client, [0241]
  • FIG. 8 is a simplified flow chart showing how an apparatus, according to the embodiment of FIG. 6, may estimate the transmission rate for a given transmission, and which describes estimations of other, more general transmission rates, [0242]
  • FIG. 9 is a simplified flow chart showing how an apparatus, according to the embodiment of FIG. 6, may estimate, from a server, the time when a client receives the HTML portion of a web page, [0243]
  • FIG. 10 is a simplified flow chart showing how an apparatus, according to the embodiment of FIG. 6, may estimate the client latency for a given download of the initial HTML text of a page, [0244]
  • FIG. 11 is a simplified flow chart showing how an apparatus, according to the embodiment of FIG. 6, may estimate the time when a client receives an entire web page, [0245]
  • FIG. 12 is a simplified flow chart showing how an apparatus, according to the embodiment of FIG. 6, may estimate the client latency for a given download of a complete web page, [0246]
  • FIG. 13 is a simplified schematic of an architecture used to test a preferred embodiment of the invention, [0247]
  • FIG. 14 is a graph of accuracies for measurements of the main page latency, and web page latency, [0248]
  • FIGS. 15A[0249] 15B 15C and 15D, 16A and 16B are cumulative distributions of latency estimation error, and RTT error, for typical web pages where the server is under a normal load, and,
  • FIGS. 17A and 17B are cumulative distributions of latency estimation error, for typical HTML, where the server is overloaded.[0250]
  • DESCRIPTION OF THE PREFERRED EMBODIMENTS
  • The present invention allows a client's latency, that is, the amount of time between a first time, when a client makes an initial request for data, and a second time when the client receives the requested data, to be estimated in association with a server serving data to the requesting client. The client's latency may be estimated using calculations, based on measurements made in association with the server. [0251]
  • Before explaining at least one embodiment of the invention in detail, it is to be understood that the invention is not limited in its application to the details of construction and the arrangement of the components set forth in the following description or illustrated in the drawings. The invention is applicable to other embodiments or of being practiced or carried out in various ways. Also, it is to be understood that the phraseology and terminology employed herein is for the purpose of description and should not be regarded as limiting. [0252]
  • Reference is now made to FIG. 1, which is a simplified schematic diagram of the connection between a client and a server using the Internet, or other like network. A [0253] content server 62 is connected to at least one Internet Service Provider (an ISP), router, switch or hub or a plurality thereof 64. The ISP, router, switch, or hub, is connected to the Internet or main network 66, which may consist of a series of other routers, hubs, switches, other network connection, and various nodes with information about the network topologies, as well as other connection devices, including various data pipes, such as Ethernet cable, fiber-optic cable, telephone lines, and satellite connections. Connected to the Internet, or other network is a client's ISP, router, switch, or hub 68. In the general case, the client's ISP, router, switch or hub may or may not be the same as the content server's ISP router, switch or hub, but often will not be. The client 70 is in turn connected to the client's ISP, router, switch or hub.
  • In general, for data to travel from one computer to another over a network, or through the Internet, that data may travel through a number of intermediaries, before that data gets to its destination. For a client to send data to a server the data may pass through the client's ISP, router, switch or [0254] hub 68, through the Internet, or various routers switches and hubs 66, through the server's ISP, router, switch or hub 64, and finally arrive at the server 62. Similarly, for data to travel from the server to the client the data may pass through the server's ISP, router, switch or hub 64, through the Internet, or various routers switches and hubs 66, through the client's ISP, router, switch or hub 68, and finally arrive at the client 70. For data to pass from a client to a server, or from a server to a client, there may be delay in any of these steps. The connection between the client and the server may have limited bandwidth, and only a certain amount of data may be able to travel from a client to a server in a given period of time. Similarly, there may be a limited bandwidth between a server and a client, which may or may not be the same as the bandwidth from the client to the server (the client's upload bandwidth, and download bandwidth, respectively). There is also a delay due to the various stops along the way from the client to the server, and from the server to the client, that is, it takes a certain amount of time for any data to travel through the network, from the client to the server, or from the server to the client, regardless of the bandwidth. Both these elements of delay may have a part in determining the client's latency.
  • Reference is now made to FIG. 2 which is a simplified flow chart showing a typical web-page download procedure, and illustrating sources of latency for a download. For simplicity, when a client sends data to a server it is referred to as a request, and when the server sends data to a client it is referred to as a response. The sum total of all exchanges that occur between a client and a server, in the client's downloading of a single web page is referred to as a web-page transaction. In order to initiate a connection with a server, a client may make a DNS lookup in order to contact a [0255] server 72. In a DNS lookup, a client sends a request to a known server (a DNS server) requesting the address of a target server (the server the client is trying to initiate a connection with). The DNS server either has the address of the server the client is looking for, or has to look up the address with another server. If the other server does not have the address it may consult a different server, which may in turn have to consult a further server, until finally a server finds the requested address, or determines that there is no server with such an address. In general, DNS lookup is a small part of the web page transaction. Assuming that the DNS server finds the address, the DNS server eventually responds to the client with the address of the target server.
  • Once the client has ascertained the address of the target server, the client may establish a connection with the target server. Using the HTTP 1.1 protocol, the client may establish persistent connections, referred to as a channels, with the [0256] target server 74. A channel is a connection that is used for multiple requests and responses, as opposed to opening a new connection for each request. The establishment of a connection takes a certain amount of time, referred to as connection latency. A client may open multiple channels with a server. Each of these channels may be opened at a different time, or may be opened simultaneously. When channels are opened simultaneously, the connection latencies, for each channel, may overlap. Using TCP/IP, establishing a connection is initiated by the client sending a request to the server. The server places the request on an uncompleted connection queue and sends an acknowledgment of the request back to the client. Upon receiving the server's acknowledgment of the initial request, the client sends a request to the server acknowledging the server's acknowledgment of the initial request. When the server receives the client's acknowledgment of the server's acknowledgment the server moves the request onto a completed connection queue 76. Normally the request does not stay in the completed connection queue for long, but if the server receives many requests at the same time, or within a short period of time, then a request may sit in the completed connections queue for some time. The time the connection remains in the server's completed connection queue is referred to as the queuing latency. The connection queue has a maximum length, and if that length is reached, then the client connection is dropped, and the channel is not established. Assuming the server does not drop the connection, the server accepts the connection. The client sends one or more requests to the server for an initial HTML page 78, called the main page. Note that this request may have been sent while the server accepts the connection, that is to say, the first request on the channel may be sent as soon as the client receives the server's initial acknowledgment of the client's connection request. Once the server accepts the connection, the server may read the request sent by the client, for the main page. There may be delay between when the request for the main page was sent, until the server receives the request for the main page 80. However, if the queuing latency is greater than the time it takes for the client to send the request, then the time for the client to send the request does not factor in to the time the client has to wait. Upon receiving the request for the main page, the server processes the request, which may consist of disk time such as finding static HTML, processing time, such as generating dynamic content, and possibly a reverse DNS lookup 82 (the server looking up the client's address). It takes time for the server to write the response to the request for the main page to the channel 84, and then it takes time for the client to receive the main page (the time it takes the response to travel from the server to the client) 86. The main page may indicate that there are other components of the page for the client to download, in which case the client may send one or more additional requests to the server for those components 88. Note that the client may send requests for additional components before the client has finished downloading the main page. When the client starts parsing the main page, the client may find a reference to an additional component of the web page, and upon recognizing the reference, the client may send a request for that component, before the client has finished downloading the entire main page. When there are additional components, the client may have to wait until the server receives requests 90 for those components, following which, the server processes those requests 92, and writes one or more responses to the additional requests 94. The client then waits for the additional responses to travel from the server to the client 96. Depending on what was in the main page, and what was in the responses to the additional requests, the client may request additional content 88. Once the client has received all the responses, the web page transaction is complete, and the client has the entire web page.
  • Reference is now made to FIG. 3, which is a simplified schematic timing diagram showing delays involved when a client establishes a connection with a server, and downloads a web page from that server, using multiple (in this case, two) channels. The client sends the server requests, to open a first [0257] 240, and a second 264, channel. Theses two requests may be sent simultaneously, or one closely following the other. The server receives the requests from the client 242, and 266, and places the requests in the uncompleted connection queue. The server sends out acknowledgments of the client's connection requests. The client receives the acknowledgments 244, and 268, and sends acknowledgments of the acknowledgements to the server, 244, and 268. The client sends the server a request for the HTML, namely the main page, on the first channel 244. The server receives the acknowledgments of the server's acknowledgment 246, and 270 and places the connections on the completed connection queue 246, and 270. The establishment of a connection comprises a 3-way handshake (1.client request, 2.server acknowledgement, 3. client acknowledgment of server acknowledgment), and the server's queuing time. The time it takes to establish a connection is referred to as the connection latency. It is noted that each channel may have connection latency, and that the connection latencies of each channel may overlap. The server accepts the connections 248, and 272, and reads the request on the first channel 248. The server processes the request, and writes a response to the client 250. The client starts receiving the response (the main page) on the first channel. When the client detects 274 a reference to a first component of the main page, such as an image, a JavaScript, a java applet, a flash object, or any other object referenced by the main page, even if the client has not finished receiving the entire HTML portion of the web page, the client may send the server a request for the referenced object on the second channel 276. The server receives the request for the component 278, and may start processing the request. The server processes the request for the first component, and starts sending the client the first component 280 on the second channel. Meanwhile the client may have finished receiving the entire HTML (the client may have received the main page) 252, may have parsed the entire HTML of the main page 254, and may send a request, for a second component referenced by the HTML, on the first channel 256. The server receives the request for the second component on the first channel 258. The server processes the request for the second component and sends a response to the request for the second component 260 on the first channel. The client receives the first component on the second channel 282, and the client receives the second component, on the first channel 262. If there is only the HTML and two components referenced by the HTML, the client will have finished downloading the page. Otherwise the client will continue sending requests to the server on as many channels as the client has, until the client has all the components of the page referenced by the HTML.
  • To compute the total time the client has had to wait, from the time the client sends the request for information, until the client has received the entire page, one needs to know when the client sent out the initial request, and when the client received the final response. The goal is to obtain as accurately as possible, such times for each client, and to obtain such information based entirely on server based measurements. [0258]
  • There are several latencies associated with a client's download that a server's administrator might be interested in determining. One such latency is the main page latency, the time between the client sending a request for a web page, and receiving the HTML portion of the web page, namely the main page. Another such latency, the web page latency, is the time between the client initially requesting the web page, and receiving the entire web page, along with all referenced components, such as images, sound files, Java Scripts, Java applets, or any other component of the web page. [0259]
  • In order to determine various latencies, it may be useful to determine a round trip time. The round trip time is defined as the time it takes for a theoretically infinitely small piece of data to go from a server to a client and return to the server. [0260]
  • An estimation of the main page latency may be provided by: [0261] Main page latency T DNS Lookup + 1.5 × RTT + T Queuing + T Server processing time + 0.5 × RTT + HTML Size Bandwidth ( eq . 1 )
    Figure US20030233445A1-20031218-M00001
  • where: [0262]
  • T[0263] DNS Lookup is the time the client takes for a DNS lookup,
  • RTT is the round trip time, [0264]
  • T[0265] Queueuing is the queuing latency
  • T[0266] Server processing time is the time it takes the server to process the request
  • HTML[0267] size is the size of the HTML being sent, and
  • Bandwidth is the bandwidth from the server to the client. [0268]
  • An estimation of the web page latency may be provided by: [0269] Web page latency T DNS Lookup + 1.5 × RTT + T Queuing + max 1 i N { E i + 0.5 × RTT + Response Size i bandwidth - S 1 } ( eq . 2 )
    Figure US20030233445A1-20031218-M00002
  • where: [0270]
  • T[0271] DNS Lookup is the time the client takes for a DNS lookup,
  • RTT is the round trip time, [0272]
  • T[0273] Queueuing is the queuing latency
  • N is the number of HTTP responses the server writes to send the client the web page, [0274]
  • E[0275] 1 is the time at which a particular response, response i is written to the channel, and
  • S[0276] 1 is the time the server received the first HTTP request from the client.
  • Both the above estimations are dependent on knowing the bandwidth from a server to a client. An estimation of bandwidth may be provided by the equation: [0277]
  • bw=Response Sizei/(t server recv i+1 −t serv send, −RTT)  (eq. 3)
  • where: [0278]
  • bw is the bandwidth, [0279]
  • Response Size[0280] i is the size of the response whose transmission bandwidth is being determined,
  • t[0281] server recv i+1 is the time the server received a first request consequent to the respone of Response Sizei,
  • t[0282] serv send, is the time the server sent the response of Response Sizei, and
  • Rtt is the round trip time. [0283]
  • [0284] Equation 3 depends on an accurate knowledge of the round-trip-time, for the given transmission. Merely using an average round trip time the above estimation could estimate, for certain transmissions an almost infinite bandwidth, or a negative bandwidth.
  • Using the available bandwidth may not give as accurate a measurement as possible, as the actual bandwidth used may vary greatly, because of dropped TCP packets that must be re-transmitted, because of network congestion, because the server is overloaded, because the client is overloaded, or for a wide variety of other possible reasons. Other means of estimating the client's latency are therefore presented. [0285]
  • Reference is now made to FIG. 4, which is a simplified schematic timing diagram of a single channel between a client and a server, showing a relationship between the times the client requests different web-page components on a channel. FIG. 4 illustrates some of the dependence between a server's response, and a client's sending a request, subsequent to the server's response, on the channel. [0286]
  • In FIG. 4, the client sends a request, request[0287] i for a component i, of a page, for instance, a web page, to the server 290, at time tclient send i. Requesti arrives at the server 292, at time, tserver recv i, which is after tclient send i. The server processes requesti and sends a response, responsei 294 to requesti, at time tserver send i. Responsei is received at the client 296 at time tclient recv i. The client processes responsei and then may send a subsequent request, 298, requesti+1 for another component, i+1, at time tclient send i+1. The server receives requesti+1 at time tserver recv i+1, processes requesti+1, and sends a response, responsesi+1 302 to requesti+1, at time tserver send i+1. The client then receives responsei+1 304, at time tclient recv i+1. It is observed that there is a dependency between tserver send i and tserver recv i+1. For example, if a client has 2 channels, A and B, the client may send a request on B for an additional component, while the client is downloading a component on A, but the client will not send a request, on channel A, until that client has finished downloading a component on channel A. The time between the server sending a response to a request (responses) and the server receiving a subsequent request (requesti+1) on the same channel is called the inter-request-time. The inter-request time may help determine the bandwidth of a client connection, as well as helping to determine the client's round trip time, as will be described below.
  • Reference is now made to FIG. 5A, which shows some of the log files a server may record. A server may typically record, in a [0288] first log 98, the IP addresses, names or other identification information of clients connecting thereto. A server may also record a second log 100 of a first line of the client's request, including the HTTP method and the URL requested. The server may record a third log 102 of a date/time stamp, which, depending on the server, can hold a representation of the time the server starts processing the request or the time when the server started writing a response. The server may record a fourth log 104 of the HTTP response status code, and a fifth log 106, of the number of bytes in the response, not including headers. Servers may optionally record a sixth log 108 of information about cookies stored on the client, and a seventh log 110, of the server's processing time.
  • Reference is now made to FIG. 5B, which shows additional logs that may be recorded by a server using a server side latency estimator, according to preferred embodiments of the present invention. The logs may be recorded by the logging mechanism already on the server, or may be recorded independently. The server records a first additional log, for each channel, of the time the server accepts the [0289] channel connection 116. For each request the client sends on each channel, the server, preferably, additionally records:
  • a [0290] first request log 112, which comprises a flag which indicates if the request is the first request on a given channel,
  • a [0291] second request log 114 comprising the client's port number,
  • a [0292] third request log 118 comprising the time the server starts processing the request,
  • a [0293] fourth request log 120 comprising the time the server completes writing a response to the channel, and
  • a [0294] fifth request log 122, comprising the number of bytes left to send when the server logs the request, that is, the number of bytes left to send when the server completes writing the response to the channel, and records the fifth request log.
  • Preferably, for each web transaction, the server also logs a client's [0295] ping time 119. Preferably, at pre-defined intervals, the server additionally logs a queuing latency, 121, as recorded by a queuing latency probe, which is described below. Preferably, the log includes a timestamp, showing the time the latency was recorded.
  • Reference is now made to FIG. 6, which is a simplified block diagram of a server side latency estimator, operative in accordance with a first preferred embodiment of the present invention. The server side latency estimator is associated with a server, for example a web server. The server side latency estimator comprises an [0296] event observer 123, which is operable to observe various pre-defined events happening at, or about, the server the device is associated with. The event observer is connected to a logging device 134, which is operable to log events, and time information relating to those events, such as when events occur, or a time duration between related events. The server side latency estimator may use the server's logging device, or the server side latency estimator may have its own, separate, logging device. The server side latency estimator may record logs, with the logging device, as described in FIGS. 5A and 5B. The server side latency estimator also comprises the queuing latency probe 130, referred to above, which may be connected to the logging device 134. The queuing latency probe preferably sends a small request to the server using a new connection, such as a new TCP connection, at pre-specified intervals. The queuing latency probe measures the time it takes for a server to accept the new connection, that time corresponding to the so-called queuing latency. It will be noted that when the new request is sent from and to the same server, or from a computer close to the server, to the server, the time for the 3-way handshake will be minimal, and the queuing time will be dominant, as the connection between the probe and the server is likely to have high bandwidth, and a small round trip time. Over a short period of time the queuing latencies are approximately the same for all requests, as the queuing time is primarily dependent on the server's conditions, namely how many requests the server is handling at a given time. The queuing latency probe measures the time it takes for the server to accept new requests, and the logging device may keep a log of time information regarding the server's responses as described above. The queuing latency estimator may also alert a server's administrator if the queuing latency estimator finds that its requests are being dropped.
  • An [0297] external pinger 132 is connected to the logging device, and preferably sends out a ping, or ping like communication, to each client connecting to the server. Sending out a ping to each client may be accomplished in a number of ways. In a preferred embodiment the external pinger runs in parallel to the web server and at low rate reads the access log. For each new client IP address appearing in the access log the external pinger sends a single ping and waits for the response. It is noted that because the pinger uses a single ping measurement and that the ping measurement may take place some time after the actual web page transaction, the measurement may not be completely accurate, but the ping measurement still serves as a good approximation of the actual round trip time. The external pinger has minimal processing and communication overhead. The external pinger may also be used to identify routing problems, by pinging a list of frequent hosts/subnets, at low rate, and reporting any problems to a server's administrator.
  • In a variation it is possible to arrange the external pinger to ping the client upon the client's initial establishment of each channel. The connection procedure is preferably modified such that on each connection establishment an interrupt is sent to the external pinger, which then pings the client. This has the advantage of giving a more accurate ping time. Also the external pinger does not then have to read the log. However the variation increases system overhead. [0298]
  • The [0299] logging device 134 preferably records a log of all the measured ping times for each client, that is, the log records time information relating to the duration between the external pinger pinging the client, and receiving the client's response.
  • Preferably there is a [0300] sentry inserter 136, a device that may modify data being sent to include a prompt for the client to make an additional request at the end of the client's reception of the data, the prompt for the additional request hereafter referred to as a sentry 136. The sentry preferably causes the client to make an additional request at the end of data receipt, the additional request thus serving as an indicator that all of the data has been received. In the first preferred embodiment, the sentry simply comprises an additional image tag, located at the end of the HTML, for an image that has no contents and may be one pixel. The request prompted by the sentry determines a latest time when the client finishes receiving the main page. The sentry may be used to provide an initial estimation for the main page latency, and, in certain cases, for example, when the web page has a large amount of HTML, and few, small images, the web page latency. In addition the sentry may generate two sequentially dependent requests which enable the rate estimator to make an estimation of the bandwidth and the round trip time, as will be described below. It is noted that the sentry has a negligible overhead on the server, the client, and the communication network.
  • When the client receives the sentry, the client preferably makes the additional request to the server. As mentioned, the image is only one pixel at the end of the page, and therefore the client is preferably able to render the page. The request generated by the client as a result of the sentry may thus serve to help determine when the client receives the last of the data, that is when the client finishes downloading the web page. [0301]
  • Generally speaking, the purpose of the ssentry is to provide a better main page latency estimator, but it provides better web-page latency estimation. If the client is using pipelining and the last image sent [not including the sentry] is large, it will finish after the client receives the sentry. Hence, in such a case, the time the client receives the sentry does not mean that he received all the web page![0302]
  • The server side latency estimator preferably has a round [0303] trip time estimator 129 associated with the logging device 134, for estimating round trip times. The server side latency estimator preferably has a data rate estimator 124, connected to the logging device, which is composed of:
  • a perceived [0304] packet rate estimator 125, connected to the logging device 134,
  • a connection perceived [0305] packet rate estimator 126, connected to the connection perceived packet rate estimator 126,
  • a web transaction perceived [0306] packet rate estimator 127, connected to the perceived packet rate estimator 125, and
  • a perceived packet [0307] line rate estimator 128, connected to the perceived packet rate estimator 125.
  • The perceived [0308] packet rate estimator 125 may be used to estimate a perceived packet rate, Rateinter-request, for each transmission.
  • The connection perceived [0309] packet rate estimator 126 may estimate a connection perceived packet rate, RateConn, which is a packet rate for each client connection, preferably based on individual channels to individual clients. Each new download of a new web page is treated separately, so that for each download the client preferably receives one ping. The packet rate is preferably based on all the estimated packet rates per transmission for each respective client.
  • The web transaction perceived [0310] packet rate estimator 127 may estimate a web transaction perceived packet rate, RateWeb-Trans, which is taken to be the average packet rate of all connection rates Rateconn. to a particular client. The perceived packet line rate estimator 128 may estimate a perceived packet line rate, Rateline, which is an average packet rate per client, based on all the estimated packet rates per transmission estimated for each respective client, wherein those packet rates fall within specific size criteria as discussed below.
  • Reference is now made to FIG. 7, which is a simplified flow chart showing how the round [0311] trip time estimator 129 may estimate a round trip time for a given client. The round trip time estimator accesses the log of ping times 140, and finds a respective ping time, as determined by the external pinger, for the client of a current web transaction 142. For simplicity we describe FIG. 7 as though the round trip time estimator only has to deal with a single ping time, the determined ping time, called T1. T1 serves as an initial estimate of the round trip time. The round trip time estimator examines the logs for the current client connection, and determines the smallest inter-request time for the current web-transaction 144, the smallest inter-request time providing a second estimated round trip time T2. The round trip time estimator checks if T1 is smaller than T2 146. If T1 is smaller, the round trip time estimator estimates the round trip time to be T1 148. Otherwise, the round trip time estimator estimates the round trip time to be T2 150. It will be noted that the estimated round trip time is not the real round trip time, rather, it is larger then the real round trip time. The estimated round trip time is biased due to the size of the transmission, and that applies to a ping, even though it is a single packet. The accuracy of the estimated round trip time depends on the bandwidth, as the data being sent and received is not infinitely small. The estimation will improve as the bandwidth increases.
  • In fact, the estimated round trip time may be smaller than the actual, average round trip time. Different packets may take different routes, and as the more packets go from the server to the client there is a tendency towards a best route. Eventually most of the packets use the smallest round trip time, which is from the best route, and this may in fact be better than the average round trip time. The RTT as estimated is just an estimator of the real one, and and is close to the average RTT but not exactly the same. [0312]
  • Reference is now made to FIG. 8 which is a simplified flow chart showing how the [0313] data rate estimator 124 of FIG. 6 may estimate various transmission rates. A transmission rate is a measure of how much data is sent over a given time period. In general transmission rates may be estimated by dividing an estimated transmission time by the amount of data transmitted. The Internet is a packet based network, so in the present embodiment the data rate estimator estimates a packet rate, a rate at which packets are sent, i.e. the number of packets sent in a given period of time. Before the server side latency estimator may start processing the logs, it may split the logs into web page transactions. Spitting the log into web-page transactions may be done by sorting the server's logs according to the timestamps of the events logged, and then splitting the logs according to the client IP addresses. In this manner the server side latency estimator may deal with only a small part of the log, for each client latency estimation. A perceived packet rate, a rate of transmission for a specific transmission, may be estimated by the formula: Preceived Packet Rate = Response Size i / Packet Size max { t server_recv i + 1 - t serv_send i - 0.5 × RTT , 0.5 × RTT } ( eq . 4 )
    Figure US20030233445A1-20031218-M00003
  • where: [0314]
  • Response Size[0315] i is the size of the present response,
  • t[0316] server recv i+1 is the time the server receives a request subsequent to the present request, i,
  • t[0317] serv send, is the time the server starts sending the response, i, and
  • RTT is the minimum round trip time, in accordance with the embodiment of FIG. 7. [0318]
  • In order to estimate the packet transmission rate, which is the perceived packet rate, alternately called Rate[0319] inter-request, for a single response action, the perceived packet rate estimator divides 162 the response size (the number of bytes in the response, or any other applicable measurement of size), by the size of the packets (or other transmission unit) being sent across the connection, giving the approximate number of packets (or other transmission units) the server may send for the response. The data rate estimator determines a time, TsubsequentReq, when a subsequent request is received on the channel the response is sent on, provided that such a subsequent request exists.
  • The data rate estimator preferably determines an initial transmission time T[0320] TransTime by subtracting the time when the server finishes writing the response with the TsubsequentReq request, from TsubsequentReq. The initial TTransTime estimation includes the time it takes for the TsubsequentReq request to get from the client to the server. The time it takes for the TsubsequentReq request to get to the server is not part of transmitting the response from the server to the client, so the data rate estimator subtracts half the estimated round trip time 164 estimated for the respective client determined by the round trip time estimator 129. The actual transmission time cannot be less than half the round trip time, so the data rate estimator estimates the transmission time to be the larger of half the round trip time, and TTransTime minus half the round trip time 166. The perceived packet rate, namely the Rateinter-request, is estimated as the approximate number of packets the server sends, divided by the estimated transmission time 168. The data rate estimator estimates a connection perceived packet rate, namely Rateconn, which is the average Rateinter-request for a given connection 169 to a given channel. The data rate estimator estimates a web transaction perceived packet rate, namely RateWeb-Trans, which is the average Rateconn for all connections 170. The data rate estimator checks if there is a sufficient number of properly sized Rateinter-request's, that is, Rateinter-request's whose transmission times are within a certain time range 171. In the preferred embodiment, there may be at least four transmission times that are not larger that six times the estimated round trip time. If there are not a sufficient number of properly sized Rateinter-requests, the data rate estimator marks the Rateline as invalid 172. Otherwise the data rate estimator estimates 174 a perceived packet line rate, namely Rateline, as the average of all properly sized Rateinter-request.
  • Reference is now made to FIG. 9, which is a simplified flow chart showing how a server side latency estimator may estimate the time when a client completes the receipt of the HTML portion of a web page. The time when a client completes receipt of the HTML portion of a web page may be estimated by the equation: [0321] t client_recv HTML = min { t server_send HTML + Response Size HTML / Packet Size Rate Conn if Rate Conn Valid t server_rec v next_inter _request - 0.5 × RTT if at least 2 request on HTML ' s conn t server_rec v Sentry_image - 0.5 × RTT ( eq . 5 )
    Figure US20030233445A1-20031218-M00004
  • where: [0322]
  • t[0323] server send HTML is the time the server sends the client the HTML, t server_recv next_inter _request
    Figure US20030233445A1-20031218-M00005
  • is the time the server receives the first request from the client, on the channel the server sent the HTML on, for an additional component of the web-page, [0324]   t server_recv Sentry_image
    Figure US20030233445A1-20031218-M00006
  • is the time the server receives the request for the sentry, and [0325]  
  • RTT is the round trip time. [0326]
  • The server side latency estimator may estimate the time the client receives the HTML portion of the page based on [0327] equation 5. The server side latency estimator estimates three different times the client may have received the main page, and sets the minimum (earliest) of these estimates as an approximation of the time the client receives the HTML potion of the web page. The server side latency estimator checks if the RateConn. is valid 182. If the Rateconn is not valid the server side latency does not make or use an estimation dependent on the Rateconn, but instead marks the first preliminary estimated time the client receives the HTML portion of the web page, T1, as invalid 184. Otherwise, in step 186, T1 is set to a time the server finishes writing the last of the HTML potion of the web page to the present channel, plus the estimated time it takes for remaining data to be sent, once the server finishes writing the HTML portion of the web page to the present channel. The time needed for sending the remaining data is estimated by dividing the size of the data to be sent by the packet size, giving a total number of packets to be sent. The number of packets to send is then divided by the Rateconn, giving an estimation of the time needed to send the remaining data. The server side latency estimator checks if there have been more than two HTTP requests 188 on the channel the server sent the HTML portion on. If there has been only one HTTP request on the channel the server sent the HTML on, the server side latency estimator does not make or use the second estimated time of receipt of the HTML, and marks T2 invalid 190. Otherwise, the server side latency estimator estimates T2 using the time the server received the second HTTP request on the present channel, minus half the round, trip time 192. Assuming the server side latency estimator is using a sentry (FIG. 6, 136) the server side latency estimator makes a third estimate of the time the client receives the HTML, T3, using the time the server received the request generated by the sentry (136) minus half the round trip time 194. The server side latency estimator then estimates the time the client receives the HTML as the minimum (earliest) of the valid estimations of T1, T2, and T3 196.
  • Reference is now made to FIG. 10 which is a simplified flow chart showing how a server side latency estimator may estimate the user perceived latency for a given download of the HTML portion, namely the main page, of a page. [0328]
  • The client's main page latency may be estimated by the equation: [0329] Main Page Latency T Queuing + 1.5 × RTT + t client_recv HTML - t serv_recv HTML ( eq . 6 )
    Figure US20030233445A1-20031218-M00007
  • where [0330]
  • T[0331] Queueing is the queuing latency,
  • RTT is the round trip time, [0332]
  • t[0333] client recv HTML is the time the client receives the HTML portion of the web page, and
  • t[0334] server recvHTML is the time the server receives the request for the HTML from the client.
  • The server side latency estimator estimates the time at which the client receives the [0335] HTML 200, as described in the embodiment of FIG. 9 The server side latency estimator subtracts the time the server accepts the connection from the client 202, as recorded in the logging arrangement of FIG. 5B, which is used as the approximate time the server received the request for the HTML 116. The server side latency estimator adds the queuing latency as determined by the queuing latency probe (130) 204, as an estimation of TQueueing and adds one and a half the estimated round trip time 206, to account for the three-way handshake time 74 (FIG. 2), necessary to establish the connection.
  • Reference is now made to FIG. 11, which is a simplified flow chart showing how a server side latency estimator may estimate the time when a client receives an entire web page. A given channel may carry one or more transmissions, and a given client may use more than one channel. Thus the time the client completes receipt of the entire web page may be estimated as: [0336] t client_recv Last_Response = max 1 j < N Conn ( t client_recv last , j ) ( eq . 7 )
    Figure US20030233445A1-20031218-M00008
  • where: [0337] t client_recv Last_Response
    Figure US20030233445A1-20031218-M00009
  • is the time the client receives the last of the server's responses, namely the time the client receives the web page, [0338] t client_recv last , j
    Figure US20030233445A1-20031218-M00010
  • is the time the client receives the last response on a given channel, j, and [0339]
  • N[0340] conn is the number of connections, namely the number of channels that the client uses, in its communication with the server.
  • [0341] Equation 7 requires an estimation of the time the client received the last response on a given channel, which may be estimated by: t client_recv last , j = { t client_recv html if N html = 1 and j = C html t serv_send last , j + Response Size last , j / Packet Size Rate line if not ( N html = 1 and j = C html ) and Rate line Valia t serv_send last j + Response Size last j / Packet Size Rate Web_Trans if not ( N html = 1 and j = C html ) and Rate line not Valia ( eq . 8 )
    Figure US20030233445A1-20031218-M00011
  • where: [0342]
  • t[0343] client recv last,j is the time the client receives the last response on a given channel, j,
  • t[0344] client recv html is the time the client receives the HTML,
  • N[0345] HTML is the number of requests on the channel of the HTML request,
  • j is a current connection, the present channel, [0346]
  • C[0347] HTML is the channel the HTML is sent on, and
  • t[0348] serv send last,j is the time the server finishes writing the last HTTP request into the channel j,
  • Response size[0349] last,j is the remaining part of the last response when the server finishes writing to connection j, i.e. the number of bytes left to send, and
  • Packet Size is the size of the packets being sent. [0350]
  • Based on the [0351] equation 8, the server side latency estimator may perform the following analysis on all the channels a client has opened with the server 210 to determine when the client received the last data on a given channel 212. The server side latency estimator checks 214 if, on the current channel:
  • (a) there has been only one request for data, and [0352]
  • (b) there has been a request for HTML. [0353]
  • If the server side latency estimator determines that the above conditions are both true, the server side latency estimator may determine the time at which the client received the last data on the current channel to be the time the client receives the HTML, that is to say the result of FIG. 9 [0354] step 196. If the above conditions are not both true, the server side latency estimator checks if Rateline is valid (FIG. 8, 172, 174). If Rateline is valid the server side latency estimator estimates the time the client receives the last data on the current channel by taking the time the server finishes writing the last data to the channel, and adding an estimate, using Rateline, of the time it takes for the data remaining to get to the client. The time it takes for the data remaining, to get to the client, is estimated by determining the number of packets left once the server has finished writing to the channel, and dividing the number of packets left by the Rateline. The number of packets left is determined by dividing the size of the data remaining, when the server has finished writing the last of the data to the channel (recorded in the embodiment of FIG. 5B, 122), by the size of the packets being sent. The number of packets left is divided by the Rate line 220, and the result is added to the time the server finishes writing the response to the channel, to give an estimated time at which the last data from the current channel is received by the client 224. If the Rateline is not valid, the server side latency estimator estimates the time the client receives the last data on the current channel by taking the time the server finishes writing the last data to the channel, and adding an estimate, using RateWeb-Trans, of the time it takes for the data remaining, to get to the client. The time it takes for the data remaining, to get to the client, is estimated by determining the number of packets left, once the server has finished writing to the channel, and dividing the number of packets left by the RateWeb-Trans. The number of packets left is determined by dividing the size of the data remaining to be sent, once the server has finished writing the last of the data to the channel (recorded in the embodiment of FIG. 5B, 122), by the size of the packets being sent. The number of packets left is divided by the Rate Web-Trans 222, the result being added to the time the server finishes writing the response to the channel, to give an estimated time at which the last data form the current channel is received by the client 224.
  • Based on [0355] equation 7, the above process is repeated for each channel the client opens with the server, and the time the client receives the last data for the entire web page is estimated to be the latest of the individual channel time estimates 226.
  • Reference is now made to FIG. 12 which is a simplified flow chart showing how a server side latency estimator may estimate a client's latency for a given download of a complete web page. FIG. 11 shows a means to estimate the time when the client receives the last of the data sent. To estimate the client's latency, it is necessary to know both when the client receives the final data, and when the client sends the initial request. From the point of view of the server there is no direct way in which to know when the client sends the initial request for the page, however the client's latency may be estimated by the equation: [0356] Web Page Latency T Queuing + 1.5 × RTT + t client_recv Last_Response - t serv_recv HTML where : ( eq . 9 )
    Figure US20030233445A1-20031218-M00012
  • where: [0357]
  • T[0358] Queueing is the queuing latency, t client_recv Last_Response
    Figure US20030233445A1-20031218-M00013
  • is the time client receives the last response from the server, and, [0359]   t serv_recv HTML
    Figure US20030233445A1-20031218-M00014
  • is the time the server receives the request for the HTML from the client. [0360]
  • The server side latency estimator adds the [0361] queuing time 230, to one and a half times the round trip time 232, and then to the time when the client receives the entire document (FIG. 11, 226) 234. The server side latency estimator subtracts the time the server receives the initial request for the first HTML 236 which is to say the time the server accepts the connection on which the HTML was sent, FIG. 5B, 116, from the previous result, thereby obtaining a total web page latency.
  • It will be noted that the preferred embodiment of a server side latency estimator does not deal with the DNS lookup time. DNS lookup time is largely a function of the various clients ISP's and is not largely affected by a web-server's behavior, provided that the web server is listed with the root DNS servers. DNS lookup times may vary widely, but are generally only a small part of the total page loading time, and thus do not affect the estimation algorithm very much. A server may, however, verify that it does not have DNS problems. The world is divided into several zones, wherein each zone is under the responsibility of a well known root DNS server. Using a program like nslookup to query each one of the root DNS servers with a server's domain name at predetermined intervals a server may report to an administrators about DNS problems in any zone. [0362]
  • In a second embodiment of the server side latency estimator, clients pipeline all requests. Pipelining involves the client sending requests for subsequent page components on a channel before completely receiving previous components on that channel. Referring to FIG. 4, t[0363] client send i+1 298, may precede tclient receive i 296. Today, most browsers do not implement pipelining, however present embodiments are still applicable, with minor changes, to a client that uses pipelining.
  • Because the client may send a request for a subsequent component, before the client has finished downloading a given component, measuring the round trip time, based on inter-request times, may become much less accurate. However, the ping time remains a valid estimation of the round trip time. Thus in the embodiment described in FIG. 7, the round trip time is estimated to be Ti (step [0364] 142). When a client uses pipelining, the server may write to non-empty buffers, and subsequent measuring of the various transmission rates may use the decrease in the buffer size over time, to determine the rate of data delivery, instead of the size of the entire buffer, and the inter-request time. Thus in FIG. 8, step 162 the response size may be determined to be the decrease in the buffer size over time, as opposed to the total buffer size, over the time when the client sends a subsequent request. It is noted that it is possible to simply determine a data rate, instead of the various packet rates, as the reduction in buffer size/time and this can be achieved without the need to log any further information. No new entries are needed since, at the time of finishing writing the images it is possible to make a determination of the current buffer size. At that determination it is possible to know how much of these belong to a current image and how much belong to the previous images.
  • In a third embodiment of the present invention the client may be served data by multiple servers. Multiple servers reduce the bottle-neck effect of a single server, and are often referred to as load balancing servers, as they balance the load from the many requests for web pages, between multiple different servers. The server side latency estimator merges the logs of the various servers. With the logs merged together the server side latency estimator proceeds in the same manner as described for a single server log. [0365]
  • In a fourth embodiment the client accesses the server through web-proxies. Multiple clients may use the same proxy to access a server, and each of those clients may have different connection rates, and connection times. The logs would make it appear that multiple clients accessing the server through the same proxy are the same client. The server side latency estimator may determine which client each request is from by assigning each client a cookie, and then using cookie information incorporated into the requests to determine which request is from which client. [0366]
  • There are some servers, which, when contacted for data, refer the client to other servers, so that a single request to a single server may lead to downloads that come from numerous servers. For example a web site may have an advertisement, and the advertisement may be served by a server, which keeps records about the advertisements. When client browsers find references to other servers, the browser may open additional connections (channels) to those servers to download material from them. These new connections (channels) may slow down the rate of the connections (channels) to the original server, if the client has a limited bandwidth. The new requests on the already established connections (channels) to the original server may suffer from decrease in rate, which may be noticeable from the access log, which may be taken into account by the server side latency estimator. In a sampling of 100 web sites, 30% of the web pages sampled have no external images, and 70% of the pages sampled have fewer then 8 external images. There are, on average 6 external images, while, on average, there are 21 embedded, or inline, images per web page. Hence, on the average, most of the images come from the original server. For 60% of the web pages the relative fraction of the external images is less then 20%. In addition it is noted that about 20% of the web pages have about 80% of their images stored in other servers. [0367]
  • The server side latency estimator may accurately estimate the latency of the HTML document, in the above cases. The server side latency estimator may accurately estimate the download time of the images located in the original server, which, for most cases is the major portion of the images. [0368]
  • EXAMPLES
  • Reference is now made to FIG. 13, which is a simplified schematic of an architecture used to test a preferred embodiment of the invention. In FIG. 13 there is a web-[0369] server 310, which is connected to a LAN 326. The LAN has four computers, 312, 314, 316, 318, which serve to send requests to the web-server. These requests may simulate various loads on the web server. The LAN is connected to a WAN, and the Internet 324, which is connected to two web-client simulators, 320, and 322, located at various distances from the server.
  • The server side latency estimator was implemented on an Apache web-[0370] server 310, running version 1.3.9, a public domain HTTP/1.1 server. Measurements were conducted at three locations, namely at the web server, referred to as TAU, 310, and at both clients 320, and 322. One web client, called MTA 320, was located relatively close to the server, and one, called UCI, 322 client, was located relatively far from the server.
  • In order to make an evaluation representative of a real client's latency it is preferable to simulate real clients with real-world web traffic characteristics (bandwidth, RTT and loss rate, loss rate being the rate of lost packets), fetching different web pages under various server loads. Also, in order to estimate the server side latency estimator's performance it is preferable to know the actual latencies the clients experience. [0371]
  • The [0372] web server 310 includes three elements, the modified Apache Web server, the queuing latency probe and the external pinger. The server 310 runs on a dedicated PC running the Linux operating system version 2.2.14. The hardware platform is an Intel Pentium III 550 MHz system, with 512M of memory and 512K cache. The disk size is 6 Gigabyte. The server 310 has a standard 100 Mbit/sec Ethernet card, which is connected to a 1 Gbit/sec LAN.
  • In order to evaluate the server side latency estimator, web clients, [0373] 320, 322, that simulate a web browser, and perform measurements of the main page latency, the web page latency and other parameters like RTT, are needed. A multi-thread web browser program utilizing a BSD-socket interface was implemented to perform these functions. The web-browser program supported HTTP/1.1 persistent connections (channels) and simulated the dependency between the request for the HTML document and the requests for the embedded images. The program fetched whole web pages (HTML document and embedded images) using a predefined number of persistent TCP connections. After fetching the whole web page all the threads closed the connections and the master program wrote in a log file, called the client log file, the RTT, the main page latency and the web page latency. In order to test real world conditions, it is necessary to generate server load, that is, it is necessary to test the server side latency estimator when the server receives a varying number of requests. To generate the load, a program called web loader was implemented. Web loader was based on the web browser program with a few modifications. Web loader used a fixed number of threads. Each thread ran in an infinite loop with the same task: open a TCP connection to the server, fetch one file from server and close the connection. Each web loader simulated several clients, from a single computer. A computer ran ten web loader threads. In order to generate the various loads that might be experienced by a server in a real world environment, several computers 312, 314, 316, 318, running the web loader program were located on the same LAN, 236, that the server, TAU, 310 was located on. FIG. 13 shows four such computers used with the web loader program to generate the appropriate load on the server. During testing, between zero and nine computers are used with the web-loader program, as will be explained in greater detail below.
  • In order to generate web pages for testing which were representative of the web, various rating sites were used to gather information about typical web browsers, and typical web pages. Information regarding typical HTML document size, the number of embedded images and their typical size, for popular web pages was gathered. Several rating sites offer statistics on popular web sites, for example, Hot100 (http://www.100hot.com), which surveys 100,000 users (40% of whom are outside the USA) and gathers data at “strategic” points on the Internet (not at the browser or server). For those top 100 sites the average HTML document size is currently given as 30K, and there are on average 21 embedded images, each with an average size of 2.5K. Using the above data, the following web page dimensions were selected as representative: combinations of HTML document sizes 10K, 30K and 60K with 5, 15, 30 and 50 embedded images with an average size of 2K-3K. Based on these selections, 12 various pages were generated as representative of typical web pages. Two additional web pages were used, 1. a web page with a 30K HTML document and 21 embedded images of average size 6K (a web page with a large number of images, creating many inter-request times), and 2. a web page, which included only 2 images, which would generate very few inter-request times. In total 14 various web pages were used in testing. To more accurately reflect real world circumstances pages from the top 100 Web pages were also selected and used in testing, and this data is used to produce the results shown in FIGS. [0374] 7-9. Estimation of a preferred embodiment of a server side latency estimator's accuracy used clients connecting through a WAN 324 In order to simulate real world web-traffic characteristics using the WAN it is necessary to simulate:
  • (a) large RTT variations, [0375]
  • (b) packet loss, and [0376]
  • (c) various bandwidth characteristics. [0377]
  • The PingER project at Stanford Accelerator Center (SLAC) (L. Cottrell, W. Matthews, and C. Logg. Tutorial on internet monitoring pinger at SLAC. Available from http://www.slac.stanford.edu/comp/net/wanmon/tutorial.html/, 1999) conducts continuous network monitoring to measure network latency and packet loss over the Internet. The PingER measurements show average loss rate of 0.5% and RTT of 30-60 milliseconds in the U.S., Asia and Europe, while between the continents the average loss rate is of 1.4% and RTT varies between 200 to 600 milliseconds. [0378]
  • The server side latency estimator experiments were conducted at various hours of the day, over a one week span. During the week of experiments, the clients' network characteristics, in terms of RTT, bandwidth and loss rate were measured. UCI's, [0379] 322, RTT was 600-620 milliseconds, bandwidth 30-40 KB/s and loss rate of 0%. MTA's, 320, RTT was 30-100 milliseconds, bandwidth 5-17 KB/s and loss rate of 2%. UCI's, 322, RTT reflect longer network RTT to the US from the server (located in Tel-Aviv). UCI's, 322, RTT, bandwidth, and loss rate showed a minor disparity. MTA, 320, showed large disparity in the RTT and in the bandwidth. The two sites chosen thus show good representation of real-world web traffic characteristics, as they cover large ranges of RTT, bandwidth and loss rate.
  • A series of test runs were conducted in the following manner: each web-client located in MTA or UCI fetched all the 14 Web pages in serial fashion. For each web page the web browser simulator first fetched the respective pages with 4 persistent connections (four channels), 5 times and later the web browser simulator fetched the respective pages, with 2 persistent connections (two channels), 5 times. Between each web page download, the web browser waited for 4 seconds. The tests were repeated under various server loads. The server load was compared by a number of web loader computers running in the server's LAN. The number of web loader computers was varied between 0-9, which means the server experienced a load of between 0 and 90 clients. Four server loads were used: Light, medium, high and overloaded, as described in the chart below: [0380]
    Average
    CPU Requests Queuing latency No. Of Web
    Load Usage [%] Per Sec [msec] Loaders
    Light
    3 7 1 0
    Medium 20 68 20 1
    High 50 75 500 4
    Overloaded 90 56 6000 9
  • Reference is now made to FIG. 14 which is a graph of the accuracies for the main page latency, and web page latency [0381]
  • FIG. 14 depicts the accuracy of the server side latency estimator's main page latency, and web page latency, estimations, for all the tests runs for both of the clients under the various server loads. The graph of FIG. 14 also shows the server side latency estimator's accuracy for the RTT (RTTmin from the inter request times, or the estimated RTT from the external pinger). It should be clear that the external pinger is an integral part of a preferred embodiment of the server side latency estimator. The performance of the server side latency estimator is preferably evaluated using the pinger for estimating the RTT More precisely, what may be used is a minimum of a ping time and the minimum inter-request time. The table below summarizes the median and average of the estimation error. The table below shows the median value of the estimation error as the average values are shifted by the few high errors in the tests runs using an overloaded server. The average latency estimation error of a preferred embodiment of the server side latency estimator, for the various tested web pages, is 4% under normal server loads, and 10% when the server is overloaded. For the tested web pages with few embedded images the server side latency estimator uses the external pinger for accurate RTT estimations. [0382]
    Average Median
    RTTmin RTTmin
    Meaning (1) Pinger (1) Pinger
    RTT Main Web Main Web Main Web Main Web
    Estimation Page Page Page Page Page Page Page Page
    Method Error Error Error Error Error Error Error Error
    All runs 8.8 11.2 4.0 4.8 1.4 3.3 0.7 3.0
  • The experiments seems to show that the accuracy of the tested embodiment of the server side latency estimator is not affected by the normal server load (light, medium and high, but not overloaded), therefore the different load results are aggregated together. [0383]
  • Reference is now made to FIGS. 15A, 15B, [0384] 15C and 15D, which are cumulative distributions of latency estimation error, and RTT error, for typical web-pages. FIG. 15A is a graph of the error of a page using 15 embedded images, from a relatively close client, MTA, 320. FIG. 15B is a graph of the error of a page using 50 embedded images, from a relatively distant client, UCI, 322. FIG. 15C is a graph of the error of a page using 30 embedded images, from a relatively close client, MTA, 320. FIG. 15D is a graph of the error of a page using large embedded images, from a relatively distant client, UCI, 322.
  • FIGS. 15A, 15B, [0385] 15C, and 15D, depict the accuracy of the server side latency estimator's, main page latency, web page latency, and RTT estimations as a function of the number of the embedded images, for both of the tested clients. The runs of client downloads of web pages with the same number of embedded images, under various server loads are aggregated to form graphs. For 15, 30 and 50 embedded images, the latency estimation errors for MTA and UCI's are similar so MTA's errors are presented for web pages with 15 and 30 embedded images and UCI's errors are presented for web pages with 50 embedded images. The below table summarizes the average estimation errors. For typical web pages the latency estimation error for both clients does not depend on the number of embedded images or on the method of estimating the RTT. Hence, the effect of the external pinger for typical web pages is negligible. For a web page with large embedded images there is no significant change in the estimation error.
    Client
    RTT MTA UCI
    Estimation RTTmin RTTmin
    Method (1) Pinger (1) Pinger
    No. of Main Web Main Web Main Web Main Web
    embedded Page Page Page Page Page Page Page Page
    images Error Error Error Error Error Error Error Error
    15 4.5 5.6 3.5 4.8 0.5 1.9 0.5 1.9
    21(2) 2.3 2.4 2.2 2.3 0.3 2.4 0.2 2.4
    30 5.5 3.4 5.4 0.2 3.2 0.2 3.2
    50 4.3 6.1 4.3 6.1 0.7 3.3 0.7 3.3
  • The average estimation error for MTA is larger than the error estimated for UCI because: 1) UCI has high bandwidth, 2) MTA has high packet loss. It is concluded that the average latency estimation error is 4% for a typical web page, [0386]
  • Reference is now made to FIGS. 16A and 16B. FIG. 16A is a graph of the cumulative distribution of latency estimation error for HTML documents, with five inline images, for the client simulator at MTA. FIG. 16B is a graph of the cumulative distribution of latency estimation error for HTML documents, with five inline images, for the client simulator at UCI. FIGS. 16A and B depict the accuracy of main page latency and web page latency as estimated by the server side latency estimator for web pages with few embedded (inline) images. The latency estimation error for web pages with two and five embedded images behave similarly so the error for web pages with five embedded images is presented. Each figure also depicts the effect of the method for estimating the RTT. The below table summarizes the average errors for UCI and MTA. In a web page with few embedded images is necessary to use the external pinger to estimate the RTT and not rely only on the RTTmin. The error decreases in some case from an average error of 90% to an average error of 6%. For web pages with few embedded images, the server side latency estimator's estimations of the main page latency, and the web page latency, are as good as the estimations for web pages with many embedded images. [0387]
    Client
    RTT MTA UCI
    Estimation RTTmin RTTmin
    Method (1) Pinger (1) Pinger
    No. of Main Web Main Web Main Web Main Web
    embedded Page Page Page Page Page Page Page Page
    images Error Error Error Error Error Error Error Error
    2 75.7 91.6 0.6 5.6 38.1 40.1 0.1 7.6
    5 32.3 46.4 2.6 3.9 19.1 23.9 0.3 2.7
  • Reference is now made to FIGS. 17A and 17B which are graphs of the cumulative distribution of latency estimation error for web pages with various numbers of embedded (inlined) images, where the estimations were made using data collected on a server that was overloaded. FIG. 17A is a graph of the cumulative distribution of latency estimation error for web pages with few embedded (inlined) images, with the estimations made using data collected on a server that was overloaded, while FIG. 17B is a cumulative distribution of latency estimation error for web pages with an intermediate number of embedded (inlined) images, with the estimations made using data collected on a server that was overloaded. [0388]
  • FIGS. 17A and 17B depict only the runs with 5 and 15 embedded images as other tests showed similar behavior, and these are representative. UCI's results are not shown, as they exhibit similar behavior. The runs of web pages with the same number of embedded images are aggregated. Each figure depicts the accuracy of an embodiment of the server side latency estimator for estimating the RTT. The below table summarizes the median and average of the latency estimation errors for the serve side latency estimator at MTA. The behavior of the latency estimation error is similar for the tests run under normal server load and overload server load as may be seen in FIGS. 15A, 15B, [0389] 15C, 15D, 16A, 16B and 17A and 17B, with the exception of the long tail in the overloaded cases FIGS. 17A and 17B. On an overloaded server there are periods of time in which the queuing latency increase rapidly in short time. The queuing latency probe of the present embodiment samples the queuing latency at a low frequency. If the queuing latency probe were to sample the queue at high frequency, the estimation accuracy of server side estimator, for an overloaded server, may go up dramatically. There is need for the external pinger only for web pages with few embedded images. The average latency estimation error is 10%, in contrast the median latency estimation error is only 4%.
    Meaning
    RTT Average Median
    Estimation RTTmin RTTmin
    Method (1) Pinger (1) Pinger
    No. of Main Web Main Web Main Web Main Web
    embedded Page Page Page Page Page Page Page Page
    images Error Error Error Error Error Error Error Error
    5 14.3 20.7 3.6 3.9 7.0 7.1 1.9 2.5
    15 10.5 9.6 10.1 9.5 2.8 2.5 2.0 2.9
    30 10.3 6.1 10.3 6.1 4.9 2.3 4.9 2.3
    50 8.6 11.5 8.5 11.5 1.2 7.7 1.2 7.7
  • In the presented tests, the server CPU overhead due to the additional fields logged is 0.15%. The queuing latency probe samples the server every four seconds, which adds an average 0.025% CPU overhead. The external pinger runs once every 30 seconds, which adds, on average 0.04% CPU overhead. Hence, the total average server CPU overhead due to our measurements is less the 0.2%. [0390]
  • There is thus provided, in accordance with the above embodiments, a system which allows for server end measurement of actual client experienced latencies in downloading data. The latencies can be used to provide a content provider with a clear picture of performance of his website. [0391]
  • It is appreciated that certain features of the invention, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the invention which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable subcombination. [0392]
  • It will be appreciated by persons skilled in the art that the present invention is not limited to what has been particularly shown and described hereinabove. Rather the scope of the present invention is defined by the appended claims and includes both combinations and subcombinations of the various features described hereinabove as well as variations and modifications thereof which would occur to persons skilled in the art upon reading the foregoing description. [0393]

Claims (99)

1. Network latency estimation apparatus for estimating a client's latency in a network communication between a server and said client, the apparatus comprising:
an event observer for observing occurrences of pre-selected events associated with said communication occurring at said server,
a logging device, associated with said event observer for logging into a data store the occurrence of said events together with respective time information, and
a latency estimator associated with said logging device for using said logged occurrences with said respective time information to arrive at an estimation of a client's latency for said communication.
2. Apparatus according to claim 1, wherein said client's latency comprises the difference between a first time at which a client sends request data and a second time at which said client completes receipt of the requested data.
3. Apparatus according to claim 1, said latency estimator further comprising a round-trip-time estimator, capable of estimating a round trip time for said client, said estimation being based on receipt at the server of a request for data from the client in consequence of data previously sent to the client by the server.
4. Apparatus according to claim 3, wherein said round-trip-time-estimator is operable to estimate the round trip time by determining the shortest duration over a plurality of durations, measured between when a server sends data to a client, and when the server receives a subsequent request from the client, said subsequent request being in consequence of said data sent to the client by the server.
5. Apparatus according to claim 3, further comprising a pinger associated with said event observer.
6. Apparatus according to claim 5, wherein said events comprise sending out a ping, and receiving a response.
7. Apparatus according to claim 6, wherein the round-trip-time estimator is operable to estimate the round trip time based on the shorter of:
(a) a duration between a first time when the server sends data to a client, and a second time when the server receives a subsequent request from the client, and
(b) the logged times of said ping event.
8. Apparatus according to claim 7, wherein said data store contains at least some of:
(a) an IP address of the client,
(b) a number of bytes in a server's response,
(c) a server's processing time,
(d) a flag indicating whether the current request is the first request on a current channel,
(e) a time the server accepts a connection from the client,
(g) a time the server starts processing a client's request,
(h) a time the server completes writing the response to the channel, and
(i) a number of bytes left to be sent when the server completes writing requests to the channel.
9. Apparatus according to claim 1, wherein said latency estimator is operable to form an initial estimate of the client's latency by adding an estimated round trip time, to a delay between when the server receives an initial request for data from said client, and when the server receives a subsequent request for data from said client.
10. Apparatus according to claim 9, comprising a receipt indicator, associated with said event observer, operable to add to the end of data being sent an automatic end of data receipt indicator, for obtaining an acknowledgment of receipt of the end of the data by said client.
11. Apparatus according to claim 10, further comprising a client-data-reception-time estimator operable to estimate a time when the client received said data sent as the time when the server receives the acknowledgment of the receipt of said data.
12. Apparatus according to claim 11, wherein said latency estimator is further operable to estimate the client's latency by using the earliest of the times determined from:
when the server receives the acknowledgment of receipt of the data, and
when the server receives any other request from the client.
13. Apparatus according to claim 11, wherein said latency estimator is further operable to estimate the client's latency by using the earliest of the times determined by when the server received the acknowledgment of receipt of the data sent, and when the server received any other request from the client wherein said other request is in consequence of said data sent.
14. Apparatus according to claim 9, further comprising a queuing latency probe, associated with said logging device, capable of sending a queue probing request to the server.
15. Apparatus according to claim 14, said pre-selected events further comprising:
sending, by the queuing latency probe, of said queue probing request, and
the server accepting said queue probing request.
16. Apparatus according to claim 15, wherein said request is an HTTP request.
17. Apparatus according to claim 15, comprising an adder for adding an elapsed time between when said queuing latency probe request is sent to the server, and when said request is accepted for handling by the server, to said initial latency as measured by the client-latency estimator.
18. Apparatus according to claim 15, further comprising a transmission-data-rate estimator, connected to the latency estimator, said data rate estimator comprising a divider and a subtractor, said subtractor being connected to receive as inputs the estimated round trip time, and the initial estimate of the client's latency, and to produce an output being the difference therebetween, said divider being connected to receive as a first input an amount of data sent by the server in the course of the transmission, and as a second input said subtractor output, and to produce an estimate of a transmission data rate by dividing therebetween.
19. Apparatus according to claim 18, further comprising an overall-client-data-rate estimator comprising an averager for averaging together a plurality of successively estimated transmission data rates for the given client connection thereby to estimate an overall data rate for said given client.
20. Apparatus according to claim 19, further comprising a client-data-reception-time estimator, comprising a multiplier and an adder, said adder being functional to add a time at which the server dispatched an end part of the requested data to the channel, to an output of the multiplier, wherein the multiplier is operable to multiply an amount of data left to send, by said overall data rate, the client-data reception time estimator thereby obtaining an estimate of client-data-reception-time.
21. Apparatus according to claim 20 further comprising a subtractor and an adder, said subtractor operable to subtract the time at which the server received an initial request for data from a client from said time the client received the last of the requested data, said adder operable to add one and a half times the estimated round trip time, and further add the latency as estimated by the queuing latency probe, to the output of the subtractor.
22. Apparatus according to claim 18, further comprising a client-connection-data-rate estimator comprising an averager for averaging together all the transmission data rates for a given client connection wherein the corresponding transmissions meet a specific size criteria.
23. Apparatus according to claim 22, further comprising a client-data-reception-time estimator comprising an adder and a multiplier, said adder for adding the time the server last dispatched data to the client, to the output of the multiplier, wherein said multiplier is operable to multiply an amount of data left to send by said data rate for a given client connection, thereby to produce an estimate of a time of receipt of said last data to the client.
24. Apparatus according to claim 23 further comprising a client latency estimator for estimating a respective client latency's, the latency estimator having a subtractor and an adder, said subtractor for subtracting the time the server received an initial request for data from a client from said estimate of a time of receipt of said last data to the client, said adder for adding together one and a half times the estimated round trip time, the latency as estimated by the queuing latency probe, and the output of the subtractor, thereby to form an estimate of said respective client's latency.
25. Apparatus according to claim 18, further comprising a global-transmission-rate estimator comprising an averager, said averager for averaging together successive data transmission rates, thereby to estimate a global transmission rate.
26. Apparatus according to claim 25, further comprising a client-data-reception-time estimator comprising an adder and a multiplier, said multiplier operable to multiply an amount of data left to send by said global transmission data rate and said adder operable to add the time the server dispatched the last data to the client, to the output of the multiplier, thereby to provide an estimate of a client data reception time.
27. Apparatus according to claim 26 further comprising a subtractor and an adder, said subtractor for subtracting the time the server received an initial request for data from a client from said time of receipt of said last data at said client, said adder connected to add one and a half times the estimated round trip time, to the latency as estimated by the queuing latency probe, thereby to provide a revision of the estimate of the client's latency.
28. Apparatus according to claim 18, further comprising a global-transmission-rate estimator comprising an averager operable to estimate a global transmission data rate, said averager operable to average together a series of successively obtained transmission data rates wherein respective transmissions meet a pre-determined size criteria.
29. Apparatus according to claim 28, further comprising a client-data-reception-time estimator operable to estimate the time the client received a last part of the requested data, the estimator comprising a multiplier and an adder, the multiplier being connected to multiply the amount of data left to send by said global transmission data rate, and to output a result to provide a first input to said adder, said adder being arranged to add said input to said time at which the server dispatched said last data part to the client, thereby to provide an estimate of said client data reception time.
30. Apparatus according to claim 29, said subtractor further being operable to subtract the time the server received an initial request for data from a client from said time the client received the last part of said requested data, said adder being further operable to add, to the output of the subtractor, one and a half times the estimated round trip time, and the time duration between a first time when the queuing latency probe sends the request to the server, and a second time when the server accepts the request sent by the probe.
31. Apparatus according to claim 18, wherein a plurality of data transmission rates are measured for a given channel, the apparatus further comprising a channel-data-rate estimator comprising an averager for averaging together all the data rates for each transmission on said given channel.
32. Apparatus according to claim 31, further comprising an overall-client-data-rate estimator comprising an averager, said averager being operable to average together a plurality of successively measured data rates for a given client's channels.
33. Apparatus according to claim 31, further comprising a global-data-rate estimator for estimating a global data rate, the estimator comprising an averager for averaging together a plurality of successively measured data rates.
34. Apparatus according to claim 18, further comprising a transmission-packet-rate estimator operable to estimate a packet rate for a transmission, the estimator comprising a divider and a subtractor, wherein said subtractor is operable to subtract the round trip time from the initial estimate of the client's latency for the transmission to produce a subtractor output, said divider being connected to divide the number of packets the server sends in the course of the transmission, by said subtractor output.
35. Apparatus according to claim 34, further comprising an overall-client-packet-rate estimator operable to estimate the overall packet rate for a given client connection, the estimator comprising an averager, for averaging together a plurality of successively measured transmission packet rates for the given client connection.
36. Apparatus according to claim 35, further comprising a client-data-reception-time estimator operable to estimate a time the client received the last of the data, the estimator comprising a multiplier and an adder, said multiplier operable to multiply a number of packets remaining to be sent by said overall packet rate, thereby to produce a multiplier output, and said adder being connected to add the time the server wrote the end of the data to the client, to said multiplier output.
37. Apparatus according to claim 36, the latency estimator being further operable to estimate the client latency, said subtractor further operable to subtract the time the server received an initial request for data from a client from said time the client received the last of the data, thereby to produce a subtractor output, said adder further operable to add to the result of the subtractor one and a half times the estimated round trip time, and further to add thereto the latency as estimated by the queuing latency probe.
38. Apparatus according to claim 34, further comprising a client-connection-packet-rate estimator operable to estimate the packet rate for a given client connection, the estimator comprising an averager operable to average together ones of a succession of packet rates for given transmissions of a client connection wherein respective transmissions meet a pre-selected size criteria.
39. Apparatus according to claim 38, further comprising a client-data-reception-time estimator operable to estimate a time the client received the last of the data, the estimator comprising an adder and a multiplier, said multiplier operable to multiply the number of packets left to send by said packet rate for a given client connection, to produce a multiplier output, said adder adding the time the server wrote the last data to the client, to said multiplier output.
40. Apparatus according to claim 39, further operable to estimate the client latency, said subtractor further operable to subtract the time the server received an initial request for data from a client from said estimated time the client received the last of the data, said adder being further operable to add to the output of the subtractor one and a half times the estimated round trip time, and said adder further operable to add thereto a queuing latency duration between a queuing latency probing request sending time when the queuing latency probe sends said probing request to the server, and a second, queuing latency probe request receipt time, when the server accepts said probing request.
41. Apparatus according to claim 34, further comprising a global-transmission-rate estimator, comprising an averager operable to estimate a global transmission packet rate, said averager operable to average together all the transmission packet rates of all the connections to the server.
42. Apparatus according to claim 41, further comprising a client-data-reception-time estimator operable to estimate the time the client received the last of the data, the estimator comprising an adder and a multiplier, said multiplier operable to multiply the number of packets left to send by said global transmission packet rate, to form a multiplier output, said adder for adding the time the server wrote the last data to the client, to said multiplier output.
43. Apparatus according to claim 42, further operable to estimate the client latency, the apparatus comprising a further adder and a further subtractor, said further subtractor operable to subtract the time the server received an initial request for data from a client from said time the client received the last of the data thereby to form a subtractor output, said adder operable to add to the output of the subtractor one and a half times the estimated round trip time, and the duration between the a first time when the queuing latency probe sends the request to the server, and a second time when the server accepts the request sent by the probe.
44. Apparatus according to claim 34, further comprising a global-transmission-rate estimator operable to estimate a global transmission packet rate, said averager being further operable to average together the transmission rates from all transmissions meeting a predetermined size criteria.
45. Apparatus according to claim 44, further comprising a client-data-reception-time estimator operable to estimate the time the client received the last of the data, the estimator comprising an adder and a multiplier, said multiplier multiplying the number of packets left to send by said global transmission packet rate, said adder operable to add the time the server wrote the last data to the client, to the output of the multiplier.
46. Apparatus according to claim 45, comprising a subtractor and a further adder, said subtractor being operable to subtract the time the server received an initial request for data from a client from said time the client received the last of the data, said adder being operable to add to the output of the subtractor one and a half times the estimated round trip time, and the time duration between the a first time when the queuing latency probe sends the latency test request to the server, and a second time when the server accepts the latency test request.
47. Apparatus according to claim 14, further comprising a channel-packet-rate estimator operable to estimate the packet rate for a given channel, the estimator comprising a subtractor, and a divider, wherein said subtractor is operable to subtract the round trip time from the latency measured for said client for said channel, thereby to produce a subtractor output, said divider being connected to divide a number of packets the server sends on the channel, by the output of the subtractor.
48. Apparatus according to claim 47, wherein each client connects via a plurality of channels, the apparatus further comprising an overall-client-packet-rate estimator operable to estimate an overall packet rate for a given client, the estimator comprising an averager for averaging together a plurality of successively measured packet rates for each of said plurality of channels.
49. Apparatus according to claim 47, further comprising a global-packet-rate estimator operable to estimate the global packet rate, the estimator comprising an averager operable to average together a plurality of successively measured packet rates.
50. Apparatus according to claim 1, wherein the apparatus is operable to log HTTP transmissions.
51. A method for estimating the latency of a user client on a network communication, using measurements made in association with a server with which said client is in communication, the method comprising the steps of:
(a) logging in association with a server, events, and the times of occurrence of said events, and
(b) processing said logged times to estimate said latency.
52. The method of claim 51, wherein said user client latency is the latency between a first time when a client sends a request for data to a server and a second time at which said client receives a last datum of the requested data.
53. The method of claim 52, further comprising a round-trip-time estimation step, comprising estimating a round trip time, said estimation being based on when a request for data from the client is received by the server in consequence of data previously sent to the client by the server.
54. The method of claim 53, wherein said round-trip-time-estimation step comprises estimating the round trip time by finding a shortest duration between a third time when the server sends data to said client, and a fourth time when the server receives a subsequent request from said client, said subsequent request being made by said client in consequence of data sent to the client by the server.
55. The method of claim 54, further comprising a step of pinging the client.
56. The method of claim 55 further comprising recording time information about a duration between sending said ping and receiving a response from a respective client.
57. The method of claim 56, comprising estimating a data round trip time based on the shorter of:
(a) a shortest duration between when the server sends data to a client, and when the server receives a corresponding subsequent request from the client, and
(b) a time duration recorded by said pinging.
58. The method of claim 52, wherein the logging step logs a selection from the group consisting of:
(a) an IP address of the client,
(b) a number of bytes in a response,
(c) a server's processing time,
(d) a flag indicating if a current request is a first request on a current channel,
(e) a time the server accepts a connection from the client,
(g) a time the server starts processing a request,
(h) a time the server completes writing a request to a channel, and
(i) a number of bytes left to be sent when the server completes writing requests to a channel.
59. The method of claim 52, further comprising a client-latency estimation step, of:
determining a duration between when the server receives an initial request for data from a client, and when the server receives a subsequent request for data from that client, as a response request duration, and
adding thereto an estimated round trip time, thereby to form an approximation of said client latency.
60. The method of claim 59, comprising adding, at the end of the data being sent, an indication to the client to send an additional request to the server.
61. The method of claim 60, further comprising a client-data-reception-time estimation step comprising approximating a time when the client receives data sent as a time when the server receives a request resulting from said indication.
62. The method of claim 61, wherein said client-latency estimation step comprises providing a first client latency estimate as an earliest of:
a time when the server receives the request in response to said indicator, and
a time when the server receives any other request from the client wherein said other request is in consequence of said data sent.
63. The method of claim 59, having a queuing latency probing step, comprising
sending a request to the server,
measuring a first time at which said request is sent to the server,
measuring a second time at which said request is handled by the server, and
recording a queuing latency as a duration between said first time and said second time.
64. The method of claim 63, wherein said request is an HTTP request.
65. The method of claim 63, wherein said client-latency estimation step further comprises adding said recorded queuing latency to said initial client latency estimate, thereby to provide a revised client latency estimation.
66. The method of claim 63, further comprising a transmission-data-rate estimation step comprising dividing an amount of data the server sends in the course of a given transmission, by said client latency estimate for said given transmission, and subtracting the round trip time, thereby to estimate a transmission data rate.
67. The method of claim 66, further comprising an overall-client-data-rate estimation step of averaging together a plurality of successively measured transmission data rates for the given client connection.
68. The method of claim 67, further comprising a client-data-reception-time estimation step comprising:
multiplying an amount of data left to send by said overall data rate, and
adding thereto a time at which the server dispatches an end of data indication to the client,
thereby to provide an approximation of a time at which the client receives a last part of the data being sent.
69. The method of claim 68, wherein the user client latency estimation step comprises:
subtracting the time the server received an initial request for data from the client from said time the client received the last of the data,
adding one and a half times the estimated round trip time, and
adding the recorded queuing latency, thereby to estimate the client latency.
70. The method of claim 66, further comprising a client-connection-data-rate estimation step comprising:
identifying ones of transmissions for a respective client connection, being larger than a predetermined size,
averaging together respective transmission data rates of said identified transmissions, thereby to provide an approximation of the data rate for a given client connection.
71. The method of claim 70, further comprising a client-data-reception-time estimation step comprising:
multiplying the amount of data remaining to send by said data rate for a given client connection, and
adding thereto a time at which the server dispatched the last data part of the requested data to the client,
thereby to provide an approximation of a time the client received the last of the requested data.
72. The method of claim 71, further comprising:
subtracting the time at which the server received an initial request for data from a client from said time at which the client received the last of the requested data,
adding thereto one and a half times the estimated round trip time, and
adding the queuing latency thereto,
thereby to provide an approximation of the client latency.
73. The method of claim 66, further comprising a global-transmission-rate estimation step, comprising estimating a global transmission data rate, by averaging together transmission rates.
74. The method of claim 73, further comprising a client-data-reception-time estimation step of:
multiplying the amount of data left to send by said global transmission data rate,
adding thereto the time of server dispatch of the last data to the client, thereby to provide an approximation of a time at which the client received the last of the requested data.
75. The method of claim 74, further comprising:
subtracting the time the server received an initial request for data from a client from said time the client received the last of the requested data,
adding thereto one and a half times the estimated round trip time, and
adding thereto the queuing latency,
thereby to provide an estimation of the client latency.
76. The method of claim 66, further comprising a global-transmission-rate estimation step, comprising
identifying ones of a plurality of transmissions exceeding a predetermined threshold size, and
averaging together transmission rates of said identified transmissions, thereby to estimates a global transmission data rate.
77. The method of claim 76, further comprising a client-data-reception-time estimation step, comprising:
multiplying the amount of data left to send by said global transmission data rate,
adding thereto the time the server wrote the last data to the client, thereby to estimate the time the client received the last of the requested data.
78. The method of claim 77, comprising:
subtracting the time the server received an initial request for data from a client from said time the client received the last of the requested data,
adding one and a half times the estimated round trip time, and
adding thereto the recorded queuing latency, thereby to estimate the client latency.
79. The method of claim 63, wherein each said client communicates via at least one channel, the method further comprising a channel-data-rate estimation step of:
dividing an amount of data the server sends, by an approximation of the client latency measured for said channel, and
subtracting therefrom the round trip time, thereby to provide an approximation of the data rate for a given channel.
80. The method of claim 79, wherein each client communicates via a plurality of channels, the method further comprising an overall-client-data-rate estimation step of:
providing data rates for each one of said plurality of channels,
averaging together said plurality of data rates from said plurality of channels, thereby to estimate an approximation of a data rate for a given client.
81. The method of claim 79, further comprising a global-data-rate estimation step comprising:
providing data rates for a plurality of clients each communicating over a plurality of channels, and
averaging together a plurality of successively measured data rates thereby to estimate the global data rate.
82. The method of claim 63, further comprising a transmission-packet-rate estimation step of:
obtaining a number of packets being sent by a server in the course of a transmission,
dividing said number of packets by said first client latency estimate,
subtracting therefrom the estimated round trip time, thereby to estimate a transmission packet rate for said transmission.
83. The method of claim 82, further comprising an overall-client-packet-rate estimation step of:
obtaining a plurality of said transmission packet rates over a plurality of transmissions for a given client connection,
averaging together said plurality of transmission packet rates, thereby to provide an estimate of an overall client connection packet rate.
84. The method of claim 83, further comprising a client-data-reception-time estimation step of:
multiplying the number of packets left to send by said overall packet rate,
adding thereto the time the server wrote the end of the data to the client, thereby to provide an approximation of a time at which the client received the last of the requested data.
85. The method of claim 84, wherein the client-latency estimation step comprises:
subtracting the time the server received an initial request for data from said client from said time the client received the last of the requested data,
adding thereto one and a half times the estimated round trip time, and
adding thereto the recorded queuing latency, thereby to provide an approximation of the client latency.
86. The method of claim 82, further comprising a client-connection-packet-rate estimation step of
obtaining a plurality of transmission packet rates for a given connection,
identifying ones of said plurality of transmission packet rates whose respective transmissions exceed a predetermined size threshold, and
averaging together said identified transmission packet rates, thereby to provide a client connection packet rate approximation.
87. The method of claim 86, further comprising a client-data-reception-time estimation step, of
multiplying the number of packets left to send by said client connection packet rate approximation, and
adding thereto the time the server dispatched the last of the requested data to the client, thereby to provide an estimate of a time the client received the last of the requested data.
88. The method of claim 87, further comprising:
subtracting the time the server received an initial request for data from a client from said time the client received the last of the requested data,
adding thereto one and a half times the estimated round trip time, and
adding thereto the recorded queuing latency, thereby to estimate the client latency.
89. The method of claim 82, further comprising a global-transmission-rate estimation step of:
obtaining a plurality of transmission packet rates, and
averaging together said plurality of transmission packet rates, thereby to estimate a global transmission packet rate.
90. The method of claim 89, further comprising a client-data-reception-time estimation step of:
multiplying the number of packets left to send by said global transmission packet rate,
adding thereto the time the server wrote the last data to the channel, thereby to provide an approximation of a time at which the client received the last of the requested data.
91. The method of claim 90, further comprising
subtracting the time the server received an initial request for data from a client from said approximation of the time at which the client received the last of the requested data,
adding thereto one and a half times the estimated round trip time, and
adding the queuing latency, thereby to provide an approximation of the client latency.
92. The method of claim 82, further comprising a global-transmission-rate estimation step of:
obtaining a plurality of transmission rates for each of a plurality of data transmissions,
identifying ones of said plurality of transmission rates whose respective transmissions exceed a predetermined size threshold, and
averaging together said identified transmission rates to form an approximation of a global transmission packet rate.
93. The method of claim 92, further comprising a client-data-reception-time estimation step of:
multiplying the number of packets left to send by said approximation of said global transmission packet rate, and
adding thereto the time the server wrote the last data to the channel, thereby to provide an estimate of the time the client received the last of the requested data.
94. The method of claim 93, further comprising:
subtracting the time the server received an initial request for data from a client from said estimate of the time the client received the last of the requested data,
adding thereto one and a half times the estimated round trip time, and
adding thereto the recorded queuing latency, thereby to provide an approximation of said client latency.
95. The method of claim 63, further comprising a channel-packet-rate estimation step, wherein said client communicates using at least one channel, the method including:
obtaining a client latency approximation for said channel,
dividing the number of packets the server is sending over the channel by the client latency approximation for said channel,
subtracting therefrom the round trip time, thereby to provide an approximation of a packet rate for a given channel.
96. The method of claim 95, said client having at least one additional channel, the method further comprising:
obtaining an approximation of a packet rate for each of said channels,
averaging together said packet rates, thereby to estimate a packet rate for said client.
97. The method of claim 95, further comprising a global-packet-rate estimation step of:
obtaining a plurality of packet rates for each one of a plurality of clients, and
averaging together said packet rates to provide an approximation of a global packet rate.
98. The method of claim 51 comprising logging HTTP transmissions.
99. A data carrier carrying data usable in combination with a general purpose computer to provide functionality capable of estimating the latency between when a client sends a request for data to a server, and when that client receives the data, using measurements made at a server, the data being usable to provide:
an event observer for observing pre-selected events associated with said communication occurring at said server,
a logging device associated with said event observer for logging into a data store the occurrence of said events together with respective time information, and
a latency estimator associated with said logging device for using said logged occurrences with said respective time information to arrive at an estimation of latency in said communication.
US10/170,460 2002-06-14 2002-06-14 Determining client latencies over a network Active 2026-02-03 US7747729B2 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US10/170,460 US7747729B2 (en) 2002-06-14 2002-06-14 Determining client latencies over a network
US11/514,378 US7676570B2 (en) 2002-06-14 2006-08-30 Determining client latencies over a network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US10/170,460 US7747729B2 (en) 2002-06-14 2002-06-14 Determining client latencies over a network

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US11/514,378 Continuation US7676570B2 (en) 2002-06-14 2006-08-30 Determining client latencies over a network

Publications (2)

Publication Number Publication Date
US20030233445A1 true US20030233445A1 (en) 2003-12-18
US7747729B2 US7747729B2 (en) 2010-06-29

Family

ID=29732504

Family Applications (2)

Application Number Title Priority Date Filing Date
US10/170,460 Active 2026-02-03 US7747729B2 (en) 2002-06-14 2002-06-14 Determining client latencies over a network
US11/514,378 Expired - Lifetime US7676570B2 (en) 2002-06-14 2006-08-30 Determining client latencies over a network

Family Applications After (1)

Application Number Title Priority Date Filing Date
US11/514,378 Expired - Lifetime US7676570B2 (en) 2002-06-14 2006-08-30 Determining client latencies over a network

Country Status (1)

Country Link
US (2) US7747729B2 (en)

Cited By (118)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050102413A1 (en) * 2003-11-10 2005-05-12 Binns Pamela A. Real-time estimation of event-driven traffic latency distributions when layered on static schedules
US20050160141A1 (en) * 2004-01-21 2005-07-21 Mark Galley Internet network banner
US20050201485A1 (en) * 2002-05-22 2005-09-15 Koninkljke Phillips Electronics N.V. Transmission method using a virtual reception buffer to absorb fluctuation of the channel transmission rate
US20060221852A1 (en) * 2005-04-01 2006-10-05 International Business Machines Corporation System and method utilizing a single agent on a non-origin node for measuring the roundtrip response time over a public or private network with HTTP/HTTPS network protocol
US20060221851A1 (en) * 2005-04-01 2006-10-05 International Business Machines Corporation System and method for measuring the roundtrip response time of network protocols utilizing a single agent on a non-origin node
US20060230134A1 (en) * 2005-04-11 2006-10-12 Tin Qian Network experience rating system and method
US20060235961A1 (en) * 2005-04-01 2006-10-19 International Business Machines Corporation System and method utilizing a single agent on a non-origin node for measuring the roundtrip response time of web pages with embedded HTML frames over a public or private network
WO2006055769A3 (en) * 2004-11-17 2007-02-01 Univ California System and method for providing a web page
US20070033213A1 (en) * 2005-08-05 2007-02-08 Sap Aktiengesellschaft Methods and systems for merging software-level objects with document-level objects in a document publishing environment
US20070064610A1 (en) * 2005-09-19 2007-03-22 Khandani Mehdi K Detection of nonconforming network traffic flow aggregates for mitigating distributed denial of service attacks
EP1807974A2 (en) * 2004-08-28 2007-07-18 Streamaware, LLC Link analysis method and system
US20070297387A1 (en) * 2004-09-02 2007-12-27 Sujit Dey Content And Channel Aware Object Scheduling And Error Control
US20070299954A1 (en) * 2006-06-27 2007-12-27 International Business Machines Corporation System, method and program for determining a network path by which to send a message
US20080104224A1 (en) * 2006-10-27 2008-05-01 Litofsky Richard T Method and apparatus for determining application responsiveness over a network
US7418492B1 (en) * 2002-06-20 2008-08-26 P-Cube Ltd. System and a method for testing network communication devices
US20100017462A1 (en) * 2003-03-19 2010-01-21 Cgi Communications, Inc. System and method for seamlessly providing video content to client systems over a network
US7782767B1 (en) * 2005-07-20 2010-08-24 Tektronix, Inc. Method and system for calculating burst bit rate for IP interactive applications
US20100261535A1 (en) * 2007-09-28 2010-10-14 Konami Digital Entertainment Co., Ltd. Game system and server
US20120078707A1 (en) * 2010-09-29 2012-03-29 Eyal Arasu Ramakrishnan Measuring Inline Ad Performance for Third-Party Ad Serving
US20120243534A1 (en) * 2010-08-06 2012-09-27 Rafsky Lawrence C Method and system for pacing, acking, timing, and handicapping (path) for simultaneous receipt of documents
US8301753B1 (en) * 2006-06-27 2012-10-30 Nosadia Pass Nv, Limited Liability Company Endpoint activity logging
US20130091268A1 (en) * 2011-10-10 2013-04-11 Rohan Bopardikar Classification of web client network bandwidth by a web server
US8438276B1 (en) * 2004-08-31 2013-05-07 Precise Software Solutions, Inc. Method of monitoring network and application performance by analyzing web clients and web servers
US8521891B1 (en) * 2007-06-21 2013-08-27 Mcafee, Inc. Network browser system, method, and computer program product for conditionally loading a portion of data from a network based on a data transfer rate
US20130242185A1 (en) * 2012-03-14 2013-09-19 Todd Stuart Roth Adaptive media delivery
KR101405472B1 (en) * 2012-11-09 2014-06-27 (주)씨디네트웍스 Method and apparatus for latency measurement based upon passive measurement
US8788527B1 (en) 2003-12-31 2014-07-22 Precise Software Solutions, Inc. Object-level database performance management
US9083743B1 (en) 2012-03-21 2015-07-14 Amazon Technologies, Inc. Managing request routing information utilizing performance information
US9106701B2 (en) 2010-09-28 2015-08-11 Amazon Technologies, Inc. Request routing management based on network components
US9130756B2 (en) 2009-09-04 2015-09-08 Amazon Technologies, Inc. Managing secure content in a content delivery network
US9135048B2 (en) 2012-09-20 2015-09-15 Amazon Technologies, Inc. Automated profiling of resource usage
US9154551B1 (en) 2012-06-11 2015-10-06 Amazon Technologies, Inc. Processing DNS queries to identify pre-processing information
US9160703B2 (en) 2010-09-28 2015-10-13 Amazon Technologies, Inc. Request routing management based on network components
US9176894B2 (en) 2009-06-16 2015-11-03 Amazon Technologies, Inc. Managing resources using resource expiration data
US9185012B2 (en) 2010-09-28 2015-11-10 Amazon Technologies, Inc. Latency measurement in resource requests
US9191458B2 (en) 2009-03-27 2015-11-17 Amazon Technologies, Inc. Request routing using a popularity identifier at a DNS nameserver
US9191338B2 (en) 2010-09-28 2015-11-17 Amazon Technologies, Inc. Request routing in a networked environment
US9208097B2 (en) 2008-03-31 2015-12-08 Amazon Technologies, Inc. Cache optimization
US9210235B2 (en) 2008-03-31 2015-12-08 Amazon Technologies, Inc. Client side cache management
US9237114B2 (en) 2009-03-27 2016-01-12 Amazon Technologies, Inc. Managing resources in resource cache components
US9246776B2 (en) 2009-10-02 2016-01-26 Amazon Technologies, Inc. Forward-based resource delivery network management techniques
US9253065B2 (en) 2010-09-28 2016-02-02 Amazon Technologies, Inc. Latency measurement in resource requests
US9251112B2 (en) 2008-11-17 2016-02-02 Amazon Technologies, Inc. Managing content delivery network service providers
US9294391B1 (en) 2013-06-04 2016-03-22 Amazon Technologies, Inc. Managing network computing components utilizing request routing
US9323577B2 (en) 2012-09-20 2016-04-26 Amazon Technologies, Inc. Automated profiling of resource usage
US9332078B2 (en) 2008-03-31 2016-05-03 Amazon Technologies, Inc. Locality based content distribution
US20160188445A1 (en) * 2014-12-30 2016-06-30 Spirent Communications, Inc. Conducting performance snapshots during test and using feedback to control test based on customer experience parameters
US9391949B1 (en) 2010-12-03 2016-07-12 Amazon Technologies, Inc. Request routing processing
US9407681B1 (en) * 2010-09-28 2016-08-02 Amazon Technologies, Inc. Latency measurement in resource requests
US9407699B2 (en) 2008-03-31 2016-08-02 Amazon Technologies, Inc. Content management
US9444759B2 (en) 2008-11-17 2016-09-13 Amazon Technologies, Inc. Service provider registration by a content broker
US9451046B2 (en) 2008-11-17 2016-09-20 Amazon Technologies, Inc. Managing CDN registration by a storage provider
US20160308959A1 (en) * 2008-06-30 2016-10-20 Amazon Technologies, Inc. Latency measurement in resource requests
US9479476B2 (en) 2008-03-31 2016-10-25 Amazon Technologies, Inc. Processing of DNS queries
US9497259B1 (en) 2010-09-28 2016-11-15 Amazon Technologies, Inc. Point of presence management in request routing
US9495338B1 (en) 2010-01-28 2016-11-15 Amazon Technologies, Inc. Content distribution network
US9515949B2 (en) 2008-11-17 2016-12-06 Amazon Technologies, Inc. Managing content delivery network service providers
US9525659B1 (en) 2012-09-04 2016-12-20 Amazon Technologies, Inc. Request routing utilizing point of presence load information
US9544394B2 (en) 2008-03-31 2017-01-10 Amazon Technologies, Inc. Network resource identification
US9571389B2 (en) 2008-03-31 2017-02-14 Amazon Technologies, Inc. Request routing based on class
US9608957B2 (en) 2008-06-30 2017-03-28 Amazon Technologies, Inc. Request routing using network computing components
US9628554B2 (en) 2012-02-10 2017-04-18 Amazon Technologies, Inc. Dynamic content delivery
US9712484B1 (en) 2010-09-28 2017-07-18 Amazon Technologies, Inc. Managing request routing information utilizing client identifiers
US9734472B2 (en) 2008-11-17 2017-08-15 Amazon Technologies, Inc. Request routing utilizing cost information
US9734134B1 (en) * 2013-09-19 2017-08-15 Amazon Technologies, Inc. Conditional promotion through frame reordering
US9742795B1 (en) 2015-09-24 2017-08-22 Amazon Technologies, Inc. Mitigating network attacks
US9774619B1 (en) 2015-09-24 2017-09-26 Amazon Technologies, Inc. Mitigating network attacks
US9785969B1 (en) 2013-09-19 2017-10-10 Amazon Technologies, Inc. Conditional promotion in multi-stream content delivery
US9787775B1 (en) 2010-09-28 2017-10-10 Amazon Technologies, Inc. Point of presence management in request routing
US9794281B1 (en) 2015-09-24 2017-10-17 Amazon Technologies, Inc. Identifying sources of network attacks
US9819567B1 (en) 2015-03-30 2017-11-14 Amazon Technologies, Inc. Traffic surge management for points of presence
US9832141B1 (en) 2015-05-13 2017-11-28 Amazon Technologies, Inc. Routing based request correlation
US9887931B1 (en) 2015-03-30 2018-02-06 Amazon Technologies, Inc. Traffic surge management for points of presence
US9887932B1 (en) 2015-03-30 2018-02-06 Amazon Technologies, Inc. Traffic surge management for points of presence
US9922006B1 (en) 2013-09-19 2018-03-20 Amazon Technologies, Inc. Conditional promotion through metadata-based priority hinting
US9930131B2 (en) 2010-11-22 2018-03-27 Amazon Technologies, Inc. Request routing processing
US9954934B2 (en) 2008-03-31 2018-04-24 Amazon Technologies, Inc. Content delivery reconciliation
US9985927B2 (en) 2008-11-17 2018-05-29 Amazon Technologies, Inc. Managing content delivery network service providers by a content broker
US9992086B1 (en) 2016-08-23 2018-06-05 Amazon Technologies, Inc. External health checking of virtual private cloud network environments
US9992303B2 (en) 2007-06-29 2018-06-05 Amazon Technologies, Inc. Request routing utilizing client location information
US10015237B2 (en) 2010-09-28 2018-07-03 Amazon Technologies, Inc. Point of presence management in request routing
US10021179B1 (en) 2012-02-21 2018-07-10 Amazon Technologies, Inc. Local resource delivery network
US10027582B2 (en) 2007-06-29 2018-07-17 Amazon Technologies, Inc. Updating routing information based on client location
US10033691B1 (en) 2016-08-24 2018-07-24 Amazon Technologies, Inc. Adaptive resolution of domain name requests in virtual private cloud network environments
US10033627B1 (en) 2014-12-18 2018-07-24 Amazon Technologies, Inc. Routing mode and point-of-presence selection service
US10049051B1 (en) 2015-12-11 2018-08-14 Amazon Technologies, Inc. Reserved cache space in content delivery networks
US10075551B1 (en) 2016-06-06 2018-09-11 Amazon Technologies, Inc. Request management for hierarchical cache
US10091096B1 (en) 2014-12-18 2018-10-02 Amazon Technologies, Inc. Routing mode and point-of-presence selection service
US10097566B1 (en) 2015-07-31 2018-10-09 Amazon Technologies, Inc. Identifying targets of network attacks
US10097448B1 (en) 2014-12-18 2018-10-09 Amazon Technologies, Inc. Routing mode and point-of-presence selection service
CN108701061A (en) * 2016-02-25 2018-10-23 瑞典爱立信有限公司 The resources control of interconnection hardware infrastructure
US10110694B1 (en) 2016-06-29 2018-10-23 Amazon Technologies, Inc. Adaptive transfer rate for retrieving content from a server
US10198348B2 (en) 2015-08-13 2019-02-05 Spirent Communications, Inc. Method to configure monitoring thresholds using output of load or resource loadings
US10205698B1 (en) 2012-12-19 2019-02-12 Amazon Technologies, Inc. Source-dependent address resolution
US10225326B1 (en) 2015-03-23 2019-03-05 Amazon Technologies, Inc. Point of presence based data uploading
US10230819B2 (en) 2009-03-27 2019-03-12 Amazon Technologies, Inc. Translation of resource identifiers using popularity information upon client request
US10257307B1 (en) 2015-12-11 2019-04-09 Amazon Technologies, Inc. Reserved cache space in content delivery networks
US10270878B1 (en) 2015-11-10 2019-04-23 Amazon Technologies, Inc. Routing for origin-facing points of presence
US10348639B2 (en) 2015-12-18 2019-07-09 Amazon Technologies, Inc. Use of virtual endpoints to improve data transmission rates
WO2019145720A1 (en) * 2018-01-25 2019-08-01 Spatialbuzz Limited Examining latency in communications networks
US20190238444A1 (en) * 2015-09-24 2019-08-01 Assia Spe, Llc Methods and apparatus for detecting internet connection problems
US10372499B1 (en) 2016-12-27 2019-08-06 Amazon Technologies, Inc. Efficient region selection system for executing request-driven code
US10447648B2 (en) 2017-06-19 2019-10-15 Amazon Technologies, Inc. Assignment of a POP to a DNS resolver based on volume of communications over a link between client devices and the POP
US10469513B2 (en) 2016-10-05 2019-11-05 Amazon Technologies, Inc. Encrypted network addresses
US10503613B1 (en) 2017-04-21 2019-12-10 Amazon Technologies, Inc. Efficient serving of resources during server unavailability
US10592578B1 (en) 2018-03-07 2020-03-17 Amazon Technologies, Inc. Predictive content push-enabled content delivery network
US10601767B2 (en) 2009-03-27 2020-03-24 Amazon Technologies, Inc. DNS query processing based on application information
US10616179B1 (en) 2015-06-25 2020-04-07 Amazon Technologies, Inc. Selective routing of domain name system (DNS) requests
US10623408B1 (en) 2012-04-02 2020-04-14 Amazon Technologies, Inc. Context sensitive object management
US10831549B1 (en) 2016-12-27 2020-11-10 Amazon Technologies, Inc. Multi-region request-driven code execution system
US10862852B1 (en) 2018-11-16 2020-12-08 Amazon Technologies, Inc. Resolution of domain name requests in heterogeneous network environments
US10938884B1 (en) 2017-01-30 2021-03-02 Amazon Technologies, Inc. Origin server cloaking using virtual private cloud network environments
US10958501B1 (en) 2010-09-28 2021-03-23 Amazon Technologies, Inc. Request routing information based on client IP groupings
US11025747B1 (en) 2018-12-12 2021-06-01 Amazon Technologies, Inc. Content request pattern-based routing system
US11075987B1 (en) 2017-06-12 2021-07-27 Amazon Technologies, Inc. Load estimating content delivery network
US11290418B2 (en) 2017-09-25 2022-03-29 Amazon Technologies, Inc. Hybrid content request routing system
US11606413B2 (en) * 2018-02-06 2023-03-14 Nippon Telegraph And Telephone Corporation Estimation apparatus, estimation method and program
US11604667B2 (en) 2011-04-27 2023-03-14 Amazon Technologies, Inc. Optimized deployment based upon customer locality

Families Citing this family (81)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TW200414737A (en) * 2002-09-27 2004-08-01 Matsushita Electric Ind Co Ltd Contents transmission system
AU2002952484A0 (en) * 2002-11-06 2002-11-21 Creative Software Solutions Pty, Ltd Network connected security system
US9549025B2 (en) * 2006-05-09 2017-01-17 International Business Machines Corporation Protocol optimization for client and server synchronization
US8413156B2 (en) 2007-04-05 2013-04-02 Ebay, Inc. Method and system for managing resource connections
US9032079B2 (en) * 2007-06-26 2015-05-12 Microsoft Technology Licensing, Llc Management and diagnosis of telephonic devices
US8904031B2 (en) * 2007-12-31 2014-12-02 Genesys Telecommunications Laboratories, Inc. Federated uptake throttling
US7860982B2 (en) * 2008-03-14 2010-12-28 Microsoft Corporation Internet connectivity verification
US8224624B2 (en) * 2008-04-25 2012-07-17 Hewlett-Packard Development Company, L.P. Using application performance signatures for characterizing application updates
US20090307347A1 (en) * 2008-06-08 2009-12-10 Ludmila Cherkasova Using Transaction Latency Profiles For Characterizing Application Updates
JP5253949B2 (en) * 2008-09-26 2013-07-31 ブラザー工業株式会社 Communication device and communication program
US8286176B1 (en) * 2008-09-29 2012-10-09 Amazon Technologies, Inc. Optimizing resource configurations
US7865594B1 (en) 2008-09-29 2011-01-04 Amazon Technologies, Inc. Managing resources consolidation configurations
US8117306B1 (en) 2008-09-29 2012-02-14 Amazon Technologies, Inc. Optimizing content management
US8051166B1 (en) 2008-09-29 2011-11-01 Amazon Technologies, Inc. Service provider optimization of content management
US7930393B1 (en) 2008-09-29 2011-04-19 Amazon Technologies, Inc. Monitoring domain allocation performance
US8316124B1 (en) 2008-09-29 2012-11-20 Amazon Technologies, Inc. Managing network data display
US8122124B1 (en) 2008-09-29 2012-02-21 Amazon Technologies, Inc. Monitoring performance and operation of data exchanges
US7925748B2 (en) * 2008-10-24 2011-04-12 International Business Machines Corporation System and method for managing system resources in a network environment
US8365062B2 (en) * 2008-11-02 2013-01-29 Observepoint, Inc. Auditing a website with page scanning and rendering techniques
US8589790B2 (en) * 2008-11-02 2013-11-19 Observepoint Llc Rule-based validation of websites
US8578019B2 (en) 2008-11-02 2013-11-05 Observepoint, Llc Monitoring the health of web page analytics code
US7930395B2 (en) * 2008-12-01 2011-04-19 International Business Machines Corporation System and method for managing system resources in a network environment
US8073655B2 (en) * 2009-01-23 2011-12-06 At&T Intellectual Property Ii, L.P. Quantifying the impact of network latency on the end-to-end response time of distributed applications
US7917618B1 (en) 2009-03-24 2011-03-29 Amazon Technologies, Inc. Monitoring web site content
US8706802B1 (en) * 2009-11-24 2014-04-22 Google Inc. Latency-guided web content retrieval, serving, and rendering
CA2927607C (en) * 2009-12-10 2021-04-06 Royal Bank Of Canada Synchronized processing of data by networked computing resources
US9940670B2 (en) 2009-12-10 2018-04-10 Royal Bank Of Canada Synchronized processing of data by networked computing resources
US8331371B2 (en) 2009-12-17 2012-12-11 Amazon Technologies, Inc. Distributed routing architecture
US8984048B1 (en) 2010-04-18 2015-03-17 Viasat, Inc. Selective prefetch scanning
EP2388947A1 (en) * 2010-05-20 2011-11-23 Thomson Licensing Method of determination of transmission quality of a communication link and corresponding apparatus
US9058323B2 (en) 2010-12-30 2015-06-16 Ss8 Networks, Inc. System for accessing a set of communication and transaction data associated with a user of interest sourced from multiple different network carriers and for enabling multiple analysts to independently and confidentially access the set of communication and transaction data
US8938534B2 (en) 2010-12-30 2015-01-20 Ss8 Networks, Inc. Automatic provisioning of new users of interest for capture on a communication network
US8972612B2 (en) 2011-04-05 2015-03-03 SSB Networks, Inc. Collecting asymmetric data and proxy data on a communication network
DE102011100793A1 (en) * 2011-05-06 2012-11-08 Vodafone Holding Gmbh Determining the transmission power in data networks
EP2639690B1 (en) * 2012-03-16 2017-05-24 Sony Corporation Display apparatus for displaying a moving object traversing a virtual display region
US9350762B2 (en) 2012-09-25 2016-05-24 Ss8 Networks, Inc. Intelligent feedback loop to iteratively reduce incoming network data for analysis
US9286047B1 (en) 2013-02-13 2016-03-15 Cisco Technology, Inc. Deployment and upgrade of network devices in a network environment
US9300430B2 (en) * 2013-10-24 2016-03-29 Harris Corporation Latency smoothing for teleoperation systems
US9830593B2 (en) 2014-04-26 2017-11-28 Ss8 Networks, Inc. Cryptographic currency user directory data and enhanced peer-verification ledger synthesis through multi-modal cryptographic key-address mapping
US10659332B2 (en) * 2014-11-26 2020-05-19 Nxp Usa, Inc. Network node, a communication system and associated methods
US9769248B1 (en) 2014-12-16 2017-09-19 Amazon Technologies, Inc. Performance-based content delivery
US10027739B1 (en) 2014-12-16 2018-07-17 Amazon Technologies, Inc. Performance-based content delivery
US10225365B1 (en) 2014-12-19 2019-03-05 Amazon Technologies, Inc. Machine learning based content delivery
US10311371B1 (en) 2014-12-19 2019-06-04 Amazon Technologies, Inc. Machine learning based content delivery
US10311372B1 (en) 2014-12-19 2019-06-04 Amazon Technologies, Inc. Machine learning based content delivery
US9575837B2 (en) 2015-02-03 2017-02-21 Uber Technologies, Inc. System and method for introducing functionality to an application for use with a network service
US10374904B2 (en) 2015-05-15 2019-08-06 Cisco Technology, Inc. Diagnostic network visualization
US9800497B2 (en) 2015-05-27 2017-10-24 Cisco Technology, Inc. Operations, administration and management (OAM) in overlay data center environments
US10089099B2 (en) 2015-06-05 2018-10-02 Cisco Technology, Inc. Automatic software upgrade
US10536357B2 (en) 2015-06-05 2020-01-14 Cisco Technology, Inc. Late data detection in data center
US9967158B2 (en) 2015-06-05 2018-05-08 Cisco Technology, Inc. Interactive hierarchical network chord diagram for application dependency mapping
US10033766B2 (en) 2015-06-05 2018-07-24 Cisco Technology, Inc. Policy-driven compliance
US10142353B2 (en) 2015-06-05 2018-11-27 Cisco Technology, Inc. System for monitoring and managing datacenters
US10158528B2 (en) 2015-10-13 2018-12-18 Uber Technologies, Inc. Application service configuration system
US11533226B2 (en) 2015-10-13 2022-12-20 Uber Technologies, Inc. Application service configuration system
US10171357B2 (en) 2016-05-27 2019-01-01 Cisco Technology, Inc. Techniques for managing software defined networking controller in-band communications in a data center network
US10931629B2 (en) 2016-05-27 2021-02-23 Cisco Technology, Inc. Techniques for managing software defined networking controller in-band communications in a data center network
US10289438B2 (en) 2016-06-16 2019-05-14 Cisco Technology, Inc. Techniques for coordination of application components deployed on distributed virtual machines
US10708183B2 (en) 2016-07-21 2020-07-07 Cisco Technology, Inc. System and method of providing segment routing as a service
US10972388B2 (en) 2016-11-22 2021-04-06 Cisco Technology, Inc. Federated microburst detection
US10708152B2 (en) 2017-03-23 2020-07-07 Cisco Technology, Inc. Predicting application and network performance
US10523512B2 (en) 2017-03-24 2019-12-31 Cisco Technology, Inc. Network agent for generating platform specific network policies
US10594560B2 (en) 2017-03-27 2020-03-17 Cisco Technology, Inc. Intent driven network policy platform
US10764141B2 (en) 2017-03-27 2020-09-01 Cisco Technology, Inc. Network agent for reporting to a network policy system
US10250446B2 (en) 2017-03-27 2019-04-02 Cisco Technology, Inc. Distributed policy store
US10873794B2 (en) 2017-03-28 2020-12-22 Cisco Technology, Inc. Flowlet resolution for application performance monitoring and management
US10680887B2 (en) 2017-07-21 2020-06-09 Cisco Technology, Inc. Remote device status audit and recovery
US10554501B2 (en) 2017-10-23 2020-02-04 Cisco Technology, Inc. Network migration assistant
US10523541B2 (en) 2017-10-25 2019-12-31 Cisco Technology, Inc. Federated network and application data analytics platform
US10594542B2 (en) 2017-10-27 2020-03-17 Cisco Technology, Inc. System and method for network root cause analysis
US11233821B2 (en) 2018-01-04 2022-01-25 Cisco Technology, Inc. Network intrusion counter-intelligence
US11765046B1 (en) 2018-01-11 2023-09-19 Cisco Technology, Inc. Endpoint cluster assignment and query generation
US10574575B2 (en) 2018-01-25 2020-02-25 Cisco Technology, Inc. Network flow stitching using middle box flow stitching
US10873593B2 (en) 2018-01-25 2020-12-22 Cisco Technology, Inc. Mechanism for identifying differences between network snapshots
US10917438B2 (en) 2018-01-25 2021-02-09 Cisco Technology, Inc. Secure publishing for policy updates
US10999149B2 (en) 2018-01-25 2021-05-04 Cisco Technology, Inc. Automatic configuration discovery based on traffic flow data
US10826803B2 (en) 2018-01-25 2020-11-03 Cisco Technology, Inc. Mechanism for facilitating efficient policy updates
US10798015B2 (en) 2018-01-25 2020-10-06 Cisco Technology, Inc. Discovery of middleboxes using traffic flow stitching
US11128700B2 (en) 2018-01-26 2021-09-21 Cisco Technology, Inc. Load balancing configuration based on traffic flow telemetry
US10565149B2 (en) * 2018-04-06 2020-02-18 Embrionix Design Inc. Standardized hot-pluggable transceiving unit, hosting unit and method for applying delays based on port positions
US10977105B2 (en) 2018-12-14 2021-04-13 Uber Technologies, Inc. Memory crash prevention for a computing device

Citations (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5570346A (en) * 1994-12-08 1996-10-29 Lucent Technologies Inc. Packet network transit delay measurement system
US5696948A (en) * 1994-07-13 1997-12-09 Bell Communications Research, Inc. Apparatus for determining round trip latency delay in system for preprocessing and delivering multimedia presentations
US6012096A (en) * 1998-04-23 2000-01-04 Microsoft Corporation Method and system for peer-to-peer network latency measurement
US6038599A (en) * 1997-04-23 2000-03-14 Mpath Interactive, Inc. Latency server and matchmaker
US6061722A (en) * 1996-12-23 2000-05-09 T E Network, Inc. Assessing network performance without interference with normal network operations
US6178160B1 (en) * 1997-12-23 2001-01-23 Cisco Technology, Inc. Load balancing of client connections across a network using server based algorithms
US6182125B1 (en) * 1998-10-13 2001-01-30 3Com Corporation Methods for determining sendable information content based on a determined network latency
US6222825B1 (en) * 1997-01-23 2001-04-24 Advanced Micro Devices, Inc. Arrangement for determining link latency for maintaining flow control in full-duplex networks
US6243761B1 (en) * 1998-03-26 2001-06-05 Digital Equipment Corporation Method for dynamically adjusting multimedia content of a web page by a server in accordance to network path characteristics between client and server
US6272539B1 (en) * 1998-11-18 2001-08-07 International Business Machines Corporation Methods, systems and computer program products for determining and visually representing a user's overall network delay in collaborative applications
US6272652B1 (en) * 1998-04-17 2001-08-07 Ameritech Corporation Method and system for adaptive interleaving
US6321264B1 (en) * 1998-08-28 2001-11-20 3Com Corporation Network-performance statistics using end-node computer systems
US20010050903A1 (en) * 2000-01-28 2001-12-13 Paul Vanlint Method and system to calculate network latency, and to display the same field of the invention
US20020026531A1 (en) * 2000-04-12 2002-02-28 John Keane Methods and systems for enabling communication between a processor and a network operations center
US6405252B1 (en) * 1999-11-22 2002-06-11 Speedera Networks, Inc. Integrated point of presence server network
US20020105911A1 (en) * 1998-11-24 2002-08-08 Parag Pruthi Apparatus and method for collecting and analyzing communications data
US20020112049A1 (en) * 2000-12-14 2002-08-15 International Business Machines Corporation Measuring response time for a computer accessing information from a network
US6438629B1 (en) * 1999-02-02 2002-08-20 Maxtor Corporation Storage device buffer access control in accordance with a monitored latency parameter
US20030046383A1 (en) * 2001-09-05 2003-03-06 Microsoft Corporation Method and system for measuring network performance from a server
US6587878B1 (en) * 1999-05-12 2003-07-01 International Business Machines Corporation System, method, and program for measuring performance in a network system
US6601098B1 (en) * 1999-06-07 2003-07-29 International Business Machines Corporation Technique for measuring round-trip latency to computing devices requiring no client-side proxy presence
US6681232B1 (en) * 2000-06-07 2004-01-20 Yipes Enterprise Services, Inc. Operations and provisioning systems for service level management in an extended-area data communications network
US6731600B1 (en) * 1999-02-08 2004-05-04 Realnetworks, Inc. System and method for determining network conditions
US6928280B1 (en) * 2000-03-20 2005-08-09 Telephia, Inc. Method and system for measuring data quality of service in a wireless network using multiple remote units and a back end processor
US7088706B2 (en) * 1999-06-30 2006-08-08 Cisco Technology, Inc. Method and apparatus for measuring latency of a computer network
US20060190598A1 (en) * 2002-03-15 2006-08-24 Microsoft Corporation Time-window-constrained multicast using connection scheduling
US7333517B2 (en) * 2000-03-29 2008-02-19 Microsoft Corporation Method and system for accurately calculating latency variation on an end-to-end path in a network
US7353272B2 (en) * 1999-06-23 2008-04-01 Savvis Communications Corporation Method and system for internet performance monitoring and analysis including user interface and periodic information measurement and collection
US7359985B2 (en) * 2000-02-07 2008-04-15 Akamai Technologies, Inc. Method and system for high-performance delivery of web content using high-performance communications protocols to optimize a measure of communications performance between a source and a destination
US7457877B1 (en) * 1998-05-26 2008-11-25 Cisco Technology, Inc. System and method for measuring round trip times in a network using a TCP packet
US7519702B1 (en) * 2000-08-10 2009-04-14 International Business Machines Corporation Method and apparatus for measuring web site performance
US7536453B2 (en) * 2000-10-25 2009-05-19 At&T Intellectual Property I, Lp Network traffic analyzer

Patent Citations (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5696948A (en) * 1994-07-13 1997-12-09 Bell Communications Research, Inc. Apparatus for determining round trip latency delay in system for preprocessing and delivering multimedia presentations
US5570346A (en) * 1994-12-08 1996-10-29 Lucent Technologies Inc. Packet network transit delay measurement system
US6061722A (en) * 1996-12-23 2000-05-09 T E Network, Inc. Assessing network performance without interference with normal network operations
US6222825B1 (en) * 1997-01-23 2001-04-24 Advanced Micro Devices, Inc. Arrangement for determining link latency for maintaining flow control in full-duplex networks
US6038599A (en) * 1997-04-23 2000-03-14 Mpath Interactive, Inc. Latency server and matchmaker
US6178160B1 (en) * 1997-12-23 2001-01-23 Cisco Technology, Inc. Load balancing of client connections across a network using server based algorithms
US6243761B1 (en) * 1998-03-26 2001-06-05 Digital Equipment Corporation Method for dynamically adjusting multimedia content of a web page by a server in accordance to network path characteristics between client and server
US6272652B1 (en) * 1998-04-17 2001-08-07 Ameritech Corporation Method and system for adaptive interleaving
US6012096A (en) * 1998-04-23 2000-01-04 Microsoft Corporation Method and system for peer-to-peer network latency measurement
US7457877B1 (en) * 1998-05-26 2008-11-25 Cisco Technology, Inc. System and method for measuring round trip times in a network using a TCP packet
US6321264B1 (en) * 1998-08-28 2001-11-20 3Com Corporation Network-performance statistics using end-node computer systems
US6182125B1 (en) * 1998-10-13 2001-01-30 3Com Corporation Methods for determining sendable information content based on a determined network latency
US6272539B1 (en) * 1998-11-18 2001-08-07 International Business Machines Corporation Methods, systems and computer program products for determining and visually representing a user's overall network delay in collaborative applications
US20020105911A1 (en) * 1998-11-24 2002-08-08 Parag Pruthi Apparatus and method for collecting and analyzing communications data
US6438629B1 (en) * 1999-02-02 2002-08-20 Maxtor Corporation Storage device buffer access control in accordance with a monitored latency parameter
US6731600B1 (en) * 1999-02-08 2004-05-04 Realnetworks, Inc. System and method for determining network conditions
US6587878B1 (en) * 1999-05-12 2003-07-01 International Business Machines Corporation System, method, and program for measuring performance in a network system
US6601098B1 (en) * 1999-06-07 2003-07-29 International Business Machines Corporation Technique for measuring round-trip latency to computing devices requiring no client-side proxy presence
US7353272B2 (en) * 1999-06-23 2008-04-01 Savvis Communications Corporation Method and system for internet performance monitoring and analysis including user interface and periodic information measurement and collection
US7088706B2 (en) * 1999-06-30 2006-08-08 Cisco Technology, Inc. Method and apparatus for measuring latency of a computer network
US6405252B1 (en) * 1999-11-22 2002-06-11 Speedera Networks, Inc. Integrated point of presence server network
US20010050903A1 (en) * 2000-01-28 2001-12-13 Paul Vanlint Method and system to calculate network latency, and to display the same field of the invention
US7359985B2 (en) * 2000-02-07 2008-04-15 Akamai Technologies, Inc. Method and system for high-performance delivery of web content using high-performance communications protocols to optimize a measure of communications performance between a source and a destination
US6928280B1 (en) * 2000-03-20 2005-08-09 Telephia, Inc. Method and system for measuring data quality of service in a wireless network using multiple remote units and a back end processor
US7333517B2 (en) * 2000-03-29 2008-02-19 Microsoft Corporation Method and system for accurately calculating latency variation on an end-to-end path in a network
US20020026531A1 (en) * 2000-04-12 2002-02-28 John Keane Methods and systems for enabling communication between a processor and a network operations center
US6681232B1 (en) * 2000-06-07 2004-01-20 Yipes Enterprise Services, Inc. Operations and provisioning systems for service level management in an extended-area data communications network
US7519702B1 (en) * 2000-08-10 2009-04-14 International Business Machines Corporation Method and apparatus for measuring web site performance
US7536453B2 (en) * 2000-10-25 2009-05-19 At&T Intellectual Property I, Lp Network traffic analyzer
US20020112049A1 (en) * 2000-12-14 2002-08-15 International Business Machines Corporation Measuring response time for a computer accessing information from a network
US20030046383A1 (en) * 2001-09-05 2003-03-06 Microsoft Corporation Method and system for measuring network performance from a server
US20060190598A1 (en) * 2002-03-15 2006-08-24 Microsoft Corporation Time-window-constrained multicast using connection scheduling

Cited By (224)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050201485A1 (en) * 2002-05-22 2005-09-15 Koninkljke Phillips Electronics N.V. Transmission method using a virtual reception buffer to absorb fluctuation of the channel transmission rate
US7418492B1 (en) * 2002-06-20 2008-08-26 P-Cube Ltd. System and a method for testing network communication devices
US20100017462A1 (en) * 2003-03-19 2010-01-21 Cgi Communications, Inc. System and method for seamlessly providing video content to client systems over a network
US20130198407A1 (en) * 2003-03-19 2013-08-01 E-Locallink, Inc. Methods for seamlessly providing content to a client system and devices thereof
US9462038B2 (en) * 2003-03-19 2016-10-04 eLocalLink, Inc. Methods for seamlessly providing content to a client system and devices thereof
US8417797B2 (en) * 2003-03-19 2013-04-09 E-Locallink, Inc. System and method for seamlessly providing video content to client systems over a network
US20050102413A1 (en) * 2003-11-10 2005-05-12 Binns Pamela A. Real-time estimation of event-driven traffic latency distributions when layered on static schedules
US7590063B2 (en) * 2003-11-10 2009-09-15 Honeywell International Inc. Real-time estimation of event-driven traffic latency distributions when layered on static schedules
US8788527B1 (en) 2003-12-31 2014-07-22 Precise Software Solutions, Inc. Object-level database performance management
US20050160141A1 (en) * 2004-01-21 2005-07-21 Mark Galley Internet network banner
EP1807974A4 (en) * 2004-08-28 2010-03-17 Streamaware Llc Link analysis method and system
EP1807974A2 (en) * 2004-08-28 2007-07-18 Streamaware, LLC Link analysis method and system
US8438276B1 (en) * 2004-08-31 2013-05-07 Precise Software Solutions, Inc. Method of monitoring network and application performance by analyzing web clients and web servers
US20070297387A1 (en) * 2004-09-02 2007-12-27 Sujit Dey Content And Channel Aware Object Scheduling And Error Control
US8060807B2 (en) 2004-09-02 2011-11-15 The Regents Of The University Of California Content and channel aware object scheduling and error control
US8010655B2 (en) 2004-11-17 2011-08-30 The Regents Of The University Of California Network monitoring system and method
JP2008521100A (en) * 2004-11-17 2008-06-19 ザ リージェンツ オブ ザ ユニバーシティ オブ カリフォルニア System and method for generating web pages
US20080104231A1 (en) * 2004-11-17 2008-05-01 Sujit Dey Network Monitoring System And Method
US20070283036A1 (en) * 2004-11-17 2007-12-06 Sujit Dey System And Method For Providing A Web Page
WO2006055769A3 (en) * 2004-11-17 2007-02-01 Univ California System and method for providing a web page
US8135829B2 (en) 2005-04-01 2012-03-13 International Business Machines Corporation Utilizing a single agent on a non-origin node for measuring the roundtrip response time of web pages with embedded HTML frames
US7519007B2 (en) * 2005-04-01 2009-04-14 International Business Machines Corporation Method utilizing a single agent on a non-origin node for measuring the roundtrip response time of web pages with embedded HTML frames over a public or private network
US7580365B2 (en) * 2005-04-01 2009-08-25 International Business Machines Corporation System and method utilizing a single agent on a non-origin node for measuring the roundtrip response time over a public or private network with HTTP/HTTPS network protocol
US20060235961A1 (en) * 2005-04-01 2006-10-19 International Business Machines Corporation System and method utilizing a single agent on a non-origin node for measuring the roundtrip response time of web pages with embedded HTML frames over a public or private network
US20060221851A1 (en) * 2005-04-01 2006-10-05 International Business Machines Corporation System and method for measuring the roundtrip response time of network protocols utilizing a single agent on a non-origin node
US20060221852A1 (en) * 2005-04-01 2006-10-05 International Business Machines Corporation System and method utilizing a single agent on a non-origin node for measuring the roundtrip response time over a public or private network with HTTP/HTTPS network protocol
US7506052B2 (en) * 2005-04-11 2009-03-17 Microsoft Corporation Network experience rating system and method
US20060230134A1 (en) * 2005-04-11 2006-10-12 Tin Qian Network experience rating system and method
US7782767B1 (en) * 2005-07-20 2010-08-24 Tektronix, Inc. Method and system for calculating burst bit rate for IP interactive applications
US8122346B2 (en) * 2005-08-05 2012-02-21 Sap Ag Methods and systems for merging software-level objects with document-level objects in a document publishing environment
US20070033213A1 (en) * 2005-08-05 2007-02-08 Sap Aktiengesellschaft Methods and systems for merging software-level objects with document-level objects in a document publishing environment
US7992208B2 (en) * 2005-09-19 2011-08-02 University Of Maryland Detection of nonconforming network traffic flow aggregates for mitigating distributed denial of service attacks
US20070064610A1 (en) * 2005-09-19 2007-03-22 Khandani Mehdi K Detection of nonconforming network traffic flow aggregates for mitigating distributed denial of service attacks
US8301753B1 (en) * 2006-06-27 2012-10-30 Nosadia Pass Nv, Limited Liability Company Endpoint activity logging
US9137043B2 (en) 2006-06-27 2015-09-15 International Business Machines Corporation System, method and program for determining a network path by which to send a message
US20070299954A1 (en) * 2006-06-27 2007-12-27 International Business Machines Corporation System, method and program for determining a network path by which to send a message
US7634562B2 (en) * 2006-10-27 2009-12-15 Cyscape, Inc. Method and apparatus for determining application responsiveness over a network
US20100088411A1 (en) * 2006-10-27 2010-04-08 Cyscape, Inc. Method and apparatus for determining application responsiveness over a network
US20080104224A1 (en) * 2006-10-27 2008-05-01 Litofsky Richard T Method and apparatus for determining application responsiveness over a network
US8521891B1 (en) * 2007-06-21 2013-08-27 Mcafee, Inc. Network browser system, method, and computer program product for conditionally loading a portion of data from a network based on a data transfer rate
US10027582B2 (en) 2007-06-29 2018-07-17 Amazon Technologies, Inc. Updating routing information based on client location
US9992303B2 (en) 2007-06-29 2018-06-05 Amazon Technologies, Inc. Request routing utilizing client location information
US8292746B2 (en) * 2007-09-28 2012-10-23 Konami Digital Entertainment Co., Ltd. Game system and server
US20100261535A1 (en) * 2007-09-28 2010-10-14 Konami Digital Entertainment Co., Ltd. Game system and server
US9544394B2 (en) 2008-03-31 2017-01-10 Amazon Technologies, Inc. Network resource identification
US10554748B2 (en) 2008-03-31 2020-02-04 Amazon Technologies, Inc. Content management
US11245770B2 (en) 2008-03-31 2022-02-08 Amazon Technologies, Inc. Locality based content distribution
US9887915B2 (en) 2008-03-31 2018-02-06 Amazon Technologies, Inc. Request routing based on class
US9888089B2 (en) 2008-03-31 2018-02-06 Amazon Technologies, Inc. Client side cache management
US11194719B2 (en) 2008-03-31 2021-12-07 Amazon Technologies, Inc. Cache optimization
US9894168B2 (en) 2008-03-31 2018-02-13 Amazon Technologies, Inc. Locality based content distribution
US9954934B2 (en) 2008-03-31 2018-04-24 Amazon Technologies, Inc. Content delivery reconciliation
US10797995B2 (en) 2008-03-31 2020-10-06 Amazon Technologies, Inc. Request routing based on class
US11909639B2 (en) 2008-03-31 2024-02-20 Amazon Technologies, Inc. Request routing based on class
US10158729B2 (en) 2008-03-31 2018-12-18 Amazon Technologies, Inc. Locality based content distribution
US10157135B2 (en) 2008-03-31 2018-12-18 Amazon Technologies, Inc. Cache optimization
US11451472B2 (en) 2008-03-31 2022-09-20 Amazon Technologies, Inc. Request routing based on class
US10305797B2 (en) 2008-03-31 2019-05-28 Amazon Technologies, Inc. Request routing based on class
US10771552B2 (en) 2008-03-31 2020-09-08 Amazon Technologies, Inc. Content management
US9208097B2 (en) 2008-03-31 2015-12-08 Amazon Technologies, Inc. Cache optimization
US9210235B2 (en) 2008-03-31 2015-12-08 Amazon Technologies, Inc. Client side cache management
US9621660B2 (en) 2008-03-31 2017-04-11 Amazon Technologies, Inc. Locality based content distribution
US9571389B2 (en) 2008-03-31 2017-02-14 Amazon Technologies, Inc. Request routing based on class
US10511567B2 (en) 2008-03-31 2019-12-17 Amazon Technologies, Inc. Network resource identification
US9479476B2 (en) 2008-03-31 2016-10-25 Amazon Technologies, Inc. Processing of DNS queries
US10645149B2 (en) 2008-03-31 2020-05-05 Amazon Technologies, Inc. Content delivery reconciliation
US9407699B2 (en) 2008-03-31 2016-08-02 Amazon Technologies, Inc. Content management
US9332078B2 (en) 2008-03-31 2016-05-03 Amazon Technologies, Inc. Locality based content distribution
US10530874B2 (en) 2008-03-31 2020-01-07 Amazon Technologies, Inc. Locality based content distribution
US20160308959A1 (en) * 2008-06-30 2016-10-20 Amazon Technologies, Inc. Latency measurement in resource requests
US9912740B2 (en) * 2008-06-30 2018-03-06 Amazon Technologies, Inc. Latency measurement in resource requests
US9608957B2 (en) 2008-06-30 2017-03-28 Amazon Technologies, Inc. Request routing using network computing components
US11115500B2 (en) 2008-11-17 2021-09-07 Amazon Technologies, Inc. Request routing utilizing client location information
US9515949B2 (en) 2008-11-17 2016-12-06 Amazon Technologies, Inc. Managing content delivery network service providers
US11811657B2 (en) 2008-11-17 2023-11-07 Amazon Technologies, Inc. Updating routing information based on client location
US9444759B2 (en) 2008-11-17 2016-09-13 Amazon Technologies, Inc. Service provider registration by a content broker
US9251112B2 (en) 2008-11-17 2016-02-02 Amazon Technologies, Inc. Managing content delivery network service providers
US10523783B2 (en) 2008-11-17 2019-12-31 Amazon Technologies, Inc. Request routing utilizing client location information
US9734472B2 (en) 2008-11-17 2017-08-15 Amazon Technologies, Inc. Request routing utilizing cost information
US9451046B2 (en) 2008-11-17 2016-09-20 Amazon Technologies, Inc. Managing CDN registration by a storage provider
US9985927B2 (en) 2008-11-17 2018-05-29 Amazon Technologies, Inc. Managing content delivery network service providers by a content broker
US10742550B2 (en) 2008-11-17 2020-08-11 Amazon Technologies, Inc. Updating routing information based on client location
US10116584B2 (en) 2008-11-17 2018-10-30 Amazon Technologies, Inc. Managing content delivery network service providers
US9787599B2 (en) 2008-11-17 2017-10-10 Amazon Technologies, Inc. Managing content delivery network service providers
US9590946B2 (en) 2008-11-17 2017-03-07 Amazon Technologies, Inc. Managing content delivery network service providers
US11283715B2 (en) 2008-11-17 2022-03-22 Amazon Technologies, Inc. Updating routing information based on client location
US10491534B2 (en) 2009-03-27 2019-11-26 Amazon Technologies, Inc. Managing resources and entries in tracking information in resource cache components
US10230819B2 (en) 2009-03-27 2019-03-12 Amazon Technologies, Inc. Translation of resource identifiers using popularity information upon client request
US10574787B2 (en) 2009-03-27 2020-02-25 Amazon Technologies, Inc. Translation of resource identifiers using popularity information upon client request
US10601767B2 (en) 2009-03-27 2020-03-24 Amazon Technologies, Inc. DNS query processing based on application information
US9191458B2 (en) 2009-03-27 2015-11-17 Amazon Technologies, Inc. Request routing using a popularity identifier at a DNS nameserver
US9237114B2 (en) 2009-03-27 2016-01-12 Amazon Technologies, Inc. Managing resources in resource cache components
US10264062B2 (en) 2009-03-27 2019-04-16 Amazon Technologies, Inc. Request routing using a popularity identifier to identify a cache component
US9176894B2 (en) 2009-06-16 2015-11-03 Amazon Technologies, Inc. Managing resources using resource expiration data
US10783077B2 (en) 2009-06-16 2020-09-22 Amazon Technologies, Inc. Managing resources using resource expiration data
US10521348B2 (en) 2009-06-16 2019-12-31 Amazon Technologies, Inc. Managing resources using resource expiration data
US10785037B2 (en) 2009-09-04 2020-09-22 Amazon Technologies, Inc. Managing secure content in a content delivery network
US9130756B2 (en) 2009-09-04 2015-09-08 Amazon Technologies, Inc. Managing secure content in a content delivery network
US10135620B2 (en) 2009-09-04 2018-11-20 Amazon Technologis, Inc. Managing secure content in a content delivery network
US9712325B2 (en) 2009-09-04 2017-07-18 Amazon Technologies, Inc. Managing secure content in a content delivery network
US10218584B2 (en) 2009-10-02 2019-02-26 Amazon Technologies, Inc. Forward-based resource delivery network management techniques
US9246776B2 (en) 2009-10-02 2016-01-26 Amazon Technologies, Inc. Forward-based resource delivery network management techniques
US9893957B2 (en) 2009-10-02 2018-02-13 Amazon Technologies, Inc. Forward-based resource delivery network management techniques
US9495338B1 (en) 2010-01-28 2016-11-15 Amazon Technologies, Inc. Content distribution network
US10506029B2 (en) 2010-01-28 2019-12-10 Amazon Technologies, Inc. Content distribution network
US11205037B2 (en) 2010-01-28 2021-12-21 Amazon Technologies, Inc. Content distribution network
US8812720B2 (en) * 2010-08-06 2014-08-19 Acquire Media Ventures Inc. Method and system for pacing, acking, timing, and handicapping (PATH) for simultaneous receipt of documents
US20120243534A1 (en) * 2010-08-06 2012-09-27 Rafsky Lawrence C Method and system for pacing, acking, timing, and handicapping (path) for simultaneous receipt of documents
US9185012B2 (en) 2010-09-28 2015-11-10 Amazon Technologies, Inc. Latency measurement in resource requests
US10097398B1 (en) 2010-09-28 2018-10-09 Amazon Technologies, Inc. Point of presence management in request routing
US9497259B1 (en) 2010-09-28 2016-11-15 Amazon Technologies, Inc. Point of presence management in request routing
US9407681B1 (en) * 2010-09-28 2016-08-02 Amazon Technologies, Inc. Latency measurement in resource requests
US10225322B2 (en) 2010-09-28 2019-03-05 Amazon Technologies, Inc. Point of presence management in request routing
US9253065B2 (en) 2010-09-28 2016-02-02 Amazon Technologies, Inc. Latency measurement in resource requests
US9191338B2 (en) 2010-09-28 2015-11-17 Amazon Technologies, Inc. Request routing in a networked environment
US11632420B2 (en) 2010-09-28 2023-04-18 Amazon Technologies, Inc. Point of presence management in request routing
US10778554B2 (en) 2010-09-28 2020-09-15 Amazon Technologies, Inc. Latency measurement in resource requests
US9787775B1 (en) 2010-09-28 2017-10-10 Amazon Technologies, Inc. Point of presence management in request routing
US11336712B2 (en) 2010-09-28 2022-05-17 Amazon Technologies, Inc. Point of presence management in request routing
US10015237B2 (en) 2010-09-28 2018-07-03 Amazon Technologies, Inc. Point of presence management in request routing
US9794216B2 (en) 2010-09-28 2017-10-17 Amazon Technologies, Inc. Request routing in a networked environment
US9800539B2 (en) 2010-09-28 2017-10-24 Amazon Technologies, Inc. Request routing management based on network components
US9160703B2 (en) 2010-09-28 2015-10-13 Amazon Technologies, Inc. Request routing management based on network components
US10931738B2 (en) 2010-09-28 2021-02-23 Amazon Technologies, Inc. Point of presence management in request routing
US9106701B2 (en) 2010-09-28 2015-08-11 Amazon Technologies, Inc. Request routing management based on network components
US9712484B1 (en) 2010-09-28 2017-07-18 Amazon Technologies, Inc. Managing request routing information utilizing client identifiers
US10079742B1 (en) 2010-09-28 2018-09-18 Amazon Technologies, Inc. Latency measurement in resource requests
US11108729B2 (en) 2010-09-28 2021-08-31 Amazon Technologies, Inc. Managing request routing information utilizing client identifiers
US10958501B1 (en) 2010-09-28 2021-03-23 Amazon Technologies, Inc. Request routing information based on client IP groupings
US9536249B2 (en) * 2010-09-29 2017-01-03 Excalibur Ip, Llc Measuring inline ad performance for third-party ad serving
US20120078707A1 (en) * 2010-09-29 2012-03-29 Eyal Arasu Ramakrishnan Measuring Inline Ad Performance for Third-Party Ad Serving
US10951725B2 (en) 2010-11-22 2021-03-16 Amazon Technologies, Inc. Request routing processing
US9930131B2 (en) 2010-11-22 2018-03-27 Amazon Technologies, Inc. Request routing processing
US9391949B1 (en) 2010-12-03 2016-07-12 Amazon Technologies, Inc. Request routing processing
US11604667B2 (en) 2011-04-27 2023-03-14 Amazon Technologies, Inc. Optimized deployment based upon customer locality
US20130091268A1 (en) * 2011-10-10 2013-04-11 Rohan Bopardikar Classification of web client network bandwidth by a web server
US9848028B2 (en) * 2011-10-10 2017-12-19 Rohan Bopardikar Classification of web client network bandwidth by a web server
US9628554B2 (en) 2012-02-10 2017-04-18 Amazon Technologies, Inc. Dynamic content delivery
US10021179B1 (en) 2012-02-21 2018-07-10 Amazon Technologies, Inc. Local resource delivery network
US9179169B2 (en) * 2012-03-14 2015-11-03 Imagine Communications Corp. Adaptive media delivery
US20130242185A1 (en) * 2012-03-14 2013-09-19 Todd Stuart Roth Adaptive media delivery
US10791348B2 (en) 2012-03-14 2020-09-29 Imagine Communications Corp. Adaptive media delivery
US9172674B1 (en) 2012-03-21 2015-10-27 Amazon Technologies, Inc. Managing request routing information utilizing performance information
US9083743B1 (en) 2012-03-21 2015-07-14 Amazon Technologies, Inc. Managing request routing information utilizing performance information
US10623408B1 (en) 2012-04-02 2020-04-14 Amazon Technologies, Inc. Context sensitive object management
US10225362B2 (en) 2012-06-11 2019-03-05 Amazon Technologies, Inc. Processing DNS queries to identify pre-processing information
US11729294B2 (en) 2012-06-11 2023-08-15 Amazon Technologies, Inc. Processing DNS queries to identify pre-processing information
US9154551B1 (en) 2012-06-11 2015-10-06 Amazon Technologies, Inc. Processing DNS queries to identify pre-processing information
US11303717B2 (en) 2012-06-11 2022-04-12 Amazon Technologies, Inc. Processing DNS queries to identify pre-processing information
US9525659B1 (en) 2012-09-04 2016-12-20 Amazon Technologies, Inc. Request routing utilizing point of presence load information
US10015241B2 (en) 2012-09-20 2018-07-03 Amazon Technologies, Inc. Automated profiling of resource usage
US9323577B2 (en) 2012-09-20 2016-04-26 Amazon Technologies, Inc. Automated profiling of resource usage
US10542079B2 (en) 2012-09-20 2020-01-21 Amazon Technologies, Inc. Automated profiling of resource usage
US9135048B2 (en) 2012-09-20 2015-09-15 Amazon Technologies, Inc. Automated profiling of resource usage
KR101405472B1 (en) * 2012-11-09 2014-06-27 (주)씨디네트웍스 Method and apparatus for latency measurement based upon passive measurement
US10205698B1 (en) 2012-12-19 2019-02-12 Amazon Technologies, Inc. Source-dependent address resolution
US10645056B2 (en) 2012-12-19 2020-05-05 Amazon Technologies, Inc. Source-dependent address resolution
US9929959B2 (en) 2013-06-04 2018-03-27 Amazon Technologies, Inc. Managing network computing components utilizing request routing
US9294391B1 (en) 2013-06-04 2016-03-22 Amazon Technologies, Inc. Managing network computing components utilizing request routing
US10374955B2 (en) 2013-06-04 2019-08-06 Amazon Technologies, Inc. Managing network computing components utilizing request routing
US9734134B1 (en) * 2013-09-19 2017-08-15 Amazon Technologies, Inc. Conditional promotion through frame reordering
US9922006B1 (en) 2013-09-19 2018-03-20 Amazon Technologies, Inc. Conditional promotion through metadata-based priority hinting
US9785969B1 (en) 2013-09-19 2017-10-10 Amazon Technologies, Inc. Conditional promotion in multi-stream content delivery
US10097448B1 (en) 2014-12-18 2018-10-09 Amazon Technologies, Inc. Routing mode and point-of-presence selection service
US10091096B1 (en) 2014-12-18 2018-10-02 Amazon Technologies, Inc. Routing mode and point-of-presence selection service
US11863417B2 (en) 2014-12-18 2024-01-02 Amazon Technologies, Inc. Routing mode and point-of-presence selection service
US10728133B2 (en) 2014-12-18 2020-07-28 Amazon Technologies, Inc. Routing mode and point-of-presence selection service
US11381487B2 (en) 2014-12-18 2022-07-05 Amazon Technologies, Inc. Routing mode and point-of-presence selection service
US10033627B1 (en) 2014-12-18 2018-07-24 Amazon Technologies, Inc. Routing mode and point-of-presence selection service
US9727449B2 (en) * 2014-12-30 2017-08-08 Spirent Communications, Inc. Conducting performance snapshots during test and using feedback to control test based on customer experience parameters
US20160188445A1 (en) * 2014-12-30 2016-06-30 Spirent Communications, Inc. Conducting performance snapshots during test and using feedback to control test based on customer experience parameters
US10225326B1 (en) 2015-03-23 2019-03-05 Amazon Technologies, Inc. Point of presence based data uploading
US11297140B2 (en) 2015-03-23 2022-04-05 Amazon Technologies, Inc. Point of presence based data uploading
US9887931B1 (en) 2015-03-30 2018-02-06 Amazon Technologies, Inc. Traffic surge management for points of presence
US10469355B2 (en) 2015-03-30 2019-11-05 Amazon Technologies, Inc. Traffic surge management for points of presence
US9819567B1 (en) 2015-03-30 2017-11-14 Amazon Technologies, Inc. Traffic surge management for points of presence
US9887932B1 (en) 2015-03-30 2018-02-06 Amazon Technologies, Inc. Traffic surge management for points of presence
US10180993B2 (en) 2015-05-13 2019-01-15 Amazon Technologies, Inc. Routing based request correlation
US10691752B2 (en) 2015-05-13 2020-06-23 Amazon Technologies, Inc. Routing based request correlation
US11461402B2 (en) 2015-05-13 2022-10-04 Amazon Technologies, Inc. Routing based request correlation
US9832141B1 (en) 2015-05-13 2017-11-28 Amazon Technologies, Inc. Routing based request correlation
US10616179B1 (en) 2015-06-25 2020-04-07 Amazon Technologies, Inc. Selective routing of domain name system (DNS) requests
US10097566B1 (en) 2015-07-31 2018-10-09 Amazon Technologies, Inc. Identifying targets of network attacks
US10884910B2 (en) 2015-08-13 2021-01-05 Spirent Communications, Inc. Method to configure monitoring thresholds using output of load or resource loadings
US10198348B2 (en) 2015-08-13 2019-02-05 Spirent Communications, Inc. Method to configure monitoring thresholds using output of load or resource loadings
US9742795B1 (en) 2015-09-24 2017-08-22 Amazon Technologies, Inc. Mitigating network attacks
US10200402B2 (en) 2015-09-24 2019-02-05 Amazon Technologies, Inc. Mitigating network attacks
US9794281B1 (en) 2015-09-24 2017-10-17 Amazon Technologies, Inc. Identifying sources of network attacks
US20190238444A1 (en) * 2015-09-24 2019-08-01 Assia Spe, Llc Methods and apparatus for detecting internet connection problems
US9774619B1 (en) 2015-09-24 2017-09-26 Amazon Technologies, Inc. Mitigating network attacks
US10567266B2 (en) * 2015-09-24 2020-02-18 Assia Spe, Llc Methods and apparatus for detecting internet connection problems
US10270878B1 (en) 2015-11-10 2019-04-23 Amazon Technologies, Inc. Routing for origin-facing points of presence
US11134134B2 (en) 2015-11-10 2021-09-28 Amazon Technologies, Inc. Routing for origin-facing points of presence
US10257307B1 (en) 2015-12-11 2019-04-09 Amazon Technologies, Inc. Reserved cache space in content delivery networks
US10049051B1 (en) 2015-12-11 2018-08-14 Amazon Technologies, Inc. Reserved cache space in content delivery networks
US10348639B2 (en) 2015-12-18 2019-07-09 Amazon Technologies, Inc. Use of virtual endpoints to improve data transmission rates
CN108701061A (en) * 2016-02-25 2018-10-23 瑞典爱立信有限公司 The resources control of interconnection hardware infrastructure
US10075551B1 (en) 2016-06-06 2018-09-11 Amazon Technologies, Inc. Request management for hierarchical cache
US10666756B2 (en) 2016-06-06 2020-05-26 Amazon Technologies, Inc. Request management for hierarchical cache
US11463550B2 (en) 2016-06-06 2022-10-04 Amazon Technologies, Inc. Request management for hierarchical cache
US10110694B1 (en) 2016-06-29 2018-10-23 Amazon Technologies, Inc. Adaptive transfer rate for retrieving content from a server
US11457088B2 (en) 2016-06-29 2022-09-27 Amazon Technologies, Inc. Adaptive transfer rate for retrieving content from a server
US10516590B2 (en) 2016-08-23 2019-12-24 Amazon Technologies, Inc. External health checking of virtual private cloud network environments
US9992086B1 (en) 2016-08-23 2018-06-05 Amazon Technologies, Inc. External health checking of virtual private cloud network environments
US10469442B2 (en) 2016-08-24 2019-11-05 Amazon Technologies, Inc. Adaptive resolution of domain name requests in virtual private cloud network environments
US10033691B1 (en) 2016-08-24 2018-07-24 Amazon Technologies, Inc. Adaptive resolution of domain name requests in virtual private cloud network environments
US11330008B2 (en) 2016-10-05 2022-05-10 Amazon Technologies, Inc. Network addresses with encoded DNS-level information
US10469513B2 (en) 2016-10-05 2019-11-05 Amazon Technologies, Inc. Encrypted network addresses
US10505961B2 (en) 2016-10-05 2019-12-10 Amazon Technologies, Inc. Digitally signed network address
US10616250B2 (en) 2016-10-05 2020-04-07 Amazon Technologies, Inc. Network addresses with encoded DNS-level information
US10372499B1 (en) 2016-12-27 2019-08-06 Amazon Technologies, Inc. Efficient region selection system for executing request-driven code
US11762703B2 (en) 2016-12-27 2023-09-19 Amazon Technologies, Inc. Multi-region request-driven code execution system
US10831549B1 (en) 2016-12-27 2020-11-10 Amazon Technologies, Inc. Multi-region request-driven code execution system
US10938884B1 (en) 2017-01-30 2021-03-02 Amazon Technologies, Inc. Origin server cloaking using virtual private cloud network environments
US10503613B1 (en) 2017-04-21 2019-12-10 Amazon Technologies, Inc. Efficient serving of resources during server unavailability
US11075987B1 (en) 2017-06-12 2021-07-27 Amazon Technologies, Inc. Load estimating content delivery network
US10447648B2 (en) 2017-06-19 2019-10-15 Amazon Technologies, Inc. Assignment of a POP to a DNS resolver based on volume of communications over a link between client devices and the POP
US11290418B2 (en) 2017-09-25 2022-03-29 Amazon Technologies, Inc. Hybrid content request routing system
WO2019145720A1 (en) * 2018-01-25 2019-08-01 Spatialbuzz Limited Examining latency in communications networks
US11606413B2 (en) * 2018-02-06 2023-03-14 Nippon Telegraph And Telephone Corporation Estimation apparatus, estimation method and program
US10592578B1 (en) 2018-03-07 2020-03-17 Amazon Technologies, Inc. Predictive content push-enabled content delivery network
US11362986B2 (en) 2018-11-16 2022-06-14 Amazon Technologies, Inc. Resolution of domain name requests in heterogeneous network environments
US10862852B1 (en) 2018-11-16 2020-12-08 Amazon Technologies, Inc. Resolution of domain name requests in heterogeneous network environments
US11025747B1 (en) 2018-12-12 2021-06-01 Amazon Technologies, Inc. Content request pattern-based routing system

Also Published As

Publication number Publication date
US7747729B2 (en) 2010-06-29
US20070073873A1 (en) 2007-03-29
US7676570B2 (en) 2010-03-09

Similar Documents

Publication Publication Date Title
US7676570B2 (en) Determining client latencies over a network
US6442141B1 (en) Network delay and loss simulator
Carter et al. Dynamic server selection using bandwidth probing in wide-area networks
US6996064B2 (en) System and method for determining network throughput speed and streaming utilization
US8135829B2 (en) Utilizing a single agent on a non-origin node for measuring the roundtrip response time of web pages with embedded HTML frames
US5913041A (en) System for determining data transfer rates in accordance with log information relates to history of data transfer activities that independently stored in content servers
US7792083B2 (en) Method and apparatus for measuring network data packet delay, jitter and loss
US6601098B1 (en) Technique for measuring round-trip latency to computing devices requiring no client-side proxy presence
US10178204B2 (en) Information processing method and device
JP2005506605A (en) Calculating response time at the server site for any application
US20020129134A1 (en) Global load balancing across mirrored data centers
US20020165956A1 (en) Traffic driven scheduling of active tests
US7827257B2 (en) System and method for automatic and adaptive use of active network performance measurement techniques to find the fastest source
EP0522211B1 (en) Testing a packet-based network
Olshefski et al. Inferring client response time at the web server
Luo et al. Design and Implementation of TCP Data Probes for Reliable and Metric-Rich Network Path Monitoring.
Marshak et al. Evaluating web user perceived latency using server side measurements
US20060221852A1 (en) System and method utilizing a single agent on a non-origin node for measuring the roundtrip response time over a public or private network with HTTP/HTTPS network protocol
Beckers et al. Generalized processor sharing performance models for Internet access lines
JP2007523508A (en) Method and apparatus for network throughput measurement
KR100499673B1 (en) Web-based Simulation Method of End-to-End VoIP Quality in Broadband Internet Service
Olshefski et al. ksniffer: Determining the Remote Client Perceived Response Time from Live Packet Streams.
Luo et al. On measuring one-way path metrics from a web server
Borzemski et al. An Empirical Study of Web Quality: Measuring the Web from Wroclaw University of Technology Campus.
Falaki et al. Traffic measurements on a local area computer network

Legal Events

Date Code Title Description
AS Assignment

Owner name: RAMOT UNIVERSITY AUTHORITY FOR APPLIED RESEARCH &

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:LEVY, HANOCH;MARSHAK, MARIK;REEL/FRAME:013012/0819;SIGNING DATES FROM 20020607 TO 20020611

Owner name: RAMOT UNIVERSITY AUTHORITY FOR APPLIED RESEARCH &

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:LEVY, HANOCH;MARSHAK, MARIK;SIGNING DATES FROM 20020607 TO 20020611;REEL/FRAME:013012/0819

AS Assignment

Owner name: KHORSABAD LOCKDOWN LLC, NEVADA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:RAMOT AT TEL AVIV UNIVERSITY LIMITED;REEL/FRAME:015797/0911

Effective date: 20041201

Owner name: KHORSABAD LOCKDOWN LLC,NEVADA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:RAMOT AT TEL AVIV UNIVERSITY LIMITED;REEL/FRAME:015797/0911

Effective date: 20041201

AS Assignment

Owner name: RAMOT AT TEL-AVIV UNIVERSITY LTD., ISRAEL

Free format text: CHANGE OF NAME;ASSIGNOR:RAMOT UNIVERSITY AUTHORITY FOR APPLIED RESEARCH & INDUSTRIAL DEVELOPMENT LTD.;REEL/FRAME:022751/0610

Effective date: 20020808

Owner name: RAMOT AT TEL-AVIV UNIVERSITY LTD.,ISRAEL

Free format text: CHANGE OF NAME;ASSIGNOR:RAMOT UNIVERSITY AUTHORITY FOR APPLIED RESEARCH & INDUSTRIAL DEVELOPMENT LTD.;REEL/FRAME:022751/0610

Effective date: 20020808

STCF Information on status: patent grant

Free format text: PATENTED CASE

CC Certificate of correction
FPAY Fee payment

Year of fee payment: 4

AS Assignment

Owner name: ZARBANA DIGITAL FUND LLC, DELAWARE

Free format text: MERGER;ASSIGNOR:KHORSABAD LOCKDOWN LLC;REEL/FRAME:037337/0894

Effective date: 20150811

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552)

Year of fee payment: 8

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 12TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1553); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 12