Advanced Topics

Additional Data Passed in ARM Function Calls

• Transaction correlation data

  You can indicate that a transaction is a component of another transaction. You can do transaction correlation within one system or across multiple systems. This permits a better understanding of the overall transaction, how much time each part of the transaction is taking, and where problems are occurring.

• Application-defined metrics

  Application-defined metrics provide additional information about the transaction, such as the number of bytes or records being processed, or about the state of the application at the moment that the transaction is being processed, such as the length of a work queue. This information is useful to better understand response times, and how the application can be tuned to perform better.

Transaction Correlation

There are two facilities that the application developer can use to provide this information to measurement agents that implement the ARM 2.0 API:

• On the same arm_start(), the application can request that the measurement agent assign and return a correlator for this instance of the transaction (that is, a parent correlator). Note that the agent has the option of not providing the correlator, because it may not support the capability (ARM Version 1.0 agents do not support correlators), or because it is operating under a policy to suppress generating them.
  

• When indicating the start of a child transaction with an arm_start(), the application can provide a correlator provided from a parent transaction. This allows the measurement agent to know the parent/child relationship.

  

  Figure: Transaction Response Time Correlation

  The figure shows the concept for a simple model. The principle can be extended to a model of arbitrary complexity.

  • Client A starts transaction T1, requesting a correlator via, and is assigned C1.

  • Client A sends a request (T1) to Server B, and includes C1 in the request.

  • Server B starts transaction T2, passing C1 as the parent. At the same time it requests a correlator and is assigned C2.

  • Server B sends a request (T2) to Server C, and includes C2 in the request.

  • Server C starts transaction T3, passing C2 as the parent.

  • T3 stops, T2 stops, and T1 stops.

  If the correlation application collects all the data about these transactions, it can put together the total picture, knowing that T1 is the parent of T2 (via C1), and T2 is the parent of T3 (via C2). The parent/child relationship could be from a client to a server, or within one program.

  An application using the ARM API need not be concerned with the format of the correlators. Measurement agents generate correlators.

Changes Needed for Transaction Correlation

• Request correlators for transactions with one or more child transactions (via arm_start()) by getting the appropriate flag.

• Send the assigned correlators to the child transaction(s) along with the data needed to invoke the child transaction(s) itself. This is done by first checking that the agent assigned a correlator, and then sending the number of bytes in the correlator. The length is stored by the agent in the Correlator Length field.

• Pass correlators received from parent transactions to the measurement agents (via arm_start()) by storing the correlator in the optional buffer and setting the appropriate flag.

To enable a correlation application to analyze the correlators coming from different systems, measurement agents follow conventions when creating correlators. Included within the correlator is information identifying the system, the transaction class (from arm_getid()), the transaction instance (from arm_start()), and some flags. The format is flexible and extendible so more conventions can be added as the need arises. See Measurement Agent Information on Measurement Agent Information, for information on the correlator format.

Correlators are passed in the arm_start() calls by utilizing the data buffer. This same data buffer is used to pass application-defined metrics, as described in Format of Data Buffer in arm_start/arm_update/arm_stop , which describes format of the data buffer in arm_start(), arm_update(), and arm_stop(). Correlators are ignored in arm_update() and arm_stop() calls.

If a correlator is being requested, the data buffer should be 256 bytes, to allow for a variable size correlator. If a correlator is being passed to the measurement agent, and none is requested, the length may be truncated based on the correlator length.

If you only wanted to do transaction correlation in your application and not provide application-defined metrics, you can zero out the metrics (set the Flags Second Byte to zero and fill with zeros 80 bytes for the metrics descriptions).

Note:

Other than the length, the correlator format need not be understood by the application developer, as it is opaque.

Application-Defined Metrics

1. Specify characteristics of the transaction that will affect the response time, or that are useful for workload planning. Examples are the number of bytes in a file transfer or print job, or the number of records being processed. A file transfer of 100 megabytes would certainly be expected to take longer than a transfer of 100 kilobytes.

2. Specify information about the current state of the application. Examples would be the length of a workload queue, the amount of memory allocated, or the number of threads being used. This information is useful for adjusting workloads by shifting work between systems, or tuning the application. If a comparison of response times versus threads shows that congestion builds and response times increase dramatically if, for example, eight threads are used instead of twelve, the application can be recompiled or instructed to use more threads, which may result in a dramatic improvement in performance.

3. Specify information that can be used in diagnosing problems. Examples are error codes returned from services invoked by the application, or information about the transaction itself such as the part number being processed.

In setting up application-defined metrics, arm_getid() is used to define the context (or meta-data) for a buffer of values that can be passed at arm_start(), arm_update() or arm_stop(). Actual values are passed in arm_start(), arm_update() and arm_stop(). The length of the buffer is specified in the data_size parameter.

Choosing a Data Type

Four general data types can be specified for each field:

• Counter

• Gauge

• Numeric id

• String

This section provides some suggestions about which data type to use.

Counter

If a counter is used, its initial value must be set in the arm_start() call. The difference between the value in the arm_start() and the arm_stop() (or the value in the last arm_update() call if no metric value is passed in arm_stop()), equals the amount attributed to this transaction. Similarly, the difference between successive arm_update() calls, or from the arm_start() to the first arm_update() call, or from the last arm_update() to the arm_stop() call, equals the value for the time period between the calls.

Here are three examples of how a counter would probably be used:

• The counter is set to zero at arm_start() and to some value at arm_stop() (or the last arm_update call). In this case, the application probably measured the value for this transaction and provided that value in the arm_stop() call. The application always sets the value to zero in the arm_start() call, so the value at arm_stop() reflects both the difference from the arm_start() value and the absolute value.

• The counter is x1 at arm_start(), x2 at its arm_stop, x2 at the next arm_start(), and x3 at its arm_stop(). In this case, the application is probably keeping a rolling counter. Perhaps this is a server application that counts the total workload. The application simply takes a snapshot of the counter at the start of a transaction and another snapshot at the end of the transaction. The agent determines the difference attributed to this transaction.

• The counter is x1 at arm_start(), x2 at arm_stop(), x3 (not equal to x2) at the next arm_start(), and x4 at arm_stop(). In this case, the application is probably keeping a rolling counter as in the previous example. But in this case the measurement represents a value affected by other users or transaction classes, so the value often changes from one arm_stop() to the next arm_start() for the same transaction class.

Gauge

Gauges can be provided on arm_start(), arm_update(), and arm_stop() calls. This creates the potential for different interpretations. If several values are provided for a transaction (one on an arm_start(), one on each arm_update(), and one on an arm_stop()), which one(s) should be used? In order to have consistent interpretation, the following conventions apply. Measurement agents are free to process the data in any way within these guidelines.

• The maximum value for a transaction will be the largest valid value passed at any time during the transaction.

• The minimum value for a transaction will be the smallest valid value passed at any time during the transaction.

• The mean value for a transaction will be the mean of all valid values passed at any time during the transaction. All values will be weighted equally.

• The median value for a transaction will be the median of all valid values passed at any time during the transaction. All values will be weighted equally.

• The last value for a transaction will be the last valid value passed at any time during the transaction.

Numeric ID

String

Format of Data Buffer in arm_getid

Data Type Definitions

ARM_Counter32

An unsigned32 value that increases up to the maximum value that the counter can hold, at which point it resets to zero and continues counting up from zero. Except for the reset back to zero, the value can never decrease. The counter is in the first four bytes, and the second four bytes are unused.

ARM_Counter64

An unsigned64 counter (see ARM_Counter32, except it is 64 bits long).

ARM_CntrDivr32

A combination of two unsigned32 integers, with ARM_Counter32 in the first four bytes, and an unsigned32 divisor in the second four bytes. The total value is ARM_CntrDivr32. The purpose of this format is to be able to represent decimal values without using floating point formats.

ARM_Gauge32

An int32 (signed) value that can increase or decrease. The gauge is in the first four bytes, and the second four bytes are unused.

ARM_Gauge64

An int64 (signed) gauge (see ARM_Gauge32, except it is 64 bits long).

ARM_GaugeDivr32

A combination of two integers, one an int32 (signed) and one an unsigned32. ARM_Gauge32 is in the first four bytes, and an unsigned32 divisor in the second four bytes. The total value is ARM_GaugeDivr32. The purpose of this format is to be able to represent decimal values without using floating point formats.

ARM_NumericID32

An unsigned32 value that should not be used in arithmetic operations because it is used as an identifier, not as a measurement. For example, a message number or error code. The numeric id is in the first four bytes, and the second four bytes are unused.

ARM_NumericID64

An unsigned64 value that should not be used in arithmetic operations because it is used as an identifier, not as a measurement. An example is a message number or error code.

ARM_String8

An 8 byte string that is not NULL terminated. If the string is less than eight bytes long, it must be padded with blanks. The character set is ASCII or EBCDIC, depending on whatever is standard for that platform. Unlike the NULL terminated character strings passed in various places in the API, these strings cannot be reliably converted to other code pages, so it is suggested you use only the common characters in the first 128 characters of the Latin code pages. See Internationalization for more information on internationalization.

ARM_String32

A 32 byte string that is not NULL terminated. If the string is less than 32 bytes long, it must be padded with blanks. The character set is ASCII or EBCDIC, depending on whatever is standard on that platform. Unlike the NULL terminated character strings passed in various places in the API, these strings cannot be reliably converted to other code pages, so it is suggested you use only the common characters in the first 128 characters of the Latin code pages. See the "Internationalization" section on page 56 for more information.

Format of Data Buffer in arm_start/arm_update/arm_stop

Format 1

Format 2

Three Ways to Instrument within a Transaction Instance

1. Instrument a transaction using arm_update() as a heartbeat, when it is an operation that takes a long time to complete (several minutes or hours) and you want to show the overall progress of the transaction in numeric form.

   If these transactions have different steps associated with processing each record, you may want to instrument these steps with component transactions (as described below), or use repeated calls to arm_update() to show the overall progress of the transaction. For example, the transaction may process a million records. A call to arm_update() could be made for every 1000 records or every minute of processing. This could show the progress of the transaction based on the number of times arm_update() was called or with one or more application-defined metrics.

2. Instrument a transaction using component transactions when it is a long transaction that has many steps. A transaction can be defined for the overall transaction and then nested transactions can be defined for each of the steps. A step might represent a single discrete operation, or it could represent a large number of operations, such as copying 1000 files. This allows for the monitoring of each of the steps as well as the overall transaction.

   For example, step 1 takes about 20 minutes, step 2 takes about 40 minutes, and step 3 takes about 10 minutes. Each step can have a defined transaction as well as the overall transaction. So you would define 3 component transactions monitoring each step, plus one transaction that monitors the overall transaction.

3. Instrument using transaction correlation when the transaction has components that span several applications or systems. This approach is more complex than the previous two as it requires changes to all the applications involved in processing components of the transaction, but it is the most accurate way to track transaction response time spanning systems.

Internationalization

The ARM API supports any code page as long as no characters are encoded with binary zero bytes (octets). This is because most strings are passed as NULL terminated strings, and the NULL terminator character is a binary zero byte. If a binary zero byte is encountered before the end of the string, the agent would interpret the zero byte as the NULL terminator and truncate the string. Most code pages meet this requirement.

These are code pages that contain binary zero bytes, but there are alternate ways to encode the characters. A well-known example is the Unicode standard. In its native format using 16 bit characters (UTC-2), there are binary zero bytes. However, the UTF-8 encoding of the same Unicode characters does not contain binary zero bytes, and this format is entirely compatible with the ARM API.

Agents that support native languages will often use the following technique. When the application links to the agent it links to a part of the agent that executes in the same process space as the application. Typically this small part of the agent communicates with the main part of the agent across an inter-process communications (IPC) channel. The small part of the agent that executes in the same process as the application can issue an operating system call to find out what code page and language the process is using. It can then pass this information to the main part of the agent, and the main part of the agent can convert from the native code page as necessary.

There are the following three restrictions on the use of native languages.

• The strings can contain no binary zero bytes except for the NULL terminator character (as was mentioned above).

• All the strings should be encoded using the same code page and language information as the process that executes the arm_init() call. This also implies that the code page and language information should not change after the arm_init() call.

• This technique does not apply to any string data passed within the optional buffers on arm_start(), arm_update(), and arm_stop(). This is because these strings are not null terminated (note that it does apply to the metric descriptions passed within the optional buffer on arm_getid()). Further, these strings are often about things that are external to the program, such as a part number or an error code, so the requirement to use the same code page and language information as the process is unacceptable. The application developer is strongly recommended to restrict these strings to the first 128 bytes of the standard Latin code pages for ASCII and EBCDIC (depending on the platform). ,iX EBCDIC