Implementation Notes

Event Encoding

A single bit might allow the following event list encoding:

0

The file has no event lists.

1

The file has an event list but there is not enough persistent storage available to encode the events.

By increasing the number of bits, well known combinations of events can be stored persistently with the file. For example, with 2 bits, the following event list encoding may exist:

00

The file has no event lists.

01

Read events will be generated for this file.

10

Write events will be generated for this file.

11

The file has an event list but there is not enough persistent storage available to encode the events.

For example, if the file has both read and write events set, then this cannot be encoded in two bits. In this case the debut event would need to be generated in order to load the event list.

The encoding of events is not defined by the DMAPI. It is expected that different DMAPI implementations would encode events differently, depending on those data management applications which would be supported.

Note:

Due to the requirement regarding implementation of the DMAPI, support for persistent non-opaque attributes is a DMAPI implementation option.

Event Ordering

If the application is multi-threaded, then the DM app writer's job is even more difficult. Synchronization primitives are necessary so that one process is not left hanging waiting for an event that has been serviced by another process.

Lock Releasing

Tokens, Messages and Handles

mmap

To get around these problems, some systems may have to adopt a paradigm that at the time the file is mapped (that is, at the time the actual mmap(2) call is executed), then if the file is non-resident it will have to be made resident. This is sub-optimal from a disk space utilization standpoint, and it also means that a large file that is mmap'd cannot be partially resident. However, the alternative is to place DMAPI call-outs deep in the VM subsystem, which is usually a very complex exercise and is not recommended unless considerable expertise is available.

Invisible I/O

In systems where pages are encached on the vnode, this can lead to troublesome locking and coherency issues. How is the actual write to disk performed? Is the page cache bypassed? How do you ensure that encached pages are dealt with correctly?

Generation of Events

In the case of managed region events, the DMAPI implementor must ensure that a poorly-behaved DM application does not cause the system to behave in an unexpected manner. For example, if a file has multiple managed regions that represent non-resident data, and a DM application only restores the data for one of those regions, the system must be sure that operations such as read-ahead do not cause multiple events to occur. For further information, see the Managed Region description in Managed Regions .

Locking Across Operations

Tokens and Multiple Handles

During recovery, a DM application can always execute dm_respond_event(), give it the offending token, and let the DMAPI release any and all access rights associated with the token. If the DM application needs to be selective about which file handles have their access rights released, then it (the DM application) must provide some mechanism external to the DMAPI to log which file handles are associated with a token. The DMAPI does not provide interfaces to identify multiple file handles from a single token.

Structure Lists

The DMAPI specifies that DM applications should use the DM_STEP_TO_NEXT macro to access variable length structures that are in a list. However, the actual implementation of this macro is not defined. One suggestion is that each variable-length structure should have a field in a well-known position (say offset zero) or use a special field name that is opaque to the DM application. For example, using the field name approach, the definition of the dm_eventmsg_t would be:

struct dm_eventmsg {
    ssize_t         _link;
    dm_type_t       ev_type;
    dm_token_t      ev_token;
    dm_vardata_t    ev_data;
};

The definition of the DM_STEP_TO_NEXT macro would then become:

#define DM_STEP_TO_NEXT(p, type) \
    ((type)((p)->_link ? (char *)(p) + (p)->_link : NULL))

Undeliverable Event Messages

• block the requesting process.

• return an error to the process that instigated the event.

• do not generate the event.

If an error is returned to the process, it may be specific to the operation that caused the event. This means that depending on the operation, two different errors can be returned for the same event type.

The implementation must also define the behavior for asynchronous events.

dm_vardata_t

struct dm_vardata {
    ssize_t   vd_offset;
    size_t    vd_length;
};
typedef struct dm_vardata     dm_vardata_t;

where the offset field (vd_offset) in dm_vardata_t records the distance in bytes from the beginning of the structure in which the variable length data begins.

In the case of an event message, it records the beginning of the actual event-specific data. For other structures, such as dm_stat_t, it indicates where the handle data can be found. The definition of the two access macros would then be:

DM_GET_VALUE(p, field, type) \
            ((type)((char *)(p) + (p)->filed.vd_offset))

DM_GET_LEN(p, field)   ((p)->field.vd_length)

NFS Daemon Starvation

A possible solution to this problem is to devise a method for the file system to notify the NFS daemon when it detects that an operation will take an unusually long time. The NFS daemon could then fork a separate thread to wait for the migration to complete, and also send a EJUKEBOX notification to the client NFS.

The dm_pending() interface allows the DMAPI application to notify the DMAPI implementation if an operation is expected to be slow. The implementation may then take appropriate steps to notify NFS.

Unmount and Shutdown Deadlock

The dt_change Field in dm_stat

Punching Holes