DCE Security Replication and Propagation

The information in this chapter assumes a knowledge of the DCE security model. Refer to DCE Security Model for a description of this model. RS Editor RPC Interfaces , "RS Editor RPC Interfaces", contains detailed information about the interfaces and datatypes used for propagation of changes to replicas, specifically in The rs_prop_acct RPC Interface to (and including) The rs_prop_replist RPC Interface , and The rs_repadm RPC Interface to and including The rs_rpladmn RPC Interface , and finally, The rs_update RPC Interface .

In a given DCE cell many security servers may run. Each one of the security servers manages its own copy of the registry database. A security server and its database are known as a replica. The servers collaborate to keep their copies of the database consistent. Only one of the replicas accepts changes, the master replica. The action of copying a change from one server to another is propagating that change. All of the changes that occur in the master registry database are propagated to all remaining replicas at the granularity of the change. That is, whenever a change is made to the master registry, such as when a new principal is added, that change is then propagated to each replica. The act of propagating changes from the master security server to all other replica security servers is considered replication.

Replication improves the system's reliability and availability. When clients bind to a replica they bind to either a read or update site. The master site is the only update site; all read only sites are called slave replicas. If a slave replica fails to respond to a query the client can then rebind to another replica.

No facility for supporting replication is specified in this document, though implementations would likely provide some sort of service functions for this purpose. Typically they would consist of support for administration and configuration functions for DCE installation.

Replication Overview

All security servers can answer queries from clients. The master server is the only server that accepts updates from clients. When a client binds to a server it must bind according to the type of access required. There are currently two methods of binding to a registry server. The client can bind to an arbitrary server using sec_rgy_site_bind(); this will bind to any available server. The client can also target the master registry for binding with sec_rgy_site_bind_update(); this will force binding to the master registry. The master propagates updates it receives from clients to the other security servers, called slaves. The replication scheme is highly available and weakly consistent.

Replication is done only within a cell. That is, within hierarchical cells or cells connected intercell a change within a cell does not force a change or a replica update in any other cell.

When an update arrives at the master, the master server applies the update to its copy of the database. It then adds the update to its propagation queue. The master then persistently tries to deliver the updates on its propagation queue to all replicas. When an update has been delivered to all replicas, the master then removes the update from its propagation queue.

Note:

The master server will maintain the update on the propagation queue until each replica server has received the update. If a replica is taken out of service without being properly retired the propagation queue will grow indefinitely. This will not stop the propagations from proceeding on all other slave replicas, however, all entries on the queue will remain until the out of service replica is in service again. This would be potentially damaging to a master replica. As the propagation queue grows without bounds there are memory considerations that must be taken into account.

The Master Replica

The master replica is responsible for maintaining and administering the several entities with regard to replication. During operation the master replica maintains the registry database, replica list and propagation queue. In addition, checkpoint entries are made in order to preserve updates, for both the registry and the replica list. No method is specified in this document for such preservation although a typical implementation might take the form of update logs.

Propagation Queue

To maintain the synchronicity between the replicas the master replica maintains a propagation queue. Each entry on the propagation queue needs to be sent to one or more replicas. The entries on the propagation queue remain until all replicas have received them. All entries on the queue are positioned on the queue as they occur, positioned in first-in, first-out order. That is, when an event occurs at the master registry it is put on the queue. During normal propagation events the following list shows the interfaces which perform normal propagation. Each entry made to the propagation queue typically would be checkpointed (or preserved) in some manner (as with the registry and replica list). The technique for doing so is not specified in this document.

The current per-modification propagation interfaces are described in RS Editor RPC Interfaces . These interfaces are:

• rs_prop_acct_* for propagating registry account information

• rs_prop_acl_* for propagating registry ACL information

• rs_prop_attr_* for propagating registry attributes

• rs_prop_attr_schema_* for propagating registry attribute schemas

• rs_prop_pgo_* for propagating registry PGO items

• rs_prop[_*]_plcy_* for propagating registry policy information

• rs_prop_replist_* for propagating registry replica list.

For more information regarding the individual propagation function calls, please see the appropriate rs_prop sections in RS Editor RPC Interfaces .

Each time a modification is made to the registry a corresponding entry is made on the propagation queue. In some instances entries are made to the propagation queue and are not propagated out to the replicas immediately. That is, when an entry is added for propagation the interface has a general argument that places the data on the queue with the specific intention of no propagation. A specific instance for the no propagation flag is during change of master. In this specific case, the sequence of events occurring during a change of master, may cause loss of data.

The propagation queue contains the following information.

• Sequence Number

  The master registry sequence number of the change. This is the coordinating entry within the update and propagation process. When the replicas are communicating their relative up-to-dateness this number is the determining factor. This number is originally generated on the master replica, it is generated when an update occurs. For each change within the master there is a sequence number generated and assigned to that change.

• Timestamp

  The time when the update happened to the master registry.

• Data

  The data that is specific to this update. The actual data that was modified in the master.

Replica List

The replica list contains an entry for each security replica in operation. Each entry within the list contains the replica's UUID, name and tower information. The master replica controls the replica list. That is, in order for a replica to be added or removed from the list, the master controls the process.

Replica List Entries

Each security server in the cell manages a replica list. Several entries are common to all replica lists. This basic information on all security servers' replica lists gives each replica an idea of the location and status of all the replicas. The entries for the replica list are outlined here; please see rs_replica_item_t and rs_replica_item_p_t which contains a complete description of replica list data items. The following entries within the replica list are common to all replicas.

• replica's name

  The cell relative name of the replica. The replica's name (which is not "well-known") falls in the form /.../cell_name/subsys/dce/sec/replica_name, where replica_name represents the name for the replica.

• replica's UUID

  The replica's instance UUID.

• replica's network address(es)

  The network address(es) for the replica. There may be multiple addresses. See the data types for rs_replica_item_t in rs_replica_item_t and rs_replica_item_p_t and rs_replica_twr_vec_t in rs_replica_twr_vec_p_t for more information.

• master

  This flag indicates whether the described replica is currently the master replica.

• deleted

  Flag indicating if the replica has been or is marked for deletion.

The server maintains special information to manage propagation of updates to a replica.

• propagation type

  Describes the current relative state of the replica. This will determine the method of updates passed from the master replica.

  • marked for initialization

    The replica is currently in the process of receiving a database from a replica.

  • marked for deletion

    The replica has been scheduled to be deleted.

  • ready for updates

    The replica is currently accepting updates.

• Sequence Number

  If the replica is ready for updates, this is the sequence number of the last update successfully delivered to the replica. (This implies, of course, that all previous sequence numbers have been successfully delivered.) See the rs_replica_prop_info_t data type definition in rs_replica_prop_info_t for information on this entry (and also the next, Time Stamp.)

• Time Stamp

  The timestamp of the last update represented by the sequence number.

• Communications Status

  The communication status of this particular replica. There are currently three levels of communication status. They are the nominal state replica_comm_ok, short term communication interruption replica_comm_short_failure, and long term communication interruption replica_comm_long_failure.

Replica Information

Each security replica has a flag that defines its current state. This state is either known or distributed to other replicas. During certain states the replica is incapable of accepting propagation information or providing database information to clients and other replicas. The complete set of replication information that each replica maintains about itself and about the system is:

• State

  As defined in the next section.

• Replica UUID

  The replica's instance UUID. This UUID may or may not be identical to that of the Master UUID. If it is, then this replica is the master replica (otherwise, it is a slave replica).

  Note:

  The master flag in the replica information rs_replica_item_t data type would also indicate TRUE if this replica is the master replica.

• Name

  Its cell-relative name.

• Sequence Number

  The sequence number of the last update provided. This is noted on the master and the client replicas to maintain a cross check of the updates. That is, the master ("thinks" it) has sent a specific number of updates and this number on the client would confirm that number.

• Timestamp

  The timestamp of the last update represented by the sequence number.

• Initialization UUID

  The UUID of the replica that provided the initialization of the replica's registry.

• Network Address(es)

  The security replica's network address(es).

• Cell's Security UUID

  This UUID is generated at the initialization of the master registry database when the cell is created. This entry is the same as the cell UUID. It uniquely determines the cell.

• Master UUID

  The UUID of the current master replica.

• Sequence Number

  The number of the event when this master became the master. This information is only relevant if this is the master replica. The master keeps the sequence number that was current at the time this replica became the master.

The rs_replica_info_t structure in rs_replica_info_t provides more information.

Replica State

Because of the variety of changes and situations that a replica can be in, the necessity of maintaining state information is critical. When a replica is attempting to communicate with a peer it needs to understand what the current state of that peer is. The concept of replica state provides this. If a new replica is going to request a database from a peer it needs to know whether that particular replica is able to be a provider. The state generally defines a series of events in the life of a replica, from initialization, name changes, slave to master changes or database key changes. The replica state defines the current condition of the replica. The data type definition of the replica states can be found in Replica States . There are 13 possible states. The following is the list of the various states.

• unknown to master

  The current state of the replica is unknown to the master.

• uninitialized

  The replica remains uninitialized while the database is being created. This is generally a temporary state during creation of a replica.

• initializing

  The replica is currently being initialized by another replica.

• in service

  The replica is currently in service. The replica may either provide information for clients or become the master.

• renaming

  This state is in effect during a renaming of the replica.

• copying database

  This state is active when the database is in the process of being copied to a new replica or a replica that has requested a new database.

• in maintenance

  The replica is in maintenance mode.

• master key changing

  The current master key is in the process of being changed.

• becoming master

  When a replica receives the request to become a master, this state is active (on the replica that is in state transition from slave to master).

• becoming slave

  During the time when a slave replica receives a request to become a master this state is active (on the master that is in state transition from master to slave).

• duplicate master

  A replica that thinks it is the master has been informed by a slave that the slave believes a different replica to be the legitimate master.

• closed

  A replica has closed its databases and is in the process of exiting.

• deleted

  The replica has been deleted from the replica list.

Slave Replica

The slave replicas maintain both the registry database and replica list in memory and on disk. Each entry is updated when the master replica propagates a change. Each change is applied to the in-memory copy and then pushed to the disk copy. For each update that is propagated from the master replica an entry is made to the update log as well.

The slave replica also maintains a list of all changes that have been made.

Creating a Replica

In DCE 1.1 a replica is created by configuring the host as a DCE client. In addition, a new ("empty" or skeletal) (security) database is created. Once the new database is created several entries from the current master are required to be cataloged. The database is initialized with the following entries:

• Cell Security ID

  The Cell well known security UUID.

• Replica ID

  The instance UUID of the replica being created. This UUID is (dynamically) created during the process of creating the replica.

• Network Towers

  The binding towers for the replica being created. See the rs_replica_twr_vec_p_t data type definition in rs_replica_twr_vec_p_t for more information.

• Replica Name

  The replica name. This name is of data type rs_replica_item_p_t. It is supplied by the administrator upon creation of the replica. See rs_replica_item_t and rs_replica_item_p_t for a more complete description.

• Sequence Number

  The master sequence number at the time of creation. This sequence number is of data type rs_update_seqno_t.

• Creator Id

  The sec_id_t of the entity creating the replica. This id is either supplied by the administrator, or it is the UUID and string name of the local cell on which the replica is created if not given by the administrator. Typically (usually) the (registry) Creator Id is supplied by the administrator.

• Cell Id

  The sec_id_t of the (local) cell. This Cell Id is initially stored in the registry database when the cell is being created. It consists of the cell's UUID and a string name identifying it. The cell name (string name) is retrieved when necessary (for instance when creating a replica). The method of retrieval is not specified in this document.

• Keyseed

  The initial keyseed for the database.

• Master Rep Information

  The rs_replica_item_t of the master replica (see rs_replica_item_t and rs_replica_item_p_t for more information).

The replica name is then created and verified with the CDS name service. This is done by creating the replica name of the form /.../cell_name/subsys/sec/replica_name. The name service is then checked to verify that the name is acceptable for use with rs_ns_entry_validate(. (See tagmref_rs_ns_entry_validate for information about this routine.) The process of creating a database notifies the master replica to add the new replica to the master's replica list. The master is notified via the rs_replist_add_replica( operation. The state of the replica is set to rs_c_state_uninitialized. The cell name (Cell Id), Replica Id (instance UUID of the replica) and binding information is then stored in the name space. In conjunction, the name of the master site is also set in the name space. The security service uses this information when contacting the master security server during initialization.

When the master site is notified of a new replica the master server guides the initialization of the replica. When the replica is added to the master's replica list it is marked for initialization using rs_rep_admin_init_replica(, which sets the replica state to rs_c_replica_prop_init. The master sends an initialization request to the replica using rs_rep_mgr_init(. This request includes a list of other replicas that the new replica can use to initialize from (see rs_rep_mgr_init() ). The new slave replica selects one of these specified replicas and sends it a request to copy its entire database using rs_rep_mgr_copy_all(. The slave replica supplying the database goes into a special copying database state, rs_c_state_copying_dbase, during which it will not accept propagations from the master replica. It copies its database to the new replica. When the copy is complete the new replica finishes its registration in the name service, goes into the state, rs_c_state_in_service, and notifies the master that it is now initialized, using rs_rep_mgr_init_done(. The master marks the new replica as ready for updates on the master replica list and records the sequence number of the last update the replica received.

Delete A Replica

A replica is deleted in DCE 1.1 under command of (initiated by) the Security Administrator. Typically, installations have a set of commands for security administration-however, they are beyond the scope of this document and are not specified here.

When the replica deletion command is given, a delete replica request is sent to the master server using rs_replist_delete_replica(. The master marks the replica for deletion by setting replica state to rs_c_replica_prop_delete and puts the delete replica update on its propagation queue. The master then propagates the delete replica update to all other replicas sites on its list. These replicas remove the entry for the deleted replica from their respective replica lists. The master then delivers the delete request to the replica being deleted. When the replica server receives the request, it destroys its database and stops running. Upon completion, the master server removes the deleted replica from it's replica list.

Ultimately there is no verification the replica deleted its database and stopped operating. But the other slave replicas and the master replica would refuse communications because it has been marked as deleted.

Master Change

By issuing the appropriate command (implementation specific items are beyond the scope of this document), the security administrator can change a slave to a master. This change is effected through rs_rep_admin_change_master() (see rs_rep_admin_change_master() for a detailed description of this function). This function causes the then current master to change its replica state into rs_c_state_becoming_slave. This stops the propagation activity and starts the transfer of the outstanding updates on the propagation queue to the new master. The becoming slave master then sends via rs_rep_mgr_become_master( a request to the selected slave to become a master. The new master replica will then request the propagation queue from the old master using the rs_rep_mgr_copy_propq(. When the change master ( rs_rep_admin_change_master() ) function call returns successfully the old master (the replica becoming the slave) writes the new master information to disk and sets its replica state to rs_c_state_in_service.

During the entire master change sequence, from initiation to completion, changes to the master registry are not accepted. If a client is attempting to change the master registry at this time and is not successful, the client attempts resubmitting the change a number of times. The number of attempts permitted is determined by the security administrator.

The slave replica selected to become the new master replica will, upon receiving the request to become the master, read the original master's replica list using rs_replist_read_full(). Once the replica list is successfully transferred to the new master a request is made to the original master to send the propagation queue using rs_rep_mgr_copy_propq(). Having successfully received the propagation queue, the new master uses the lowest number on the propagation queue as it's master sequence number, and commits to being the master by writing the new master information to disk, sending updates to clients and accepting updates from clients.

Authentication between Replicas

Communication between replicas is secure. The master server authenticates to the slaves as the dce-rgy principal and the slaves authenticate to the master using the host principal of the machine on which they run. By default slaves need i, m and I ACL rights to the replica list (/.:sec/replist)-see RS Protected Objects and their ACL Manager Types for more information on ACL rights. The replica's information and credentials are acquired using the rs_rep_mgr_get_info_and_creds() function call (see rs_rep_mgr_get_info_and_creds() for more information).

Name Service Registration

Each replica has a server entry name in CDS. The default is /.:/sec. When binding to a security server, using this default will cause a binding to the cell's master replica. (An installation (cell) can change this default to any of the security server names registered in CDS. This is typically done via installation-supplied functions that set the default (and which are beyond the scope of this document) to a specific replica in /.../cell_name/subsys/dce/sec.)

The /.../cell_name/subsys/dce/sec node maps the replica's name to its location, its replica UUID (replica ID), and the cell's security object UUID (Cell Security ID), for any replicas that have been registered. When a replica server is first created it validates its server entry's information.

The security RPC group name is /.../cell_name/sec. This name is not "well-known", but by convention it is named "/.:/sec" (see Binding to TCB Servers for more detail on group names (and cell-profiles)). All initialized security server entry names appear in the security group. The cell profile, /.../cell_name/cell-profile (a well-known CDS node), maps a few security interface UUIDs to the security group name as follows: (For more information regarding RPC Profiles please reference the DCE RPC Specification. It's complete title can be found in the Referenced Documents preface section of this specification.)

 


         UUID                         Vers       Name            Priority  Interface
{{d46113d0-a848-11cb-b863-08001e046aa5 2.0}  /.../cell_name/sec      0     rs_bind}
{{0d7c1e50-113a-11ca-b71f-08001e01dc6c 1.0}  /.../cell_name/sec-v1   0     secidmap}
{{8f73de50-768c-11ca-bffc-08001e039431 1.0}  /.../cell_name/sec      0     krb5rpc}
{{b1e338f8-9533-11c9-a34a-08001e019c1e 1.0}  /.../cell_name/sec      0     ps_request}
{{b1e338f8-9533-11c9-a34a-08001e019c1e 1.1}  /.../cell_name/sec      0     ps_request}

Sample Cell Profile Entries

The CDS name /.../cell_name/sec-v1 is an RPC Group designating the master security server. For more information regarding RPC Groups please reference the DCE RPC Specification.

Locate a Security Server

When a client needs to find a security server replica it does so by looking up a special security service interface UUID in the CDS cell profile /.../cell_name/cell-profile . This special interface UUID in the cell profile maps to the cell's security group name /.../cell_name/sec. The client binding code tries to bind to one of the servers in the security group.

Note:

During initialization and configuration of a site, the client cannot locate a security server through the CDS name service as that information is not yet available. For these instances, installation-specific information is used to locate the servers. The handling of such information is not specified in this document.

Registry Database Encryption

Each replica maintains its own master key to encrypt the data it stores on disk. The key is initially generated via system administrator input. This key can be changed with the routine rs_rep_admin_mkey().

When a database is initially created by the administrator, as part of the creation process (for both slave and master replicas), the administrator command usually typically requires the specification of a keyseed in order to create the key for the database. In DCE 1.1, if a keyseed is not specified, the administrator is asked to input one as part of the creation process. This keyseed is a character string up to 1024 bytes in length that is then used to seed the random key generator in order to create the master key for the database being created (master or slave). This master key is used to encrypt account passwords. Note that each instance of a replica has its own master key.