Man Pages

clshmd Man Page

CLSHMD(1M)                                                          CLSHMD(1M)

NAME
     clshmd - Daemon for inter-partition shared-memory communication over
     CrayLink.

SYNOPSIS
     clshmd <unit>

DESCRIPTION
     clshmd is a daemon which facilitates the setting up and tearing down of
     shared-memory communication over CrayLink between different partitions of
     a partitioned Origin-2000 machine.  clshmd should not be started up
     directly from the command-line. Rather, the startup function of
     clshmctl(1m) should be used to set up this daemon.

     Each partition of a partitioned Origin-2000 runs a distinct kernel.
     CrayLink Shared-Memory allows low-latency shared-memory communication
     between applications running on the distinct partitions. For such
     communication to be effected, physical memory from one partition is
     mapped to user buffers in a different partition. The different kernels
     which are involved in such mappings have to exchange information for the
     mapping to take place.  clshmd allows the kernels to communicate over
     CrayLink using an RPC-like mechanism. It should be noted that the kernels
     need to communicate only when the inter-partition-mapped buffers are set
     up, torn down, when a new partition comes up dynamically, or when an
     existing partition is brought down for administrative maintainence or
     other reasons. Neither clshmd, nor the different kernels are involved
     during steady-state inter-partition shared-memory communication. The
     clshmd daemon contains some state necessary to allow the xp_func(3p) user
     library to have a system V-like interface which includes part_func(3p)
     helper functions.  It also has a shared memory segment that is shared
     between it and the clshm kernel driver.  It receives messages from the
     driver in this area and sends messages to the driver through ioctl(2)
     calls.

     clshmd takes only one parameter.  This parameter is the <unit> number of
     the clshm driver.  This should always be "0".  Other numbers are reserved
     for future expansion.  All other controlling state of the daemon can be
     set with the different parameters of the clshmctl(1m) command before the
     clshmctl startup command is issued.

     clshmd uses two hwgraph files.  /hw/clshm/0/ctl and /hw/clshm/0/shm are
     used by the clshm.  The "0" in the paths are the unit number of the clshm
     driver.  ctl stands for "control" file which is used for setting up
     shared memory between the driver and daemon and the shm stands for
     "shared memory" file which is used by the user library to mmap shared
     memory segments into user processes.

FILES
     /hw/clshm/

SEE ALSO
     clshmctl(1m), xp_func(3p), part_func(3p), clshm_diag(1m), mkpart(1m)

clshmctl Man Page

CLSHMCTL(1M)                                                      CLSHMCTL(1M)

NAME
     clshmctl - Administration tool for CrayLink Shared Memory

SYNOPSIS
     clshmctl startup
     clshmctl shutdown
     clshmctl getcfg
     clshmctl maxpgs <num of pages>
     clshmctl maxdpgs <num daem pages>
     clshmctl timeout <daemon timeout>

DESCRIPTION
     clshmctl is a system-administrator's tool for controlling the resources
     associated with CrayLink-shared-memory (clshm) -- a means of cache-
     coherent, shared-memory communication between partitions on an Origin-
     2000.  Each partition of an Origin-2000 has its own hostname, IP address
     and operating-system, and behaves as a stand-alone host.  clshm has a
     system V shared memory interface in the form of a user library
     (xp_func(3p) and part_func(3p)) that can be used directly by user-
     applications.

     clshmctl performs various functions needed to administer the software
     devices and daemon associated with clshm and returns operational status
     information.

     clshmctl is invoked with one of the following function names, and the
     associated parameters:

     startup     Configure clshm on the local partition, and start up the
                 clshm devices and clshmd(1m) daemon with default parameters.
                 The value of the defaults can be examined by using the
                 clshmctl getcfg function, as explained later. The startup
                 step is performed at system boot-up by /etc/init.d/sn0start
                 and is required before any inter-partition communication can
                 be effected over CrayLink Shared Memory. This startup at
                 bootup time must be enabled using the chkconfig(1m) command
                 for clshmd(1m).  It also needs to be done if clshm has been
                 reset by a clshmctl shutdown command.  clshmctl startup also
                 creates and initializes the software devices through which
                 shared-memory communication is performed.

     shutdown    Forces a reset of the clshm software on the local partition.
                 The clshmd(1m) daemon is made to exit and the resources
                 associated with the clshm software devices are released.
                 clshmctl shutdown should be used carefully since any user-
                 program using the clshm devices will be killed when this
                 function is called.

     getcfg      Returns the configuration of clshm on the local partition.
                 The parameters configurable through clshmctl are the maximum
                 number of 16K pages of shared-memory that are supported, the
                 number of 16K pages shared between the daemon and the kernel
                 driver, and the timeout time before the clshmd(1m) daemon
                 wakes up from sleeping and checks for a message from the
                 driver when there is no user activity going on.

     maxpgs <num of pages>
                 Sets the maximum number of pages of shared memory that can be
                 associated with any of the channels on the local partition.
                 maxpgs <num of pages> will fail if it is called after
                 clshmctl startup has been called (without another clshmctl
                 shutdown).  In order to set this parameter, the clshmd(1m)
                 daemon must be in the shutdown state.

     clshmctl maxdpgs <num daem pages>
                 Sets the maximum number of shared memory pages that the clshm
                 driver can use to communicate with the clshmd(1m) daemon.
                 Like maxpgs, this call will fail if made after a clshmctl
                 startup call has been made.

     clshmctl timeout <daemon timeout>
                 Sets the timeout value in milliseconds that the clshmd(1m)
                 daemon will sleep before checking for messages in the shared
                 memory segment from the driver.  If there are messages coming
                 in on either other daemon or user sockets, then the driver
                 will not sleep the full amount of the timeout, but will
                 process all messages if fines on all sockets and in the
                 shared memory segment.

WARNINGS
     If the clshmd(1m) daemon aborts abnormally a clshmctl shutdown must be
     called before a clshmctl startup can be called again to restart the
     shared-memory mechanism.

FILES
     /etc/init.d/sn0start

SEE ALSO
     clshmd(1m), chkconfig(1m), xp_func(3p), part_func(3p), clshm_diag(1m),
     mkpart(1m)

clshm_diag Man Page

CLSHM_DIAG(1M)                                                  CLSHM_DIAG(1M)

NAME
     clshm_diag - Diagnostic tool for CrayLink Shared Memory

SYNOPSIS
     clshm_diag

DESCRIPTION
     clshm_diag is a system-administrator's tool for ensuring that the
     daemons, devices and driver required for communication between partitions
     using shared-memory over CrayLink (clshm), have been properly set up.

     clshm_diag causes the exchange of information between the clshmd(1m)
     daemon on the local partition, and its counterpart on every remote
     partition known to the local partition. Services are known to be up if we
     can send and receive hostname information from other partitions' daemons.
     The daemons will not come up if the driver is not present.  If the
     exchange of hostnames can occur then all services are up and ready to
     run.  clshm_diag prints out the hostnames of all partitions that we can
     successfully communicate and indicates all partitions that it has
     knowledge, but cannot communicate with, so that the system-administrator
     can investigate the problem further.

SEE ALSO
     clshmctl(1m), clshmd(1m), xp_func(3p), part_func(3p), mkpart(1m)

xp_* Man Page

XP_FUNC(3P)                                                        XP_FUNC(3P)

NAME
     xp_func: xp_ftok, xp_shmget, xp_shmat, xp_shmdt, xp_shmctl - CrayLink
     Shared Memory Functions

SYNOPSIS
     #include <xp_shm.h>

     key_t xp_ftok(const char *path, int id);

     int   xp_shmget(key_t key, size_t size, int shmflg);

     void *xp_shmat(int shmid, const void *shmaddr, int shmflg);

     int   xp_shmdt(const void *shmaddr);

     int   xp_shmctl(int shmid, int cmd, .../*struct shmid_ds *buf*/);

DESCRIPTION
     xp_func calls are a user library interface for CrayLink-shared-memory
     (clshm) -- a means of cache-coherent, shared-memory communication between
     partitions on an Origin-2000.  The functions try to be very much "system
     V-like" in their interface.

     xp_ftok creates a key of type key_t given a path and an id much like the
     ftok(3c) call.  The path should either be /hw/clshm/partition/<2 digit
     hex partition #> or NULL.  The shared memory associated with this key
     will be placed on the specified partition.  If no partition was specified
     it will simply be placed on the partition that the process is running.
     The id should always be passed in a "1".  Everytime this function is
     called a different key will be returned.

     xp_shmget must be passed a key that is created by xp_ftok in order to
     create a shared memory segment and its associated shmid.  The size that
     is passed in, must be the same size as all other xp_shmget calls that
     have been called with the same key.  The shmflg parameter is ignored.

     xp_shmat must be called with a shmid that was returned from a xp_shmget
     call in order to attach a shared memory segment.  The shmaddr and shmflg
     parameters have the same semantics as the shmat(2) function.

     xp_shmdt detaches the given shared memory segment for the given shmaddr.
     The last detach does not automatically remove the shared memory segment
     unless the IPC_AUTORMID command was sent to the xp_shmctl function.

     xp_shmctl is much like the shmctl(2) call.  But it only implements the
     IPC_RMID, command which removes a shared memory segment associated with
     the shmid passed in.  It also implements the IPC_AUTORMID, command which
     removes the shared memory segment associated with shmid after the last
     xp_shmdt call.  If the last xp_shmdt call has already been made, then the
     shared memory segment is removed immediately.

     All xp_func calls will fail and not perform any function if one of the
     following are true:

     ENOMEM        A malloc(3c) call failed.

     EPROTO        This version of the library is not compatable with the
                   local clshmd(1m) daemon's version.

     EIO           There has been a communication problem with the local
                   clshmd(1m) daemon.

     EINVAL        The function was sent invalid arguments.

     ENOSYS        The argument passed in is not supported at this time.

     EFAULT        There has been an internal problem with the user library or
                   clshmd(1m) daemon.

     ENAMETOOLONG  The path is too long to put into the specified buffer.

     These functions can also fail with the errno returned from various unix
     calls that the library makes.

WARNINGS
     If xp_shmat is called after the last xp_shmdt and the IPC_AUTORMID
     command has been sent through xp_shmctl, then the behavior is undefined.

SEE ALSO
     part_func(3p), clshmctl(1m), clshmd(1m), mkpart(1m)

DIAGNOSTICS
     Upon successful completion, the return values are as follows:

          xp_ftok returns a valid key_t type key.

          xp_shmget returns a valid shmid.

          xp_shmat returns the data segment start address of the attached
          shared memory segment.

          xp_shmdt returns 0.

          xp_shmctl returns 0.

     Otherwise, a value of -1 is returned and errno is set to indicate the
     error.

part_* Man Page

PART_FUNC(3P)                                                    PART_FUNC(3P)

NAME
     part_func: part_getid, part_getcount, part_getlist, part_gethostname,
     part_getpart, part_setpmo, part_getnode - CrayLink Shared Memory Helper
     Functions

SYNOPSIS
     #include <xp_shm.h>

     partid_t part_getid(void);

     int part_getcount(void);

     int part_getlist(partid_t *part_list, int max_parts);

     int part_gethostname(partid_t part, char *name, int max_len);

     partid_t part_getpart(int shmid);

     int part_setpmo(key_t key, pmo_handle_t pmo_handle, pmo_type_t pmo_type);

     int part_getnode(int shmid, char *hwpath, int maxlen);

DESCRIPTION
     part_func calls are part of the xp_func(3p) user library interface for
     Craylink shared memory (clshm).  They are functions that do not map
     directly to system V shared memory like calls that the xp_func(3p) calls
     do.

     part_getid gets the partition id number for the partition that the
     process is running on.

     part_getcount gets the number of partitions that are currently active
     according to the partition that the process is running on.

     part_getlist returns the number of partitions that are currently active.
     The partition numbers of the active partitions are placed int the
     part_list array whose lenth is passed in through the max_parts parameter.

     part_gethostname takes a active partition number and places the hostname
     of that partition in the name string.  The maximum length this string can
     be filled up to is passed in the max_len parameter.

     part_getpart takes a shmid and returns the partition id of the partition
     that hosts the memory associated with this shmid.

     part_setpmo takes a key, pmo_handle, and pmo_type and associates the pmo
     information with the key.  The key must come from xp_ftok(3p) and must be
     hosted by the partition that the current process is running on.  This
     call must be made before any xp_shmget(3p) calls are made to associate a
     shmid with this key.  See the mld(3) interface for how to use memory
     locality domains to control the placement of memory for this key.

     part_getnode takes a shmid and hwpath string with maximum length of
     max_len to fill in the hwgraph(4) path of the node that the shmid shared
     memory has been allocated on.  This is only guaranteed to return a valid
     string if both part_setpmo and xp_shmget both succeeded for the shmid and
     the part_getnode function is being called from the partition that hosts
     the shmid shared memory.  However, if all memory associated with a shared
     memory segment is placed on the same node without part_setpmo being
     called, then this call will also return a valid hwgraph(4) string, given
     that it is called from the correct partition.

     All part_func calls will fail and not perform any function if one of the
     following are true:

     ENOMEM        A malloc(3c) call failed.

     EPROTO        This version of the library is not compatable with the
                   local clshmd(1m) daemon's version.

     EIO           There has been a communication problem with the local
                   clshmd(1m) daemon.

     EINVAL        The function was sent invalid arguments.

     ENOSYS        The argument passed in is not supported at this time.

     EFAULT        There has been an internal problem with the user library or
                   clshmd(1m) daemon.

     ENAMETOOLONG  The path is too long to put into the specified buffer.

     These functions can also fail with the errno returned from various unix
     calls that the library makes.

SEE ALSO
     xp_func(3p), mld(3), hwgarph(4), clshmctl(1m), clshmd(1m), mkpart(1m)

DIAGNOSTICS
     Upon successful completion, the return values are as follows:

          part_getid returns a valid partition id.

          part_getcount returns the number of active partitions.

          part_getlist returns the number of active partitions and fills in
          the part_list array.

          part_gethostname returns 0.

          part_getpart returns the partition number of the partition hosting
          the shmid memory.

          part_setpmo returns 0.

          part_getnode returns 0.

     Otherwise, a value of -1 is returned and errno is set to indicate the
     error.