Configure a Liberty collective

  1. Overview
  2. Create and configure a controller
  3. Add the server to the collective as a member
  4. Add more servers to the collective as members


Overview

We can organize Liberty servers into collectives to support clustering, administration, and other operations that act on multiple Liberty servers. A Liberty collective is a set of Liberty servers configured as part of the same administrative and operational domain. Configuration and state data about a Liberty collective is housed in an active operational repository.

For distributed systems, the collectiveController-1.0 feature and its capabilities are available only in WebSphere Application Server Network Deployment Liberty. The feature is not available in WebSphere Application Server Liberty Base, or WebSphere Application Server Liberty Core. If we have a WAS ND Liberty installation, we can use its collectiveController-1.0 feature to work with collective members from WAS Liberty Base, or WAS Liberty Core installations.

The collective create command can create default certificates and keys. Alternatively, we can create our own certificates and keys using another method.

  • We can store them in a keystore file.
  • We can store them in a SAF key ring (z/OS).

The Liberty server default keystore type is PKCS12. The JKS keystore type also works. Depending on when the collective keystores were created, they might be JKS keystores that have the .jks file extension or they might be the PKCS12 keystores with the .p12 file extension. For example, the rootKey.jks keystore file might be the rootKeys.p12 keystore in the configuration and file system.

Membership in a Liberty collective is optional. Liberty servers join a collective by registering with a collective controller to become members. Members share information about themselves through the operational repository of the controller.

The following rules apply:

  • A Liberty server can be a member of only one collective.
  • Different Liberty servers on the same host can be in different collectives.
  • Liberty servers on the same host that are members of a collective can coexist with Liberty servers that are not members of a collective.
  • Collectives make decisions using a quorum process. This requires that collective controllers can communicate among themselves so they can make decisions about the state of collective members. For communication among collective controllers, we need a reliable and low latency network connection for correct functioning of several network deployment features.

IBM strongly recommends that collectives be fully contained within a single data center.


Create and configure a controller

  1. Specify the user directory for the controller server by setting the WLP_USER_DIR environment variable.

    We want to place the controller in its own Liberty user directory, separate from the member servers for the controller. If necessary, customize the server.env file to contain the correct values for the JAVA_HOME environment variable and other settings.

  2. Create a server manually.

      cd /path/to/liberty/wlp/bin
      ./server create controller1

    This server will act as the collective controller.

    Create the following items:

    • A started task cataloged procedure (What is this?)
    • Appropriate security controls

  3. Create the collective controller configuration.

    To create a collective that uses self-signed certificate keystores...

      wlp/bin/collective create controller1 --keystorePassword=controllerKSPassword

    This command creates XML statements written to the console. We can these add to the server.xml file of the controller.

    Alternatively redirect the statments for controller1 into an .xml file using the --createConfigfile parameter: 

      wlp/bin/collective create controller1 \
    --keystorePassword=controllerKSPassword \
    --createConfigFile=$WLP_USER_DIR/servers/controller1/controller.xml

    Put this .xml file in the server directory of the controller.

    This collective command generates public and private SSH keys for collective-wide use.

    We can use already existing keystores and truststores: 

      wlp/bin/collective create controller1 
    --keystorePassword=controllerKSPassword 
    --createConfigFile=$WLP_USER_DIR/servers/controller1/controller.xml 
    --serverIdentityKeystore="keystore_filespec" 
    --serverIdentityKeystoreAlias="certificate_alias" 
    --collectiveTrustKeystore="truststore_filespec" 
    [-certificateRdn="xxx"]

    For more information run:

      collective help create

  4. Update the server.xml file of the collective controller.

    If the collective create command wrote the configuration output to the console, copy this output and paste it into the controller server.xml file. If we instead used the --createConfigFile parameter to write the configuration to a file, use an include statement to specify this file in the server.xml file. For example:

      <include location="c:\wlp\usr\servers\controller1\controller.xml" />

    To provide an administrative user name and password for the collective, edit the following statement in the server.xml file or the included file:

    Change this statement by specifying an administrative user name and password like the following example:

    Alternatively, add registry information to the server.xml file for the controller. Specify our own administrative ID and password through the registry.

  5. Start the collective controller server.

      wlp/bin/server start controller1

  6. Verify that the collective controller server started correctly and is ready to receive members.

    1. Open an editor on the collective controller messages log...

    2. Look for the following message:

        CWWKX9003I: CollectiveRegistration MBean is available.

    The following SSL configuration is generated automatically. The memberConnectionConfig is used when communicating with the members and controllerConnectionConfig is used when communicating with the controllers and replicas. 

      <ssl id="controllerConnectionConfig" keyStoreRef="serverIdentity" trustStoreRef="collectiveTrust" /> 
<ssl id="memberConnectionConfig" keyStoreRef="serverIdentity" trustStoreRef="collectiveTrust" />


Add the server to the collective as a member

  1. If the server does not exist, create it.

      wlp/bin/server create server1

  2. Ensure that we have these items:

    • A network connection to the collective controller.
    • An administrative user ID and password to perform MBean operations on the controller.
    • Values for the --host, --port, -user, and --password parameters. Found in server.xml file of the collective controller.

  3. Make sure the collective controller server.xml file defines a wildcard (*) host on the httpEndpoint element: 

      <httpEndpoint id="defaultHttpEndpoint"
      host="*"
      httpPort="9080"
      httpsPort="9443" />

    Otherwise, the join command might fail with the following message:

      The specified port 9443 could not be reached

  4. Add the server to the collective as a member.

    The collective join command adds the server to the collective configuration as a member, and creates server-side XML to define the communication path to the collective controller. 

      wlp/bin/collective join server1 \
    --host=controllerHostname \
    --port=9443 \
    --user=adminUser \
    --password=adminPassword \
    --keystorePassword=memberKSPassword \
   [--hostname=memberHostname]

    On the --keystorePassword parameter, set a value for the member keystore password, such as memberKSPassword. We can specify different --keystorePassword values for each server that we join to the collective.

    Set the optional --hostname parameter only if the system for the member server has multiple host names or does not have a host name configured. If we set the parameter, the value must match the defaultHostName variable defined in the server.xml file.

    To reduce the number of parameters needed, use the --controller parameter instead of the --user, --password, --host, and --port parameters: 

       
wlp/bin/collective join server1  \
    --controller=user[:password]@host:httpsPort \
    --keystorePassword=memberKSPassword \
   [--hostname=memberHostname]

    By default, self-signed certificates are created. If we already set up our own keystore and truststore files, we can add more parameters to the collective join command to specify those files. 

      wlp/bin/collective join server1 \
    --controller=user[:password]@host:httpsPort \
    --keystorePassword=controllerKSPassword \
    --serverIdentityKeystore="keystore_filespec" \
    --serverIdentityKeystoreAlias="certificate_alias" \
    --collectiveTrustKeystore="truststore_filespec" \
    [-certificateRdn="xxx"]

    We can specify deployment variables using the genDeployVariable parameter.

    To write the output of this collective command to a file, instead of to a console screen, specify an optional --createConfigFile parameter. Then, include the outputted file in the collective configuration by adding an include statement to the member server.xml file: 

      <include location=outputFilePath />

    For more information run the collective help join command:

      collective help join

    By default, the join operation leaves remote procedure call (RPC) credentials undefined. We must specify values for the rpcUser parameter, the rpcUserPassword parameter. We must also specify values for the operating system login user ID and password for the host on which the member server exists. If the member host is registered with the collective controller and the member host is not enabled for SSH, specify an optional --useHostCredentials parameter. When we specify this parameter, the member inherits the RPC credentials from its host registration on the controller. Typically, Linux hosts are enabled for SSH and Windows hosts are not enabled for SSH; thus, the --useHostCredentials parameter is useful for Windows member hosts. Specifying --useHostCredentials adds...

    ...to the member server.xml file. We can then run collective member server commands such as start or stop without specifying RPC credentials because the member inherits credentials from its host.

    See:

  5. If prompted to accept the certificate chain, enter y (yes).

  6. Update the member server.xml file.

    • If the collective join command wrote the configuration output to the console, copy this output and paste it into the server.xml file of the member server.

    • If we specified the --createConfigFile parameter to put the member configuration in a file, add an include statement to the server.xml file of the member server. The following example is of an include statement: 

        <include location="c:\wlp\usr\servers\server1\member.xml" />

  7. If we did not specify the --useHostCredentials parameter on the collective join command and the member host is not enabled for SSH, set RPC credentials for the hostAuthInfo element.

    Set these credentials in the member server.xml file or in the outputted file. We can set RPC credentials for the member server in either of two ways:

    • Set the RPC user and password values on the hostAuthInfo element.

      Set the rpcUser option to an operating system login user ID for the host on which the member server exists.

      Set the rpcUserPassword option to the operating system login password for the user ID. For example, if you log in to the member computer with the test1 user ID and the test1pwd password, then change the hostAuthInfo element: 

        <hostAuthInfo rpcUser="test1" rpcUserPassword="test1pwd" />

    • If the member host is registered with the collective controller, set the useHostCredentials option on the hostAuthInfo element to a value of true for the member server to inherit RPC credentials from its host: 

        <hostAuthInfo useHostCredentials="true" />

    See Overriding Liberty server host information for information about the hostAuthInfo element and for an example that shows how to register a member host and run the join command with the --useHostCredentials parameter.

    To reduce the number of parameters needed, use the --controller parameter instead of the --user, --password, --host, and --port parameters: 

      wlp/bin/collective \
    updateHost \
    controllerHostname \
    --controller=adminUser:adminPassword@controllerHostname:9443 \
    --hostWritePath=/dir1 \
    --rpcUser=rpcUser \
    --rpcUserPassword=rpcUserPassword  \
    --hostJavaHome=JAVA_HOME

    The --hostWritePath parameter specifies the directories to which the collective controller can write. Paths specified by the parameter are also readable.

    The --hostJavaHome parameter specifies the absolute path to where the JVM is on the system and where the member server runs, for example...

      /usr/lpp/java/java_1.7_64

  8. Start the member server. 

      wlp/bin/server start member1

  9. Verify that the member server started correctly and is publishing information to the controller.

    1. Review the member messages log

    2. Look for the following messages in any order:

        CWWKX8112I: The server's host information was successfully published to the collective repository.
        CWWKX8114I: The server's paths were successfully published to the collective repository.
        CWWKX8116I: The server STARTED state was successfully published to the collective repository.

    3. Optional: Use the testConnection command to validate connectivity. The command validates RXA connectivity between the controller and the host where the member exists. It also validates JMX secure connectivity between the collective controller and the collective member. 

        wlp/bin/collective \
    testConnection \
    hostName,usrDirPath,serverName  \
    --host=controllerHost
    --port=controllerHTTPSPort \
    --user=controllerAdmin 
    --password=controllerAdminPassword\
    --autoAcceptCertificates

      Where the hostName,usrDirPath,serverName tuple describes the member server just joined: The hostName variable is the name of the host where the collective member exists. The userDirPath variable is the user directory of the collective member. serverName is the name of the collective member.Alternatively, use the simplified --controller parameter to provide the controller-specific information. 

        wlp/bin/collective \
    testConnection \
    hostName,usrDirPath,serverName \
    --controller=user[:password]@host:HttpsPort \
    --autoAcceptCertificates

      The following SSL configuration is generated automatically. The controllerConnectionConfig is used when communicating with the controllers. 

        <ssl id="controllerConnectionConfig" \
    keyStoreRef="serverIdentity" \
    trustStoreRef="collectiveTrust" />


Add more servers to the collective as members

Repeat the previous step to add more servers to the collective.


Subtopics


Parent topic: Set up the server-management environment for Liberty using collectives


Related information