Network Deployment (Distributed operating systems), v8.0 > Troubleshoot and support > Troubleshoot web services
Detecting and fixing problems with WS-ReliableMessaging
The nature of WS-ReliableMessaging is that network and server failures are assumed, and therefore the target web service or message store might not be available. In these cases, message sequences cannot be completed and collections of web service messages are held awaiting transmission. We can use the SystemOut.log file, system events, and the runtime administrative panels to monitor the system and detect and fix problems with WS-ReliableMessaging.
If a sequence fails, a message is written to the application server SystemOut.log file and a system event is generated. Therefore you can detect failed sequences by looking at the SystemOut.log file, or by writing an event listener (or by using third party software) to monitor system events.
With WAS v8 you can configure the server to use the HPEL log and trace infrastructure instead of using SystemOut.log , SystemErr.log, trace.log, and activity.log files or native z/OS logging facilities. If you are using HPEL, you can access all of your log and trace information using $PROFILE/bin/LogViewer.sh.
After a sequence has been established, WS-ReliableMessaging provides retransmission of messages to a service. However if the sequence is not established (that is, if the initial CreateSequence request is refused) then the messages are not transmitted to the service. See the troubleshooting tip A sequence is not established, and therefore WS-ReliableMessaging cannot ensure messages are transmitted.
For more detailed status information at run time, and facilities to help you fix problems, use the WS-ReliableMessaging administrative console runtime panels. These panels are available at many different scopes (for example cell; application server; messaging engine). For a full list of the WS-ReliableMessaging runtime panels, and details of the scopes at which they are available, see WS-ReliableMessaging - administrative console panels.
At all scopes, the parent panel is Reliable messaging state settings. From this panel you can investigate each of the three key runtime aspects of reliable messaging:
- Message stores
- Inbound sequences
- Outbound sequences
The following icons are displayed here and on several other reliable messaging runtime panels:
| OK | Everything here, and (if there is a link) in all runtime panels below this link, is running normally. |
|---|---|---|
| Warning | Something here, or (if there is a link) in one of the runtime panels below this link, is in an unusual state and you might have to take some action to resolve it.
For example, the system might be awaiting a response from an endpoint. In this case, either the response will be received (in which case you need take no action and the runtime information will be updated to "OK") or the reliable messaging destination has stopped acknowledging messages (in which case we have to take some action to resolve the failed sequence). |
| Error | There is a definite error that take some action to resolve, either here or (if there is a link) in one of the runtime panels below this link. |
To use the reliable messaging runtime panels to detect and fix problems with WS-ReliableMessaging, complete one or more of the following steps:
Procedure
- Investigate problems with message stores.
In the navigation pane, click one of the paths to this panel. For example Servers > Server Types > WebSphere application servers > server_name
> [Additional Properties] Reliable messaging state > Runtime > Message store . The list of reliable messaging storage managers for the current scope is displayed in the Message store collection form.
For the managed qualities of service, the messages are written to a messaging engine. For the unmanaged non-persistent quality of service, the messages are stored in memory. For in-memory stores the only possible value is "Running". For messages stored by a messaging engine, the possible values are "Running" or "Messaging engine not contactable", probably because the messaging engine is not running. The "OK" icon indicates that the message store is running. If the messaging engine is not contactable, the "Error" icon is displayed.
For each message store in the list, the name of the associated reliable messaging application is given in the description column. If a messaging engine is not contactable, restart the message store for that application.
- Investigate problems with inbound sequences.
In the navigation pane, click one of the paths to this panel. For example Servers > Server Types > WebSphere application servers > server_name
> [Additional Properties] Reliable messaging state > Runtime > Inbound sequences . The runtime state of each of the inbound sequences for the current scope is displayed in the Inbound sequence collection form.
We can use a filter to look at sequences that are in a particular state (for example Failed due to missing message) or that have a large number of messages awaiting dispatch to applications. If the sequence status is Error, there is a problem with the sequence and the source server hosting the other end of the sequence has terminated it. If the sequence is active and there are a large number of messages awaiting dispatch to the application, then there might be a problem with the application or, if in-order delivery is specified, delivery might be held up because the sequence has gaps in it.
We can select one or more sequences, then use the buttons provided to dispatch the messages to their associated applications, to export the messages to compressed files, to close or terminate the selected sequences, or to delete the selected sequences and all their messages.
Attention: Delete or terminate sequences only if necessary. If you delete or terminate an active sequence, the resulting messaging behavior is unpredictable and can cause loss of messages. If you are not sure whether you can safely delete or terminate a sequence, do not delete or terminate it; the system automatically deletes sequences that have been inactive for 12 hours.
To see more detailed information about a particular sequence, click the Sequence identifier field. The Inbound sequences settings form is displayed. This detailed information includes addressing information to help you identify the source of the sequence, and the value (true or false) for in-order delivery for the sequence. From this panel you can also display the following forms:
- The Acknowledgement state collection form. (The ranges of message sequence numbers received from the WS-ReliableMessaging source. If more than one range is displayed, this indicates a gap in the messages received. If "In-order delivery" is selected for the sequence manager, messages with a sequence number greater than the lowest gap cannot be delivered to the application until the gap is closed.)
- The Inbound message collection form.
(The messages on the inbound sequence. We can use this form to delete individual messages.)
- The Message settings form. (The contents of an individual message in the sequence.)
- The Message settings form. (The contents of an individual message in the sequence.)
For more guidance on diagnosing problems with inbound sequences, see Diagnosing the problem when a reliable messaging source cannot deliver its messages
- Investigate problems with outbound sequences.
In the navigation pane, click one of the paths to this panel. For example Servers > Server Types > WebSphere application servers > server_name
> [Additional Properties] Reliable messaging state > Runtime > Outbound sequences . The runtime state of each of the outbound sequences for the current scope is displayed in the Outbound sequence collection form.
We can use a filter to look at sequences that are in a particular state. For example, the state Cannot contact the remote endpoint indicates that the sequence has been established but the reliable messaging destination has stopped acknowledging messages (which, coupled with a high number of messages awaiting transmission, might indicate a potential problem). If the sequence status is Error, there is a problem with the sequence and the server hosting the other end of the sequence has terminated it.
We can select one or more sequences, and use one of the buttons provided to export the messages to compressed files, to close or terminate the selected sequences, or to delete the selected sequences and all their messages. For more information about deleting sequences, see Delete a failed WS-ReliableMessaging outbound sequence.
Attention: Delete or terminate sequences only if necessary. If you delete or terminate an active sequence, the resulting messaging behavior is unpredictable and can cause loss of messages. If you are not sure whether you can safely delete or terminate a sequence, do not delete or terminate it; the system automatically deletes sequences that have been inactive for 12 hours.
To see more detailed information about a particular sequence, click the Sequence identifier field. The Outbound sequences settings form is displayed. This detailed information includes addressing information to help you identify the server at which the sequence is targeted. From this panel you can also display the following forms:
- The Outbound message collection form.
(The messages on the outbound sequence. We can use this form to delete individual messages.)
- The Message settings form. (The contents of an individual message in the sequence.)
- The Message settings form. (The contents of an individual message in the sequence.)
For more guidance on diagnosing problems with outbound sequences, see Diagnosing and recovering a WS-ReliableMessaging outbound sequence that is in retransmitting state.
- The Outbound message collection form.
(The messages on the outbound sequence. We can use this form to delete individual messages.)
Related
WS-ReliableMessaging sequence reallocation
Diagnosing the problem when a reliable messaging source cannot deliver its messages
Diagnosing and recovering a WS-ReliableMessaging outbound sequence that is in retransmitting state
Delete a failed WS-ReliableMessaging outbound sequence
WS-ReliableMessaging troubleshooting tips
WS-ReliableMessaging
Add assured delivery to web services through WS-ReliableMessaging
Use HPEL to troubleshoot applications
Configure WS-SecureConversation to work with WS-ReliableMessaging
Related
WS-ReliableMessaging - requirements for interaction with other implementations
WS-ReliableMessaging - administrative console panels
WS-ReliableMessaging: supported specifications and standards