Troubleshooting the proxy server

This topic helps you to solve problems that you might encounter with your proxy server.

Overview

Proxy server errors are logged in the SystemOut.log , proxy.log, or local.log files. Consult the following list if you are having problems with your proxy server.

Procedure

• The proxy server was created successfully, but I am unable to start it. Check the SYSOUT file for port conflicts. Use the netstat –a command to see if any of the endpoints that are associated with the proxy server are already being used. You can find the ports in the administrative console by clicking Servers > Proxy servers > <server_name> > Ports.If the proxy server fails to start when attempting to start it as a non-privleged user on UNIX systems, check for the following message in the logs:

  ChannelFramew E   CHFW0029E: Error initializing chain HTTPS_PROXY_CHAIN because of 
  exception 
  com.ibm.wsspi.channel.framework.exception.RetryableChannelException: Permission 
  denied
  TCPPort       E   TCPC0003E: TCP Channel TCP_7 initialization failed.  The socket 
  bind failed for host * and port 80.  The port may already be in use.
  

  Change the ports of the PROXY_HTTP_ADDRESS and PROXY_HTTPS_ADDRESS transport chains to values greater than 1024.

• The proxy server routes requests to the Web container over an administration port. The proxy server is located in front of several Web containers. The configuration requires that the Web containers listen to the non-default ports such as 9061, 9081, and so on. This scenario is the default case when multiple application servers are on the same machine, which forces new and different ports to be used in the configuration. In this scenario, the proxy server might route an application request to the Web container over the administration port of 9061, instead of using the expected port of 9081.

  Add the listening port numbers of the Web container to the virtual host that is associated with the target application. This process will ensure that the proxy server routes the request to the Web container over the correct port number.

• The proxy server started, but I am unable to access the application resources through the endpoints for the proxy server. Ensure that the endpoints for the proxy server are among the host aliases in the virtual host that are associated with the application.

• The proxy server routes to another core group. Verify that core group bridges exist between the core groups in the cell, and that the processes that are chosen to be bridges are restarted. If there is a firewall between the core group, verify that the correct ports are open for core group bridge traffic.

• The proxy server routes to another cell. Review the core group bridge settings. Verify that the peer access point group names match in each cell. Check the peer ports against the bridge interfaces to verify that they are correct. If bridge interfaces or peer ports are added or changed, restart all bridge interfaces.

• Receiving a blank page when making a request to the proxy. Consider the following actions:

  • Update the virtual host. Ensure that the target application and routing rule are assigned to a virtual host that includes the proxy server listening ports (default: HTTP 80, HTTPS 443). Add the proxy server listening ports to the application, or routing rule virtual host, or use the proxy_host virtual host.

  • Stop the conflicting process. Check your system to ensure that no other process (for example, Apache, IBM HTTP Server, and so on) is running that uses the proxy server ports (default: HTTP 80, HTTPS 443). If this problem occurs, the proxy server seems to start normally, but is unable to receive requests on the affected listening port. Check your system as follows:

    1. Stop the proxy server.

    2. Query your system using netstat and ps commands to determine if an offending process is using the port on which the proxy server is listening.

    3. If an offending process is found, stop the process and configure your system so that the process is not started during system startup.

  • Enable proxy routing. Ensure that proxy routing is enabled for the Web module of the application. Proxy routing is enabled by default, so if no proxy properties are modified, disregard this solution. Otherwise, see Customizing routing to applications for instructions on modifying the proxy properties.

  • Test direct request. Ensure that the target application is installed by making a request directly to the application server. If a response is not received, then the problem is with the application server and not the proxy server. Verify this case by going through the proxy server after you can receive a response directly from the application server.

• HTTP 404 (File not found) error received from the proxy server. Consider the following actions:

  • Update the virtual host. Ensure that the target application and routing rule are assigned to a virtual host that includes the proxy server listening ports (default: HTTP 80, HTTPS 443). Add the proxy server listening ports to the application, or routing rule virtual host, or use the proxy_host virtual host.

  • Enable proxy routing. Ensure that proxy routing is enabled for the Web module of the application. Proxy routing is enabled by default, so if no proxy properties are modified, disregard this solution. Otherwise, see Customizing routing to applications for instructions on modifying the proxy properties.

  • Test direct request. Ensure that the target application is installed by making a request directly to the application server. If a response is not received, then the problem is with the application server and not the proxy server. Verify this case by going through the proxy server when you can receive a response directly from the application server.

• Unable to make Secure Sockets Layer (SSL) requests to application or routing rule. Ensure that the virtual host of the application or routing rule includes a host alias for the proxy server SSL port (default: 443).

• Unable to connect to the proxy server...request times out. Stop the conflicting process. Check your system to ensure that no other process (for example, Apache, IBM HTTP Server, and so on) is running that uses the proxy server ports (default: HTTP 80, HTTPS 443). If this situation occurs, the proxy server seems to start normally, but is unable to receive requests on the affected listening port. Check your system, as follows:

  1. Stop the proxy server.

  2. Query your system using netstat and ps commands to determine if an offending process is using a port on which the proxy server is listening.

  3. If an offending process is found, stop the process and configure your system so that the process is not started during system startup.

• Did not receive a response from the error page application when the HTTP error occurred (for example, 404). Ensure that the error page URI is entered correctly. Also, make sure that the Handle remote errors option is selected if you are handling HTTP error responses from back-end servers. For more detailed information, refer to Overview of the custom error page policy and the custom error page policy section of Proxy server settings.

• What packages do I enable when tracing the proxy server? All of the following packages are not needed for every trace, but if unsure, use all of them:

  • *=info

  • WebSphere Proxy=all

  • GenericBNF=all

  • HAManager=all

  • HTTPChannel=all

  • TCPChannel=all

  • WLM*=all

  • DCS=all

  • ChannelFrameworkService=all

  • com.ibm.ws.dwlm.*=all

  • com.ibm.ws.odc.*=all

• How do I enable SSL on/off load? SSL on/off load is referred to as the transport protocol in the administrative console, and transport protocol is a Web module property. Refer to Customizing routing to applications to see how to configure Web module properties. No SSL on/off load or transport protocol properties exist for routing rules because the transport protocol is inherent to the generic server cluster that is specified in the routing rule.

• When fronted by IBM HTTP Server or a plug-in, how do I configure the proxy server so I do not have to add a port for it to the virtual host? For the proxy server to trust the security-related information, for example WebSphere Application Server private headers, of a request, add the originator of the request to the proxy server trusted security proxies list. For example, add an IBM HTTP Server or a plug-in sending requests to the proxy server to the proxy server trusted security proxies list. The plug-in sends WebSphere Application Server private header information that among other things, contains the virtual host information of a request. If the proxy does not trust the WebSphere Application Server private headers from the plug-in (or any client), the proxy server adds its own WebSphere Application Server private headers, which requires the addition of proxy server ports (HTTP and HTTPS) to the virtual host. Most likely, when using the plug-in with the proxy server, the intent is to use the proxy server as a back-end server. Be sure to add the WebSphere Application Server plug-in as a trusted security proxy to avoid having to expose the proxy server ports. Refer to Routing requests from a plug-in to a proxy server for more information about configuring the WebSphere Application Server plug-in to use with the proxy server. Refer to Proxy server settings for more information about trusted security proxies.

• The proxy server seems to "hang" under stress, or "Too Many Files Open" exceptions display in ffdc or SystemErr.log. Under high connection loads, the number of file system descriptors might become exhausted and the proxy server may seem to hang and drop "Too Many Files Open" exceptions in the ffdc directory or in the SystemError.log file because it is unable to open a socket. The problem can be alleviated by setting certain tuning parameters at the operating system level and at the proxy server level that optimize the use of connections for the proxy server:

  • Proxy server tuning

    • Persistent requests - A persistent request is one that is sent over an existing TCP connection. You can maximize performance by increasing the number of requests that are received over a TCP connection from a client. The value should represent the maximum number of embedded objects, for instance GIF and so on, in a Web page +1.

      View or set this value in the WebSphere Application Server administrative console by clicking Servers > Proxy Servers > server_name > Proxy server transports > HTTP_PROXY_CHAIN/HTTPS_PROXY_CHAIN

      Default value	100
      Recommended value	A value that represents the maximum number of embedded objects in a Web page + 1.

    • Outbound connection pool size - The proxy server pools outbound connections to target servers and the number of connections that resides in the pool is configurable. If the connection pool is depleted or empty, the proxy server creates a new connection to the target server. Under high concurrent loads, increase the connection pool size should to a value of the expected concurrent client load to achieve optimal performance.

      View or set this value in the WebSphere Application Server administrative console by clicking Servers > Proxy Servers > server_name > HTTP Proxy Server Settings. In the Content Server Connection section, increase the maximum connections per server field to a value that is equal to or greater than the expected maximum number of connected clients. Save your changes, synchronize the changes to the proxy server node, and restart the proxy server.

      Recommended value

      Value consistent to the expected concurrent client load.

    • Outbound request time-out - Often times, the back-end application servers that are fronted by the proxy server may be under high load and may not respond in an adequate amount of time, therefore the connections on the proxy server may be tied up from waiting for the back-end application server to respond. Alleviate this by configuring the amount of time the proxy server waits for a response from the target server. This is the Outbound Request Time-out value. By managing the amount of time the proxy server waits for a slow back-end application server, connections are freed up faster and used for other request work.View or set this value in the WebSphere Application Server administrative console by clicking Servers > Proxy Servers server_name > HTTP Proxy Server Settings. In the Content Server Connection section, set Outbound Request Time-out to a value that represents the acceptable response time from the point of view of the client.

      Default value	120
      Recommended value	A value that represents the acceptable response time from the point of view of the client.

• HMGR0149E: An attempt by another process to connect to this process via the core group transport has been rejected. The connecting process provided a source core group name of network1server1a.com9353network2server2a.com9353, a target of <null> a member name of 10.42.45.18:9353 and an IP address of /10.42.45.18. If you receive this error message, your application requires security between the Web modules and EJB modules. You must configure WebSphere Application Security by doing the following:

  1. Review the time, date, and time zone on all machines in both cells. The machines in the primary and backup cells must be within five minutes of each other.

  2. Configure a user registry with an LDAP server, OS security or a custom user registry.

  See the topic entitled Exporting Lightweight Third Party Authentication keys for more information.

Related information