Problem determination: Debugging HTTP communication using Tracing, HTTP error, and NCSA access log settings

During your tenure as a WebSphere Application Server Administrator or support engineer, you will find yourself needing to help other teams debug applications. JEE provides many frameworks which can be used. Some application are clients requesting resources from the WAS Web container, others use both the ore traditional HTTP POST and GET options. Either way, whether your deployed application contains web services or simple servlet you will need to use the following WAS features to ensure you have enough logging to provide enough information which can be aid problem determination i.e. finding the root cause of a problem.

Tracing

1. To turn on tracing, navigate to the Troubleshooting session located in the left-handed-side of the Administrative console and click the Logs and trace link.

Image may be NSFW.
Clik here to view.

1. You will then presented with a list of servers that exist in you Cell (is using WAS ND). Note if you are using WAS base then you would only see one server.
   

Image may be NSFW.
Clik here to view.

1. Click on the appropriate server for example server1
   
2. Once you are presented with the Logging and Tracing screen, select Change Log Details Levels link.
   

Image may be NSFW.
Clik here to view.

1. Within, the Change Log Detail Levels screen you have two tabs. The configuration and Runtime tables.To add tracing immediately to the running JVM you can use the Runtime table. Within the runtime tab there is an option to save runtime changes to the configuration as well. When you turn on tracing it can affect performance, so if it is just a quick debug session and you cannot restart the server, you can use the runtime tab. If you want the trace.log to continually log data over time for retrospective analysis.The Configuration tab is essentially the same accept when you apply the tracing settings (Log detail levels) then it will saved to the Was configuration and will still be enabled upon the JVM being restarted,

Image may be NSFW.
Clik here to view.

1. To set appropriate logging levels, you can select the appropriate components by expanding the groups available.In the case of debugging most HTTP issues you can enables the Finest logging level for the following:com.ibm.ws.http.HttpConnection=finest:
   com.ibm.ws.http.HttpRequest=finest:
   com.ibm.ws.http.HttpResponse=finest

Image may be NSFW.
Clik here to view.

1. Click Apply.
   
2. If you have used the runtime option, then the immediate result will a new log file called trace.log which wil be located the JMV’s log directory which is normally <was_root>/profiles/<[profile_name>/logs/<server_name>/trace.log
   

NOTE: WAS logs can be located where ever you wish, however they have to be configured to do so. How to do this is covered in another module.

NCSA Logging

The next level of HTTP logging which can be useful is the NCSA logging approach. This logging is similar to IBM HTTP logs (Apache style logs) and is how we can view GET and POST requests to the WAS Web Container.

1. To view this administrative console page, click Servers | Server Types | WebSphere application servers | <server_name>
   

Image may be NSFW.
Clik here to view.

1. Once you have opened the server configuration page. Under Troubleshooting, click NCSA access and HTTP error logging. This console page has separate sections for each type of logging.
   

Image may be NSFW.
Clik here to view.

1. Within the NCSA access and HTTP error logging screen there are few settings. The most important thing to do is turn on Enable logging service at serer start up if you wish to keep logging running.
   
2. If you are happy for the logs to be created in the same folder as the JVM logs then all you need to do is enable Access Logging and Error Logging. These two settings are immediate.
   

NOTE: If you anticipate large logs it is recommend that you locate all your WAS logs on a separate file system that WAS runtime binaries, this ensure that when the logs fills up the file system Was can still keep running.

Image may be NSFW.
Clik here to view.

The HTTP error log contains a record of HTTP processing errors that occur. The level of error logging that occurs is dependent on the value that is selected for the Error log level field.

The NCSA access log contains a record of all inbound client requests that the HTTP transport channel handles

All of the messages that are contained in a NCSA access log are in NCSA format.

Options:

Avoid trouble: The settings for any of these logs can also be modified on the settings page for a specific HTTP inbound channel. Any changes that you make on the HTTP inbound channel settings page only apply to that specific inbound channel. and override any global configuration settings that you specify on this page.

Enable logging service at server start-up

Select this option if you want any of the following logging to start when the server starts:

• NCSA access logging
  
• HTTP error logging
  

Avoid trouble: Even if you select this option, you must explicitly enable the type of logging that you want to occur on this page and on the settings page for the HTTP transport channel for which you want that type of logging to occur.

Enable NCSA access logging

When selected, a record of inbound client requests that the HTTP transport channel handles is kept in the NCSA access log.

NCSA access log file path

Specifies the directory path and name of the NCSA access log. Standard variable substitutions, such as $(SERVER_LOG_ROOT), can be used when specifying the directory path.

NCSA access log maximum size

Specifies the maximum size, in megabytes, of the NCSA access log. When the content of the NCSA access log reaches the specified maximum size limit, a log_name.1 archive log is created. The current content of the NCSA access log is then copied to this archive log.

The next time the content in the NCSA access log reaches the specified maximum log size, the content of the NCSA access log is again copied to the log_name.1 archive log. The copy process overwrites the current content of the archive file with the most current content of the NCSA access log.

Maximum number of historical files

Specifies the maximum number of historical versions of the NCSA access log file that are kept for future reference.

NCSA access log format

Specifies which NCSA format is used when logging client access information. If you select Common, the log entries contain the requested resource and a few other pieces of information, but does not contain referral, user agent, and cookie information. If you select Combined, referral, user agent, and cookie information is included.

Enable error logging

When selected, HTTP errors that occur while the HTTP channel processes client requests are recorded in the HTTP error log.

Error log file path

Specifies the directory path and the name of the HTTP error log. Standard variable substitutions, such as $(SERVER_LOG_ROOT), can be used when specifying the directory path.

Error log maximum size

Specifies the maximum size, in megabytes, of the HTTP error log. When the content of the HTTP error log reaches the specified maximum size limit, a log_name.1 archive log is created. The current content of the HTTP error log is then copied to this archive log.

The next time the content in the HTTP error log reaches the specified maximum log size, the content of the HTTP error log is again copied to the log_name.1 archive log. The copy process overwrites the current content of the archive file with the most current content of the HTTP error log.

Maximum number of historical files

Specifies the maximum number of historical versions of the Error log file that are kept for future reference.

Error log level

Specifies the type of error messages that are included in the HTTP error log.

You can select:

• Critical
  
  • Only critical failures that stop the Application Server from functioning properly are logged.
    
• Error
  
  • The errors that occur in response to clients are logged. These errors require Application Server administrator intervention if they result from server configuration settings.
    
• Warning
  
  • Information on general errors, such as socket exceptions that occur while handling client requests, are logged. These errors do not typically require Application Server administrator intervention.
    
• Information
  
  • The status of the various tasks that are performed while handling client requests is logged.
    
• Debug
  
  • More verbose task status information is logged. This level of logging is not intended to replace RAS logging for debugging problems, but does provide a steady status report on the progress of individual client requests. If this level of logging is selected, you must specify a large enough log file size in the Error log maximum size field to contain all of the information that is logged.
    

Transport Chains Configuration

After you configure the HTTP error logs and NCSA access logs, you must explicitly enable each type of logging on the settings page for the HTTP channels for which you want specific types of logging to occur.

To view the settings page for an HTTP channel, click Servers | Server Types | Application servers | <server_name> | Web Container Settings | Web container transport chains | HTTP inbound channel.

Image may be NSFW.
Clik here to view.

Image may be NSFW.
Clik here to view.

Image may be NSFW.
Clik here to view.

Image may be NSFW.
Clik here to view.

Now the HTTP logging is configured. You can use these features of WebSphere application to debug situations when you are hitting the WAS Web Container directly.