Troubleshooting: Introduction

Overview of ways to get help with troubleshooting Policy Manager, Network Director, and Policy Manager for DataPower.

Valid in version: 7.0

Troubleshooting PM Troubleshooting ND Database Queries

Table of Contents

  1. Overview
  2. Documentation Summary
  3. Customer Support
  4. Monitoring Tabs: Alerts and Logs
  5. Log Files
  6. stdout.txt File
  7. Monitoring Tool
  8. Restarting the Container: General Information
  9. Knowledge Base
  10. Release Notes
  11. Product Documentation

Overview

Over the course of using Policy Manager, Network Director, Policy Manager for DataPower, or other Akana products, you're likely to have to do some troubleshooting from time to time. Due to the relationship between the various containers and a selection of different type of web services, security, databases, and networks, there is a wide range of issues that might occur.

These products are deployed in a wide range of environments, and interface with a wide selection of products and versions, including operating systems, databases, servers, firewalls, security mechanisms, and others. It is important to take an orderly approach to installation, deployment, and troubleshooting.

If you encounter errors, check this publication and the resources referenced here.

This document also includes information about contacting technical support and the support process. Finally, it includes some resource material such as firewall settings and database queries for your reference.

back to top

Documentation Summary

The table below provides a summary of the information in this troubleshooting documentation and how it is organized.

This section... Provides this information...
Introduction General information about information resources available, information about working with Support, general information about basic troubleshooting tools.
Troubleshooting Policy Manager Troubleshooting information for Policy Manager
Troubleshooting Network Director Troubleshooting information for Network Director
Reference: Database Queries Examples of database queries you can use to resolve issues or optimize.

back to top

Customer Support

This section provides information about working with Akana technical support, including:

Contacting Technical Support

If you experience an issue with an Akana product, you can contact Akana Technical Support. Akana offers a variety of support services by email and phone. Support options and details are listed in the table below.

For the latest information on how to contact Technical Support, go to the Akana docs site, http://docs.akana.com/. In the footer of every page is a link to Customer Support. On that page, you'll see the contact numbers and email addresses listed. Use the information in the Akana section.

When sending an email to Customer Support, please provide the following information:

  • Your name
  • Email address
  • Subject
  • As much information as possible regarding your question

You'll receive an acknowledgment email and one of our support staff will contact you.

Logging a Support Ticket

For information about logging a support ticket, go to the Akana documentation site, http://docs.akana.com/, and follow the Customer Support link in the page footer.

The page includes information about how to file a support request, email Technical Support, or call.

When you log a support ticket, provide clear and specific details about the issue you are having, with as much background information as possible. Include:

  • Product version and update
  • Database version
  • Operating system
  • A clear subject and description of the issue and, if possible, steps to reproduce your issue so that Support can troubleshoot it more effectively.
  • The appropriate log files based on the type of issue being reported. Screen captures are also helpful.
  • If applicable, other related files such as the WSDL file for an API service.

Support Tickets: Customer Responsibilities

When logging a support ticket, please bear in mind these additional points and customer responsibilities:

  • Please make sure that the issue is related to the Akana product. In some cases, issues are caused by other factors such as network, firewall, or security certificates.
  • In case of a Production Critical issue, you can contact Akana Technical Support immediately and one of our knowledgeable support staff will help you troubleshoot your problem and collect information for further diagnosis. If you are reporting the issue by email, specify in the subject line that it is Production Critical. A production critical issue is defined as follows:
    • Actual or potential complete failure of traffic on a critical route due to failure of a system or network element.
    • Complete or partial loss of visibility/control of network elements.
    • Loss or impairment of control/monitoring equipment.
  • Document the scenario/steps to reproduce the issue. If it's not possible to reproduce the issue, explain what was happening at the time you experienced the issue and what then occurred.
  • Provide the appropriate log files from all Akana containers that are involved in the request flow.
  • Collect any other information that you think will be useful for Akana engineers to understand and troubleshoot the issue.
  • Report the issue to Akana Technical Support using one of the options listed earlier in this section.

Notes for Support Customers

  1. For the response time and actions taken based on ticket priority, refer to the Response Times table in the general Support Policy section of the Support Site.
  2. If you urgently need a quick response (for example, in the case of a Production Critical issue), please call Akana Technical Support, or submit a ticket and indicate it on the ticket.
  3. If screen sharing or an online session is needed, please specify this in the ticket so that Akana Technical Support can be prepared.
  4. In the case of screen sharing or an online session, Akana Technical Support might need to control the console to demonstrate how to resolve the issue.
  5. If you allow Akana Technical Support to access your system directly, remember to also provide the needed access information such as VPN or authentication information.

back to top

Monitoring Tabs: Alerts and Logs

Monitoring information, including alerts and logs, is available at the following levels:

At each level, a monitoring tab gives you access to alerts, logs, and other information so that you can view the state of functions in real time.

Organization Monitoring Tab

The highest level of monitoring information is available via the monitoring tab for an organization. This lets you view all logs and alerts sent by services and sub-organizations within the organization you are viewing.

This tab includes three types of alerts:

  • Service Alerts
  • SLA Alerts
  • Container Alerts

If there is an error with one of your services, the monitoring tab is a good place to look first, to see if the alerts and log entries can help you identify the problem.

An example of the monitoring tab for an organization is shown below.

Monitoring tab for an organization

Service-Level Monitoring Tab

Each service also has its own monitoring tab, with alerts and logs relating only to that service and its operations, as shown below.

If the basic auditing policy is being used, the Monitoring -> Logs tab also shows usage data for the service. However, as a best practice this should only be used while troubleshooting or in non-production environments as the payload data is stored in the database.

Service-level monitoring tab

If the detailed auditing policy is being used, you can also view the request and response payload in the Logs tab. Double-click a specific message to see the Usage Data Details overlay. This includes usage detail, recorded messages, and transaction events. In the Recorded Messages tab you can see the individual request and response message. You can also choose to view Raw Format, which includes the HTTP headers. An example is shown below.

viewing request and response in the Logs tab

Monitoring Tab for the Container

If there is an issue with a specific container, alerts are displayed in the container's monitoring tab as well. You also see the container alerts when you log in to the Policy Manager console.

The example below shows the monitoring tab for a container.

Monitoring tab for a container

In some cases the information on the monitoring tab can help you discover a deeper error occurring within the container or service.

The next step in troubleshooting an instance is to make use of the logging system.

Monitoring Tab for the Contract

A monitoring tab is also available for each contract, giving access to the logs applicable to the contract.

back to top

Log Files

By default, Policy Manager and Network Director only log errors (exceptions) that happen over the course of normal usage. If you are having any runtime processing errors or issues while performing some action in the Policy Manager console, applicable errors will generally be logged in the log file for the applicable container.

This section includes the following information about log files:

Note: There is another type of log that you can enable if needed. In the Policy Manager Administration Console, Configuration tab, choose the configuration category of com.soa.transport.jetty and enable the NCSA Access log (set the ncsa.access.log.enable property to true). Then, in the ncsa.access.log.filename field, specify the location for the log file. After that, access to any page in the Policy Manager Console or the Akana Administration Console for the Policy Manager instance generates an entry to the specified log file.

File Location

Each instance has its own set of logs at the following default location:

{installation directory}/sm60/instances/{instance name}/log

The default behavior for the logging system is to have a maximum of ten backup logs at 4.7 MB (5000000 bytes) each. When a log reaches 4.7 MB in size, the logging information rolls over into the next file. Once the total number of log files reaches 10, the oldest file is deleted when the new one starts.

Modifying the Default Logging Behavior

You can modify the default settings for logging behavior, along with the level of logging and other customization, in the Akana Administration Console for the Policy Manager and Network Director instance.

Modifying the default logging behavior

To modify the default logging behavior
  1. Log in to the Akana Administration Console for the Policy Manager or Network Director instance.
  2. Click the Configuration tab.
  3. From the configuration categories on the left, find com.soa.log.
  4. In the properties panel on the right, the two properties below control the number of backups and/or the maximum size for each log file. Modify as needed:
    • log4j.appender.FILE.MaxBackupIndex: the number of backup files that are kept
    • log4j.appender.FILE.MaxFileSize: the maximum size for each file
  5. Click Apply Changes.

Turning Trace Logging On

If a problem with a container persists, you could enable trace logging in the Akana Administration Console. Trace logging is enabled dynamically and does not require a container restart.

Depending on the category for which trace logging is enabled, detailed information is collected in the log file, including such activity as:

  • Internal communication between Akana containers
  • Database queries
  • Incoming requests
  • Certificate information
  • Scheduled jobs

When the troubleshooting is complete, trace logging for the specific category should set back to the default setting of error.

A good practice is to figure what action is causing specific symptoms in the container, and turn on trace logging only while that action is occurring. For example, if a service detail page is coming up blank, you might want to see what Policy Manager is doing when you click on the service detail page. You would set the logging level to trace, click on the service detail page, and then change the level back to error and analyze the logs.

To turn trace logging on or off
  1. Log in to the Akana Administration Console for the Policy Manager or Network Director instance.
  2. Click the Configuration tab.
  3. From the configuration categories on the left, find com.soa.log.
  4. In the properties panel on the right, modify this property to enable or disable trace for all runtime activity on the container:
    • To enable: log4j.category.com.soa: Switch from ERROR to TRACE
    • To disable: log4j.category.com.soa: Switch from TRACE to ERROR
  5. Click Apply Changes.

back to top

stdout.txt File

If there is an issue with the bundles not starting, you can check the stdout.txt file to get additional information for troubleshooting purposes.

This file is created whenever the container starts up. It is stored in the instances folder (instances/{container name}/log/stdout.txt).

Normally the file contains a one-line message stating that the bundles have started. However, if the bundles fail to load, the errors that occur during the container initialization process are recorded in this file. Errors relating to bundles loading do not appear in the Policy Manager log files, since logging of messages starts when the container has started.

back to top

Monitoring Tool

All Policy Manager 6.x containers include an optional Monitoring Tool to help troubleshoot issues related to the container resources. It is not installed by default but you can easily install it. You can use this tool to monitor and analyze the following:

  • Incoming HTTP connections (com.soa.transport.httpclient)
  • Database thread pool (com.soa.database.config.{db-config-id}-mon)
  • Active/idle Policy Manager processes (com.soa.framework)
  • Container memory usage (com.soa.vmstats)
  • Outgoing HTTP connections (com.soa.transport.jetty)
  • Monitoring queues (com.soa.usage)
  • JMS connections (com.soa.transport.jms)
To install the monitoring tool
  1. Log in to the Akana Administration Console for the Policy Manager or Network Director instance.
  2. Click the Available Features tab.
  3. From the Filter drop-down list at the top of the left panel, choose Tool.
  4. Click the checkbox for the Akana Admin Monitoring Tool and click Install Feature.
  5. Restart the container.
  6. After restart, verify that the Monitoring tab is now present in the Akana Administration Console, as shown below.

    Making sure the Monitoring tab is present

Note: This tool does not require additional machine or container resources to run. Before closing the tool, set the polling interval to 0.

back to top

Restarting the Container: General Information

Some types of changes that you might make will require restarting of the container before the changes go into effect. Other types of changes are effective immediately, without restarting the container.

In most cases, specific procedures and issue resolution notes in this document state whether you need to restart the container or not. In general, configuration changes do not require restart unless they include changes to the container listener or database. If you add or remove container features you'll need to restart the container for the changes to go into effect.

Examples of changes that require restart:

  • Adding the monitoring tool in the Akana Administration Console for the Policy Manager instance
  • Changing database properties such as username, password, or hostname
  • Changing the port number for the container listener (for Policy Manager versions 6.0 and prior)

Examples of changes that do not require restart:

  • Increasing the log level to TRACE
  • Adding an HTTP route configuration file to the /instances/ {ND}/deploy folder
  • Adding an identity system such as LDAP to the Policy Manager Workbench
  • Changing the port number for the container listener (for Policy Manager version 6.1)

Determining Where to Look for Error Information

When trying to narrow down information for troubleshooting purposes, it might be useful to know what symptoms are likely to relate to which container types.

You might find info about these types of errors... In this location...
Issues with Policy Manager (for example, usage writer or container configuration), user interface issues, search results, and some database issues.

Policy Manager log files.

These types of issues are generally a problem with the Policy Manager instance.

404 when invoking a service, bad context paths, virtual service authentication errors, authorization errors, or routing issues.

Network Director log files.

Possibly also Policy Manager log files.

These issues are likely to relate to the Network Director. However, since the Network Director communicates with the Policy Manage to retrieve information, in many cases the Policy Manager logs are helpful as well.

Container initialization.

stdout console or the stdout file.

Any errors that occur during the container initialization process are written to stdout.

back to top

Knowledge Base

The Rogue Wave Akana knowledge base, https://library.roguewave.com/display/SUPPORT/Akana+-+Knowledgebase, includes many type of information such as:

  • How-to articles
  • Product compatibility matrices
  • Troubleshooting articles

back to top

Release Notes

It's possible that you could encounter a bug that might have been resolved in a later version of the product. For this and other reasons, it's a good idea to check the release notes for versions later than yours.

The release notes for each product version include information about the bugs/issues that have been fixed in that version, as well as information about new product features and enhancements. You might find that the problem you encountered was resolved in a later version.

To view release notes
  1. Log in to the Rogue Wave Support Center (https://library.roguewave.com).

    Note: If you don't have a login for the site, contact Akana technical support to get access.

  2. In the My Resources section, click Product Downloads, then click Akana - Product Downloads (https://library.roguewave.com/display/SUPPORT/Akana+-+Product+Downloads).
  3. Find the release notes and download the file.

back to top

Product Documentation

When you download your installation executable files, make sure you get and read the product documentation. The documentation for each product includes general information about installation and often includes troubleshooting information for the specific product.

Updates to documents are available from time to time on the Support site.

Back to top