Detecting expensive HTTP threads

The HTTP Connection Handler can detect expensive threads that spend an unusually long time processing HTTP requests. This helps identify potential performance bottlenecks and responsiveness issues in the PingAuthorize Server.

When request processing takes longer than a configured threshold, the PingAuthorize Server can automatically generate a thread dump. A thread dump is a snapshot of all active application threads, indicating whether each thread is actively running, blocked waiting for a resource, or waiting for another task to complete. Thread dumps also include the sequence of method calls each thread is currently executing.

This diagnostic information is helpful for troubleshooting slowdowns caused by issues such as:

Delays in connecting to external data sources
Long-running policy evaluations
Resource contention or deadlocks
Infrastructure problems affecting responsiveness

Analyzing a thread dump enables you to pinpoint the specific operations responsible for performance degradation.

How it works

The PingAuthorize Server uses the following process to detect expensive threads:

At a regular interval, the server checks the status of all active HTTP threads.
If a thread is still handling the same request it was processing during the previous check, the server flags that thread as expensive.
If the number of concurrently expensive threads meets a configured threshold, the server creates a thread dump.
Each thread dump is saved as a separate file in the /logs/thread-dumps directory. To avoid excessive logging, the server waits for a configured interval before creating another thread dump.

Thread dump files are named with the following format:

expensive-operation-dump-<timestamp>.log

The thread dump file path and naming format aren’t configurable.

Configuring expensive thread detection

The HTTP Connection Handler provides the following settings for configuring expensive thread detection:

Setting name dsconfig parameter name Description Default value

Expensive Thread Check Interval

expensive-thread-check-interval

Specifies how frequently the server checks thread activity. If a thread is processing the same request for two consecutive checks, the server flags it as expensive.

You must specify a non-zero value for this setting to enable expensive thread detection.

0 ms

Expensive Thread Minimum Concurrent Count

expensive-thread-minimum-concurrent-count

Specifies the minimum number of simultaneously expensive threads required to generate a thread dump.

1

Expensive Thread Hold Off Interval

expensive-thread-hold-off-interval

Specifies the cooldown period the server waits after creating a thread dump before it can create another. This setting helps prevent excessive disk usage.

60000 ms

You can configure expensive thread detection with the administrative console or with the dsconfig command.

Admin console
dsconfig

Steps

Go to System > Connection Handlers.
Click HTTP Connection Handler or HTTPS Connection Handler.
Scroll down and configure the following:
- Expensive Thread Check Interval
- Expensive Thread Minimum Concurrent Count
- Expensive Thread Hold Off Interval

Steps

Run the dsconfig set-connection-handler-prop command with the following arguments:

dsconfig set-connection-handler-prop \
  --handler-name "HTTP Connection Handler" \
  --set "expensive-thread-check-interval:<time-interval> ms" \
  --set "expensive-thread-minimum-concurrent-count:<thread-count>" \
  --set "expensive-thread-hold-off-interval:<time-interval> ms"

You must restart the server or the HTTP Connection Handler for the expensive thread detection settings to take effect.

Analyzing thread dump files

Each thread dump file is organized into three main parts:

Header summary: Provides context about why the thread dump was generated. It includes the name of the detector that triggered the dump, the number of expensive threads identified, and the time interval over which they were detected.
Expensive thread list: Lists the threads flagged as expensive immediately after the summary. Each entry includes a thread ID, thread name, and correlation ID to help trace related request activity across other logs.

Save the thread IDs for use during analysis.
Full stack trace: Shows a complete snapshot of all threads running in the PingAuthorize Server’s Java process at the time of the thread dump. This section provides detailed request context to help identify performance bottlenecks or blocked threads.

Use the thread IDs from the expensive thread list to locate and analyze each expensive thread’s state.

For example, the following header summary and expensive thread list identify a single expensive thread, detected over an interval of 5000 milliseconds:

HTTP Connection Handler :80 worker Expensive HTTP Thread Detector (10.3.0.0-20250611152739.000Z-364a3822) detected that 1 threads were found to be processing the same HTTP request over an interval of at least 5000 ms:
* Thread id=176 name='HTTP Connection Handler :80 worker 6 (10.3.0.0-20250611152739.000Z-364a3822) requestID=0 correlationID="3bf39de1-7bc4-4937-b99a-62785a41f55d"

The following is a truncated version of the expensive thread’s stack trace entry:

"HTTP Connection Handler :80 worker 6 (10.3.0.0-20250611152739.000Z-364a3822)" priority=5 id=176
   java.lang.Thread.State: WAITING
        at java.base@17.0.10/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@17.0.10/java.util.concurrent.locks.LockSupport.park(LockSupport.java:211)
        at java.base@17.0.10/java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1864)
        at java.base@17.0.10/java.util.concurrent.ForkJoinPool.unmanagedBlock(ForkJoinPool.java:3465)
        at java.base@17.0.10/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3436)
        at java.base@17.0.10/java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1898)
        at java.base@17.0.10/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2072)
        at app//com.pingidentity.authorize.decisionengine2.evaluation.effects.Inner.get(Inner.java:32)

A thread state of WAITING indicates the thread is idle, waiting indefinitely for another thread to perform an action. Learn more about thread states in the Enum Thread.State reference in the Java documentation.

Thread dump file management

The PingAuthorize Server doesn’t automatically manage the thread dump files it creates. You must implement your own retention policy for these files, including any log rotation, compression, or cleanup.

In a containerized deployment, make sure a mechanism is in place, such as a volume mount, to persist thread dump files and make them available for analysis outside the container.