Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -390,7 +390,25 @@ default Duration getFailedIpCooldownPeriod() {
boolean isStrict302Handling();

/**
* @return the maximum time an {@link AsyncHttpClient} will keep connection in the pool, or negative value to keep connection while possible.
* The maximum age of a connection, or a value that is not positive (the default is negative) to keep connections
* for as long as they last. Resolution is milliseconds.
* <p>
* An HTTP/1.1 connection older than this is closed instead of being returned to the pool, and the pool cleaner
* closes one that passes it while idle in the pool; its age counts from the first time it was pooled. An HTTP/2
* connection is retired once it has been open for between 90% and 100% of this value, the spread keeping
* connections opened together from being replaced together. New requests then go to a new connection, and the
* retired one closes once the streams open on it have finished, along with the requests already waiting for one
* of its stream slots; a GOAWAY from the server fails those waiting requests, as it does on any connection. A
* request that reaches a retired connection it can no longer use is sent again on a new one. Retirement never
* interrupts an exchange in flight, so a long-lived response keeps its connection open past this age.
* <p>
* A retired connection no longer counts against {@code maxConnections} or {@code maxConnectionsPerHost}, so its
* replacement can open while its streams finish. The bound on requests waiting for a stream slot is per
* connection, a retired one included. Requests that arrive between the retirement and the registration of the
* replacement each open a connection of their own, as they do on the first request to a host and after a GOAWAY;
* all but one of those close after the request they were opened for.
*
* @return the maximum age of a pooled HTTP/1.1 or multiplexed HTTP/2 connection
*/
Duration getConnectionTtl();

Expand Down
22 changes: 12 additions & 10 deletions client/src/main/java/org/asynchttpclient/LoadBalance.java
Original file line number Diff line number Diff line change
Expand Up @@ -63,17 +63,19 @@ public enum LoadBalance {
* IPs, and a request pinned to any further IP cannot acquire a per-host permit — it multiplexes
* onto one of those K sibling connections (multiplexing onto an existing connection takes no
* permit) instead of opening another, failing with {@code TooManyConnectionsPerHostException}
* only if none is available. Those K connections are held open (kept alive by HTTP/2 PING), so
* they stay pinned to their K IPs rather than rotating across all of them; set
* {@code maxConnectionsPerHost} at least as large as the resolved-IP count for full per-IP
* spreading. The default is unlimited, so this bound only applies when a finite per-host limit is
* configured.</li>
* only if none is available. Those K connections stay pinned to their K IPs for as long as they
* are open, rather than rotating across all of them; {@code connectionTtl}, when set, retires them
* so that the set moves across the IPs over time. Set {@code maxConnectionsPerHost} at least as
* large as the resolved-IP count for full per-IP spreading. The default is unlimited, so this
* bound only applies when a finite per-host limit is configured.</li>
* <li>Since the per-host permit is held for the connection's lifetime, each live round-robin HTTP/2
* connection also occupies a global {@code maxConnections} slot until it closes, and pool
* {@code connectionTtl}/idle reaping do not apply (these connections live in the HTTP/2 registry,
* kept alive by PING, until GOAWAY or the connection drops). When a GOAWAY starts draining a
* connection its permit is released immediately, so a replacement can be opened without waiting
* for in-flight streams to finish.</li>
* connection also occupies a global {@code maxConnections} slot until it is retired or
* drained. These connections live in the HTTP/2 registry rather than the pool, so idle reaping
* does not apply: one stays open until the server closes it, a GOAWAY drains it, or it reaches
* {@link AsyncHttpClientConfig#getConnectionTtl() connectionTtl}, and HTTP/2 PING, when enabled,
* keeps an idle one alive. When a GOAWAY or {@code connectionTtl} retires a connection its permit
* is released immediately, so a replacement can be opened without waiting for in-flight streams to
* finish; until they do, the old and the new connection are both open.</li>
* <li>The address order comes straight from the configured
* {@link io.netty.resolver.InetNameResolver}; this mode does not re-sort it. For the
* rotation to map consistently across requests, use a resolver that returns the addresses
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,9 @@ public final class NettyResponseFuture<V> implements ListenableFuture<V> {
private static final AtomicIntegerFieldUpdater<NettyResponseFuture> ON_THROWABLE_CALLED_FIELD = AtomicIntegerFieldUpdater
.newUpdater(NettyResponseFuture.class, "onThrowableCalled");
@SuppressWarnings("rawtypes")
private static final AtomicIntegerFieldUpdater<NettyResponseFuture> HTTP2_REDISPATCHED_FIELD = AtomicIntegerFieldUpdater
.newUpdater(NettyResponseFuture.class, "http2Redispatched");
@SuppressWarnings("rawtypes")
private static final AtomicReferenceFieldUpdater<NettyResponseFuture, TimeoutsHolder> TIMEOUTS_HOLDER_FIELD = AtomicReferenceFieldUpdater
.newUpdater(NettyResponseFuture.class, TimeoutsHolder.class, "timeoutsHolder");
@SuppressWarnings("rawtypes")
Expand Down Expand Up @@ -118,6 +121,8 @@ public final class NettyResponseFuture<V> implements ListenableFuture<V> {
private volatile int contentProcessed;
@SuppressWarnings("unused")
private volatile int onThrowableCalled;
@SuppressWarnings("unused")
private volatile int http2Redispatched;
private volatile TimeoutsHolder timeoutsHolder;
@SuppressWarnings("unused")
private volatile @Nullable NettyResponseBodyControl responseBodyControl;
Expand Down Expand Up @@ -622,6 +627,17 @@ public boolean incrementRetryAndCheck() {
return maxRetry > 0 && CURRENT_RETRY_UPDATER.incrementAndGet(this) <= maxRetry;
}

/**
* Claims the one re-dispatch this exchange is allowed when the HTTP/2 connection it was bound for was retired at
* {@code connectionTtl} before a stream could be opened. Separate from the retry count: the request was never
* sent, so re-sending it is not a retry.
*
* @return {@code true} the first time only
*/
public boolean claimHttp2Redispatch() {
return HTTP2_REDISPATCHED_FIELD.compareAndSet(this, 0, 1);
}

/**
* Return true if the {@link Future} can be recovered. There is some scenario
* where a connection can be closed by an unexpected IOException, and in some
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@
import io.netty.util.concurrent.GlobalEventExecutor;
import io.netty.util.concurrent.ImmediateEventExecutor;
import io.netty.util.concurrent.Promise;
import io.netty.util.concurrent.ScheduledFuture;
import io.netty.util.internal.PlatformDependent;
import org.asynchttpclient.AsyncHandler;
import org.asynchttpclient.AsyncHttpClientConfig;
Expand Down Expand Up @@ -101,13 +102,15 @@
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.net.SocketAddress;
import java.time.Duration;
import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.Map;
import java.util.Map.Entry;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ThreadFactory;
import java.util.concurrent.ThreadLocalRandom;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.function.Consumer;
Expand Down Expand Up @@ -146,6 +149,9 @@ public class ChannelManager {
// initializer instead of via Bootstrap#option to avoid Netty's synchronized per-connect options map (issue #2218).
private final Map.Entry<ChannelOption<?>, Object>[] channelOptions;
private final long handshakeTimeout;
// connectionTtl in milliseconds, bounding the age of every HTTP/2 connection; not positive means unbounded,
// the same conversion and test DefaultChannelPool applies to HTTP/1.1 connections.
private final long http2ConnectionTtl;
private final @Nullable AddressResolverGroup<InetSocketAddress> addressResolverGroup;

private final ChannelPool channelPool;
Expand Down Expand Up @@ -217,6 +223,17 @@ public ChannelManager(final AsyncHttpClientConfig config, Timer nettyTimer) {
this.channelPool = channelPool;
openChannels = new DefaultChannelGroup("asyncHttpClient", GlobalEventExecutor.INSTANCE);
handshakeTimeout = config.getHandshakeTimeout();
// Null was never rejected, and only DefaultChannelPool dereferences it, so a client with its own pool or
// without keep-alive could be built with it.
Duration connectionTtl = config.getConnectionTtl();
long connectionTtlMillis;
try {
connectionTtlMillis = connectionTtl != null ? connectionTtl.toMillis() : -1;
} catch (ArithmeticException tooLongForMillis) {
// A Duration beyond Long.MAX_VALUE milliseconds (ChronoUnit.FOREVER's, say) is as good as unbounded.
connectionTtlMillis = -1;
}
http2ConnectionTtl = connectionTtlMillis;

// check if external EventLoopGroup is defined
ThreadFactory threadFactory = config.getThreadFactory() != null ? config.getThreadFactory() : new DefaultThreadFactory(config.getThreadPoolName());
Expand Down Expand Up @@ -1244,17 +1261,19 @@ public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception
int lastStreamId = goAwayFrame.lastStreamId();
Http2ConnectionState connState = ctx.channel().attr(Http2ConnectionState.HTTP2_STATE_KEY).get();
if (connState != null) {
connState.setDraining(lastStreamId);
Object pk = connState.getPartitionKey();
if (pk != null) {
removeHttp2Connection(pk, ctx.channel());
}
// Free the round-robin per-host permit at drain start instead of at channel close.
// A draining connection rejects new streams but can stay open while in-flight
// streams finish, and holding the permit that whole time blocks a replacement
// connection (issue #2214). Once-only: the closeFuture release becomes a no-op,
// and in DEFAULT mode no hook is installed so this does nothing.
// and in DEFAULT mode no hook is installed so this does nothing. Freed before the
// connection stops being offered: a request refused the permit tries it once more
// when it finds nothing to multiplex onto, and must find it free then.
connState.releasePermitOnce();
connState.setDraining(lastStreamId);
Object pk = connState.getPartitionKey();
if (pk != null) {
removeHttp2Connection(pk, ctx.channel());
}
// Fail requests still queued for a stream slot: a draining connection accepts no
// new streams, so they can never be opened here and would otherwise wait until the
// connection finally closes. Fail them now so they retry on a fresh connection (the
Expand All @@ -1280,6 +1299,55 @@ public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception
pipeline.addLast("http2-idle-state", new IdleStateHandler(0, 0, pingIntervalMs, TimeUnit.MILLISECONDS));
pipeline.addLast("http2-ping", new Http2PingHandler());
}

// Bound the connection's age by connectionTtl. Nothing else would: the registry has no idle reaper, and PING,
// when enabled, keeps an idle connection alive. Up to a tenth earlier than configured, never later, so that
// connections opened together (one per IP in LoadBalance.ROUND_ROBIN, or a fleet started at once) are not all
// replaced at the same instant every time.
if (http2ConnectionTtl > 0) {
long delayMillis = http2ConnectionTtl - ThreadLocalRandom.current().nextLong(http2ConnectionTtl / 10 + 1);
pipeline.addLast("http2-connection-ttl", new Http2ConnectionTtlHandler(delayMillis));
}
}

/**
* Retires an HTTP/2 connection that reached {@code connectionTtl}. It is no longer handed out, the streams open
* on it and the openers queued for it still run on it, and it closes once it has neither. Runs on the
* connection's event loop.
*/
private void retireHttp2Connection(Channel channel) {
Http2ConnectionState state = channel.attr(Http2ConnectionState.HTTP2_STATE_KEY).get();
if (state == null) {
return;
}
// As at GOAWAY: free the round-robin permit now, so a replacement can open while streams finish here. Before
// the registry removal: a request refused the permit tries it once more when it finds nothing to multiplex
// onto, and a poll that misses this connection must mean the permit is free again.
state.releasePermitOnce();
// Must leave the registry, not just be flagged. While it is registered and active it stays the canonical
// connection for its key, and registerHttp2Connection marks any newcomer for that key redundant: the
// replacement would serve one request and close, and so would every connection after it until this one closed.
// Removed before it is marked retired, so that a request which sees it retired and is sent again cannot find
// it in the registry.
Object partitionKey = state.getPartitionKey();
if (partitionKey != null) {
removeHttp2Connection(partitionKey, channel);
}
if (!state.retire()) {
return;
}
LOGGER.debug("HTTP/2 connection {} reached connectionTtl, retiring it", channel);
// No GOAWAY now. RFC 9113 section 6.8 asks for one before closing, and Netty's close() sends GOAWAY(NO_ERROR)
// when this connection is closed idle. An early one would cost requests and buy nothing: a request that looked
// this connection up just before it was retired still opens a stream on it, and a server may treat a client
// GOAWAY as the start of its own shutdown and drop streams opened after it, as Go's HTTP/2 server does.
// Until then a replacement may be open alongside. Section 9.1 asks clients not to open more than one
// connection per origin and lists replacements only for stream-identifier exhaustion, TLS rekeying and
// errors, so this overlap is a deliberate deviation of the kind LoadBalance.ROUND_ROBIN already documents,
// bounded by the streams still in flight here.
if (state.closeIfRetiredAndIdle()) {
closeChannel(channel);
}
}

/**
Expand Down Expand Up @@ -1412,6 +1480,40 @@ private static final class ConnectionCounts {
private long idleConnectionCount;
}

/**
* Retires its HTTP/2 connection once it has been open for the given delay. The task runs on the connection's own
* event loop, so it is serialized with the GOAWAY and stream-close handling there, and it is cancelled when the
* pipeline is torn down on close, so it never outlives the connection. A connection that closed before the handler
* was added gets no task: its pipeline may be torn down already, and nothing would cancel it then.
*/
private final class Http2ConnectionTtlHandler extends ChannelInboundHandlerAdapter {

private final long delayMillis;
private @Nullable ScheduledFuture<?> expiry;

private Http2ConnectionTtlHandler(long delayMillis) {
this.delayMillis = delayMillis;
}

@Override
public void handlerAdded(ChannelHandlerContext ctx) {
if (!ctx.channel().isActive()) {
return;
}
expiry = ctx.executor().schedule(() -> retireHttp2Connection(ctx.channel()),
delayMillis, TimeUnit.MILLISECONDS);
}

@Override
public void handlerRemoved(ChannelHandlerContext ctx) {
ScheduledFuture<?> scheduled = expiry;
if (scheduled != null) {
scheduled.cancel(false);
expiry = null;
}
}
}

private static final class ClientHttp2FrameCodecBuilder extends Http2FrameCodecBuilder {

private final SuspensionAwareHttp2LocalFlowController flowController;
Expand Down
Loading
Loading