Whatever
language or client library you're using, you should be able to set the
timeout on network socket operations, typically split into a connect
timeout, read timeout, and write timeout.
However, although you should be able to make these timeouts as small
as you want, the connect timeout in particular has an effective maximum
value for any given kernel. Beyond this point, higher timeout values
you might request will have no effect - connecting will still time out
after a shorter time.
The reason TCP connects are special is that the establishment of a
TCP connection has a special sequence of packets starting with a SYN
packet. If no response is received to this initial SYN packet, the
kernel needs to retry, which it may have to do a couple of times. All
kernels I know of wait an increasing amount of time between sending SYN
retries, to avoid flooding slow hosts.
All kernels put an upper limit on the number of times they will retry
SYNs. On BSD-derived kernels, including Mac OS X, the standard pattern
is that the second SYN will be second 6 seconds after the first, then a
third SYN 18 seconds after that, then the connect times out after a
total of around 75 seconds.
On Linux however, the default retry cycle ends after just 20 seconds.
Linux does send SYN retries somewhat faster than BSD-derived kernels -
Linux supposedly sends 5 SYNs in this 20 seconds, but this includes the
original packet (the retries are after 3s, 6s, 12s, 24s).
The end result though is that if your application wants a connect
timeout shorter than 20s, no problem, but if your application wants a
connect timeout longer than 20s, you'll find that the default kernel
configuration will effectively chop it back to 20s.
Changing this upper timeout limit is easy, though it requires you to
change a system configuration parameter and so you will need to have
root access to the box (or get the system administrators to agree to
change it for you).
The relevant sysctl is
tcp_syn_retries, which for IP v4 is
net.ipv4.tcp_syn_retries.
Be conservative in choosing the value you change it to. Like BSD,
the SYN retry delays increase in time (albeit doubling rather than
tripling), so a relatively small increase in the number of retries leads
to a large increase in the maximum connect timeout. In a perfect
world, there would be no problem with having a very high timeout because
applications' connect timeouts will come into play.
However, many applications do not set an explicit connect timeout,
and so if you set the kernel to 10 minutes, you're probably going to
find something hanging for ages sooner or later when a remote host goes
down!
I recommend that you set it to a value of 6, 7, or at most 8. 6
gives an effective connect timeout ceiling of around 45 seconds, 7 gives
around 90 seconds, and 8 gives around 190 seconds.
To change this in a running kernel, you can use the /proc interface:
# cat /proc/sys/net/ipv4/tcp_syn_retries
5
# echo 6 > /proc/sys/net/ipv4/tcp_syn_retries
Or use the sysctl command:
# sysctl net.ipv4.tcp_syn_retries
net.ipv4.tcp_syn_retries = 5
# sysctl -w net.ipv4.tcp_syn_retries=6
net.ipv4.tcp_syn_retries = 6
To make this value stick across reboots however you need to add it to
/etc/sysctl.conf:
net.ipv4.tcp_syn_retries = 6
Most Linux installations support reading sysctls from files in
/etc/sysctl.d,
which is usually better practice as it makes it easier to administer
upgrades, so I suggest you put it in a file there instead.
(I see no reason you'd want to reduce this sysctl, but note that
values of 4 or less all seem to be treated as 4 - total timeout 9s.)
