Skip to content

Commit

Permalink
libcurl-thread.3: Consolidate thread safety info
Browse files Browse the repository at this point in the history
This is a new document to consolidate our thread safety information from
several documents (curl-www:features, libcurl.3, libcurl-tutorial.3).
Each document's section on multi-threading will now point to this one.
  • Loading branch information
jay authored and bagder committed Jul 28, 2015
1 parent 7db03e5 commit 279965c
Show file tree
Hide file tree
Showing 5 changed files with 127 additions and 79 deletions.
42 changes: 22 additions & 20 deletions docs/libcurl/Makefile.am
Expand Up @@ -29,18 +29,19 @@ man_MANS = curl_easy_cleanup.3 curl_easy_getinfo.3 curl_easy_init.3 \
curl_formadd.3 curl_formfree.3 curl_getdate.3 curl_getenv.3 \
curl_slist_append.3 curl_slist_free_all.3 curl_version.3 \
curl_version_info.3 curl_escape.3 curl_unescape.3 curl_free.3 \
curl_strequal.3 curl_mprintf.3 curl_global_init.3 curl_global_cleanup.3 \
curl_multi_add_handle.3 curl_multi_cleanup.3 curl_multi_fdset.3 \
curl_multi_info_read.3 curl_multi_init.3 curl_multi_perform.3 \
curl_multi_remove_handle.3 curl_share_cleanup.3 curl_share_init.3 \
curl_share_setopt.3 libcurl.3 libcurl-easy.3 libcurl-multi.3 \
libcurl-share.3 libcurl-errors.3 curl_easy_strerror.3 \
curl_strequal.3 curl_mprintf.3 curl_global_init.3 \
curl_global_cleanup.3 curl_multi_add_handle.3 curl_multi_cleanup.3 \
curl_multi_fdset.3 curl_multi_info_read.3 curl_multi_init.3 \
curl_multi_perform.3 curl_multi_remove_handle.3 curl_share_cleanup.3 \
curl_share_init.3 curl_share_setopt.3 libcurl.3 libcurl-easy.3 \
libcurl-multi.3 libcurl-share.3 libcurl-errors.3 curl_easy_strerror.3 \
curl_multi_strerror.3 curl_share_strerror.3 curl_global_init_mem.3 \
libcurl-tutorial.3 curl_easy_reset.3 curl_easy_escape.3 \
curl_easy_unescape.3 curl_multi_setopt.3 curl_multi_socket.3 \
curl_multi_timeout.3 curl_formget.3 curl_multi_assign.3 \
curl_easy_pause.3 curl_easy_recv.3 curl_easy_send.3 \
curl_multi_socket_action.3 curl_multi_wait.3 libcurl-symbols.3
curl_multi_socket_action.3 curl_multi_wait.3 libcurl-symbols.3 \
libcurl-thread.3

HTMLPAGES = curl_easy_cleanup.html curl_easy_getinfo.html \
curl_easy_init.html curl_easy_perform.html curl_easy_setopt.html \
Expand All @@ -60,27 +61,28 @@ HTMLPAGES = curl_easy_cleanup.html curl_easy_getinfo.html \
curl_easy_unescape.html curl_multi_setopt.html curl_multi_socket.html \
curl_multi_timeout.html curl_formget.html curl_multi_assign.html \
curl_easy_pause.html curl_easy_recv.html curl_easy_send.html \
curl_multi_socket_action.html curl_multi_wait.html libcurl-symbols.html
curl_multi_socket_action.html curl_multi_wait.html \
libcurl-symbols.html libcurl-thread.html

PDFPAGES = curl_easy_cleanup.pdf curl_easy_getinfo.pdf \
curl_easy_init.pdf curl_easy_perform.pdf curl_easy_setopt.pdf \
curl_easy_duphandle.pdf curl_formadd.pdf curl_formfree.pdf \
curl_getdate.pdf curl_getenv.pdf curl_slist_append.pdf \
curl_slist_free_all.pdf curl_version.pdf curl_version_info.pdf \
curl_getdate.pdf curl_getenv.pdf curl_slist_append.pdf \
curl_slist_free_all.pdf curl_version.pdf curl_version_info.pdf \
curl_escape.pdf curl_unescape.pdf curl_free.pdf curl_strequal.pdf \
curl_mprintf.pdf curl_global_init.pdf curl_global_cleanup.pdf \
curl_multi_add_handle.pdf curl_multi_cleanup.pdf curl_multi_fdset.pdf \
curl_multi_info_read.pdf curl_multi_init.pdf curl_multi_perform.pdf \
curl_multi_remove_handle.pdf curl_share_cleanup.pdf curl_share_init.pdf \
curl_share_setopt.pdf libcurl.pdf libcurl-multi.pdf libcurl-easy.pdf \
libcurl-share.pdf libcurl-errors.pdf curl_easy_strerror.pdf \
curl_multi_strerror.pdf curl_share_strerror.pdf \
curl_global_init_mem.pdf libcurl-tutorial.pdf curl_easy_reset.pdf \
curl_easy_escape.pdf curl_easy_unescape.pdf curl_multi_setopt.pdf \
curl_multi_socket.pdf curl_multi_timeout.pdf curl_formget.pdf \
curl_multi_assign.pdf curl_easy_pause.pdf curl_easy_recv.pdf \
curl_easy_send.pdf curl_multi_socket_action.pdf curl_multi_wait.pdf \
libcurl-symbols.pdf
curl_multi_remove_handle.pdf curl_share_cleanup.pdf \
curl_share_init.pdf curl_share_setopt.pdf libcurl.pdf \
libcurl-multi.pdf libcurl-easy.pdf libcurl-share.pdf \
libcurl-errors.pdf curl_easy_strerror.pdf curl_multi_strerror.pdf \
curl_share_strerror.pdf curl_global_init_mem.pdf libcurl-tutorial.pdf \
curl_easy_reset.pdf curl_easy_escape.pdf curl_easy_unescape.pdf \
curl_multi_setopt.pdf curl_multi_socket.pdf curl_multi_timeout.pdf \
curl_formget.pdf curl_multi_assign.pdf curl_easy_pause.pdf \
curl_easy_recv.pdf curl_easy_send.pdf curl_multi_socket_action.pdf \
curl_multi_wait.pdf libcurl-symbols.pdf libcurl-thread.pdf

m4macrodir = $(datadir)/aclocal
dist_m4macro_DATA = libcurl.m4
Expand Down
1 change: 1 addition & 0 deletions docs/libcurl/index.html
Expand Up @@ -17,6 +17,7 @@ <h2>Overviews</h2>
<br><a href="libcurl-share.html">libcurl-share</a>
<br><a href="libcurl-errors.html">libcurl-errors</a>
<br><a href="libcurl-tutorial.html">libcurl-tutorial</a>
<br><a href="libcurl-thread.html">libcurl-thread</a>

<H2>Library Functions (A-Z)</H2>
<a href="curl_easy_cleanup.html">curl_easy_cleanup</A>
Expand Down
100 changes: 100 additions & 0 deletions docs/libcurl/libcurl-thread.3
@@ -0,0 +1,100 @@
.\" **************************************************************************
.\" * _ _ ____ _
.\" * Project ___| | | | _ \| |
.\" * / __| | | | |_) | |
.\" * | (__| |_| | _ <| |___
.\" * \___|\___/|_| \_\_____|
.\" *
.\" * Copyright (C) 2015, Daniel Stenberg, <daniel@haxx.se>, et al.
.\" *
.\" * This software is licensed as described in the file COPYING, which
.\" * you should have received as part of this distribution. The terms
.\" * are also available at http://curl.haxx.se/docs/copyright.html.
.\" *
.\" * You may opt to use, copy, modify, merge, publish, distribute and/or sell
.\" * copies of the Software, and permit persons to whom the Software is
.\" * furnished to do so, under the terms of the COPYING file.
.\" *
.\" * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
.\" * KIND, either express or implied.
.\" *
.\" **************************************************************************
.\"
.TH libcurl-thread 3 "13 Jul 2015" "libcurl" "libcurl thread safety"
.SH NAME
libcurl-thread \- libcurl thread safety
.SH "Multi-threading Issues"
libcurl is thread safe with the following exceptions:

Do \fBNOT\fP use library functions to access or modify the same handle
concurrently from multiple threads. If concurrent access to the same handle
from multiple threads could be an issue then you must implement your own
locking to ensure it won't happen.

Shared objects. You can share certain data between multiple handles by using
the share interface but you must implement your own locking and set
\fIcurl_share_setopt(3)\fP CURLSHOPT_LOCKFUNC and CURLSHOPT_UNLOCKFUNC.

SSL/TLS handlers. If you are accessing HTTPS or FTPS URLs in a multi-threaded
manner, you are then of course using the underlying SSL library multi-threaded
and those libs might have their own requirements on this issue. You may need to
provide one or two functions to allow it to function properly:

.RS
.IP OpenSSL

http://www.openssl.org/docs/crypto/threads.html#DESCRIPTION

http://curl.haxx.se/libcurl/c/opensslthreadlock.html

.IP GnuTLS

http://gnutls.org/manual/html_node/Thread-safety.html

.IP NSS

is claimed to be thread-safe already without anything required.

.IP PolarSSL

Required actions unknown.

.IP yassl

Required actions unknown.

.IP axTLS

Required actions unknown.

.IP Secure-Transport

The engine is fully thread-safe, and no additional steps are required.
.RE

Signals. Signals are used for timing out name resolves (during DNS lookup) -
when built without using either the c-ares or threaded resolver backends. When
using multiple threads you should set the \fICURLOPT_NOSIGNAL(3)\fP option to 1
for all handles. Everything will or might work fine except that timeouts are
not honored during the DNS lookup - which you can work around by building
libcurl with c-ares support. c-ares is a library that provides asynchronous
name resolves. On some platforms, libcurl simply will not function properly
multi-threaded unless this option is set.

gethostby* functions and other system calls. These functions, provided by your
operating system, must be thread safe. It is very important that libcurl can
find and use thread safe versions of these and other system calls, as otherwise
it can't function fully thread safe. Some operating systems are known to have
faulty thread implementations. We have previously received problem reports on
*BSD (at least in the past, they may be working fine these days). Some
operating systems that are known to have solid and working thread support are
Linux, Solaris and Windows.

curl_global_* functions. These functions are not thread safe. If you are using
libcurl with multiple threads it is especially important that before use you
call \fIcurl_global_init(3)\fP to explicitly initialize the library and its
dependents, rather than rely on the "lazy" fail-safe initialization that takes
place the first time \fIcurl_easy_init(3)\fP is called. For an in-depth
explanation refer to \fIlibcurl(3)\fP section \fBGLOBAL CONSTANTS\fP.

\fICURLOPT_DNS_USE_GLOBAL_CACHE(3)\fP is not thread-safe.
54 changes: 2 additions & 52 deletions docs/libcurl/libcurl-tutorial.3
Expand Up @@ -256,58 +256,8 @@ complication for you. Given simply the URL to a file, libcurl will take care
of all the details needed to get the file moved from one machine to another.

.SH "Multi-threading Issues"
The first basic rule is that you must \fBnever\fP simultaneously share a
libcurl handle (be it easy or multi or whatever) between multiple
threads. Only use one handle in one thread at any time. You can pass the
handles around among threads, but you must never use a single handle from more
than one thread at any given time.

libcurl is completely thread safe, except for two issues: signals and SSL/TLS
handlers. Signals are used for timing out name resolves (during DNS lookup) -
when built without using either the c-ares or threaded resolver backends.

If you are accessing HTTPS or FTPS URLs in a multi-threaded manner, you are
then of course using the underlying SSL library multi-threaded and those libs
might have their own requirements on this issue. Basically, you need to
provide one or two functions to allow it to function properly. For all
details, see this:

OpenSSL

http://www.openssl.org/docs/crypto/threads.html#DESCRIPTION

GnuTLS

http://gnutls.org/manual/html_node/Thread-safety.html

NSS

is claimed to be thread-safe already without anything required.

PolarSSL

Required actions unknown.

yassl

Required actions unknown.

axTLS

Required actions unknown.

Secure Transport

The engine is fully thread-safe, and no additional steps are required.

When using multiple threads you should set the \fICURLOPT_NOSIGNAL(3)\fP
option to 1 for all handles. Everything will or might work fine except that
timeouts are not honored during the DNS lookup - which you can work around by
building libcurl with c-ares support. c-ares is a library that provides
asynchronous name resolves. On some platforms, libcurl simply will not
function properly multi-threaded unless this option is set.

Also, note that \fICURLOPT_DNS_USE_GLOBAL_CACHE(3)\fP is not thread-safe.
libcurl is thread safe but there are a few exceptions. Refer to
\fIlibcurl-thread(3)\fP for more information.

.SH "When It Doesn't Work"
There will always be times when the transfer fails for some reason. You might
Expand Down
9 changes: 2 additions & 7 deletions docs/libcurl/libcurl.3
Expand Up @@ -111,13 +111,8 @@ libcurl works
.B exactly
the same, on any of the platforms it compiles and builds on.
.SH "THREADS"
Never ever call curl-functions simultaneously using the same handle from
several threads. libcurl is thread-safe and can be used in any number of
threads, but you must use separate curl handles if you want to use libcurl in
more than one thread simultaneously.

The global environment functions are not thread-safe. See \fBGLOBAL
CONSTANTS\fP below for details.
libcurl is thread safe but there are a few exceptions. Refer to
\fIlibcurl-thread(3)\fP for more information.

.SH "PERSISTENT CONNECTIONS"
Persistent connections means that libcurl can re-use the same connection for
Expand Down

0 comments on commit 279965c

Please sign in to comment.