Permalink
Browse files

FTP: WILDCARDMATCH/CHUNKING/FNMATCH added

  • Loading branch information...
Pavel Raiskup authored and bagder committed May 12, 2010
1 parent 04cb15a commit 0825cd80a62c21725fb3615f1fdd3aa6cc5f0f34
@@ -84,6 +84,54 @@ If this option is set and libcurl has been built with the standard name
resolver, timeouts will not occur while the name resolve takes place.
Consider building libcurl with c-ares support to enable asynchronous DNS
lookups, which enables nice timeouts for name resolves without signals.
.IP CURLOPT_WILDCARDMATCH
Set this option to 1 if you want to transfer multiple files according to a
file name pattern. The pattern can be specified as part of the \fICURLOPT_URL\fP
option, using an fnmatch-like pattern (Shell Pattern Matching) in the last part
of URL (file name).
By default, libcurl uses its internal implementation of fnmatch(). You can
provide your own matching function by the \fICURLOPT_FNMATCH_FUNCTION\fR option.
This feature is only supported by the FTP download for now.
A brief introduction of its syntax follows:
.RS
.IP "\fB*\fR - ASTERISK"
\&ftp://example.com/some/path/\fB*.txt\fR (for all txt's from the root
directory)
.RE
.RS
.IP "\fB?\fR - QUESTION MARK"
Question mark matches any (exactly one) character.
\&ftp://example.com/some/path/\fBphoto?.jpeg\fR
.RE
.RS
.IP "\fB[\fR - BRACKET EXPRESSION"
The left bracket opens a bracket expression. The question mark and asterisk have
no special meaning in a bracket expression. Each bracket expression ends by the
right bracket and matches exactly one character. Some examples follow:
\fB[a-zA-Z0\-9]\fR or \fB[f\-gF\-G]\fR \- character interval
\fB[abc]\fR - character enumeration
\fB[^abc]\fR or \fB[!abc]\fR - negation
\fB[[:\fR\fIname\fR\fB:]]\fR class expression. Supported classes are
\fBalnum\fR,\fBlower\fR, \fBspace\fR, \fBalpha\fR, \fBdigit\fR, \fBprint\fR,
\fBupper\fR, \fBblank\fR, \fBgraph\fR, \fBxdigit\fR.
\fB[][-!^]\fR - special case \- matches only '\-', ']', '[', '!' or '^'. These
characters have no special purpose.
\fB[\\[\\]\\\\]\fR - escape syntax. Matches '[', ']' or '\\'.
Using the rules above, a file name pattern can be constructed:
\&ftp://example.com/some/path/\fB[a-z[:upper:]\\\\].jpeg\fR
.RE
.PP
.SH CALLBACK OPTIONS
.IP CURLOPT_WRITEFUNCTION
@@ -424,6 +472,43 @@ in 7.20.0)
.IP CURLOPT_INTERLEAVEDATA
This is the stream that will be passed to \fICURLOPT_INTERLEAVEFUNCTION\fP when
interleaved RTP data is received. (Added in 7.20.0)
.IP CURLOPT_CHUNK_BGN_FUNCTION
Function pointer that should match the following prototype: \fBlong function
(const void *transfer_info, void *ptr, int remains)\fR. This function
gets called by libcurl before a part of the stream is going to be transferred
(if the transfer supports chunks).
This callback makes sense only when using the \fICURLOPT_WILDCARDMATCH\fR
option for now.
The target of transfer_info parameter is a "feature depended" structure. For the
FTP wildcard download, the target is curl_fileinfo structure (see
\fIcurl/curl.h\fR).
The parameter ptr is a pointer given by \fICURLOPT_CHUNK_DATA\fR. The parameter
remains contains number of chunks remaining per the transfer. If the feature is
not available, the parameter has zero value.
Return \fICURL_CHUNK_BGN_FUNC_OK\fR if everything is fine,
\fICURL_CHUNK_BGN_FUNC_SKIP\fR if you want to skip the concrete chunk or
\fICURL_CHUNK_BGN_FUNC_FAIL\fR to tell libcurl to stop if some error occurred.
.IP CURLOPT_CHUNK_END_FUNCTION
Function pointer that should match the following prototype:
\fBlong function(void *ptr)\fR. This function gets called by libcurl as soon as
a part of the stream has been transferred (or skipped).
Return \fICURL_CHUNK_END_FUNC_OK\fR if everything is fine or
\fBCURL_CHUNK_END_FUNC_FAIL\fR to tell the lib to stop if some error occurred.
.IP CURLOPT_CHUNK_DATA
Pass a pointer that will be untouched by libcurl and passed as the ptr argument
to the \fICURL_CHUNK_BGN_FUNTION\fR and \fICURL_CHUNK_END_FUNTION\fR.
.IP CURLOPT_FNMATCH_FUNCTION
Function pointer that should match \fBint function(const char *pattern, const
char *string)\fR prototype (see \fIcurl/curl.h\fR). It is used internally for
the wildcard matching feature.
Return \fICURL_FNMATCHFUNC_MATCH\fR if pattern matches the string,
\fICURL_FNMATCHFUNC_NOMATCH\fR if not or \fICURL_FNMATCHFUNC_FAIL\fR if an error
occurred.
.SH ERROR OPTIONS
.IP CURLOPT_ERRORBUFFER
Pass a char * to a buffer that the libcurl may store human readable error
@@ -218,6 +218,16 @@ return code is only returned from \fIcurl_easy_recv(3)\fP and
Failed to load CRL file (Added in 7.19.0)
.IP "CURLE_SSL_ISSUER_ERROR (83)"
Issuer check failed (Added in 7.19.0)
.IP "CURLE_FTP_PRET_FAILED (84)"
PRET command failed
.IP "CURLE_RTSP_CSEQ_ERROR (85)"
Mismatch of RTSP CSeq numbers.
.IP "CURLE_RTSP_SESSION_ERROR (86)"
Mismatch of RTSP Session Identifiers.
.IP "CURLE_FTP_BAD_FILE_LIST (87)"
Unable to parse FTP file list (during FTP wildcard downloading).
.IP "CURLE_CHUNK_FAILED (88)"
Chunk callback reported error.
.IP "CURLE_OBSOLETE*"
These error codes will never be returned. They were used in an old libcurl
version and are currently unused.
View
@@ -198,6 +198,96 @@ typedef size_t (*curl_write_callback)(char *buffer,
size_t nitems,
void *outstream);
/* enumeration of file types */
typedef enum {
CURLFILETYPE_FILE = 0,
CURLFILETYPE_DIRECTORY,
CURLFILETYPE_SYMLINK,
CURLFILETYPE_DEVICE_BLOCK,
CURLFILETYPE_DEVICE_CHAR,
CURLFILETYPE_NAMEDPIPE,
CURLFILETYPE_SOCKET,
CURLFILETYPE_DOOR, /* is possible only on Sun Solaris now */
CURLFILETYPE_UNKNOWN /* should never occur */
} curlfiletype;
#define CURLFINFOFLAG_KNOWN_FILENAME (1<<0)
#define CURLFINFOFLAG_KNOWN_FILETYPE (1<<1)
#define CURLFINFOFLAG_KNOWN_TIME (1<<2)
#define CURLFINFOFLAG_KNOWN_PERM (1<<3)
#define CURLFINFOFLAG_KNOWN_UID (1<<4)
#define CURLFINFOFLAG_KNOWN_GID (1<<5)
#define CURLFINFOFLAG_KNOWN_SIZE (1<<6)
#define CURLFINFOFLAG_KNOWN_HLINKCOUNT (1<<7)
/* Content of this structure depends on information which is known and is
achievable (e.g. by FTP LIST parsing). Please see the url_easy_setopt(3) man
page for callbacks returning this structure -- some fields are mandatory,
some others are optional. The FLAG field has special meaning. */
struct curl_fileinfo {
char *filename;
curlfiletype filetype;
time_t time;
int32_t perm;
int uid;
int gid;
curl_off_t size;
long int hardlinks;
struct {
/* If some of these fields is not NULL, it is a pointer to b_data. */
char *time;
char *perm;
char *user;
char *group;
char *target; /* pointer to the target filename of a symlink */
} strings;
int32_t flags;
/* used internally */
char * b_data;
size_t b_size;
size_t b_used;
};
/* return codes for CURLOPT_CHUNK_BGN_FUNCTION */
#define CURL_CHUNK_BGN_FUNC_OK 0
#define CURL_CHUNK_BGN_FUNC_FAIL 1 /* tell the lib to end the task */
#define CURL_CHUNK_BGN_FUNC_SKIP 2 /* skip this chunk over */
/* if splitting of data transfer is enabled, this callback is called before
download of an individual chunk started. Note that parameter "remains" works
only for FTP wildcard downloading (for now), otherwise is not used */
typedef long (*curl_chunk_bgn_callback)(const void *transfer_info,
void *ptr,
int remains);
/* return codes for CURLOPT_CHUNK_END_FUNCTION */
#define CURL_CHUNK_END_FUNC_OK 0
#define CURL_CHUNK_END_FUNC_FAIL 1 /* tell the lib to end the task */
/* If splitting of data transfer is enabled this callback is called after
download of an individual chunk finished.
Note! After this callback was set then it have to be called FOR ALL chunks.
Even if downloading of this chunk was skipped in CHUNK_BGN_FUNC.
This is the reason why we don't need "transfer_info" parameter in this
callback and we are not interested in "remains" parameter too. */
typedef long (*curl_chunk_end_callback)(void *ptr);
/* return codes for FNMATCHFUNCTION */
#define CURL_FNMATCHFUNC_MATCH 0 /* string corresponds to the pattern */
#define CURL_FNMATCHFUNC_NOMATCH 1 /* pattern doesn't match the string */
#define CURL_FNMATCHFUNC_FAIL 2 /* an error occurred */
/* callback type for wildcard downloading pattern matching. If the
string matches the pattern, return CURL_FNMATCHFUNC_MATCH value, etc. */
typedef int (*curl_fnmatch_callback)(const char *pattern,
const char *string);
/* These are the return codes for the seek callbacks */
#define CURL_SEEKFUNC_OK 0
#define CURL_SEEKFUNC_FAIL 1 /* fail the entire transfer */
@@ -409,6 +499,8 @@ typedef enum {
CURLE_FTP_PRET_FAILED, /* 84 - a PRET command failed */
CURLE_RTSP_CSEQ_ERROR, /* 85 - mismatch of RTSP CSeq numbers */
CURLE_RTSP_SESSION_ERROR, /* 86 - mismatch of RTSP Session Identifiers */
CURLE_FTP_BAD_FILE_LIST, /* 87 - unable to parse FTP file list */
CURLE_CHUNK_FAILED, /* 88 - chunk callback reported error */
CURL_LAST /* never use! */
} CURLcode;
@@ -1322,6 +1414,23 @@ typedef enum {
/* Let the application define a custom write method for RTP data */
CINIT(INTERLEAVEFUNCTION, FUNCTIONPOINT, 196),
/* Turn on wildcard matching */
CINIT(WILDCARDMATCH, LONG, 197),
/* Directory matching callback called before downloading of an
individual file (chunk) started */
CINIT(CHUNK_BGN_FUNCTION, FUNCTIONPOINT, 198),
/* Directory matching callback called after the file (chunk)
was downloaded, or skipped */
CINIT(CHUNK_END_FUNCTION, FUNCTIONPOINT, 199),
/* Change match (fnmatch-like) callback for wildcard matching */
CINIT(FNMATCH_FUNCTION, FUNCTIONPOINT, 200),
/* Let the application define custom chunk data pointer */
CINIT(CHUNK_DATA, OBJECTPOINT, 201),
CURLOPT_LASTENTRY /* the last unused */
} CURLoption;
View
@@ -4,6 +4,7 @@ CSOURCES = file.c timeval.c base64.c hostip.c progress.c formdata.c \
cookie.c http.c sendf.c ftp.c url.c dict.c if2ip.c speedcheck.c \
ldap.c ssluse.c version.c getenv.c escape.c mprintf.c telnet.c \
netrc.c getinfo.c transfer.c strequal.c easy.c security.c krb4.c \
curl_fnmatch.c fileinfo.c ftplistparser.c wildcard.c \
krb5.c memdebug.c http_chunks.c strtok.c connect.c llist.c hash.c \
multi.c content_encoding.c share.c http_digest.c md5.c curl_rand.c \
http_negotiate.c http_ntlm.c inet_pton.c strtoofft.c strerror.c \
@@ -18,6 +19,7 @@ HHEADERS = arpa_telnet.h netrc.h file.h timeval.h qssl.h hostip.h \
progress.h formdata.h cookie.h http.h sendf.h ftp.h url.h dict.h \
if2ip.h speedcheck.h urldata.h curl_ldap.h ssluse.h escape.h telnet.h \
getinfo.h strequal.h krb4.h memdebug.h http_chunks.h curl_rand.h \
curl_fnmatch.h wildcard.h fileinfo.h ftplistparser.h \
strtok.h connect.h llist.h hash.h content_encoding.h share.h \
curl_md5.h http_digest.h http_negotiate.h http_ntlm.h inet_pton.h \
strtoofft.h strerror.h inet_ntop.h curlx.h curl_memory.h setup.h \
Oops, something went wrong.

0 comments on commit 0825cd8

Please sign in to comment.