TclCurl - Tcl Bindings for libcurl

TclCurl provides Tcl bindings for the libcurl library, allowing Tcl scripts to perform HTTP requests, WebSocket connections, and other network operations.

Features

• Support for HTTP methods: GET, POST, PUT, DELETE, HEAD, OPTIONS, PATCH
• HTTP/2 and HTTP/3 support with dedicated convenience commands
• WebSocket support for real-time bidirectional communication
• File transfer protocols: FTP, SFTP, SCP with dedicated command interfaces
• SSH support with connection management, remote execution, tunneling, and authentication options
• Support for multiple protocols including LDAP, SMTP, POP3, IMAP, and more
• Handle interface for fine-grained control
• Direct interface for simple requests
• File upload and download capabilities
• URL manipulation (parsing, building, escaping)
• Custom headers support
• SSL/TLS configuration
• Authentication options
• Proxy support
• Compression settings
• Extensive information and utility functions

Installation

1. Make sure you have libcurl (8.16) installed on your system.
2. Build the TclCurl library:

   cd tclcurl
   make
   

3. Add the library to your Tcl script:

   lappend auto_path [file dirname [info script]]
   package require tclcurl

Basic Usage

Direct Interface

# GET request
set response [curl::get "https://example.com"]

# POST request
set response [curl::post "https://example.com" "name=value&other=data"]

# PUT request
set response [curl::put "https://example.com" "name=value&other=data"]

# DELETE request
set response [curl::delete "https://example.com"]

# HEAD request
set response [curl::head "https://example.com"]

# OPTIONS request
set response [curl::options "https://example.com"]

# PATCH request
set response [curl::patch "https://example.com" "name=value&other=data"]

# HTTP/2 request
set response [curl::http2 "https://example.com"]

# HTTP/3 request
set response [curl::http3 "https://example.com"]

Handle Interface

# Create a handle
set handle [curl::create]

# Set options
$handle setopt URL "https://example.com"
$handle setopt FOLLOWLOCATION 1
$handle setopt TIMEOUT 30

# Set HTTP version
$handle setopt HTTP_VERSION "2"    ;# HTTP/2
$handle setopt HTTP_VERSION "3"    ;# HTTP/3

# Perform the request
set response [$handle perform]

# Get information from the response
set status [$handle getinfo RESPONSE_CODE]
set type [$handle getinfo CONTENT_TYPE]

# Reset the handle
$handle reset

# Clean up
$handle cleanup

API Reference

HTTP Methods

curl::get

Performs an HTTP GET request.

curl::get url ?options?

• url: The URL to request
• options: Optional key-value pairs for custom settings

curl::post

Performs an HTTP POST request.

curl::post url data ?options?

• url: The URL to request
• data: The data to send in the request body
• options: Optional key-value pairs for custom settings

curl::put

Performs an HTTP PUT request.

curl::put url data ?options?

• url: The URL to request
• data: The data to send in the request body
• options: Optional key-value pairs for custom settings

curl::delete

Performs an HTTP DELETE request.

curl::delete url ?options?

• url: The URL to request
• options: Optional key-value pairs for custom settings

curl::head

Performs an HTTP HEAD request.

curl::head url ?options?

• url: The URL to request
• options: Optional key-value pairs for custom settings

curl::options

Performs an HTTP OPTIONS request.

curl::options url ?options?

• url: The URL to request
• options: Optional key-value pairs for custom settings

curl::patch

Performs an HTTP PATCH request.

curl::patch url data ?options?

• url: The URL to request
• data: The data to send in the request body
• options: Optional key-value pairs for custom settings

curl::http2

Performs an HTTP/2 request.

curl::http2 url ?options?

• url: The URL to request
• options: Optional key-value pairs for custom settings

curl::http3

Performs an HTTP/3 request.

curl::http3 url ?options?

• url: The URL to request
• options: Optional key-value pairs for custom settings

HTTP Version Support

TclCurl supports HTTP/2 and HTTP/3 protocols through both convenience commands and the handle interface.

HTTP Version Options

The following HTTP version options are supported:

• "none" or "1.0": HTTP/1.0
• "1.1": HTTP/1.1 (default)
• "2" or "2.0": HTTP/2
• "2tls": HTTP/2 over TLS
• "2-prior-knowledge": HTTP/2 without upgrade
• "3" or "3.0": HTTP/3
• "3-only": HTTP/3 only (no fallback)

Using HTTP Version with Handle Interface

# Create a handle
set handle [curl::create]

# Set HTTP/2
$handle setopt URL "https://example.com"
$handle setopt HTTP_VERSION "2"
$handle perform

# Set HTTP/3
$handle setopt URL "https://example.com"
$handle setopt HTTP_VERSION "3"
$handle perform

# Check HTTP version used
set version [$handle getinfo HTTP_VERSION]
if {$version == 2000} {
    puts "Used HTTP/2"
} elseif {$version == 3000} {
    puts "Used HTTP/3"
} else {
    puts "Used HTTP/1.1"
}

$handle cleanup

HTTP Version Requirements

• HTTP/2: Requires libcurl compiled with nghttp2 support
• HTTP/3: Requires libcurl compiled with nghttp3 and QUIC support
• Fallback: If the requested version is not supported by the server, libcurl will automatically fall back to HTTP/1.1

File Transfer Protocols

curl::ftp

Performs FTP operations.

curl::ftp operation url ?args?

• operation: One of list, nlst, get, put, mkdir, delete, rmdir, or rename
• url: The FTP URL
• args: Additional arguments depending on operation

Examples:

# List directory contents
set listing [curl::ftp list "ftp://example.com/path/"]

# Download a file
set content [curl::ftp get "ftp://example.com/path/file.txt"]

# Upload a file
curl::ftp put "ftp://example.com/path/file.txt" "/local/path/file.txt"

# Create a directory
curl::ftp mkdir "ftp://example.com/new_dir/"

# Delete a file
curl::ftp delete "ftp://example.com/path/file.txt"

# Remove a directory
curl::ftp rmdir "ftp://example.com/old_dir/"

# Rename a file
curl::ftp rename "ftp://example.com/oldname.txt" "newname.txt"

curl::sftp

Performs SFTP operations.

curl::sftp operation url ?args?

• operation: One of list, get, put, mkdir, rmdir, delete, or rename
• url: The SFTP URL
• args: Additional arguments depending on operation

Examples:

# List directory contents
set listing [curl::sftp list "sftp://example.com/path/"]

# Download a file
set content [curl::sftp get "sftp://example.com/path/file.txt"]

# Upload a file
curl::sftp put "sftp://example.com/path/file.txt" "/local/path/file.txt"

# Create a directory
curl::sftp mkdir "sftp://example.com/new_dir/"]

# Delete a file
curl::sftp delete "sftp://example.com/path/file.txt"]

curl::scp

Performs SCP operations.

curl::scp operation url ?args?

• operation: Either get or put
• url: The SCP URL
• args: Additional arguments depending on operation

Examples:

# Download a file
set content [curl::scp get "scp://example.com/path/file.txt"]

# Upload a file
curl::scp put "scp://example.com/path/file.txt" "/local/path/file.txt"

curl::smtp

Performs SMTP operations for sending emails.

curl::smtp operation args ?options?

• operation: Either send or verify
• args: Additional arguments depending on operation
• options: Optional key-value pairs for custom settings

Examples:

# Using the dedicated SMTP interface
# Send an email
set emailContent "From: sender@example.com\r\n"
append emailContent "To: recipient@example.com\r\n"
append emailContent "Subject: Test Email\r\n"
append emailContent "\r\n"
append emailContent "This is a test email from TclCurl SMTP."

curl::smtp send "smtp.example.com" $emailContent \
    -mail "sender@example.com" \
    -recipients [list "recipient@example.com"] \
    -username "user" \
    -password "pass" \
    -sslVerify 1 \
    -timeout 30

# Send a secure email using SMTPS
curl::smtps send "smtp.example.com" $emailContent \
    -mail "sender@example.com" \
    -recipients [list "recipient@example.com"] \
    -username "user" \
    -password "pass" \
    -timeout 30
    
# Note: With curl::smtps, SSL is automatically enabled and the URL 
# is automatically converted to use the smtps:// scheme

# Verify an email address
curl::smtp verify "smtp.example.com" "user@example.com" \
    -username "user" \
    -password "pass"

# Verify an email address securely
curl::smtps verify "smtp.example.com" "user@example.com" \
    -username "user" \
    -password "pass"

# Using the handle interface for SMTP
set handle [curl::create]
$handle setopt URL "smtp://smtp.example.com"
$handle setopt USERNAME "user"
$handle setopt PASSWORD "password"
$handle setopt MAIL_FROM "sender@example.com"
$handle setopt MAIL_RCPT [list "recipient@example.com"]
$handle setopt READDATA "From: sender@example.com\r\nTo: recipient@example.com\r\nSubject: Test\r\n\r\nMessage body"
$handle perform
$handle cleanup

Testing SMTP with Gmail

Gmail's SMTP servers can be used for testing the SMTP functionality:

Setup Requirements

1. Gmail Account Setup:

   • If you have 2-Factor Authentication enabled, create an "App Password" in your Google Account:
     1. Go to your Google Account → Security → App Passwords
     2. Select "Mail" and your device type
     3. Use the generated 16-character password in your application
   • If you don't have 2FA, you may need to enable "Less secure app access" in your Google Account Security settings

2. Connection Settings:

   • For SMTP (with STARTTLS): smtp.gmail.com:587
   • For SMTPS (SSL/TLS): smtps://smtp.gmail.com:465

Gmail Testing Example

# Gmail configuration
set smtpServer "smtp.gmail.com"
set username "your.email@gmail.com"  # Your Gmail address
set password "your-app-password"     # App Password from Google
set fromEmail $username              # From address must match your Gmail
set toEmails [list $username]        # Send to yourself for testing

# Create test email
set emailContent "From: $fromEmail\r\n"
append emailContent "To: [lindex $toEmails 0]\r\n"
append emailContent "Subject: TclCurl SMTP Test\r\n"
append emailContent "\r\n"
append emailContent "This is a test email sent using TclCurl SMTP functionality.\r\n"

# Test with SMTPS (recommended for Gmail)
set result [curl::smtps send $smtpServer $emailContent \
    -mail $fromEmail \
    -recipients $toEmails \
    -username $username \
    -password $password]
puts "Send result: $result"

Important Notes

• When testing, send emails to your own Gmail address to avoid spamming others
• Gmail has daily sending limits for regular accounts
• Most modern SMTP servers (including Gmail) disable the VRFY command, so the verify operation may not work
• Connection testing with CONNECT_ONLY option is always safe as it doesn't send any emails

POP3/POP3S Protocol

POP3 (Post Office Protocol) is used for retrieving email.

URL Syntax

pop3://[user:password@]hostname[:port][/message_number]
pop3s://[user:password@]hostname[:port][/message_number]

Key Options

$handle setopt URL "pop3://pop.example.com"
$handle setopt USERNAME "user"
$handle setopt PASSWORD "password"
$handle setopt CUSTOMREQUEST "RETR"        ;# RETR, LIST, STAT, etc.

POP3 Methods

# Connect and authenticate
$handle setopt URL "pop3://pop.example.com"
$handle setopt USERNAME "user"
$handle setopt PASSWORD "password"
$handle perform

# List all messages
$handle setopt URL "pop3://pop.example.com"
$handle setopt USERNAME "user"
$handle setopt PASSWORD "password"
$handle setopt CUSTOMREQUEST "LIST"
$handle perform

# Retrieve a specific message
$handle setopt URL "pop3://pop.example.com/1"  ;# Message number 1
$handle setopt USERNAME "user"
$handle setopt PASSWORD "password"
$handle perform

# Delete a message
$handle setopt URL "pop3://pop.example.com/1"
$handle setopt USERNAME "user"
$handle setopt PASSWORD "password"
$handle setopt CUSTOMREQUEST "DELE"
$handle perform

IMAP/IMAPS Protocol

IMAP (Internet Message Access Protocol) is used for accessing email messages.

URL Syntax

imap://[user:password@]hostname[:port][/mailbox][/message_uid][?query]
imaps://[user:password@]hostname[:port][/mailbox][/message_uid][?query]

Key Options

$handle setopt URL "imap://imap.example.com/INBOX"
$handle setopt USERNAME "user"
$handle setopt PASSWORD "password"
$handle setopt CUSTOMREQUEST "FETCH"       ;# FETCH, APPEND, SEARCH, etc.

IMAP Methods

# Connect and select a mailbox
$handle setopt URL "imap://imap.example.com/INBOX"
$handle setopt USERNAME "user"
$handle setopt PASSWORD "password"
$handle perform

# Fetch a message by UID
$handle setopt URL "imap://imap.example.com/INBOX/;UID=1"
$handle setopt USERNAME "user"
$handle setopt PASSWORD "password"
$handle perform

# Search for messages
$handle setopt URL "imap://imap.example.com/INBOX?SUBJECT%20Test"  ;# URL-encoded: SUBJECT Test
$handle setopt USERNAME "user"
$handle setopt PASSWORD "password"
$handle perform

# Append a message to a mailbox
$handle setopt URL "imap://imap.example.com/INBOX"
$handle setopt USERNAME "user"
$handle setopt PASSWORD "password"
$handle setopt UPLOAD 1
$handle setopt READDATA "From: sender@example.com\r\nTo: recipient@example.com\r\nSubject: Test\r\n\r\nMessage body"
$handle setopt CUSTOMREQUEST "APPEND"
$handle perform

LDAP/LDAPS Protocol

LDAP (Lightweight Directory Access Protocol) is used for accessing and maintaining directory information.

URL Syntax

ldap://[hostname[:port]]/[distinguished_name][?attribute][?scope][?filter]
ldaps://[hostname[:port]]/[distinguished_name][?attribute][?scope][?filter]

Key Options

$handle setopt URL "ldap://ldap.example.com/dc=example,dc=com?cn?sub?(objectClass=*)"
$handle setopt USERNAME "user"         ;# For binding (optional)
$handle setopt PASSWORD "password"     ;# For binding (optional)

LDAP Methods

# Simple query
$handle setopt URL "ldap://ldap.example.com/dc=example,dc=com?cn?sub?(objectClass=*)"
$handle perform

# Query with authentication
$handle setopt URL "ldap://ldap.example.com/dc=example,dc=com?cn?sub?(objectClass=*)"
$handle setopt USERNAME "cn=admin,dc=example,dc=com"
$handle setopt PASSWORD "password"
$handle perform

MQTT Protocol

MQTT (Message Queuing Telemetry Transport) is a lightweight messaging protocol for IoT devices.

MQTT Usage

# Connect to an MQTT broker
curl::mqtt connect "mqtt://test.mosquitto.org:1883" \
    -username "user" \
    -password "pass"

# Publish a message to a topic
curl::mqtt publish "mqtt://test.mosquitto.org:1883" "my/topic" "Hello World" \
    -username "user" \
    -password "pass"

# Subscribe to a topic
curl::mqtt subscribe "mqtt://test.mosquitto.org:1883" "my/topic" \
    -username "user" \
    -password "pass"

# Receive messages from a broker
curl::mqtt receive "mqtt://test.mosquitto.org:1883" \
    -timeout 5 \
    -username "user" \
    -password "pass"

# Disconnect from a broker
curl::mqtt disconnect "mqtt://test.mosquitto.org:1883" \
    -username "user" \
    -password "pass"

MQTT Operations

• connect: Establish a connection to an MQTT broker

  curl::mqtt connect broker_url ?options?

• publish: Publish a message to a topic

  curl::mqtt publish broker_url topic message ?options?

• subscribe: Subscribe to a topic

  curl::mqtt subscribe broker_url topic ?options?

• receive: Receive messages from subscribed topics

  curl::mqtt receive broker_url ?options?

• disconnect: Disconnect from an MQTT broker

  curl::mqtt disconnect broker_url ?options?

MQTT Options

The following options are supported for MQTT operations:

• -username: Username for authentication
• -password: Password for authentication
• -timeout: Timeout in seconds for receive operation

MQTT Testing

The MQTT implementation has been tested with public MQTT brokers:

1. test.mosquitto.org: A widely used public MQTT broker

   curl::mqtt publish "mqtt://test.mosquitto.org:1883" "test/topic" "Hello"

2. broker.emqx.io: Alternative public MQTT broker

   curl::mqtt publish "mqtt://broker.emqx.io:1883" "test/topic" "Hello"

License

This software is provided under the same license as libcurl.

WebSocket Interface

TclCurl provides support for WebSocket connections, allowing for real-time, bidirectional communication.

WebSocket Direct Usage

# Source the WebSocket library
source [file join [file dirname [info script]] "websocket.tcl"]

# Connect to a WebSocket server
set ws [websocket::connect "wss://example.com/ws"]

# Send a message
websocket::send $ws "Hello, WebSocket!"

# Receive a response (with timeout in milliseconds)
set response [websocket::receive $ws 5000]
puts "Received: [dict get $response data]"

# Close the connection
websocket::close $ws

WebSocket API Reference

websocket::connect

Establishes a WebSocket connection to the specified URL.

websocket::connect url ?timeout?

• url: WebSocket URL (ws:// or wss://)
• timeout: Connection timeout in seconds (default: 30)

Returns a curl handle that can be used with other websocket commands.

websocket::send

Sends a WebSocket message.

websocket::send handle message ?type?

• handle: WebSocket handle from websocket::connect
• message: The message to send
• type: Message type (default: "text")
  • "text": Text frame
  • "binary": Binary frame
  • "ping": Ping frame
  • "pong": Pong frame
  • "close": Close frame

Returns the number of bytes sent.

websocket::receive

Receives a WebSocket message.

websocket::receive handle ?timeout?

• handle: WebSocket handle from websocket::connect
• timeout: Receive timeout in milliseconds (default: 10000)

Returns a dictionary with the following keys:

• data: The received message data
• bytes: Number of bytes received
• type: Message type as string (text, binary, close, ping, pong)
• flags: Message type as integer

Note: Ping frames are automatically responded to with pong frames.

websocket::close

Closes a WebSocket connection.

websocket::close handle

• handle: WebSocket handle from websocket::connect

Returns 1 on success.

websocket::debug

Enables or disables debug mode.

websocket::debug enable

• enable: 1 to enable, 0 to disable

Low-Level WebSocket Commands

TclCurl also provides low-level commands for WebSocket handling:

curl::wsconnect

Enables WebSocket mode on a curl handle.

curl::wsconnect handle

• handle: A curl handle

Returns a message indicating success.

$handle wssend

Sends a WebSocket message using a curl handle.

$handle wssend data ?type?

• data: The data to send
• type: The type of frame (text, binary, ping, pong, close)

Returns the number of bytes sent.

$handle wsrecv

Receives a WebSocket message using a curl handle.

$handle wsrecv

Returns a dictionary with the received message data and metadata.