WebSocket in Curl

 

WebSocket in Curl

URL WebSocket Communication

URL WebSocket communication with libcurl is achieved by setting up a transfer to a URL using ws:// or wss:// URL schemes. The latter is the secure version, handled over HTTPS.

When using wss:// (WebSocket over HTTPS), standard TLS and HTTPS options are acknowledged for the CA and certificate verification.

WebSocket communication is done by upgrading a connection from either HTTP or HTTPS. When given a WebSocket URL, libcurl considers it a transfer failure if the upgrade procedure fails. This means that a plain HTTP 200 response code is considered an error for this work.

API

The WebSocket API is described in the individual man pages for the new API.

libcurl can handle WebSocket communication in two ways:

1. Get the WebSocket frames from the server sent to the write callback. You can then respond with curl_ws_send() from within the callback (or outside of it).
2. Set CURLOPT_CONNECT_ONLY to 2L (new for WebSocket). This makes libcurl perform an HTTP GET + Upgrade: request plus response in the curl_easy_perform() call before it returns. In this case, you can use curl_ws_recv() and curl_ws_send() to receive and send WebSocket frames from and to the server.

New Options to curl_easy_setopt():

• CURLOPT_WS_OPTIONS: To control specific behavior.
• CURLWS_RAW_MODE: To make libcurl provide all WebSocket traffic raw in the callback.
• CURLWS_NOAUTOPONG: To disable automatic PONG replies.

New Function Calls:

• curl_ws_recv() - receive a WebSocket frame.
• curl_ws_send() - send a WebSocket frame.
• curl_ws_meta() - return WebSocket metadata within a write callback.

Max Frame Size

The current implementation only supports frame sizes up to a max (64K right now). This is because the API delivers full frames, and libcurl cannot manage a full $2^{63}$ bytes size.

If we decide to support much larger frames than 64K, we need to adjust the API accordingly to be able to deliver partial frames in both directions.

Errors

If the given WebSocket URL (using ws:// or wss://) fails to get upgraded via a 101 response code and instead gets another response code back from the HTTP server, the transfer returns CURLE_HTTP_RETURNED_ERROR for that transfer. Note that even 2xx response codes are considered errors since it failed to provide a WebSocket transfer.

Test Suite

The following section outlines the original approach:

I looked for an existing small WebSocket server implementation with maximum flexibility to dissect and cram into the test suite, but decided that extending the existing test suite server (sws) to deal with WebSocket might be the better way. This server is already integrated and working in the test suite.

Command Line Tool: WebSocket

The plan is to make curl do WebSocket similar to telnet/nc. This part of the work has not been started yet.

Ideas:

• Read stdin and send off as messages.
• Consider newline as the end of a fragment.
• (Default to text? Offer an option to set binary.)
• Respond to PINGs automatically.
• Issue PINGs at some default interval (option to switch off/change interval?).
• Allow -d to specify (initial) data to send (should the format allow for multiple separate frames?).
• Exit after N messages received, where N can be zero.

Future Work:

• Verify the Sec-WebSocket-Accept response. It requires a SHA-1 function.
• Verify Sec-WebSocket-Extensions and Sec-WebSocket-Protocol in the response.
• Consider a curl_ws_poll().
• Make sure WebSocket code paths are fuzzed.
• Add client-side PING interval.
• Provide an option to disable PING-PONG automation.
• Support compression (CURLWS_COMPRESS).

Why Not libWebSocket?

libWebSocket is said to be a solid, fast, and efficient WebSocket library with a vast amount of users. My plan was originally to build upon it to skip having to implement the low-level parts of WebSocket myself.

Here are the reasons why the decision was made to move forward with WebSocket in curl without using libWebSocket:

• Doxygen Generated Docs: Only makes them hard to navigate. There are no tutorials or clearly written explanatory pages for specific functions.
• Integration: It seems (too) tightly integrated with a specific TLS library, while we want to support WebSocket with whatever TLS library libcurl was already made to work with.
• Integration: It seems (too) tightly integrated with event libraries.
• Complexity: The references to threads and thread-pools in code and APIs indicate too much logic for our purposes.
• Bloated Library: It is a huge library that is actually more lines of code than libcurl itself.
• WebSocket Simplicity: WebSocket is a fairly simple protocol on the network/framing layer, so making a homegrown handling of it should be fine.