FT_HTTP

FT_HTTP

HTTP protocol filter.

Category: Protocol filter

Supported flags:
FF_DONT_FILTER_IN
FF_DONT_FILTER_OUT
FF_READ_ONLY_IN
FF_READ_ONLY_OUT
FF_HTTP_KEEP_PIPELINING
FF_HTTP_INDICATE_SKIPPED_OBJECTS
FF_HTTP_BLOCK_SPDY
FF_HTTP_FILTER_WEBSOCKET
FF_HTTP_DONT_UNCOMPRESS_CONTENT

Object types:
OT_HTTP_REQUEST - outgoing HTTP data
OT_HTTP_RESPONSE - incoming HTTP data
OT_HTTP_SKIPPED_REQUEST_COMPLETE - a skipped HTTP request is completed
OT_HTTP_SKIPPED_RESPONSE_COMPLETE - a skipped HTTP response is completed
OT_WEBSOCKET_REQUEST - outgoing WebSocket data
OT_WEBSOCKET_RESPONSE - incoming WebSocket data

Indicates object parts: yes

HTTP objects contain 3 streams (indexed by ePF_HttpStream):
HS_STATUS (0) - first string of a request or response
HS_HEADER (1) - HTTP header
HS_CONTENT (2) - decoded and uncompressed content

HTTP responses contain the following custom fields in header:
X-EXHDR-REQUEST - First string from appropriate HTTP request
X-EXHDR-REQUEST-HOST - Host field from appropriate HTTP request header

Between receiving HTTP header and full content the library indicates object parts via dataPartAvailable event. dataAvailable is called when full object is available for filtering, unless dataPartAvailable returned DPCR_BYPASS, DPCR_BLOCK or DPCR_UPDATE_AND_BYPASS. In this case dataAvailable is called only when the flag FF_HTTP_INDICATE_SKIPPED_OBJECTS is specified for the filter, after completion of the current HTTP object. The completion is indicated as objects of types OT_HTTP_SKIPPED_REQUEST_COMPLETE/OT_HTTP_SKIPPED_RESPONSE_COMPLETE. For these objects the filter fills two streams: HS_STATUS and HS_HEADER. The objects of specified types are always read-only.

It is possible to update object content in dataPartAvailable and return DPCR_UPDATE_AND_BYPASS or DPCR_UPDATE_AND_FILTER_READ_ONLY code. In this case the changes are applied to the available part of data. The rest of HTTP request/response is bypassed or the full object will be indicated in read-only mode via dataAvailable, according to returned code. This method is useful in case if it is necessary to change only small part of HTTP request/response at the beginning without waiting until the full response is downloaded from server, e.g. insert a script to HTML page.

By default the filter sends pipelined requests by one, after receiving a response from server for a previous request. The flag FF_HTTP_KEEP_PIPELINING instructs the filter to send all pipelined requests as-is.

The filter does not decompress the objects having Content-Type: application/* in header. This is because some servers specify invalid Content-Type for this MIME category. The filtering application can analyze value of Content-Encoding header value and decompress the data if needed after deeper analysis.

The flag FF_HTTP_DONT_UNCOMPRESS_CONTENT disables decompression for HTTP content.

Google Chrome and other browsers can switch to UDP based QUIC instead of TCP and HTTP. As a workaround it is possible to block the protocols, which cannot be filtered. In this case the browsers switch to TCP and HTTP/1.x and/or HTTP/2.

SPDY protocol can be disabled by adding FF_HTTP_BLOCK_SPDY flag for HTTP filter:
pf_addFilter(FT_HTTP, FF_HTTP_BLOCK_SPDY);

To disable QUIC add blocking UDP rules for the default remote ports 80/443:

C/C++:

NF_RULE rule;

memset(&rule, 0, sizeof(rule));
rule.protocol = IPPROTO_UDP;
rule.remotePort = htons(80);
rule.filteringFlag = NF_BLOCK;
nf_addRule(&rule, TRUE);

memset(&rule, 0, sizeof(rule));
rule.protocol = IPPROTO_UDP;
rule.remotePort = htons(443);
rule.filteringFlag = NF_BLOCK;
nf_addRule(&rule, TRUE);

Delphi:

FillChar(rule, sizeof(rule), 0);

rule.protocol := IPPROTO_UDP;
rule.remotePort := ntohs(80);
rule.filteringFlag := NF_BLOCK;
nf_addRule(rule, 1);

rule.protocol := IPPROTO_UDP;
rule.remotePort := ntohs(443);
rule.filteringFlag := NF_BLOCK;
nf_addRule(rule, 1);

C#:

rule = new NF_RULE();
rule.protocol = (int)ProtocolType.Udp;
rule.remotePort = (ushort)IPAddress.HostToNetworkOrder((Int16)80);
rule.filteringFlag = (uint)NF_FILTERING_FLAG.NF_BLOCK;
NFAPI.nf_addRule(rule, 0);

rule = new NF_RULE();
rule.protocol = (int)ProtocolType.Udp;
rule.remotePort = (ushort)IPAddress.HostToNetworkOrder((Int16)443);
rule.filteringFlag = (uint)NF_FILTERING_FLAG.NF_BLOCK;
NFAPI.nf_addRule(rule, 0);

HTTP/2 is disabled by default, because SSL filter ignores the attempts to negotiate the next protocol via TLS extension.

When FF_HTTP_FILTER_WEBSOCKET flag is specified for FT_HTTP filter, WebSocket protocol is parsed after initial HTTP handshake with protocol upgrade. By default HTTP filter is removed from filtering chain after start of WebSocket data exchange, so the flag FF_HTTP_FILTER_WEBSOCKET is required to filter WebSocket data. The decoded WebSocket requests and responses are indicated via dataAvailable event as OT_WEBSOCKET_REQUEST and OT_WEBSOCKET_RESPONSE objects.

WebSocket objects contain 2 streams (indexed by ePF_WebSocketStream):
WSS_PAYLOAD (0) - decoded and decompressed WebSocket payload data
WSS_HEADER (1) - WebSocket frame flags, formatted as HTTP header

WebSocket header fields:
OPCODE - frame type, containing one of the values: OpCodeContinuation = 0x0, OpCodeText = 0x1, OpCodeBinary = 0x2, OpCodeClose = 0x8, OpCodePing = 0x9, OpCodePong = 0xA, OpCodeInvalid = 0x10. See RFC6455 for descriptions.
MASK - 1 or 0, specifies if initially the payload data was encrypted
COMPRESS - 1 or 0, specifies if initially the payload data was compressed
FINAL - 1 or 0, specifies if the payload data is the final fragment in a message

The current implementation of WebSocket filter only decompresses the payload data, but doesn't compress the generated WebSocket frames. To avoid this modification it is possible to use FF_READ_ONLY_IN and FF_READ_ONLY_OUT flags for HTTP filter.

References:

http://www.faqs.org/rfcs/rfc2616.html
http://www.faqs.org/rfcs/rfc6455.html