FT_HTTP¶

HTTP protocol filter.

Category:
Protocol filter

Supported flags

FF_DONT_FILTER_IN

FF_DONT_FILTER_OUT

FF_READ_ONLY_IN

FF_READ_ONLY_OUT

FF_HTTP_KEEP_PIPELINING

FF_HTTP_INDICATE_SKIPPED_OBJECTS

FF_HTTP_BLOCK_SPDY

FF_HTTP_FILTER_WEBSOCKET

FF_HTTP_DONT_UNCOMPRESS_CONTENT

Object types

OT_HTTP_REQUEST
outgoing HTTP data

OT_HTTP_RESPONSE
incoming HTTP data

OT_HTTP_SKIPPED_REQUEST_COMPLETE
a skipped HTTP request is completed

OT_HTTP_SKIPPED_RESPONSE_COMPLETE
a skipped HTTP response is completed

OT_WEBSOCKET_REQUEST
outgoing WebSocket data

OT_WEBSOCKET_RESPONSE
incoming WebSocket data

Indicates object parts:
yes

HTTP objects contain 3 streams (indexed by ePF_HttpStream):

HS_STATUS (0): first string of a request or response
HS_HEADER (1): HTTP header
HS_CONTENT (2): decoded and uncompressed content

HTTP responses contain the following custom fields in header:

X-EXHDR-REQUEST: First string from appropriate HTTP request
X-EXHDR-REQUEST-HOST: Host field from appropriate HTTP request header

Between receiving HTTP header and full content the library indicates object parts via dataPartAvailable event. dataAvailable is called when full object is available for filtering, unless dataPartAvailable returned DPCR_BYPASS, DPCR_BLOCK or DPCR_UPDATE_AND_BYPASS. In this case dataAvailable is called only when the flag FF_HTTP_INDICATE_SKIPPED_OBJECTS is specified for the filter, after completion of the current HTTP object. The completion is indicated as objects of types OT_HTTP_SKIPPED_REQUEST_COMPLETE/OT_HTTP_SKIPPED_RESPONSE_COMPLETE. For these objects the filter fills two streams: HS_STATUS and HS_HEADER. The objects of specified types are always read-only.

It is possible to update object content in dataPartAvailable and return DPCR_UPDATE_AND_BYPASS or DPCR_UPDATE_AND_FILTER_READ_ONLY code. In this case the changes are applied to the available part of data. The rest of HTTP request/response is bypassed or the full object will be indicated in read-only mode via dataAvailable, according to returned code. This method is useful in case if it is necessary to change only small part of HTTP request/response at the beginning without waiting until the full response is downloaded from server, e.g. insert a script to HTML page.

FF_HTTP_KEEP_PIPELINING¶

By default the filter sends pipelined requests by one, after receiving a response from server for a previous request. The flag FF_HTTP_KEEP_PIPELINING instructs the filter to send all pipelined requests as-is.

FF_HTTP_DONT_UNCOMPRESS_CONTENT¶

The filter does not decompress the objects having Content-Type: application/* in header. This is because some servers specify invalid Content-Type for this MIME category. The filtering application can analyze value of Content-Encoding header value and decompress the data if needed after deeper analysis.

The flag FF_HTTP_DONT_UNCOMPRESS_CONTENT disables decompression for HTTP content.

QUIC protocol¶

Google Chrome and other browsers can switch to UDP based QUIC instead of TCP and HTTP. As a workaround it is possible to block the protocols, which cannot be filtered. In this case the browsers switch to TCP and HTTP/1.x and/or HTTP/2.

SPDY protocol can be disabled by adding FF_HTTP_BLOCK_SPDY flag for HTTP filter: pf_addFilter(FT_HTTP, FF_HTTP_BLOCK_SPDY);

To disable QUIC add blocking UDP rules for the default remote ports 80/443:

C/C++:

NF_RULE rule;

memset(&rule, 0, sizeof(rule));
rule.protocol = IPPROTO_UDP;
rule.remotePort = htons(80);
rule.filteringFlag = NF_BLOCK;
nf_addRule(&rule, TRUE);

memset(&rule, 0, sizeof(rule));
rule.protocol = IPPROTO_UDP;
rule.remotePort = htons(443);
rule.filteringFlag = NF_BLOCK;
nf_addRule(&rule, TRUE);

Delphi:

FillChar(rule, sizeof(rule), 0);

rule.protocol := IPPROTO_UDP;
rule.remotePort := ntohs(80);
rule.filteringFlag := NF_BLOCK;
nf_addRule(rule, 1);

rule.protocol := IPPROTO_UDP;
rule.remotePort := ntohs(443);
rule.filteringFlag := NF_BLOCK;
nf_addRule(rule, 1);

C#:

rule = new NF_RULE();
rule.protocol = (int)ProtocolType.Udp;
rule.remotePort = (ushort)IPAddress.HostToNetworkOrder((Int16)80);
rule.filteringFlag = (uint)NF_FILTERING_FLAG.NF_BLOCK;
NFAPI.nf_addRule(rule, 0);

rule = new NF_RULE();
rule.protocol = (int)ProtocolType.Udp;
rule.remotePort = (ushort)IPAddress.HostToNetworkOrder((Int16)443);
rule.filteringFlag = (uint)NF_FILTERING_FLAG.NF_BLOCK;
NFAPI.nf_addRule(rule, 0);

HTTP/2¶

HTTP/2 is disabled by default, because SSL filter ignores the attempts to negotiate the next protocol via TLS extension. In result the browsers switch to generic HTTP/1.x. FT_HTTP2 filter must be added in tcpConnected event to enable negotiation.

FF_HTTP_FILTER_WEBSOCKET¶

When FF_HTTP_FILTER_WEBSOCKET flag is specified for FT_HTTP filter, WebSocket protocol is parsed after initial HTTP handshake with protocol upgrade. By default HTTP filter is removed from filtering chain after start of WebSocket data exchange, so the flag FF_HTTP_FILTER_WEBSOCKET is required to filter WebSocket data. The decoded WebSocket requests and responses are indicated via dataAvailable event as OT_WEBSOCKET_REQUEST and OT_WEBSOCKET_RESPONSE objects.

WebSocket objects contain 2 streams (indexed by ePF_WebSocketStream):

WSS_PAYLOAD (0)
decoded and decompressed WebSocket payload data

WSS_HEADER (1)
WebSocket frame flags, formatted as HTTP header

WebSocket header fields:

OPCODE
frame type, containing one of the values: OpCodeContinuation = 0x0, OpCodeText = 0x1, OpCodeBinary = 0x2, OpCodeClose = 0x8, OpCodePing = 0x9, OpCodePong = 0xA, OpCodeInvalid = 0x10. See RFC6455 for descriptions.

MASK
1 or 0, specifies if initially the payload data was encrypted

COMPRESS
1 or 0, specifies if initially the payload data was compressed

FINAL
1 or 0, specifies if the payload data is the final fragment in a message

The current implementation of WebSocket filter only decompresses the payload data, but doesn’t compress the generated WebSocket frames. To avoid this modification it is possible to use FF_READ_ONLY_IN and FF_READ_ONLY_OUT flags for HTTP filter.

References: