FT_HTTP¶

HTTP protocol filter.

Category:
Protocol filter

Supported flags

FF_DONT_FILTER_IN

FF_DONT_FILTER_OUT

FF_READ_ONLY_IN

FF_READ_ONLY_OUT

FF_HTTP_KEEP_PIPELINING

FF_HTTP_INDICATE_SKIPPED_OBJECTS

FF_HTTP_BLOCK_SPDY

FF_HTTP_FILTER_WEBSOCKET

FF_HTTP_DONT_UNCOMPRESS_CONTENT

Object types

OT_HTTP_REQUEST
outgoing HTTP data

OT_HTTP_RESPONSE
incoming HTTP data

OT_HTTP_SKIPPED_REQUEST_COMPLETE
a skipped HTTP request is completed

OT_HTTP_SKIPPED_RESPONSE_COMPLETE
a skipped HTTP response is completed

OT_WEBSOCKET_REQUEST
outgoing WebSocket data

OT_WEBSOCKET_RESPONSE
incoming WebSocket data

Indicates object parts:
yes

HTTP objects contain 3 streams (indexed by ePF_HttpStream):

HS_STATUS (0): first string of a request or response
HS_HEADER (1): HTTP header
HS_CONTENT (2): decoded and uncompressed content

HTTP responses contain the following custom fields in the header:

X-EXHDR-REQUEST: First string from the corresponding HTTP request
X-EXHDR-REQUEST-HOST: Host field from the corresponding HTTP request header

Between receiving the HTTP header and the full content, the library indicates object parts via the dataPartAvailable event. dataAvailable is called when the full object is available for filtering, unless dataPartAvailable returned DPCR_BYPASS, DPCR_BLOCK or DPCR_UPDATE_AND_BYPASS. In this case, dataAvailable is called only when the FF_HTTP_INDICATE_SKIPPED_OBJECTS flag is specified for the filter, after completion of the current HTTP object. Completion is indicated as objects of types OT_HTTP_SKIPPED_REQUEST_COMPLETE/OT_HTTP_SKIPPED_RESPONSE_COMPLETE. For these objects, the filter fills two streams: HS_STATUS and HS_HEADER. The objects of the specified types are always read-only.

It is possible to update object content in dataPartAvailable and return the DPCR_UPDATE_AND_BYPASS or DPCR_UPDATE_AND_FILTER_READ_ONLY code. In this case, the changes are applied to the available part of the data. The rest of the HTTP request/response is bypassed, or the full object will be indicated in read-only mode via dataAvailable, according to the returned code. This method is useful if it is necessary to change only a small part of the HTTP request/response at the beginning without waiting until the full response is downloaded from the server, e.g. to insert a script into an HTML page.

FF_HTTP_KEEP_PIPELINING¶

By default, the filter sends pipelined requests one by one, after receiving a response from the server for the previous request. The FF_HTTP_KEEP_PIPELINING flag instructs the filter to send all pipelined requests as-is.

FF_HTTP_DONT_UNCOMPRESS_CONTENT¶

The filter does not decompress objects having Content-Type: application/* in the header. This is because some servers specify an invalid Content-Type for this MIME category. The filtering application can analyze the value of the Content-Encoding header and decompress the data if needed after deeper analysis.

The FF_HTTP_DONT_UNCOMPRESS_CONTENT flag disables decompression for HTTP content.

QUIC protocol¶

Google Chrome and other browsers can switch to the UDP-based QUIC protocol instead of TCP and HTTP. As a workaround, it is possible to block protocols that cannot be filtered. In this case, browsers switch to TCP and HTTP/1.x and/or HTTP/2.

SPDY protocol can be disabled by adding FF_HTTP_BLOCK_SPDY flag for HTTP filter: pf_addFilter(FT_HTTP, FF_HTTP_BLOCK_SPDY);

To disable QUIC add blocking UDP rules for the default remote ports 80/443:

C/C++:

NF_RULE rule;

memset(&rule, 0, sizeof(rule));
rule.protocol = IPPROTO_UDP;
rule.remotePort = htons(80);
rule.filteringFlag = NF_BLOCK;
nf_addRule(&rule, TRUE);

memset(&rule, 0, sizeof(rule));
rule.protocol = IPPROTO_UDP;
rule.remotePort = htons(443);
rule.filteringFlag = NF_BLOCK;
nf_addRule(&rule, TRUE);

Delphi:

FillChar(rule, sizeof(rule), 0);

rule.protocol := IPPROTO_UDP;
rule.remotePort := ntohs(80);
rule.filteringFlag := NF_BLOCK;
nf_addRule(rule, 1);

rule.protocol := IPPROTO_UDP;
rule.remotePort := ntohs(443);
rule.filteringFlag := NF_BLOCK;
nf_addRule(rule, 1);

C#:

rule = new NF_RULE();
rule.protocol = (int)ProtocolType.Udp;
rule.remotePort = (ushort)IPAddress.HostToNetworkOrder((Int16)80);
rule.filteringFlag = (uint)NF_FILTERING_FLAG.NF_BLOCK;
NFAPI.nf_addRule(rule, 0);

rule = new NF_RULE();
rule.protocol = (int)ProtocolType.Udp;
rule.remotePort = (ushort)IPAddress.HostToNetworkOrder((Int16)443);
rule.filteringFlag = (uint)NF_FILTERING_FLAG.NF_BLOCK;
NFAPI.nf_addRule(rule, 0);

HTTP/2¶

HTTP/2 is disabled by default because the SSL filter ignores attempts to negotiate the next protocol via the TLS extension. As a result, browsers switch to generic HTTP/1.x. The FT_HTTP2 filter must be added in the tcpConnected event to enable negotiation.

FF_HTTP_FILTER_WEBSOCKET¶

When the FF_HTTP_FILTER_WEBSOCKET flag is specified for the FT_HTTP filter, the WebSocket protocol is parsed after the initial HTTP handshake with protocol upgrade. By default, the HTTP filter is removed from the filtering chain after the start of WebSocket data exchange, so the FF_HTTP_FILTER_WEBSOCKET flag is required to filter WebSocket data. The decoded WebSocket requests and responses are indicated via the dataAvailable event as OT_WEBSOCKET_REQUEST and OT_WEBSOCKET_RESPONSE objects.

WebSocket objects contain 2 streams (indexed by ePF_WebSocketStream):

WSS_PAYLOAD (0)
decoded and decompressed WebSocket payload data

WSS_HEADER (1)
WebSocket frame flags, formatted as HTTP header

WebSocket header fields:

OPCODE
Frame type, containing one of the values: OpCodeContinuation = 0x0, OpCodeText = 0x1, OpCodeBinary = 0x2, OpCodeClose = 0x8, OpCodePing = 0x9, OpCodePong = 0xA, OpCodeInvalid = 0x10. See RFC6455 for descriptions.

MASK
1 or 0, specifies if the payload data was initially encrypted

COMPRESS
1 or 0, specifies if the payload data was initially compressed

FINAL
1 or 0, specifies if the payload data is the final fragment in a message

The current implementation of the WebSocket filter only decompresses the payload data, but doesn’t compress the generated WebSocket frames. To avoid this modification, it is possible to use the FF_READ_ONLY_IN and FF_READ_ONLY_OUT flags for the HTTP filter.

References: