FT_HTTP ============== HTTP protocol filter. Category: Protocol filter .. admonition:: **Supported flags** :collapsible: closed FF_DONT_FILTER_IN FF_DONT_FILTER_OUT FF_READ_ONLY_IN FF_READ_ONLY_OUT FF_HTTP_KEEP_PIPELINING FF_HTTP_INDICATE_SKIPPED_OBJECTS FF_HTTP_BLOCK_SPDY FF_HTTP_FILTER_WEBSOCKET FF_HTTP_DONT_UNCOMPRESS_CONTENT .. admonition:: **Object types** :collapsible: closed OT_HTTP_REQUEST outgoing HTTP data OT_HTTP_RESPONSE incoming HTTP data OT_HTTP_SKIPPED_REQUEST_COMPLETE a skipped HTTP request is completed OT_HTTP_SKIPPED_RESPONSE_COMPLETE a skipped HTTP response is completed OT_WEBSOCKET_REQUEST outgoing WebSocket data OT_WEBSOCKET_RESPONSE incoming WebSocket data Indicates object parts: yes HTTP objects contain 3 streams (indexed by ePF_HttpStream): HS_STATUS (0) first string of a request or response HS_HEADER (1) HTTP header HS_CONTENT (2) decoded and uncompressed content HTTP responses contain the following custom fields in header: X-EXHDR-REQUEST First string from appropriate HTTP request X-EXHDR-REQUEST-HOST Host field from appropriate HTTP request header Between receiving HTTP header and full content the library indicates object parts via dataPartAvailable event. dataAvailable is called when full object is available for filtering, unless dataPartAvailable returned DPCR_BYPASS, DPCR_BLOCK or DPCR_UPDATE_AND_BYPASS. In this case dataAvailable is called only when the flag FF_HTTP_INDICATE_SKIPPED_OBJECTS is specified for the filter, after completion of the current HTTP object. The completion is indicated as objects of types OT_HTTP_SKIPPED_REQUEST_COMPLETE/OT_HTTP_SKIPPED_RESPONSE_COMPLETE. For these objects the filter fills two streams: HS_STATUS and HS_HEADER. The objects of specified types are always read-only. It is possible to update object content in dataPartAvailable and return DPCR_UPDATE_AND_BYPASS or DPCR_UPDATE_AND_FILTER_READ_ONLY code. In this case the changes are applied to the available part of data. The rest of HTTP request/response is bypassed or the full object will be indicated in read-only mode via dataAvailable, according to returned code. This method is useful in case if it is necessary to change only small part of HTTP request/response at the beginning without waiting until the full response is downloaded from server, e.g. insert a script to HTML page. FF_HTTP_KEEP_PIPELINING ------------------------ By default the filter sends pipelined requests by one, after receiving a response from server for a previous request. The flag FF_HTTP_KEEP_PIPELINING instructs the filter to send all pipelined requests as-is. FF_HTTP_DONT_UNCOMPRESS_CONTENT ------------------------------- The filter does not decompress the objects having Content-Type: application/* in header. This is because some servers specify invalid Content-Type for this MIME category. The filtering application can analyze value of Content-Encoding header value and decompress the data if needed after deeper analysis. The flag FF_HTTP_DONT_UNCOMPRESS_CONTENT disables decompression for HTTP content. QUIC protocol -------------- Google Chrome and other browsers can switch to UDP based QUIC instead of TCP and HTTP. As a workaround it is possible to block the protocols, which cannot be filtered. In this case the browsers switch to TCP and HTTP/1.x and/or HTTP/2. SPDY protocol can be disabled by adding FF_HTTP_BLOCK_SPDY flag for HTTP filter: pf_addFilter(FT_HTTP, FF_HTTP_BLOCK_SPDY); To disable QUIC add blocking UDP rules for the default remote ports 80/443: C/C++: .. code-block:: NF_RULE rule; memset(&rule, 0, sizeof(rule)); rule.protocol = IPPROTO_UDP; rule.remotePort = htons(80); rule.filteringFlag = NF_BLOCK; nf_addRule(&rule, TRUE); memset(&rule, 0, sizeof(rule)); rule.protocol = IPPROTO_UDP; rule.remotePort = htons(443); rule.filteringFlag = NF_BLOCK; nf_addRule(&rule, TRUE); Delphi: .. code-block:: FillChar(rule, sizeof(rule), 0); rule.protocol := IPPROTO_UDP; rule.remotePort := ntohs(80); rule.filteringFlag := NF_BLOCK; nf_addRule(rule, 1); rule.protocol := IPPROTO_UDP; rule.remotePort := ntohs(443); rule.filteringFlag := NF_BLOCK; nf_addRule(rule, 1); C#: .. code-block:: rule = new NF_RULE(); rule.protocol = (int)ProtocolType.Udp; rule.remotePort = (ushort)IPAddress.HostToNetworkOrder((Int16)80); rule.filteringFlag = (uint)NF_FILTERING_FLAG.NF_BLOCK; NFAPI.nf_addRule(rule, 0); rule = new NF_RULE(); rule.protocol = (int)ProtocolType.Udp; rule.remotePort = (ushort)IPAddress.HostToNetworkOrder((Int16)443); rule.filteringFlag = (uint)NF_FILTERING_FLAG.NF_BLOCK; NFAPI.nf_addRule(rule, 0); HTTP/2 -------- HTTP/2 is disabled by default, because SSL filter ignores the attempts to negotiate the next protocol via TLS extension. In result the browsers switch to generic HTTP/1.x. FT_HTTP2 filter must be added in tcpConnected event to enable negotiation. FF_HTTP_FILTER_WEBSOCKET -------------------------- When FF_HTTP_FILTER_WEBSOCKET flag is specified for FT_HTTP filter, WebSocket protocol is parsed after initial HTTP handshake with protocol upgrade. By default HTTP filter is removed from filtering chain after start of WebSocket data exchange, so the flag FF_HTTP_FILTER_WEBSOCKET is required to filter WebSocket data. The decoded WebSocket requests and responses are indicated via dataAvailable event as OT_WEBSOCKET_REQUEST and OT_WEBSOCKET_RESPONSE objects. WebSocket objects contain 2 streams (indexed by ePF_WebSocketStream): WSS_PAYLOAD (0) decoded and decompressed WebSocket payload data WSS_HEADER (1) WebSocket frame flags, formatted as HTTP header WebSocket header fields: OPCODE frame type, containing one of the values: OpCodeContinuation = 0x0, OpCodeText = 0x1, OpCodeBinary = 0x2, OpCodeClose = 0x8, OpCodePing = 0x9, OpCodePong = 0xA, OpCodeInvalid = 0x10. See RFC6455 for descriptions. MASK 1 or 0, specifies if initially the payload data was encrypted COMPRESS 1 or 0, specifies if initially the payload data was compressed FINAL 1 or 0, specifies if the payload data is the final fragment in a message The current implementation of WebSocket filter only decompresses the payload data, but doesn't compress the generated WebSocket frames. To avoid this modification it is possible to use FF_READ_ONLY_IN and FF_READ_ONLY_OUT flags for HTTP filter. References: - http://www.faqs.org/rfcs/rfc2616.html - http://www.faqs.org/rfcs/rfc6455.html