FT_SSL

SSL decoder/encoder.

Category: Preprocessor

Supported flags:
FF_SSL_TLS
FF_SSL_SELF_SIGNED_CERTIFICATE
FF_SSL_INDICATE_HANDSHAKE_REQUESTS
FF_SSL_TLS_AUTO
FF_SSL_COMPATIBILITY
FF_SSL_VERIFY
FF_SSL_SUPPORT_CLIENT_CERTIFICATES
FF_SSL_INDICATE_SERVER_CERTIFICATES
FF_SSL_INDICATE_EXCEPTIONS
FF_SSL_ENABLE_ALPN
FF_SSL_INDICATE_CLIENT_CERT_REQUESTS
FF_SSL_DECODE_ONLY
FF_SSL_KEEP_SERIAL_NUMBERS
FF_SSL_DONT_IMPORT_SELF_SIGNED
FF_SSL_DISABLE_TLS_1_0
FF_SSL_DISABLE_TLS_1_1
FF_SSL_TLS_COMPATIBILITY
FF_SSL_STRICT_VERIFICATION
FF_SSL_SCT_VERIFICATION
FF_SSL_INDICATE_ALPN_SELECT_PROTOCOL

Object types:
OT_SSL_HANDSHAKE_OUTGOING
OT_SSL_HANDSHAKE_INCOMING
OT_SSL_INVALID_SERVER_CERTIFICATE
OT_SSL_SERVER_CERTIFICATE
OT_SSL_EXCEPTION
OT_SSL_CLIENT_CERT_REQUEST
OT_SSL_HANDSHAKE_OUTGOING_PROTOCOL

Indicates object parts: yes

The filter uses man in the middle scheme for decoding the encrypted content. It detects SSL handshakes automatically and generates a compatible SSL certificate for each server. The generated certificates are saved to cert.db file in SSL subfolder of configuration folder specified in pf_init call. The certificates in cert.db are encrypted. By default the filter generates a chained certificate, inherited from a common root with default subject "NetFilterSDK". It is possible to change this name using pf_setRootSSLCertSubject function. The generated root certificate is added to automatically to the following storages:
- Windows certificate storage.
- The certificate storages used by Mozilla products (Firefox, Thunderbird) and other products using the same library for SSL communications. There must be NSS folder containing Mozilla libraries, in the same directory with executable.
- Opera certificate storage.

To disable warnings in the applications using own certificate storages, import ProtocolFilters root certificate to their storage manually. The root certificate is stored in <configuration folder>\SSL\<root name>.cer file, where <configuration folder> is a folder specified in pf_init call, and <root name> is a name specified in pf_setRootSSLCertSubject call. The default root certificate file name is NetFilterSDK.cer. The private key is the same for generated root certificates, unless RSIF_GENERATE_ROOT_PRIVATE_KEY flag is specified in pf_setRootSSLCertImportFlags call before the first run of pf_setRootSSLCertSubject.

When the flag FF_SSL_SELF_SIGNED_CERTIFICATE is specified for FT_SSL filter, the library generates self signed certificates for filtered SSL sessions. The applications using Windows certificate storage like Internet Explorer work without warnings in this case too. But the other applications like Firefox show alerts and do not allow to continue transmitting the data until receiving a confirmation from user.

In some cases it is not possible to filter SSL protected data. For example when a client denies the generated server certificates. In this case the filter counts unsuccessful attempts to filter encrypted connection from some process to some host. After second unsuccessful attempt it stops trying to decode SSL, and creates a record in x.db file in SSL folder to avoid blocking the data transmittion in future.


When FF_SSL_INDICATE_HANDSHAKE_REQUESTS flag is specified in pf_addFilter, the filter indicates SSL handshake requests via dataPartAvailable event. The indicated objects contain one stream with a host name, specified in extended TLS field by a client application. The stream contains IP address in case if the client application doesn't specify host name during handshake. The object type is OT_SSL_HANDSHAKE_OUTGOING for outgoing connections, OT_SSL_HANDSHAKE_INCOMING for incoming connections. dataPartAvailable can return one of the following codes:
DPCR_FILTER - continue filtering SSL and decode/encode the transmitted data.
DPCR_BYPASS - don't filter SSL, remove SSL filter from chain and bypass the transmitted packets as-is.
DPCR_BLOCK - close the connection.
DPCR_FILTER_READ_ONLY - enable the flag FF_SSL_DECODE_ONLY and continue filtering SSL without encoding the data between proxy and server.

When the flag FF_SSL_DECODE_ONLY is enabled, the traffic between proxy and server is not encoded. It is useful for example when a local or remote server should handle both HTTP and HTTPS traffic as generic HTTP. It is possible to enable the flag dynamically, by returning DPCR_FILTER_READ_ONLY from dataPartAvailable for OT_SSL_HANDSHAKE_OUTGOING object, containing some known domain name.

When the flag FF_SSL_TLS_AUTO is enabled, the filter tries to detect TLS handshake automatically in first 8 kilobytes of packets, transmitted after adding the filter. It is useful for decoding TLS for unknown protocols, which establish encoded session dynamically using a special protocol command.

The flag FF_SSL_COMPATIBILITY is required to avoid issues with RPC over HTTPS protocol in MS Outlook. It is necessary to use this flag during filtering HTTPS connections of Outlook. When this flag is enabled, the filter uses RC4 algorithm for SSL session and doesn't change the size of transmitted packets. In other cases RPC over HTTPS sometimes hangs in the middle of communicating with Exchange via proxy.

When FF_SSL_VERIFY flag is specified, SSL filter verifies the server certificates using Windows API. When a server returns invalid certificate, for example revoked, expired, self-signed or with untrusted root, the filter calls dataPartAvailable with object having type OT_SSL_INVALID_SERVER_CERTIFICATE.

The indicated objects with type OT_SSL_INVALID_SERVER_CERTIFICATE have the following streams:
SSL_ICS_SUBJECT = 0 - Certificate subject name
SSL_ICS_DOMAIN = 1 - Domain name from TLS SNI field 
SSL_ICS_ERROR = 2 - String with hexademical error number. It is a bitmask with CERT_TRUST_* error bits 
SSL_ICS_CERTIFICATE = 3 - Server certificate bytes in DER format

dataPartAvailable can return one of the following codes:
DPCR_FILTER - continue filtering SSL and decode/encode the transmitted data. In this case SSL filter substitutes the invalid certificate with own generated certificate having the same time values, but valid root.
DPCR_BYPASS - don't filter SSL. Block the connection once, add host to the list of exceptions and don't filter the connections with this host:port in future. The connection is blocked because there is no way to bypass the data immediately at the stage when server certificate is already requested.
DPCR_BLOCK - close the connection.

If the certificate is not revoked, SSL filter validates it each 24 hours.

The flag FF_SSL_SUPPORT_CLIENT_CERTIFICATES can be specified for filtering SSL in case when a server requires client certificate. By default SSL filter adds exception in this case. If FF_SSL_SUPPORT_CLIENT_CERTIFICATES flag is specified, the filter requests a client certificate, tries to find an appropriate certificate with exportable private key in Windows storage, then uses the found certificate for filtering SSL. So in this case SSL filter requires the following:
- The client certificate must be imported to Windows storage with exportable private key, even if the client application uses own certificate storage. I.e. it is necessary to mark the private key as exportable during importing the certificate from *.pfx or *.p12 file.
- To have access to all user certificate storages the filtering application must run as a service under LocalSystem account.

When FF_SSL_INDICATE_SERVER_CERTIFICATES flag is specified in pf_addFilter, the filter indicates SSL server certificates via dataPartAvailable event.
The indicated objects with type OT_SSL_SERVER_CERTIFICATE have 3 streams:
SSL_SCS_CERTIFICATE = 0 - Server certificate bytes
SSL_SCS_SUBJECT = 1 - Certificate subject name
SSL_SCS_ISSUER = 2 -  Certificate issuer name

The filtering application can analyze the certificate and return one of the following codes:
DPCR_FILTER - continue filtering SSL and decode/encode the transmitted data.
DPCR_BYPASS - don't filter SSL. Block the connection once, add host to the list of exceptions and don't filter the connections with this host:port in future. The connection is blocked because there is no way to bypass the data immediately at the stage when server certificate is already requested.
DPCR_BLOCK - close the connection.

When FF_SSL_INDICATE_CLIENT_CERT_REQUESTS flag is enabled, and server requests a client certificate, SSL filter indicates via dataPartAvailable an object of type OT_SSL_CLIENT_CERT_REQUEST containing the following streams:
SSL_CCS_DOMAIN = 0 -  Domain name from TLS SNI field
SSL_CCS_CERTIFICATE = 1 - Certificate bytes in DER format
SSL_CCS_PKEY = 2 - Private key bytes in DER format

The stream SSL_CCS_DOMAIN contains server domain name (or IP address the name is not specified in SNI). If dataPartAvailable returns DPCR_BYPASS, SSL filter adds host to internal list of exceptions and doesn't try to filter SSL for this host in future. If dataPartAvailable returns DPCR_FILTER, it is possible to specify a client side certificate in SSL_CCS_CERTIFICATE stream and private key in SSL_CCS_PKEY stream, in DER (ASN.1) format. If the certificate is not specified, the request is directed to client application. For any other return code the certificate requests are handled as described above when enabled the flag FF_SSL_SUPPORT_CLIENT_CERTIFICATES.

When FF_SSL_INDICATE_EXCEPTIONS flag is specified in pf_addFilter, the filter calls dataPartAvailable each time when server or client closes a connection during SSL handshake. In such cases SSL filter adds the appropriate pair remote host:port to the list of exceptions after 2 unsuccessful attempts to establish SSL connection.
The indicated objects with type OT_SSL_EXCEPTION have one stream, containing a remote host name from TLS SNI field, or remote endpoint string host:port in case if SNI field is empty.
dataPartAvailable can return DPCR_BLOCK to avoid adding the host to list of exceptions. If it returns any other code, the exception is added as usual. 

FF_SSL_ENABLE_ALPN flag enables TLS extension for negotiating next protocol (e.g. HTTP/2,SPDY,HTTP/1.1). The flag is disabled by default to block HTTP/2 and SPDY. In this case the browsers use generic HTTP 1.0 and 1.1 protocols, supported by HTTP filter.
When FT_HTTP2 filter is added, the flag FF_SSL_ENABLE_ALPN is added automatically. Also h2 protocol is negotiated in this case, and http/1.1 if FT_HTTP is added too.
The flag FF_SSL_INDICATE_ALPN_SELECT_PROTOCOL enables indication of each ALPN protocol offered by client via dataPartAvailable as OT_SSL_HANDSHAKE_OUTGOING_PROTOCOL objects. Each protocol is indicated as a separate OT_SSL_HANDSHAKE_OUTGOING_PROTOCOL object. The objects contain a protocol name in a single stream. If dataPartAvailable returns DPCR_FILTER for a protocol from OT_SSL_HANDSHAKE_OUTGOING_PROTOCOL, the protocol is added for negotiation with server. Any other return code blocks the negotiation of the protocol, so it cannot be used in the appropriate session.

FF_SSL_KEEP_SERIAL_NUMBERS flag instructs to copy serial numbers from original server certificates. It is useful in case if they are required in client applications.

FF_SSL_DONT_IMPORT_SELF_SIGNED - don't import the generated self signed certificates to trusted storages.

FF_SSL_DISABLE_TLS_1_0 - disable TLS 1.0 protocol support for the connection with remote servers.

FF_SSL_DISABLE_TLS_1_1 - disable TLS 1.1 protocol support for the connection with remote servers.

FF_SSL_TLS_COMPATIBILITY - enable automatic switching to old protocol TLS 1.0 when server doesn't support newer versions. When TLS handshake fails, SSL filter tries to establish TLS 1.0 session. If it succeeds, SSL filter adds TLS exception (EXC_TLS), and uses TLS 1.0 for next sessions with the appropriate host:port.

FF_SSL_STRICT_VERIFICATION - use strict rules when FF_SSL_VERIFY flag is enabled. In result the connections of clients and servers using old versions of TLS protocol may fail. The full DH key checks are executed no more than once a day for each host.

FF_SSL_SCT_VERIFICATION - enables Signed Certificate Timestamps verification for server certificates when FF_SSL_VERIFY flag is enabled.

There are known issues with SSL filter:

- Some versions of Windows Update use additional certificate checks. SSL is filtered properly, but after establishing a connection and transmitting several packets it is blocked by Windows Update service. To avoid issues don't add SSL filter for svchost.exe.
- Skype TCP connections via HTTPS proxy cannot be filtered properly. Don't add SSL filter for the process Skype.exe.
- PokerStars client establishes many SSL connections with different servers, which cannot be filtered. The filter is unable to add such hosts to the list of exceptions automatically. Add pokerstars.exe to the list of SSL exceptions.

In other cases when some application doesn't work properly with SSL filter, it is possible to implement in your application a whitelist of processes/domains/IPs/ports, to avoid filtering SSL in such cases. It is possible to use FF_SSL_INDICATE_HANDSHAKE_REQUESTS flag for SSL filter, analyze the whitelist parameters in dataPartAvailable event and return DPCR_BYPASS to disable filtering SSL on the handshake stage. The domain names taken from SNI field of TLS extensions are indicated in OT_SSL_HANDSHAKE_OUTGOING objects.