ICAP

Overview

ICAP (Internet Content Adaptation Protocol) is a lightweight HTTP-like protocol to transport HTTP messages to 3rd party services. The server runs its transformation service on messages and sends back responses to the client, usually with modified messages. For more information on ICAP, refer to RFC3507.

ICAP is supported for HTTP requests processing through NSX Advanced Load Balancer. With the implementation of the ICAP client functionality within the NSX Advanced Load Balancer, the following use-cases are supported:

  • Antivirus scanning - via 3rd party antivirus scan engine
  • Content sanitization - via 3rd party content sanitization service
  • Other request modification options via ICAP services, for example, URL filtering

Starting with NSX Advanced Load Balancer version 21.1.3, ICAPs is supported.

Configuring NSX Advanced Load Balancer for ICAP

NSX Advanced Load Balancer, as an ICAP client, supports the followings:

  • Preview functionality
  • Streaming of payload
  • Content rewrite

The followings are the main configuration components for enabling ICAP for a virtual service on an NSX Advanced Load Balancer:

  • Configuring an ICAP pool group
  • Configuring an ICAP profile (attached to the virtual service)
  • Configuring an HTTP Policy for the virtual service with the action set as Enable ICAP
  • Associating the ICAP profile to the virtual service
  • Configuring HTTP security policy for ICAP
  1. Configuring ICAP Pool Group
    Navigate to Application > Pool Group, create a pool group, as shown below. The field for the Fail Action under the Pool Group Failure Settings needs to be empty. pool-group
  2. Configuring ICAP Pool
    Navigate to create an ICAP pool, as shown below. Configure the default port as 1344. Multiple Servers can be added as Pool members.

  3. Configuring ICAP Profile

    Refer to the following table for the various attributes used in the ICAP profile configuration.

    Config Item Description Example
    Name Name of this profile ICAP-APPX
    Cloud Defines, which cloud object this is associated with Default-Cloud
    Pool Group Pool group of all ICAP server pools ICAP-Pool-Group
    Vendor Vendor specific configuration if a vendor is supported OPSWAT
    Generic-ICAP
    Service URL ICAP Server service URL When using OPSWAT: /OMSScanReq-AV
    Request Buffer Size Maximum buffer size for request body Default: 51200 (50 MB)
    Enable ICAP Preview Enable ICAP Preview functionality, where the ICAP server can make decisions by examining the preview size payload Default: Enabled (Boolean)
    Preview Size Payload size for ICAP preview Default: 5000 (5 MB)
    Response Timeout When this threshold is hit, the request will be handled as an error and the failure action will be run Default: 60000 (60 seconds)
    Slow Response warning Threshold When this threshold is hit, the request will cause a significant log entry, but will still be served Default: 10000 (10 seconds)
    Actions
    Failure Action Handling of error with ICAP server. If failed closed, a 503 will be sent if an error is occurring. Fail Closed/ Fail Open
    Large Upload Failure Action Handling of size exceeded error. If fail closed, a 413 will be sent. Fail Closed/ Fail Open


    Navigate to Virtual Service > Edit > ICAP Profile or Templates > Profiles > ICAP Profile to create an ICAP profile. icap-profile

  4. Navigate to the Application > Virtual Service, select the required virtual service, and select the ICAP profile (created in the previous step). vs
  5. Creating HTTP Security Policy
    Create a security policy to define the rules based on which the ICAP scanning should be performed. Navigate to Application > Virtual Service, select the desired virtual service, and click on Edit. Select Policies > HTTP Security, and create a new rule with the following options:
    • Select a match criteria for the ICAP requests
    • Select Enable ICAP as the action

    security-policy

    Note: The rule name configured in this step will appear in the logs, so it is recommended to make it self-explanatory for ease of troubleshooting.

  6. Save the virtual service configuration.

With these steps, the ICAP configuration for the virtual service is complete. Incoming requests on the virtual service that match the rule or the match criteria of the HTTP security policy will use ICAP.

NSX Advanced Load Balancer supports the following ICAP servers (3rd party AV-Malware/CDR vendors):

  • OPSWAT
  • MetaDefender ICAP Server (with MetaDefender Core)

To set up an OPWSWAT server for ICAP scanning, refer to the OPSWAT documentation.

Limitations

The followings are the limitations for ICAP support on NSX Advanced Load Balancer:

  • ICAP is not supported for HTTP/2 virtual services.
  • ICAP client does not work in the response context.

ICAP Support for NSX Defender

Starting with NSX Advanced Load Balancer release 21.1.1, ICAP support is available for NSX Defender’s server for preventing malicious file uploads. When compared with OPSWAT, there are minor difference in the method using which the NSX Defender sends file back to the NSX Advanced Load Balancer. This section covers the followings:

  • NSX Defender ICAP configuration
  • NSX Advanced Load Balancer integration with NSX Defender
  • Required visibility changes for NSX Defender reported information.

Configuring NSX Defender for ICAP

Login to the NSX Defender and navigate to Appliances > Admin > Configuration > Proxy as shown below.

Select the ENABLED option for the following knobs on the UI:

  • ICAP Server
  • INLINE ANALYSIS

Under the Blocking pages section enable the follwoings:

  • BLOCKED PAGE DETAILS
  • X-LASTLINE HEADER
  • LASTLINE LOGO

icap-nsx-config

The followings are the blocking types available on the NSX Defender. For more information, refer to the NSX Defender documentation.

  • PASSIVE – No blocking is attempted on this type of file, but any relevant content will be analyzed.

  • SENSOR-KNOWN – Block all artifacts known to be malicious by the Sensor (listed in its local cache). This method offers the lowest levels of protection but ensures minimal lag.

  • MANAGER-KNOWN – Block all artifacts known to be malicious by the NSX Defender Manager. These data are listed in the Manager cache and shared across all managed appliances.

  • FULL – — This mode allows the proxy to stall an ICAP request for as long as necessary to provide a verdict on the file, within the limits set by the ICAP timeout. Depending on the client implementation, this may cause the transaction to appear as unresponsive for long periods of time (in the order of minutes in some cases).
    This blocking mode is particularly suitable for the integration with third party proxies that implement mechanisms to improve the user experience. Such mechanisms may include data trickling or “patience pages”, providing feedback to the user.

  • FULL WITH FEEDBACK – — This mode will generate “patience pages” that provide feedback to the user on the analysis progress. These mechanisms have been tested exclusively with the squid proxy. They may lead to unwanted side-effects when using third-party proxies, which may implement caching mechanisms that disrupt the NSX Defender operation. Such third party proxies often implement their own mechanisms to improve user experience, and therefore may perform better with the Full blocking mode.

Configuring NSX Advanced Load Balancer for NSX Defender

The following configuration ICAP server specific options are required to enable ICAP scanning using NSX Defender.

  • Service URI – This needs to be set to /lastline to use NSX Defender service.
  • ICAP Pool – ICAP pool needs to point to NSX Defender ICAP server:port.
  • Status URL – Only applicable to NSX Defender and has default value of https://user.lastline.com/portal#/analyst/task/$uuid/overview

Rest all the configuration options are generic and not tied to any particular ICAP server.

Visibility Changes for NSX Defender Reported Information

X-Lastline HTTP headers Pages analyzed by the ICAP instance may contain additional information on the analysis status by means of additional HTTP headers. The presence of such headers can be disabled from the ICAP configuration.

  • X-Lastline-Status – Provides information on the state of the object at the time of analysis. The following values are possible:
    • new — The specific file hash has not been recently analyzed by NSX Defender and a score is not currently available.
    • known — The specific file is known and a score is associated to it.
    • blacklist — The contacted remote endpoint has low reputation.
    • timeout — The process reached its timeout while waiting for the analysis of the file.
    • error — An error is preventing the analysis of the file.
  • X-Lastline-Score — The score currently associated with the file, if known, expressed as a value between 0 and 100.
  • X-Lastline-Task — The NSX Defender task UUID associated with the analysis of the file. It is possible to use this UUID in order to access the analysis details from the NSX Defender Portal/Manager Web UI. The following is the REST API to access information about any upload using UUID:
    *https://user.lastline.com/portal#/analyst/task//overview*

Following is the snapshot of Analysis overview based on the task UUID:

icap-nsx-header

ICAP Response Header

NSX Defender may also send following ICAP headers as part of ICAP response as per the ICAP extensions draft.

  • X-Infection-Found:Type=0;Resolution=1;Threat=LastlineArtifact(score=XX;md5=;uuid=)
  • X-Virus-ID: LastlineArtifact(score=100;md5=;task_uuid=)

Logs and Troubleshooting

This section discusses the various logs and troubleshooting options available for ICAP on NSX Advanced Load Balancer. The NSX Advanced Load Balancer UI and NSX Advanced Load Balancer CLI can be used to check logs and error messages for analytics and troubleshooting.

Log for the requests that were handled by the ICAP server has an icap_log section populated.

If the ICAP server blocks or modifies a request, the consequent log entry is significant. The following example shows details of the available logs on NSX Advanced Load Balancer. As shown under the Response Information, the overall request is blocked, and a 403 response code is sent back to the client.

logs1

  • The following log exhibits ICAP scan detects an infection (JSON log file):

"icap_log": {
    "action": "ICAP_BLOCKED",
    "request_logs": [
        {
        "icap_response_code": 200,
        "icap_method": "ICAP_METHOD_REQMOD",
        "http_response_code": 403,
        "http_method": "HTTP_METHOD_POST",
        "icap_absolute_uri": "icap://100.64.3.15:1344/OMSScanReq-AV ",
        "complete_body_sent": true,
        "pool_name": {
        "val": "ICAP-POOL-GROUP",
        "crc32": 1799851903
        },
        "pool_uuid": "poolgroup-c7dd3b93-60c1-4190-b6d6-26c22d55dc30",
        "latency": "1275",
        "icap_headers_sent_to_server": "Host: 100.64.3.15:1344\r\nConnection: close\r\nPreview: 653\r\nAllow: 204\r\nEncapsulated: req-hdr=0, req-body=661\r\n",
        "icap_headers_received_from_server": "Date: Thu, 19 Nov 2020 13:55:00 G11T\r\nServer: Metadefender Core V4\r\nISTag: \"001605794100\"\r\nX-ICAP-Profile: File process\r\nX-Response-Info: Blocked\r\nX-Response-Desc: Infected\r\nX-Blocked-Reason: Infected\r\nX-Infection-Found: Type=0",
        "action": "ICAP_BLOCKED",
        "reason": "Infected",
        "threat_id": "EICAR-Test-File (not a virus)"
        }]
},
  • The following is the log entry when the ICAP server modifies the ICAP request: icap-modified
  • The following log shows that the ICAP scan is performed successfully. The action field for the icap_log exhibits the value as ICAP_PASSED.

{"icap_log": 
    {"action": "ICAP_PASSED", "request_logs": 
        [{
          "icap_response_code": 204, 
          "icap_method": "ICAP_METHOD_REQMOD",
          "http_method": "HTTP_METHOD_POST",
          "icap_absolute_uri": 
          "icap://100.64.3.15:1344/OMSScanReq-AV ", 
          "complete_body_sent": true, 
          "pool_name": {"val": "ICAP-POOL-GROUP", "crc32": 1799851903}, 
          "pool_uuid": "poolgroup-c7dd3b93-60c1-4190-b6d6-26c22d55dc30", 
          "latency": "456", 
          "icap_headers_sent_to_server": "Host: 100.64.3.15:1344\r\nConnection: close\r\nPreview: 0\r\nAllow: 204\r\nEncapsulated: req-hdr=0, null-body=661\r\n", 
          "icap_headers_received_from_server": "Date: Wed, 18 Nov 2020 12:54:06 G11T\r\nServer: Metadefender Core V4\r\nISTag: \"000000000096\"\r\nX-Response-Info: Allowed\r\nEncapsulated: null-body=0\r\n", "action": "ICAP_PASSED"}]}
  • The log entries will show the action for icap_log as ICAP_DISABLED if ICAP feature is not enabled.
 
"icap_log": {"action": "ICAP_DISABLED"}

Log Analytics

When ICAP is enabled, the log analytics on NSX Advanced Load Balancer provides an additional overview. All data items are clickable and allow the quick addition of filters for a detailed log view.

icap-analytics

Troubleshooting

ICAP Server Connection Failed
The following example shows a log error message for a failed ICAP server connection. The ICAP Error is logged against as the Significance field. To solve this issue, check the direct connectivity from the Service Engines to the ICAP servers.

connection-failed ICAP Server Error The following example shows the ICAP Rrequest is blockedmisconfiguration of the ICAP server will exhibit the action for the ICAP log as ICAP_BLOCKED. The reason for the action is No security rule matched as available in the ICAP header.


"icap_log":
    {"action": "ICAP_BLOCKED", 
    "request_logs":
      [{
        "icap_response_code": 200, 
        "icap_method": "ICAP_METHOD_REQMOD", 
        "http_response_code": 403, 
        "http_method": "HTTP_METHOD_POST", 
        "icap_absolute_uri": "icap://100.64.3.15:1344/OMSScanReq-AV ", 
        "complete_body_sent": true, "pool_name": {"val": "ICAP-POOL-GROUP", "crc32": 1799851903}, "pool_uuid": "poolgroup-c7dd3b93-60c1-4190-b6d6-26c22d55dc30", "latency": "17", 
        "icap_headers_sent_to_server": "Host: 100.64.3.15:1344\r\nConnection: close\r\nPreview: 0\r\nAllow: 204\r\nEncapsulated: req-hdr=0, null-body=661\r\n", 
        "icap_headers_received_from_server": "Date: Thu, 19 Nov 2020 13:25:15 G11T\r\nServer: Metadefender Core V4\r\nISTag: \"001605792300\"\r\nX-Response-Info: Blocked\r\nX-Response-Desc: No security rule matched\r\nEncapsulated: res-hdr=0, res-body=91\r\n", "action": "ICAP_BLOCKED"}]}

To solve this issue, refer to the documentation of the ICAP server used for the deployment.

ICAPs

Starting with NSX Advanced Load Balancer version 21.1.3, ICAPs is supported. ICAP traffic can now be encrypted using SSL.

The following are the configuration components for enabling ICAPs:

  • To configure ICAPs on NSX Defender, enable Secure ICAP in Proxy configurations as shown below:
    nsx-defender-icaps

  • To configure ICAPs on OPSWAT, click here.

  • In NSX Advanced Load Balancer, when configuring a pool for ICAPs, ensure SSL is enabled in the Pool, that is referred in the ICAP profile (has IPs of ICAP servers) and configure the default port as 11344 as shown below:
    icaps-enable-ssl

Starting with Avi Vantage version 21.1.3, the ICAP supports HTTP2 traffic to the virtual service. If the virtual service has HTTP2 enabled for any port, and ICAP is configured, then the HTTP2 traffic will be subjected to the ICAP server.

Document Revision History

Date Change Summary
December 20, 2021 Added ICAPs section for 21.1.3