THIS FIELD NOTICE IS PROVIDED ON AN "AS IS" BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY, INCLUDING THE WARRANTY OF MERCHANTABILITY. YOUR USE OF THE INFORMATION ON THE FIELD NOTICE OR MATERIALS LINKED FROM THE FIELD NOTICE IS AT YOUR OWN RISK. CISCO RESERVES THE RIGHT TO CHANGE OR UPDATE THIS FIELD NOTICE AT ANY TIME.
Revision | Publish Date | Comments |
---|---|---|
1.0 |
14-Mar-16 |
Initial Release |
10.0 |
18-Oct-17 |
Migration to new field notice system |
Affected OS Type | Affected Release | Affected Release Number | Comments |
---|---|---|---|
NON-IOS |
9 |
9.1.0,9.6.0,9.7.0,9.7.1 |
|
NON-IOS |
8 |
8.5.5,8.5.6,8.5.7 |
Defect ID | Headline |
---|---|
CSCuy44285 | Too many connection resets to SDS servers |
A change in the server pool used by the URL Reputation Feature might cause the Email Security Appliance (ESA) to start to back up the work queue. This change in the server pool takes place on April 4th, 2016.
A change in the server pool used by the URL Reputation Feature might cause the ESA to start to back up the work queue. This change in the server pool takes place on April 4th, 2016.
If URL filtering is enabled and these behaviors are observed, the ESA experiences the issue and the workaround provided has to be applied.
The Work Queue on the ESA Backs Up
The ESA stores the Web Client logs under the web_client log subscription. If this message is present in this log it is a good indication that the appliance faces the described issue.
[ Unexpected workqueue growth ]
A Large Number of 'Request already expired' Messages are in the Web Client Logs
Thu Feb 11 23:00:49 2016 Warning: cache: THR: cache_manager-cloud_connector: SRC: antispam: MID: 99: REQ_ID: 64878.1455231634: Error requesting ?http://www.someURL.com': (Request already expired)
Thu Feb 11 23:00:49 2016 Warning: cache: THR: cache_manager-cloud_connector: SRC: work_queue: MID: 100: REQ_ID: 61581051: Error requesting ?http://www.someWebSitecom': (Request already expired)
Thu Feb 11 23:00:49 2016 Warning: cache: THR: cache_manager-cloud_connector: SRC: work_queue: MID: 100: REQ_ID: 61581051: Error requesting 'http:/wwww.anotherURL.com: (Request already expired)
Thu Feb 11 23:00:49 2016 Warning: cache: THR: cache_manager-cloud_connector: SRC: work_queue: MID: 100: REQ_ID: 61581051: Error requesting ?http://www.funysites.com/funny/URL': (Request already expired)
Thu Feb 11 23:00:49 2016 Info: rpc_server: THR: request_handler_5: SRC: antispam: MID: 100: REQ_ID: 64881.1455231634: URL lookup success = False; whitelisted = 0, cache hits = 0, cache misses = 2, ignored or errored= 2, time taken = 7.029145 sec
Alerts Similar to These Examples are Sent by the Appliance
The Warning message is:
Unable to connect to Cisco Web Security Service.
URL Filtering will not work correctly.
Please verify all network, proxy and firewall settings.
Connection to "v2.sds.cisco.com" failed.
The last error seen on this connection: "Server busy or service unavailable"
Version: 9.7.0-125
Serial Number: 8xxxxxxx.9yyyy
Timestamp: 11 Feb 2016 23:00:03 +0000
or
The Warning message is:
Unable to connect to Cisco Web Security Service.
URL Filtering will not work correctly.
Please verify all network, proxy and firewall settings.
Connection to "v2.sds.cisco.com" failed.
The last error seen on this connection: "Request failed with code: 28 (SSL_write() error: error:1409F07F:SSL routines:SSL3_WRITE_PENDING:bad write retry)"
Unable to connect to Cisco Web Security Service.
URL Filtering will not work correctly.
Please verify all network, proxy and firewall settings.
Connection to "v2.sds.cisco.com" failed.
The last error seen on this connection: "Unknown error"
Last message occurred 46 times between Fri Feb 12 22:13:54 2016 and Fri Feb 12 22:16:27 2016.
Version: 9.7.0-125
Serial Number: 8xxxxxxx.9yyyy
Timestamp: 12 Feb 2016 23:07:14 +0000
As part of this migration, the OpenSSL software will be patched in order to improve the pool's security posture. A bug has been uncovered between the software used by the ESA to talk to these servers and the OpenSSL patch.
In order to avoid the workqueue backups, you can reduce the amount of URLs sent for verification at the same time. Complete these steps in order to make this change:
1. Secure Shell (SSH) into the appliance.
2. Enter the command websecurityadvancedconfig.
3. Change the value for "Enter the threshold value for outstanding requests" from the default to 5.
4. Do not change any other option.
5. Commit the change.
ESA>websecurityadvancedconfig
Enter URL lookup timeout (includes any DNS lookup time) in seconds:
[5]>
Enter the URL cache size (no. of URLs):
[810000]>
Do you want to disable DNS lookups? [N]>
Enter the maximum number of URLs that should be scanned:
[100]>
Enter the Web security service hostname:
[v2.sds.cisco.com]>
Enter the threshold value for outstanding requests:
[50]>5
Do you want to verify server certificate? [Y]>
Enter the default time-to-live value (seconds):
[30]>
Do you want to include additional headers? [N]>
Enter the default debug log level for RPC server:
[Info]>
Enter the default debug log level for URL cache:
[Info]>
Enter the default debug log level for HTTP client:
[Info]>
ESA> commit
Please enter some comments describing your changes:
[]> updated the threshold value for outstanding requests to 5
Do you want to save the current configuration for rollback? [Y]>
Changes committed: Thu Mar 03 18:40:57 2016 GMT
If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods:
Cisco Notification Service—Set up a profile to receive email updates about reliability, safety, network security, and end-of-sale issues for the Cisco products you specify.
Unleash the Power of TAC's Virtual Assistance