 |
 |
Please Sign in: |
|
|
|
Abuse Prevention System |
 |
Abuse Prevention System
The Abuse Prevention System (APS) of TheWebWatcher is designed to prevent web sites from beiing checked too often and reduce the remote bandwidth traffic.
The system uses several levels of abuse protection.
First of all TheWebWatcher monitoring agent checks if the remote web server supports the "If-Modified-Since" and "If-Match" header fields as defined in RFC-2616 sec. 14. As it is mentioned in RFC2616, these headers are used "to allow efficient updates of cached information with a minimum amount of transaction overhead". When the monitoring agent issues requests to servers that supports these headers, he sends requests to see if the page is modified since his last visit and if there are no changes the agent don't fetches the content of the web page at all. The alternative of the If-Modified-Since tag is the If-Match tag which is also used by TheWebWatcher to find if there are any modifications since the last visit. The If-Match header field is used with ETag value which is returned from the remote server when the last modifications are tracked.
The second level of protection is used for popular sites that are checked too often with the aid of internal rules which defines the minimal monitoring interval for a specific web pages or for entire domains. When the monitoring agent needs to issue a request to site or domain that has defined rules in the APS, he gets the date and time of the last check and if this date fulfills the rule for this site, the agent downloads the page. Otherwise the agent is trying to use a cached version of that page. If there's no cached version of this page, the check is marked as forbidden and another request is issued after a while in regard to the monitoring frequency of the watch-filter that monitors this resource.
The APS system rules can be defined to totally ban sites or domains, which are excluded from checks at all.
|
|
|
