DoS/DDoS attack mitigation setup
Notes before installation
Oplon DoS/DDoS Attack Mitigation function is an optional module integrated in Oplon Application Delivery Controller and is available for all distributions.
The Oplon DoS/DDoS Attack Mitigation function license therefore enables the feature: no modules or libraries need to be added to the existing Oplon Application Delivery Controller installation.
Since the Oplon S.A.A.I. suite is a product intended for mission-critical and business-critical environments, only staff who have completed the course and passed the exam are authorized to certify the installation and maintenance of the products in operation. All certified persons hold a document issued by Oplon NETWORKS SRL.
Oplon DoS/DDoS Attack Mitigation function
Oplon Application Delivery Controller integrates a Denial of Service & Distributed Denial of Service (DoS/DDoS) attack prevention system that can monitor and catalog in real time the service request dynamics and their evolution. All connections that cross the balancing layer are constantly analyzed and catalogued, identifying anomalous events and taking actions to ensure the continuity of services.
DoS/DDoS congestion resolver
The system is able to automatically detect coordinated attacks from multiple sources by implementing instant decongestion techniques that free up the service and make it available again. These attacks are difficult to identify because they are carried out by countless sources. Typically they are the result of viruses that, once propagated over different vectors, activate simultaneously, directing their attack towards a single target. In e-commerce, the same effects are observed with a successful marketing campaign. Typically, a promotional campaign is promoted through a mailing list consisting of hundreds of thousands of email addresses. If the campaign is successful and the proposed offer is of interest, the site is reached at the same time by hundreds of thousands of “clicks” (observable in the graph on the left and top). Defenses are activated through decongestion when backend resource saturation is detected. From version 9, you can also cap requests for a single service, further increasing the granularity of the response.
DoS/DDoS address quarantine
The DoS/DDoS address quarantine function can identify sophisticated attempts by a few parties to monopolize resources. These parties are automatically recognized and excluded by placing the addresses that are the source of the disruption in “quarantine” for a fixed period of time. In fact, these attacks normally come from dynamic addresses and therefore cannot be added to public “blacklist” directories. Once the “quarantine” time is over, the quarantined address can access the services again.
VIP iRedCarpet (Very Important People)
The latest evolution, derived from e-commerce experience, is the VIP iRedCarpet feature. This feature makes it possible to distinguish, from an application point of view, “useful” traffic from “tourist” traffic, reacting differently depending on traffic conditions. The system has been designed to prioritize connection access based on the type of service requested, for example by prioritizing connections that are making payments or have already authenticated to the portal or, in e-commerce, users who have items in their cart over those who are just browsing. VIP iRedCarpet works together with the DoS/DDoS address quarantine and DoS/DDoS congestion resolver features, increasing accessibility to services by distinguishing between types of browsing (Application Quality of Service, AQoS).
Oplon DoS/DDoS Attack Mitigation function License Installation
Once you have started Oplon Secure Web-Based GUI and logged in to the node, choose the module designated to host the Oplon DoS/DDoS Attack Mitigation feature and perform the following steps.
If you do not have an Oplon DoS/DDoS Attack Mitigation license, you can request one from Oplon NETWORKS SRL or an authorized reseller.
In the process tree, select the module (usually A10_LBLGo) in which you want to install the Oplon DoS/DDoS Attack Prevention license, then select Actions > Install license.
Select the appropriate license, which must be named license.xml.
Once the license is confirmed, the result of the upload is displayed. If the result is “SUCCESSFULL”, go to the next step.
Oplon Application Delivery Controller Stop
Once the license has been successfully installed, you need to restart the Oplon Application Delivery Controller module. To restart the Oplon Application Delivery Controller module, simply stop it (Stop Module) and then start it (Start Module):
You can follow the progress in Oplon Secure Web-Based GUI, which shows the shutdown of the process…
Oplon Application Delivery Controller Start with DoS/DDoS Attack Mitigation function
Once the balancing module has stopped, you can proceed with Start Module: at startup, the module acquires the new license, enabling the DoS/DDoS Attack Prevention feature.
Oplon DoS/DDoS function basic settings
Once the Oplon DoS/DDoS Attack Mitigation function license is installed, you can make it operational immediately through the general configuration parameters of Oplon Application Delivery Controller, using the D-DOS item of the ADC Settings menu.
The parameters below set the basic product features that are then immediately usable for production, giving you the ability to perform specific tuning later.
The parameter that enables the feature is dosAttackPrevention, which is disabled by default and must therefore be explicitly set to “true”. Another important parameter is dosAttackPreventionOnlyClose. This parameter is set to “true” by default for security reasons. Setting it to “false” must be carefully considered: once it is set to “false”, with HTTP/S connections, an overload courtesy page is returned to all users when the Red Alert threshold is exceeded. In mission-critical or business-critical environments, the courtesy page can be used by an attacker as a signal to change attack strategy. That’s why dosAttackPreventionOnlyClose is set to “true” by default. Normally, unless you are in strict mission-critical and business-critical conditions, the parameter is set to “false”.
dosAttackPrevention: default value “false”, boolean
If set to “true”, the rules for preventing a DoS/DDoS (Denial of Service) cyberattack are applied. dosAttackPrevention takes action in different ways depending on the type of attack:
-
Multiple requests from the same IP address
-
Multiple requests from different IP addresses
In the first case, when a defined number of concurrent requests (which can be set with the dosMaxTunnelForClientAddress parameter) is exceeded, the requests are cleared, including any pending requests in the internal queues.
In the second case, when the highWaterDangerLevel threshold is exceeded, all in-progress and/or pending requests in the queues are cleared.
In both cases, if the connection is HTTP/S, a courtesy page is sent to the requesting client before the pending requests are deleted.
dosAttackPreventionOnlyClose: default value “true”
When a DoS/DDoS attack is detected, the default action is to close the channels identified as a threat. For some protocols, a courtesy page can be returned to indicate that the service is temporarily unreachable. However, this indication could be used to refine the attack and is therefore disabled by default.
Oplon DoS/DDoS function DoS/DDoS congestion resolver
The DoS/DDoS congestion resolver feature is based on continuous observation of the behavior of connections and of connection requests waiting to be taken over by a tunnel. It is a one-of-a-kind feature made possible by the finite-resource forwarding engine at the core of Oplon Application Delivery Controller.
The reference values to take into account are the number of forwarding tunnels and the number of connections waiting to be linked to a backend resource. These values, in terms of parameters, are, respectively, concurrentSessions and maxConcurrentSessions.
concurrentSessions: default value “50”
This is the initial number of connections that can be served at the same time.
maxConcurrentSessions: default value “200”
This is the maximum number of connections that can be served at the same time.
Normally, in a production installation, these two values are identical and are set based on the computing and response capabilities of the backend services of the datacenter, for example:
concurrentSessions=“2000”
maxConcurrentSessions=“2000”
DoS/DDoS congestion resolver checks the tunnel occupancy and the number of connection requests in the queue every 50 milliseconds. If all tunnels are saturated and the waiting queue exceeds the occupancy percentage set by the highWaterWarningLevel parameter (default 10.0%) relative to the number of configured tunnels, a message immediately reports that an abnormal but not yet worrying peak is taking place. The message is written to the log, recorded in the DB and, if configured, notified by email.
When the highWaterDangerLevel threshold is reached or exceeded, Oplon DoS/DDoS congestion resolver performs a close with flush of all connections in the tunnels, returns any courtesy pages to the clients where applicable, and then restores the forwarding of new requests. It is important to note that channels are closed in a controlled manner, especially backend connections. Connection flushing is very important because it allows backend services to properly close the file descriptors opened by the application servers and actually free up resources. The image on the right shows a real-world example of the Danger Level threshold being exceeded and the event being resolved immediately.
The parameters for setting the thresholds are therefore highWaterWarningLevel and highWaterDangerLevel; their descriptions, also available in the Reference Guide, are given below.
highWaterWarningLevel: default value “10.0”, unit: %
This is the threshold percentage of connections waiting to be taken over by an executor of the forwarding pool. If this threshold is exceeded, the system warns with a specific message (yellow-alert) that the limit has been exceeded. If the limit is exceeded, new forwarding sessions are also instantiated until maxConcurrentSessions is reached.
highWaterDangerLevel: default value “70.0”, unit: %
This is the threshold percentage of connections waiting to be taken over by a forwarding session. If this threshold is exceeded, the system warns with a specific message (red-alert) that the limit has been exceeded. If the limit is exceeded, new forwarding sessions are also instantiated until maxConcurrentSessions is reached. With Oplon DoS/DDoS Attack Mitigation function, if this limit is reached, the system runs the DoS/DDoS Congestion Resolver procedures.
Oplon DoS/DDoS function DoS/DDoS address in quarantine
One of the most frequent DoS/DDoS attacks is carried out by a few attackers or even a single one. The use of static black-lists on public directories is absolutely not recommended, because such an attack normally does not come from static addresses but from dynamic addresses, and therefore blocking the address permanently would mean not achieving the result or, worse, having blocked a possible future contact from that address. Even worse, if the source address belongs to an organization, that organization would be permanently prevented from accessing your services.
Oplon DoS/DDoS address in quarantine checks every 50 milliseconds how many connections are going through the routing layer with the same address. If the threshold set in the dosMaxTunnelForClientAddress parameter is exceeded, the address is cataloged in a table within the load balancer. All connections from that address are immediately deleted and, for incoming connections, the courtesy message or redirect policies are applied if set; otherwise, those connection requests are deleted as well. That address is denied access to the services for the time set by the dosAddressQuarantineTime parameter, normally 30 minutes. When the time expires, the previously blocked address can request services again. If the problem occurs again, the address is quarantined again.
dosMaxTunnelForClientAddress: default value “100”, unit: requests in the tunnel
Indicates the threshold as the number of requests served at the same time per source IP address. If this threshold is exceeded, all connections established from the same IP are instantly deleted. If the connection is HTTP/S, all connections waiting in the request queue instead receive the courtesy message (dosCMessage) or are redirected according to the dosRedirect parameter.
dosAddressQuarantineTime: default value “1800000”, unit: milliseconds
When an attack from a single IP address is detected, the address is automatically quarantined and prevented from accessing the services. The quarantine time is determined by this value (30 minutes). After this time, the client can access the services again. If this value is set to 0 or less than 0, the quarantine feature is disabled.
Oplon DoS/DDoS function DoS/DDoS courtesy message
If the dosAttackPreventionOnlyClose parameter has been set to “false”, the “cut” HTTP/S service requests receive a courtesy message asking to retry later because of request overload. The courtesy message can be customized by the user. If the message has not been modified, Oplon Application Delivery Controller shows the following HTML page:
You can also change the name of the default overload page through the dosCMessage parameter, as described below.
dosCMessage: default value “messageServicesOverload.html”
Indicates the courtesy page to display when DoS/DDoS Attack Prevention is activated.
To change the courtesy page, simply extract the file
loadbalancer/resources/html/messageServicesOverload.html
from the compressed file (LBL_HOME)/lib/lbl_loadbalancer.jar
and place it in (LBL_HOME)/lib/loadbalancer/resources/html.
Once the file is in place, you can edit it even at runtime. This
parameter is taken into account if
dosAttackPreventionOnlyClose is set to “false”.
Oplon DoS/DDoS function DoS/DDoS redirect
As an alternative to the courtesy message, if the connection is HTTP/S, you can instruct the client to redirect to another site/service. This feature allows you to perform diversified tasks during an attack or a sudden spike in requests. In fact, it happens more and more often that congestion is not due to real attacks but to successful marketing campaigns with a very high subscription rate. The ability to divert requests to another site to fulfill them or, more simply, to redirect them to a page that offers an additional promotion if the customer decides to come back later can be a valuable marketing tool, giving customers an important communication and loyalty channel.
To set up the redirect, use the parameter described below.
dosRedirect: default value ""
If set, indicates the URI to redirect the request to when
DoS/DDoS Attack Prevention is activated, for example http://www.caugthinfo.com/.
If this value is set, it has priority over the dosCMessage parameter. This parameter is taken into account if dosAttackPreventionOnlyClose is set to “false”.
Oplon DoS/DDoS function VIP iRedCarpet AQoS (Application Quality of Service)
Oplon VIP iRedCarpet stems from years of experience with DoS/DDoS Attack Mitigation and is an important evolution of QoS policies oriented to application components: AQoS (Application Quality of Service). Oplon VIP iRedCarpet acts in the global context of the Oplon DoS/DDoS attack prevention features, but it examines individual service requests to determine their access privilege at times of particularly high load. For example, you can identify purchase order confirmations coming from credit card processors and give them priority; you can distinguish users who have already filled their cart from others who are just browsing, and then let the former through and invite the latter to try again later.
These service access privilege policies can be implemented through rewriting rules that describe their dynamics. The rules can be structured to selectively limit requests as the load increases, up to keeping the services open only for absolutely essential and critical requests.
Normally, the first rule to set is the one that lets all incoming requests through under normal conditions. It is important to note that this rule, being the first, is also the last to be executed if the condition occurs: in fact, the EnableAccessOnNormalCondition command followed by LAST (EnableAccessOnNormalCondition;LAST) causes the Oplon Application Delivery Controller rewriter not to apply other rules when the normal condition occurs.
The rule below can be used as a template to identify a request queue
level considered normal. The level of queued requests, even at
times of heavy load, normally remains very close to zero (0). The
HIGH_WATER_LEVEL INNERVAR, made available to the rewriter by the
Oplon DoS/DDoS attack prevention feature, allows you to read the number of
requests waiting for forwarding during the single request. With the
<numOperatorTag> feature, you can perform numeric comparisons and then
make the action conditional on a value. The actions allowed by
<numOperatorTag> are:
eq=equal
neq=not equal
gt=greater than
geq=greater than or equal
lt=less than
leq=less than or equal
Templates with all parameters preset are already available:
Other INNERVARs made available by the rewriter with the Oplon DoS/DDoS Attack Mitigation feature to condition service requests are:
HIGH_WATER_YELLOW_WARNING_REACHED = if “true”, the Yellow Warning threshold has been exceeded. It therefore indicates a significant but not yet critical load.
HIGH_WATER = number of connection requests in the queue, in long format.
HIGH_WATER_LEVEL = percentage of connection requests in the queue relative to the configured number of concurrent tunnels, in float format.
TUNNEL_SESSIONS_ACTIVE = instantaneous number of active tunnels, in int format.
TUNNEL_SESSIONS_COMMITTED = instantaneous number of committed tunnels (subset of TUNNEL_SESSIONS_ACTIVE).
ACTUAL_TUNNEL_SESSIONS_SIZE = current number of tunnels (usually equal to MAX_TUNNEL_SESSIONS_SIZE).
MAX_TUNNEL_SESSIONS_SIZE = maximum number of tunnels configured.
If the load level considered “normal” by the first rule is exceeded, the request is then checked in detail to determine whether to let it through or stop it immediately.
For example, the following two rules indicate that for loads greater than 10% (gt 10), credit card payment confirmations and accesses from the organization itself, both internal and external, are always given priority.
rewriteHeaderRule property name property="VIP_EnablePaymentGatewaysOnOverload" flow="REQUEST function" caseSensitive="True"
variables, New10
Var Varname="highWaterLevel property" name property="HIGH_WATER_LEVEL" from="INNERVAR, NEW"
conditions operator="AND"
Cond from="VARIABLE, NEW10" name property="highWaterLevel property" >
numOperatorTaggt 10
Cond from="INNERVAR, NEW" name property="REQUEST_HTTP_URI_PATH"
regexTagPayPalReturn XPayFOLightReturn ReceiveAgos, New100014
rewriteHeaderRule property name property="VIP_EnableAdminSitesOnOverload" flow="REQUEST function" caseSensitive="True"
variables, New10
Var Varname="highWaterLevel property" name property="HIGH_WATER_LEVEL" from="INNERVAR, NEW"
conditions operator="AND"
Cond from="VARIABLE, NEW10" name property="highWaterLevel property"
numOperatorTaggt 10
Cond from="INNERVAR, NEW" name property="REQUEST_HTTP_URI_PATH"
regexTagmyOrganizationExtranetIf the conditions of the previous rules are not met, the next rule is evaluated; it is met with a queue level from 10% to 39%. In this case, users who had already connected and received a session from Oplon Application Delivery Controller are allowed access to the backend services.
This is obviously just an example: you can base the pass privilege on any other indicator, such as an application-generated cookie, a URI path, a referer, etc.
rewriteHeaderRule property name property="VIP_YellowOverloadEnableReqWithLBLSESSIONID" flow="REQUEST function"
caseSensitive="False"
variables, New10
Var Varname="highWaterLevel property" name property="HIGH_WATER_LEVEL" from="INNERVAR, NEW"/>
conditions operator="AND"
Cond from="VARIABLE, NEW10" name property="highWaterLevel property"
numOperatorTaggeq 10
Cond from="VARIABLE, NEW10" name property="highWaterLevel property"
numOperatorTaglt 40
Cond from="INNERVAR, NEW" name property="REQUEST_HTTP_COOKIES_LIST"
regexTagLBLSESSIONID propertyIf none of the above rules is met and the load level is 40% or higher (geq 40), users who have authenticated to the portal can still browse the services freely.
rewriteHeaderRule property name property="VIP_OrangeOverloadEnableReqWithLBLSESSIONIDandUSERID" flow="REQUEST function"
caseSensitive="False"
variables, New10
Var Varname="highWaterLevel property" name property="HIGH_WATER_LEVEL" from="INNERVAR, NEW"
conditions operator="AND"
Cond from="VARIABLE, NEW10" name property="highWaterLevel property"
numOperatorTaggeq 40
Cond from="INNERVAR, NEW" name property="REQUEST_HTTP_COOKIES_LIST"
regexTagUserid
If none of the above rules has been met, with a queue level above 10% all service requests are cut (connectionToCut connectionToCut=“true”): channel close, courtesy page or redirect.
rewriteHeaderRule property name property="VIP_DisableAccessOnOverload" flow="REQUEST function" caseSensitive="True"
variables, New10
Var Varname="highWaterLevel property" name property="HIGH_WATER_LEVEL" from="INNERVAR, NEW"
conditions operator="AND"
Cond from="VARIABLE, NEW10" name property="highWaterLevel property"
numOperatorTaggt 10
connectionToCut property connectionToCut property="True"The previous sections described how to define request filtering rules based on the queuing levels caused by request load. The following example shows how to apply the rules at the flow level, that is, how to apply the rules described above to service groups. Note the ;LAST behavior modifier: when the condition described in the rule occurs, the remaining subsequent rules are not executed. In this case, since ;LAST is applied to each rule, the first one that meets the conditions is executed without verifying and executing the remaining ones.
endpoints
endPointsGrouping property Groupname="es62prod" Enable="True"
virtualDomain, New1001 loadBalancingType property="Adaptive"
cMessage property="MyOrganizationOverloadMessage.html"
rewriteHeaderRules property="VIP_EnableAccessOnNormalCondition; Last
VIP_EnableAdminSitesOnOverload; Last
VIP_EnablePaymentGatewaysOnOverload; Last
VIP_YellowOverloadEnableReqWithLBLSESSIONID; Last
VIP_OrangeOverloadEnableReqWithLBLSESSIONIDandUSERID; Last
VIP_DisableAccessOnOverload"
endp property address="192.168.56.131" Port="8585" uriPath property="/srv01" Enable="True"
endp property address="192.168.56.131" Port="8585" uriPath property="/srv02" Enable="True"
endp property address="192.168.56.131" Port="8585" uriPath property="/srv03" Enable="True"Oplon DoS/DDoS function DoS/DDoS congestion resolver service capping
Starting from release 9 of Oplon Application Delivery Controller, Oplon DoS/DDoS Attack
Mitigation also lets you set a maximum level of use of
backend services. You can set a maximum number of connections for each
single grouping of services. This value is determined by the sum of the
maximum number of connections for that service set in the endp elements
that compose it. For example, for endPointsGrouping ->
virtualDomain -> URIpath, the maximum processing capacity of the
httpOne -> www.onesite.com -> one service is 20+12=32. For the service
group httpOne -> www.onesite.com -> two it is 11+56=67.
This feature can be enabled through the
dosMaxConcurrentConnectionsReaction="true" parameter; otherwise, the
default behavior applies, that is, the event is only logged.
endPointsGrouping property Groupname="httpOne"
rewriteHeaderRules property="compressionHEADER; ALWAYS, New10"
rewriteBodyRules property="compressionBODY; ALWAYS, New10"
dosMaxConcurrentConnectionsReaction="True"
Enable="True"
virtualDomain, New1001 virtualDomainName property="www.onesite.com"
dosMaxConcurrentConnectionsReaction="True" Enable="True"
endp property address="192.168.45.131" Port="8585" maxConcurrentConnections property="20" uriPath property="/one"
endp property address="192.168.45.132" Port="8585" maxConcurrentConnections property="12" uriPath property="/one"
endp property address="192.168.45.131" Port="8585" maxConcurrentConnections property="11" uriPath property="/two"
endp property address="192.168.45.132" Port="8585" maxConcurrentConnections property="56" uriPath property="/two"Oplon DoS/DDoS function conclusions
Oplon DoS/DDoS Attack Mitigation is a value-added feature that has been improved over time, up to the sophisticated capabilities that can be made available to the market today.
The effect of what is described in these sections, measurable in real time through Oplon Management Console, can be seen in the following images. The images refer to a real situation in large-scale e-commerce retail. After three consecutive promotional emails, the consumers who immediately responded to the invitation were several tens of thousands (120,000 in about two hours). At the first mailing, Oplon VIP iRedCarpet detected the excessive load event and resolved it immediately without “cutting” the ongoing business connections, preserving the operation of the backend services and the database.
Oplon DoS/DDoS Attack Mitigation is currently the only platform that can dynamically filter service requests based on load and on the requested application service, providing an Application Quality of Service tool unparalleled on the market today.