Skip to Content
DocsApplication Delivery ControllerHow to balance protocols

How to balance protocols

Introduction

This document describes the Oplon ADC settings to balance the most commonly used network protocols at OSI layer 4 (port forwarding). Handling of the HTTP/S protocol at OSI layer 7 is not covered in this manual, since it is native to Oplon ADC and widely described in the white paper, with its many configurations and balancing policies. For details on the individual parameters, refer to the Oplon ADC Reference Guide.

This document is meant as a reference, since each protocol has many configuration variants depending on how you want to use it, the result you want to achieve, and the characteristics of the network infrastructure and of the existing services.

The descriptions of how connections evolve in the different protocols are deliberately limited to the elements needed to balance, and make highly available, the services provided with the protocols considered. For more details, refer to the original RFC and IETF specifications.

Oplon ADC lets you set different parameters for each protocol and/or service group. You can therefore use different backend services more flexibly, giving each of them its own session affinity characteristics and tuning the time-outs to the overall quality of the specific service.

You can also use templates, which make configuration much easier. A template can start from an existing connection profile, change only a few parameters and generate a new profile from the differences. At startup, Oplon ADC automatically creates some profiles that you can use immediately, without any further action. You can also use these profiles as templates to create new profiles tailored to your specific needs. The currently preset profiles are: HTTP/S; FTP; RDP with session affinity; RDP without session affinity; SSH; Telnet; DNS; LDAP; UDP.

For simplicity, this document uses the preset profiles where available. The chapter on the LDAP protocol contains an example of reusing, through “templates”, parameters derived from another group. This technique can of course be used whenever some parameters do not meet the specific needs of the installation. The LDAP chapter also contains an example of the “FailOver” balancing policy, which can also be used with the protocols described here or with other custom protocols.

File Transfer Protocol (FTP)

The FTP protocol is a protocol designed to exchange files between heterogeneous machines.

The protocol distinguishes two flows with independent connections: the Command Flow and the Data Flow. Independent connections are established for each of these flows. These connections can evolve in two ways: Active-mode or Passive-mode.

  1. Active-mode FTP connections

In Active-mode, also called “client-managed”, the FTP information flow begins with the client sending the server an address/port known to the client, to which the server must send the data. The diagram below shows this type of flow, highlighting in red the side that takes the initiative and establishes the connection.

  1. The Client connects to the FTP Server on port 21 and sends the Server a proposed temporary address/port (e.g. 6060) to which the Server must connect to run the data transfer command

  2. The Server responds with an ACK.

  3. The Server connects to the temporary address/port (e.g.6060) passed earlier to run the command

  4. Depending on the command, data is transferred through the connection established to the temporary address/port (e.g.6060)

Active-mode is normally the default setting of an FTP server. This type of connection shows its limits when there are one or more Firewalls, because in a datacenter normally only a few known ports are open outbound. On the client side, listening connections are in any case normally blocked by the client’s own Firewall, which in practice makes communication impossible.

For this reason, a different way of evolving the connection has also been developed to be able to overcome these limitations: FTP “Passive-mode”.

  1. Passive-mode FTP connections

In Passive-mode, or “server-managed”, the communication flow begins with the Client connecting to the Server, which replies to the Client with one of its own predefined addresses/ports on which the Client can open new connections to transfer the Data flow, depending on the command used.

  1. The Client connects to the FTP Server on port 21 and sends the Server a proposed connection to a temporary address/port (e.g. 6060) to which the Server should connect to run the data transfer command

  2. The Server ignores the client’s proposal and sends its own PASV connection proposal, to its own address/port (e.g. 6161), for the data transfer

  3. The Client opens a new connection to the Server and follows up on the command

  4. Depending on the command, the Server sends an ACK and continues the flow

This second way of establishing the communication flow, in which the Client takes the initiative and opens the connections to the Server, is the solution (slightly more elaborate, as shown below) that lets Datacenters preserve the efficiency of the FTP protocol in data transfer while also preserving scalability, high availability and security, since NAT (Network Address Translation) addresses can be used behind a Firewall. In fact, in step [2] the FTP Server can suggest not only the port but also an address to connect to, which may differ from its own: for example, the address on which Oplon ADC is listening.

The diagrams below show two moments of the same scenario: in the first, Oplon ADC schedules the first transfer request on Server A; in the next image, the request is scheduled on Server B.

  1. The Client connects to the FTP Server on port 21 and sends the Server a proposed temporary address/port (e.g. 6060) to which the Server should connect to run the data transfer command

  2. Based on its routing policies, Oplon ADC opens a connection to Server A on port 2121

  3. Server A ignores the Client’s proposal and replies with its own PASV (passive) connection proposal to the Oplon ADC address/port (e.g. 6161) for the data transfer

  4. Oplon ADC transfers the information to the Client

  5. The Client opens a new connection to Oplon ADC (port 6161) and follows up on the command

  6. Based on its routing policies, Oplon ADC schedules the data connection to Server A/port (e.g. 6161)

  7. Server A transfers data to Oplon ADC from port 6161

  8. Oplon ADC forwards the data to the Client from port 6161

Below is the diagram of the same flow at the second connection and transfer request.

Note that the Data Listeners on Servers A and B are not always present: they are activated only when Data must be transferred, while the command Listeners are always active.

FTP Servers are set to “Active-mode” by default, so to get this functionality they must be set to “Passive-mode”. Check the manual of your FTP Server for the specific setting.

An important consideration concerns the consistency of the information stored on Server A and Server B and exposed by the corresponding FTP Servers. Load balancing does not look into the transmitted content, so the information handled by the respective FTP Servers/Services must be shared at Storage level, with a network Filesystem (CIFS/NFS), or with a Global Filesystem.

Below is an example of an Oplon ADC configuration file that achieves the result described above.

Attention: After setting the port range, restart the FTP service from Services: restarting it from the Internet Information Services (IIS) panel is NOT enough.

From the GUI, simply copy the template to the desired instance and change the addresses and ports on which connection requests will be accepted.

To set up a preconfigured endpoint grouping, simply go to the FTP template group and copy it to the desired instance. Once it is copied, you only need to change the number and addresses of the endpoints to use it.

parameters -FTP

Bind listenType-"NAT"
address"192.168.43.141" port"21, 65000-65010"
portForwarding-"true"
osiLayer"4"
protocol-"ftpDATA"
endPointsGrouping"FTP"
transport-"tcp"
          transportSessionAffinity-"true"
          enableVirtualDomain"false"
          enable-"true"/>
endPointsGrouping property groupName"FTP" loadBalancingType"Adaptative" enable"true"
virtualDomain, New1001 enable-"true"
endp property address"192.168.45.141" healthCheck"false" enable"true"
endp property address"192.168.45.143" healthCheck"false" enable"true"

Remote Desktop Protocol (RDP)

Remote desktop services provide multiple client sessions served by a host system.

Today this protocol has concrete applications, since the large amounts of computing power and network throughput available at affordable costs make a centralized desktop solution, known as VDI (Virtual Desktop Infrastructure), competitive.

Desktop centralization is not meant as a solution for every need, but with the growing availability of infrastructure it can now be applied in many Intranet and Extranet environments.

Centralization certainly offers undoubted advantages, first and foremost the ability to make highly available a service that, by its nature, could not be by definition if it relied on individual productivity tools.

In this context, Oplon ADC makes it quick and simple to handle both load balancing and high availability for Remote Desktop services.

The RDP protocol starts with a connection in which negotiation takes place, followed by a disconnection and then by a new connection based on the result of the negotiation.

The new connections are highlighted in red in the diagram below.

  1. The client connects to the server

  2. Negotiation

  3. Ack and disconnect

    1. (A) Connection with negotiated parameters

    2. (A) Connection evolution

    3. (A) Connection evolution…

This simple diagram makes it clear that, in an architecture meant to scale not only in terms of resource usage but also over time, through successive server-side and Client-side releases, the initial connection-negotiation-disconnection phase, in an environment with multiple Terminal-Server systems, must take place on the same machine as the initial connection.

Based on this consideration, in Oplon ADC version 9.0 you can choose between two different behaviors, depending on your needs. The first favors connection balancing at the expense of the persistence of the connection to a terminal; the second instead favors the persistence of the last connection, reconnecting to the existing “terminal” if it is still active.

For this reason, you must use the “session-affinity” parameter and limit it to the time it takes to make the first negotiation connection and the next data connection.

The two diagrams show the evolution of a connection flow that, for negotiation consistency, must take place on the same server on which the first negotiation connection was made.

To achieve this, Oplon ADC must remember, at least for as long as necessary, to establish the connection on the same Server on which the negotiation connection was made.

Below is an example of an Oplon ADC configuration file that achieves the result described above. It is followed by a configuration file that favors persistence of the last connection, returning to the same service after disconnection, within the session timeout.

Here too, there are ready-made templates that you can use directly through the Copy function of the HTML 5 interface: copying and customizing the listener(s)

Copying and customizing endpoints

RDP example (with session affinity)

Bind 
listenType-"NAT"
description-"Sample RDP with session affinity" 
address-"0.0.0.0" port"3389"
osiLayer"4"
protocol-"rdp-session-affinity"
endPointsGrouping"sample-rdp-mysession-affinity"
transport-"tcp"
transportSessionAffinity-"true"

endPointsGrouping property groupName"sample-rdp-mysession-affinity" enable-"true"
virtualDomain, New1001 enable-"true"
endp address"localhost" port"3389" enable"true"
	endp address"localhost" port"3389" enable"true"

Unlike a single-Server environment, in a balanced environment you cannot always be sure to return to the same server at the next connection request (either deliberately, when working without session affinity, or unintentionally, because the session has expired). This would raise some considerations about the session time-out and the position of the balancer within the architecture, which are covered in the certification course required to install Oplon ADC in a production environment.

In this regard, it is important to know that, in a typical architecture shielded by a firewall with NAT addresses, to get a good trade-off between resource usage and “terminal-server” sessions spread across all servers, the RDP Server must be set up so that sessions that are no longer used because they are disconnected are logged off, freeing up resources.

For this purpose, the RDP-server service (aka terminal-server) provides some parameters that automatically log off disconnected sessions, or set a time-out when a connection is not used for a specified time. In environments with many “centralized” workstations, these two parameters ensure that system resources are committed only to the sessions actually in use.

The parameters are very easy to set; the screenshots below show where to make these behavior changes.

Path:

Control Panel > Administrative Tools > Terminal Services Configuration > Connections > RDP-Tcp > Sessions

Also set:

Override user settings > When session limit is reached or connection is broken: End session

With many RDP-Clients, you should also set the time-out for unused terminals. The time-out value depends on how the workstation is used and on the resources available. The following example sets the session time-out to 2 days.

  • Parameters with rdp-nosession-affinity

    • Override user settings > End a disconnected session: 1 minute
    • Active session limit: Never
    • Idle session limit: 2 days

  • Parameters with rdp-session-affinity

    • Override user settings > End a disconnected session: 30 minutes
    • Active session limit: Never
    • Idle session limit: 2 days

This way, after 2 days of inactivity the Client is disconnected, and after 1 or 30 minutes the terminal server session on the server is logged off and its resources are freed.

Telnet, SSH & SFTP (Secure Shell & Secure File Transfer Protocol)

Secure Shell is a secure protocol for remote login and other services over the network.

This protocol, often used to provide secure remote command-line terminal functionality, has been designed to deliver multiple communication services between computers, as defined by RFC 4254, in particular:

  • “shell” for terminal shells

  • SFTP and exec requests (including SCP transfers)

  • “direct-tcpip” for client-to-server forwarded connections

  • “forwarded-tcpip” for server-to-client forwarded connections

In its most common form of use, the protocol uses a port listening in SSL where, depending on the command used (hereafter called Channel, as in the RFC 4254 specifications), the connection can evolve into different types of service.

Below is a graphical representation of how the connection evolves with this protocol, with the connection and its direction highlighted in red.

  1. The SSH Client connects to the Server on port 22

  2. The Server establishes a secure connection

  3. The Client specifies the Channel and its features

  4. Evolution of the connection with the features of the Channel

In this simplified representation, which is sufficient for the purpose of this manual, note that once the connection is established, it evolves within the same connection regardless of the Channel used. This behavior is very useful for delivering heterogeneous services through firewalls, and it also simplifies the Oplon ADC configuration, since there is a single channel to balance.

Below is a graphical representation of how a balanced SSH connection evolves. The connection and its direction are highlighted in red.

  1. The Client connects to Oplon ADC on port 22

  2. Oplon ADC connects to Server A, according to the load-balancing policies, on port 22

  3. Server A, with the SSH service listening on port 22, responds and establishes a secure connection

  4. Oplon ADC forwards the response from Server A

  5. The Client specifies the Channel and its characteristics to Oplon ADC

  6. Oplon ADC forwards the specifications to Server A

  7. Evolution of the connection with the characteristics of the Channel on the channel established with Oplon ADC

  8. Oplon ADC forwards the evolution of the connection to Server A

The following is the evolution of the connection at the next service request:

Configuring Oplon ADC to deliver balanced, highly available SSH services is very simple. Just a few considerations about the TimeOut: in the configuration below, the TimeOut, that is, the time after which an unused connection is closed, is set to 1 hour. You can adjust this parameter depending on the required security and on the Channel mainly used.

Here too, there are ready-made templates that you can use directly through the Copy function of the HTML 5 interface: copying and customizing the listener(s)

Copying and customizing endpoints

iproxy.xml -SSH-SFTP example

Bind listenType-"NAT"
description-"Sample SSH"
address-"0.0.0.0" port"222"
osiLayer"4"
protocol-"ssh"
endPointsGrouping"sample-ssh"
transport-"tcp"
transportSessionAffinity-"true"
enable-"true"
endPointsGrouping property groupName"sample-ssh" enable-"true"
virtualDomain, New1001 enable-"true"
endp property address-"localhost" port-"22" enable-"true"
endp property address-"localhost" port-"22" enable-"true"

Lightweight Directory Access Protocol (LDAP)

This protocol has been designed to provide directory services and has evolved over time due to the need to centralize large volumes of profile information.

As far as connection evolution is concerned, LDAP is a modern protocol, so balancing it presents no protocol difficulties. The only caveat is that the LDAP Server product must support multimaster functionality, or in any case be able to scale out across multiple systems. Since Oplon ADC version 5.0, you can also use LDAP servers with “active-passive” functionality by associating the “FailOver” balancing policy (see the white paper to learn more about FailOver).

Normally an LDAP Server is used to make many reads and few writes. It is therefore optimized for this type of function. However, in some circumstances there may be a need to implement an LDAP-based infrastructure with high write needs, and in these situations it is appropriate to adapt the balancing policies, as described below.

The port on which an LDAP Server normally listens is TCP port 389 and TCP port 636 for SSL on which the entire communication evolves. Below is the evolution of the protocol with the connections highlighted in red. These examples take port 389 into account, but the concept also applies to port 636 where the SSL connection evolves.

  1. The client connects to port 389

  2. The LDAP server establishes the transmission

  3. Protocol evolution

To balance this protocol, you therefore need a two-way forwarding engine; the evolution of the protocol in multimaster environments is shown in the diagram below.

  1. The client connects to port 389, on which Oplon ADC is listening

  2. Based on the balancing policies, Oplon ADC forwards the connection to the LDAP service on Server A

  3. The LDAP service responds to Oplon ADC with an ACK

  4. Oplon ADC forwards the ACK message

  5. Evolution of the client protocol to Oplon ADC

  6. Forwarding of the protocol evolution

The following image shows a new connection evolving, with the new request balanced on Server B.

As briefly mentioned in the introduction, these architectures are normally used mostly for reading, and in these situations balancing can be used without paying special attention to the balancing policies. It is a different matter with multimaster services where you need to make many writes followed by immediate rereads. Multimaster installations replicate the data repository so that they can effectively cope with failures without the need for clusters and shared global repositories. While this reduces the complexity of installation and maintenance, replication services are normally asynchronous, so if the number of updates and writes is very high, an immediate read of the same data on another node may not be consistent.

For this purpose, in both situations you can provide an appropriate balancing solution using session affinity. Note that LDAP Servers are normally used for countless reads and few writes, so in 98% of cases you will use balancing without session affinity, to make the most of the potential of this architecture.

Here too, there are ready-made templates that you can use directly through the Copy function of the HTML 5 interface: copying and customizing the listener(s)

Copying and customizing endpoints

iproxy.xml -LDAP example

Bind listenType-"NAT"
description"LDAP no session affinity"
address-"0.0.0.0" port"389"
endPointsGrouping"ldap-services"
enableVirtualDomain"false"
protocol-"ldap"
transport-"tcp"
osiLayer"4"
enable-"true"
endPointsGrouping property enable-"true" groupName-"ldap-services"
virtualDomain, New1001 enable-"true" virtualDomainName-"default"
endp property address-"localhost" enable-"true" port-"389"
endp property address-"localhost" enable-"true" port-"389"

iproxy.xml -LDAP with session affinity example

Bind listenType-"NAT"
description"LDAP no session affinity"
address-"0.0.0.0" port"389"
endPointsGrouping"ldap-services"
enableVirtualDomain"false"
protocol-"ldap"
transport-"tcp"
transportSessionAffinity-"true"
osiLayer"4"
enable-"true"
endPointsGrouping property enable-"true" groupName-"ldap-services"
virtualDomain, New1001 enable-"true" virtualDomainName-"default"
endp property address-"localhost" enable-"true" port-"389"
endp property address-"localhost" enable-"true" port-"389"

iproxy.xml -LDAP FailOver example

Bind listenType-"NAT"
description"LDAP no session affinity"
address-"0.0.0.0" port"389"
endPointsGrouping"ldap-services-failover"
enableVirtualDomain"false"
protocol-"ldap"
transport-"tcp"
osiLayer"4"
enable-"true"
endPointsGrouping property enable-"true" groupName-"ldap-services-failover" 
         loadBalancingType:"FailOver"
virtualDomain, New1001 enable-"true" virtualDomainName-"default"
endp property address-"localhost" enable-"true" port-"389"
endp property address-"localhost" enable-"true" port-"389"

Database listeners Oracle & Oracle RAC, universal DBMS

Transmission protocols that allow information to be exchanged between clients and the Database Management System (DBMS) are generally connection-oriented.

For Oplon ADC, these protocols too present no particular critical issues in tunneling and traffic balancing.

What is described in this section is used to balance connections to Oracle RAC and classic Oracle listeners. The same parameters can also be used for other databases whose transmission behavior does not differ from these.

Data traffic balancing techniques for a database are often not about traffic balancing in the strict sense, which the vendor also provides, but about decoupling the service delivery point from the requests, on a “cross” transport layer between applications, to provide high availability, disaster recovery, security, expandability during operation and, not least, simplicity.

Taking expandability (scalability) during operation as an example, decoupling clients from services makes it easy to add RAC nodes without changing client settings, which is highly valued in enterprise environments and in distributed client/server environments. Changing client settings, in fact, becomes very costly, especially when the clients are distributed (client/server applications), or is at least a delicate operation on application servers in production.

The evolution of the protocol, from a connection and transport point of view, can be summarized as follows:

  1. The client connects to port 1521

  2. The DB Server listener establishes the transmission

  3. Protocol evolution

  4. Information exchange

The following image highlights the role of the DB client connectors themselves in a multi-tier environment, where the connections to the database are made by application servers that in turn provide application services. Normally, the DB clients in the connection pool connect as soon as they start and are made available, already connected, to the applications that request them.

  1. The client connects to the service provided by the application server

  2. A DB client connection from the pool makes the request

  3. The DB Server listener establishes the transmission

  4. Protocol evolution

  5. Information exchange between the application within the application server, the pool DB client and the DB Server

  6. Result of the application processing sent to the application clients that requested the service

The scenario below summarizes the evolution of the connection, introducing the balancing layer in a completely transparent way.

  1. The client connects to the service provided by the application server

  2. A DB client connection from the pool makes the request to Oplon ADC

  3. Oplon ADC forwards the request to one of the two nodes

  4. The DB Server listener establishes the transmission with Oplon ADC

  5. Oplon ADC responds to the client that requested it

  6. Protocol evolution

  7. Information exchange between the client and the application within the Application Server

  8. The application server makes requests to Oplon ADC

  9. Oplon ADC forwards the request to the DB server listener

  10. Information exchange between Oplon ADC and the DB server listener

  11. Information exchange between the application server and Oplon ADC

  12. Information exchange between the client and the application server

  13. Maintaining the connection even after the client’s application activities are closed

Precisely because the protocol is connection-oriented, and therefore uses as few connections/disconnections as possible, it is important to ensure that connections are properly distributed across the nodes when they are opened. For this purpose, the “Adaptative” balancing policy always ensures that connections are evenly distributed across all DB nodes.

Here too, there are ready-made templates that you can use directly through the Copy function of the HTML 5 interface: copying and customizing the listener(s)

Copying and customizing endpoints

iproxy.xml -ORAC example

Bind listenType-"NAT"
description-"ORAC"
address-"0.0.0.0" port"51122"
endPointsGrouping:"ORAC"
enableVirtualDomain"false"
protocol-"ORAC"
transport-"tcp"
transportSessionAffinity"false"
osiLayer"4"
enable-"true"
endPointsGrouping property enable-"true" groupName-"ORAC" loadBalancingType-"Adaptative"
virtualDomain, New1001 enable-"true" virtualDomainName-"default"
endp property address-"localhost" enable-"true" port-"51222"
endp property address-"localhost" enable-"true" port-"51222"

For all other (non-peer) database types, the DBMS connection type is available since version 7.0. Its configuration does not differ from that of the Oracle/Oracle RAC protocol: the difference, of course, lies in the type of balancing, which cannot be peer-to-peer and must therefore be FailOver. A connection to the secondary server is attempted only when the connection to the primary server fails.

This solution works best when combined with Oplon Commander, which can switch service operations from one server to another in coordination with network activity.

iproxy.xml -DBMS example

Bind listenType-"NAT"
description-"DBMS"
address-"0.0.0.0" port"51122"
endPointsGrouping:"DBMS"
enableVirtualDomain"false"
protocol-"DBMS"
transport-"tcp"
transportSessionAffinity"false"
osiLayer"4"
enable-"true"
endPointsGrouping property enable"true" groupName"DBMS" loadBalancingType"FailOver"
virtualDomain, New1001 enable-"true" virtualDomainName-"default"
endp property address-"localhost" enable-"true" port-"51222"
endp property address-"localhost" enable-"true" port-"51222"

Bibliography

  1. Network Working Group J. Postel Request for Comments: 959 J. Reynolds ISI [http://www.w3.org/Protocols/rfc959/ ]

  2. Unix Network Programming, W.Richard Stevens, Prentice-Hall, 1998

  3. Unix System V, Network Programming, Stephen Rago, Addison-Wesley, 2000

  4. Web Proxy Servers, Ari Luotenen, Prentice Hall, 1998

  5. Load Balancing Servers, Firewalls, and Caches, Chandra Kopparapu, Wiley, 2001

  6. Reti di calcolatori (Italian edition of Computer Networks: A Systems Approach), Larry L. Peterson, Bruce S. Davie, Apogeo, 2004

  7. Network Working Group S. Lehtinen Request for Comments: 4250 SSH Communications Security Corp [http://www.ietf.org/rfc/rfc4250.txt ]

  8. Network Working Group T. Ylonen Request for Comments: 4251 SSH Communications Security Corp [http://www.ietf.org/rfc/rfc4251.txt ]

  9. Network Working Group T. Ylonen Request for Comments: 4252 SSH Communications Security Corp [http://www.ietf.org/rfc/rfc4252.txt ]

  10. Network Working Group T. Ylonen Request for Comments: 4253 SSH Communications Security Corp [http://www.ietf.org/rfc/rfc4253.txt ]

  11. Network Working Group T. Ylonen Request for Comments: 4254 SSH Communications Security Corp [http://www.ietf.org/rfc/rfc4254.txt ]

  12. Network Working Group W. Yeong Request for Comments: 1487 Performance Systems International [http://tools.ietf.org/html/rfc1487 ]

  13. Network Working Group W. Yeong Request for Comments: 1777 Performance Systems International [http://tools.ietf.org/html/rfc1777 ]

HTTP CompressionLayer 2/3 Setup