DokuWiki

The page here contains full installation details of how to set up a server to do the following:

• Create a captive portal using iptables and PHP
• Install a set of PHP scripts to manage the captive portal
• Traffic shape the internet connection to optimise its speed
• Create a “splash page” to display adverts to users
• Load share between multiple connections

The page referred to above is purely a list of instructions on how to set up the server, with little explanation of the scripts used. This page is designed to describe the scripts in more detail.

The PortalShaper package consists of:

• A set of iptables/iproute2 rules written in Bash, all packaged as portalshaper-sh.tar.gz
• A set of PHP scripts, all packaged as portalshaper-www.tar.gz
• A handful of other scripts, all detailed in the installation page above
• Customisation of Squid, again detailed in the installation page above

To install the scripts, please see the above installation page.

If you want more information as to how the scripts work, then carry on reading.

Packet marking is used extensively in the scripts to both traffic shape as well as route packets. As each packet can only have one mark value, the mark is split up, with different bits used for different aspects. A mask is used to differentiate between uses. The mark value is used as follows:

When following the instructions, a number of Bash scripts are installed into /usr/local/portalshaper/. Those scripts consist of the following:

This script contains all the settings. It is the only script you should have to edit. The settings are:

IPTABLES=/sbin/iptables    # The location of the iptables binary
TC=/sbin/tc                # The location of the tc binary
IP=/sbin/ip                # The location of the ip binary
IPSET=/usr/sbin/ipset      # The location of the ipset binary
MODPROBE=/sbin/modprobe    # The location of the modprobe binary

IF_LOCALNET=eth0           # The interface of the local network

The default gateway (primary internet interface). A default gateway needs to be set in order for routing to work initially. This will then be changed for the purpose of load balancing as packets traverse iptables. This value will be overwritten by the default-gw script if one of the interfaces fails.

IF_INTERNET_GW=ppp0

All the internet facing interfaces. Specify multiple ones for load balancing.

declare -a IF_INTERNET=( ppp0 ppp1 )

These are the relevant speeds for the interfaces above. The speeds should be slightly less than the ADSL line maximum to prevent buffering of the line at the remote end.

declare -a IF_INTERNET_DOWNSPEED=( 3800 3800 )
declare -a IF_INTERNET_UPSPEED=( 550 550 )

This script pulls all the other scripts together. Its operation should be obvious by looking at it.

This script is based on Jesper Dangaard Brouer's ADSL-optimizer project. It is used to calculate the amount of bandwidth to reserve in the upstream for ACK packets. If an upstream link gets congested (including ACK packets), then downstream downloads are adversely affected. This prevents that by giving upstream ACK packets a high priority.

This script sets up the captive portal iptables rules. It works by creating a new chain called “internet”. All packets are routed through this chain during PREROUTING. If the packets are from a MAC address that is recognised, then RETURN is called. If a packet traverses through the whole chain and the MAC address isn't recognised, then the packet is marked as per the table above.

A rule is added to the FORWARD chain to drop the packets that have the captive portal bit set. They can't be DROPed during PREROUTING, as there is no filter table.

Packets to Squid are DROPed in the INPUT table to prevent them browsing the web.

This script is used for general filtering rules. The following is carried out here:

• ICMP ping packets to the local host are accepted from any interface
• SSH connections to the local host are accepted from any interface
• SMTP connections are prevented from the local network (to prevent a client unknowingly sending spam)
• Each internet facing interface is configured in a loop to:
  • Accept a variety of outgoing traffic
  • Drop any other outgoing traffic
  • Accept any incoming related connections
  • Drop all other incoming connections

This script sets up the required routing for load balancing over multiple internet facing interfaces. The actual balancing is done in the mark-packets script, by marking different packet streams with a mark value (as per the table above) that changes according to the number of external interfaces.

The mark value is used within this script as a rule to route packets to different routing tables depending on the value. The routing table will then route the packets out over that particular interface.

SNAT is also done here, to ensure that the packets are transmitted with the correct source address.

This script does all the packet marking. The first part of the script is a “do” loop that iterates the same number of times as there are external interfaces. The loop sets up the load-balancing packets by doing the following:

• Applies a conntrack mark to any new TCP connection
• Restores the mark for the returning packet from the interface (all the scripts use the packet mark not the connection mark)
• Applies a mark to any other protocol (connectionless)
• Marks the same packets returning
• Does the same for locally generated packets from the DNS server and Squid.

The rest of the script does the remaining load balancing configuration, namely restoring the packet mark for TCP connections and setting the mark for DNS packets to the local network. The Squid packets do not need setting. These are retained by Squid according to the connmark value, which allows shaping to be done based on the external interface that was used.

A number of rules then follow to mark the traffic according to its type. This page shows the full details.

This script does the actual traffic shaping, based on the mark value.

The internet facing interfaces all have their own root qdisc applied. Ingress shaping is done using egress shaping on the local network interface. Because all the traffic passes through the one interface, multiple layers of classes are used, which use the full mark value (both the interface value and the traffic type value) to filter the traffic. An example for the local network is as follows:

1:1                  <   3800Kbit -   3800Kbit >    235.4 kbit/s  ( 32pps)
   1:101                <  760000bit -  760000bit >      6.3 kbit/s  (  2pps)
   1:301                <   1064Kbit -   3040Kbit >     48.5 kbit/s  ( 13pps)
   1:401                <  836000bit -   3040Kbit >      0.0 kbit/s  (  0pps)
   1:501                <  760000bit -   3610Kbit >      0.2 kbit/s  (  0pps)
   1:6661               <  380000bit -   3800Kbit >    180.5 kbit/s  ( 16pps)
1:2                  <   3800Kbit -   3800Kbit >   2776.6 kbit/s  (269pps)
   1:102                <  760000bit -  760000bit >      0.0 kbit/s  (  0pps)
   1:302                <   1064Kbit -   3040Kbit >    158.5 kbit/s  ( 26pps)
   1:402                <  836000bit -   3040Kbit >      0.0 kbit/s  (  0pps)
   1:502                <  760000bit -   3610Kbit >   1896.1 kbit/s  (181pps)
   1:6662               <  380000bit -   3800Kbit >    721.9 kbit/s  ( 62pps)

Full details of the HTB rules are contained at this page. It should be noted that a “flow hash keys” rule is used to split the bandwidth in classes evenly per client IP address rather than connection stream.

The PHP scripts are used to:

• Present the user with the captive portal pages
• Administer users of the system
• Create token codes for access to the system
• Edit the announcement splash pages

The only page that should need editing is the “settings.php” scipt.

Each script is described briefly here:

• admin.php - the administration interface
• announce.html - the HTML of the splash page. Should be writeable by the web server
• aup.txt - the Acceptable Use Policy
• functions.php - all the various functions
• jscripts - the TinyMCE HTML editor
• library - a tree that simulates to Apple device users that there device is connected
• portal-schema.sql - the SQL database schema
• settings.php - all the settings
• announce_days.txt - the days on which the splash page should be displayed
• announce.php - the splash page
• disable.php - a script to disable a PC from the captive portal
• index.php - the main captive portal page
• key.php - a key for encrypting the passwords
• logo-rnrmc.gif - a logo for the captive portal page
• reset.php - a page to email a user their password
• style.css - the CSS stylesheet
• validate.php - validate functions for emails and names

Squid is used as a web proxy, not just to cache pages, but also to display a splash page to users.

The configuration of Squid is described in the full instructions. For reference, the following parameters are configured:

• server_persistent_connections off - This prevents multiple traffic streams to the same server being seen as one connection by iptables, for the purpose of classifying traffic.
• qos_flows mark miss=0x1000000/0xF000000 - This does 2 things. Firstly it enables mark retention by Squid of the connection mark. Secondly, it sets an extra packet on the retained mark when it is output to the local network. This stops the packets being wrongly re-routed in the load-balancing tables.
• The splash page is configured in the ACL settings. The configuration is described fully in the section below.

The splash page makes use of Squid's session helper. The session helper runs a database of client IP addresses, and when they were last seen. In order to gain access to the web, a client's IP address must be “logged in” to the session helper. It then times-out after a predetermined time interval.

The session helper is configured with these rules:

external_acl_type session_active_def concurrency=100 ttl=3 %SRC /usr/lib/squid3/ext_session_acl -a -T 10800 -b /var/lib/squid/session/
acl session_is_active external session_active_def

Another external helper is used to only show the splash page on specific days (the inbuilt time ACL is not used so that configuration is possible without re-loading Squid). The datetime external helper is a simple perl script that returns a value depending whether the day is that specified in the file written to by the admin.php page:

external_acl_type session_day_def ttl=60 %SRC /usr/lib/squid3/datetime_acl.pl /var/www/announce_days.txt
acl session_day external session_day_def

This ACL is used to allow images regardless, so that external images can be used in the Splash pages:

acl images urlpath_regex -i (\.gif$|\.jpg$|\.png$|\.jpeg$)

The final rules pull the above rules together and specify the location of the splash page:

deny_info http://cwdwr.wardroom/announce.php?url=%u session_day session_is_active images
http_access deny session_day !session_is_active !images 

You will notice that there is no rule to “login” the client IP address. This is achieved in the PHP script of the splash page, which shells out to the same external session helper and “logs in” the client IP address. The reason for this is to ensure that a user actually clicks on the “continue” button themself, otherwise things such as automatic downloads force the disappearance of the splash page.

The script test-ppp monitors each external interface by attempting to ping external servers through that interface. If an interface fails then it is automatically restarted.

The script is designed to work with a Solos PCI ADSL modem. Experience has shown that sometimes the whole module will fail, in which case it will need unloading and reloading. If all interfaces have failed then this is exactly what is done.

A check is done at the end of the script to re-run the master script if the number of available interfaces has changed. This will cause a count of the number of available interfaces to be performed, and the load balancing updated as required. The default-gw script will also be run to change the main default gateway if required.

The script is not perfect, because if the interface with the default gateway on fails, then the DNS lookup cannot be performed and it therefore looks like all interfaces have failed.

A better solution is possibly LSM (http://lsm.foobar.fi/) which I have not looked at yet.

If everything is working correctly, when a user connects to the network, they will first be presented with the captive portal login page. Once they have created an account and credited it, they can use the “use internet” button to enable access to the internet.

Once they start to use the internet, there traffic will be categorised and prioritised appropriately, as well as balanced across the multiple external interfaces. Every 3 hours or so, they will be presented with the splash screen with adverts or other information on.

http://www.sysresccd.org/Sysresccd-Networking-EN-Iptables-and-netfilter-load-balancing-using-connmark