Traffic Accounting and Quotas

FireRack Firewalls incorporate a flexible Realm and Entity based IP accounting system.

Introduction

IP traffic accounting is concerned with the efficient and accurate measurement of volumes of traffic within certain broad categories. It is not concerned with logging the exact nature of network activity. (That would be audit logging which is provided in FireRack by the Argus traffic flow logging system.)

'Traffic accounting' is not synonymous with 'traffic charging', however, there may often be a strong relationship between the two. 'Traffic accounting' is the mechanism by which a 'traffic charging' policy could be implemented. FireRack stores its traffic accounting statistics in specially optimized custom database format, and provides API calls and command-line tools to allow customers to extract the summarized data themselves and process as they wish in order to implement any charging policy they may have.

FireRack categorizes its accounting data in two ways; accounting entities and accounting realms. An accounting entity is simply a set of IP addresses for which a single set of accounting data should be collected. An accounting realm describes a network or set of networks. The FireRack will record separate byte, packet and connection counters for all traffic to and from each realm

Accounting Entities

FireRack currently supports two types of Accounting entities:

• A Security Zone
• A Registered Host

Accounting for zones and registered hosts is enabled from the 'Settings' tab under the 'Traffic Statistics' section of the management console, or alternatively under the 'User Quotas' tab of the zone configuration. To treat all hosts within the zone as a single entity turn on 'Per Zone' accounting. To treat registered hosts in the zone as separate entities from each other and from unregistered hosts, turn on 'Per IP' address accounting. For registered hosts the entity is defined by its zone, user and IP address; if any of the parameters change for a registered host then future traffic will be accounted to a new entity.

You should not turn on accounting for zones which represent your external and Internet connections, as this would create entities encompassing the whole world. Ideally you should only have entities representing your local networks and/or hosts. FireRack always attempts to account traffic to a single 'responsible' entity.

In the case where traffic is flowing between two local hosts, there are usually two entities involved. FireRack will only account the traffic to the entity that INITIATED the connection in this case, since the initiator is considered to be taking responsibility for the traffic. (Special handling is done for active FTP: the data connections should always be accounted to the entity that initiated the control connection.)

In the case where a local host is talking to an external remote host, the network administrator is typically more interested in knowing the total traffic usage by the local host. Since there is normally no entity associated with the remote host, the local host is always considered to be held responsible for the traffic regardless of which end initiated the connection, and hence the traffic is accounted to the corresponding local entity.

Entities are mutually exclusive, i.e. the same traffic can never simultaneously be accounted to more than one entity.

Accounting Realms

A realm is essentially an arbitrary set of IP addresses and/or subnets in relation to which traffic for entities must be accounted. For example, a realm might simply be defined as the Internet excluding your local subnets. This would allow you to account for use of your Internet backbone by local entities (zones or registered hosts) and thus be in a position to charge them for Internet traffic.

Realms are mutually exclusive.  Any given IP address can only lie within exactly one realm.  A 'default realm' encompasses all address space which is not part of an explicitly configured realm.

Realms are configured via the IP routing system: each route defined in a zone can be given a realm override. It is possible to have multiple routes via the same device or gateway to different subnets with different realm assignments.

An example of where this flexibility might prove very useful is in a university department, attached to the university network via a single router. The university network provides a link to the Internet; it is desirable for the department to account traffic to other networks within the university separately from Internet traffic, as it only has to pay for the latter. The department would therefore need to define separate 'University' and 'Internet' realms, and specify 'University' realm routes to each network range in use throughout the university in addition to a default 'Internet' realm route. All the routes would have the same gateway address.

Whereas a packet traversing the firewall is never accounted to more than one entity, for that entity the packet is usually accounted twice; once against the source realm of the packet and once against the destination realm.  Consider the case where a packet is received from realm A and sent to realm B and entity E is held responsible for this traffic. In order for this routing to occur E must lie either within realm A or within realm B, but it does not matter which is the case as it does not affect how the traffic is accounted. What happens is E's receive counters for realm A are increased accordingly and E's transmit counters for realm B are increased by the same amount.

It is therefore commonly observed that the total received counters for an entity across all realms will be the same as the total transmit counters for the same entity across all the realms. There will be some exceptions to this as it some traffic may be received and never forwarded (because it is destined for the firewall itself or was blocked) and some traffic can be sent without ever having been received (because it was generated by the firewall).

Accounting Database

It is the responsibility of the FireRack Management Server (FMS) to collect regular accounting data samples from the firewalls, and store these in a database.  FireRack uses a proprietary database format which is specially optimized for storing accounting data.  The data format has one file per firewall configuration, per 24 hour period starting at midnight UTC.  Each file contains 288 samples recording cumulative byte, connection and packet counters for every entity versus every realm at five minute intervals since midnight UTC.  It is hence possibly to efficiently determine the total counts between any two five minute aligned times on the same day, and generate time-series figures at various intervals.

The 'Traffic Statistics' section of the management console provides tools to examine the accounting database statistics and generate graphs showing average traffic rates for an entity and realm.  These same statistics and graphs are also available to users of registered hosts via the self-registration module if this module is installed.  A command-line tool ('adbquery') and XML-RPC API calls are also provided to facilitate programmed retrieval of this data.  

Accounting Data Collection

FireRack firewalls internally use cumulative traffic counters, which allows them some flexibility with data collection resolution.  The FMS typically collects a fresh sample from the firewall every five minutes, however, the firewall will automatically collect samples itself at hourly intervals and also in response to certain events, such as configuration pushes and reboot requests (the samples are stored in flash).

The FMS will compare each sample with the previous sample from the firewall.  It will calculate differentials for all the counters and interpolate these into the correct five minute slots in the accounting database.  Hence the exact timing of data collection is not critical.

When the FMS has been off-line or out of communication with the firewall for a period of time, it will collect all the samples automatically stored by the firewall at the earliest opportunity.  Because the firewall is storing cumulative counters there will be no loss of data: the total counters in the database should still reflect the true amount of traffic.  There will, however, be loss of resolution in the data.  This scenario can be witnessed as a smooth section a the daily traffic graph: the data plotted will effectively show hourly average traffic rates instead of 5 minute average rates.  If the FMS is out of contact with the firewall for very long periods of time, the resolution of the data may be further reduced as the firewall may delete intermediate samples to make room for new ones.

Command-line query tools

Several command line tools and scripts are included in the standard FireRack accounting package on the FMS to allow you to query the database. 

/usr/lib/firerack/acct/adbquery is the main tool that can be used for querying the database.   It outputs its results in a pipe ('|') seperated ASCII format, making it suitable for importing into a spreadsheet or database or processing with a script.  There are two main modes of operation for adbquery.  In both modes you must specifiy a time period over which you want the data.   Period end-times may absolute dates and times or relative to the current date and time (e.g. now or last midnight) or the time of the last full sample inserted into the database.  Start times may be absolute or relative to the end time (e.g. 24hr period up to end-time).

You may run /usr/lib/firerack/acct/adbquery without any options to get its syntax:

Usage: /usr/lib/firerack/acct/adbquery (<realm id>|ALL|ALLKNOWN|SUMALL|SUMKNOWN) <start time> <end time> <output> <firewall config name> (<entity id>)

<start time> can be: 
     absolute seconds since epoch,
     absolute local date and time in format DD/MM/YYYY_HH:MM:SS, 
     -x(h|m|s|d) where x is time prior to end and 'd','h','m', or 's' are units, 
     'midnight', implying 00:00:00 local time immediately prior to <end time>,
or   'midnightgmt', implying 00:00:00 GMT immediately prior to <end time>.

<end time> can be: 
     absolute seconds since epoch,
     absolute local date and time in format DD/MM/YYYY_HH:MM:SS, 
     'latest', implying the end of last full timeslot,
     'midnight', implying 00:00:00 local time immediately prior to current time,     'midnightgmt', implying 00:00:00 GMT immediately prior to current time,
or   'now', implying the current system time.

<output> is one of: 
     'total', giving total usage per entity between start and end 
     peak-rate-X(d|h|m) 
     total-X(d|h|m),
     rate-X(d|h|m) where X is size of timeslot for timeseries output,
        giving usage per entity per timeslot of specified size. 

'total' mode will give a summary of total traffic between the two specified time points.  (Where both time points lie within the same accounting database, this is simply taken as the difference between the cummulative counters at each time, thus making this a reasonably fast query mode.)
If the time points are not exactly aligned on the 5 minute sample period boundaries, interpolation is used to estimate the appropriate values.  Totals may be given for a specific realm ID, all realms, all realms excluding the default realm ('ALLKNOWN'), as a sum of all the realms ('SUMALL') or as a sum of all realms except the default ('SUMKNOWN').  Realms are placed across the columns, with each realm having a columns for each direction for bytes, packets and connections.  Entities are listed down the rows; by default all entities are included although an optional entity ID may be specified to limit the output.

For viewing the output of adbquery from the command line the unix 'column' command can be useful for reformating.  E.g.:

 /usr/lib/firerack/acct/adbquery SUMALL midnight now total name.of.firewall | column -n -t '-s|' |less

The other mode of operation is to give timeseries data, which can take three forms; absolute quantities ('total-...'), average rates ('rate-...') and peak rates ('peak-rate-...' : the maximum average rate over any 5 minute sample period).  For each form the desired resolution can be specified in days, hours or minutes.  E.g. 'peak-rate-1d' will give a 1 day sample resolution and output the peak data rate for each 24 hour period (aligned to midnight GMT).  

The timeseries output format is '|' delimited with no header line and one row of output per sample period.  The columns are as follows:

number of last 5 minute sample slot in range (base 0)
time at start of sample range (seconds sinch epoch)
time at end of sample range (seconds sinch epoch)
received bytes
sent bytes
packets in
packets out
connections in
connections out

For rates, the above counters are in units per second.

Special Handling for Fail-over Configurations

Where two firewalls constitute a fail-over pair, only data from the master firewall will be inserted in the accounting database.  However, the FMS will periodically connect to the slave firewall to collect any automatic samples it may have taken.  The most recent of these samples is always kept as it may be used as a statistics baseline should the master and slave firewalls be swapped.  Each sample records the mode of the firewall (master or slave) at the time the sample was taken, so the FMS should handle a transition correctly even if it was out of contact with the firewalls at the time.

User Bandwidth Quota Management

User Bandwidth Quota Management is an optional module that can be used to apply limits on the volume of data transferred by registered hosts within definable time periods.

A quota specification can be defined with the 'User Quotas' section of the configuration for a security zone.  You can define multiple quotas which are applied at the same time, thus allowing you to have different quotas for different time periods (e.g. you could allow 1GB in any 24 hour period but no more than 3GB in any 7 day period) or to trigger different actions (e.g. notify by email at 1GB and begin throttling at 1.2 GB).

A quota specification has the following parameters:

• A description.  
• The data transfer limit, specified in KB
• The period length, specified in hours.  Quotas are applied on a rolling basis so the quota system will always query the accounts data for the total byte counts over the period starting at the time of the last sample less this time.
• The accounting realm.
• The direction of the quota; the quota can be applied only to traffic received from the realm, to traffic sent to the realm or the sum of the two.
• The user login name.  Normally you would not specify this: a quota without a user specified is a default quota for the zone and applies to any users with registered hosts in the zone.  If you define quotas for a specific user these will override all the default quotas for that user.
• The scope of the quota; whether it should be applied to each IP address individually ('per IP') or to the totals for all hosts registered to each user ('per user').
• An optional dynamic group, to which any IP addresses deemed over quota will be added.  As you can write firewall rules that depend on membership of these groups this is extremely powerful, and can be used to selectively block, redirect or throttle connections to or from hosts which are over quota.
• Whether or not to e-mail the user with a notification message that they are over quota.  The messages are generated from a template on the FMS which can be modified.  It is also possible to have multiple custom templates corresponding to individual quotas.