Tech Tip (LISTSERV) – Issue 1 – 2007

Q: How can I limit the rate at which LISTSERV delivers mail?

Answer by Liam Kelly
Senior Consulting Analyst, L-Soft

L-Soft has designed LISTSERV HPO to be the fastest and most resource-efficient mailing list manager available. However, there may be situations in which the LISTSERV administrator needs to slow LISTSERV's delivery of outbound mail to the SMTP server to avoid saturating the outbound Internet connection. Starting with LISTSERV version 15.0, LISTSERV is able to do such rate limiting. (This feature is available only with the LISTSERV High Performance Option, and only on non-VM platforms. To upgrade to LISTSERV HPO, consult your L-Soft sales representative.)

SMTP rate limiting works by limiting the volume of traffic that LISTSERV's SMTP delivery workers send to the outbound SMTP server(s) and is controlled by the SMTP_RATE_LIMIT site configuration keyword. In the simplest case, we can limit all outbound traffic leaving LISTSERV. For example, to limit outbound LISTSERV bandwidth to 12 Megabits per second:

The rate limiting setting accepts units of 'kbps' (kilobits per second), 'Mbps' (Megabits per second), or 'Gbps' (Gigabits per second). The units are not case-sensitive.

Note that the SMTP_RATE_LIMIT settings affects only the delivery of email from LISTSERV to the SMTP server; it does not directly affect the delivery of messages from the SMTP server to the Internet. Note also that SMTP_RATE_LIMIT requires that one or more SMTP_FORWARD_n hosts be defined in the LISTSERV site configuration. (Synchronous delivery through the fallback SMTP_FORWARD settings ignores SMTP rate limiting. The old ASYNCH_SMTP settings is deprecated and should not be used with SMTP_RATE_LIMIT.)

Because rate limiting is designed as a means to limit Internet bandwidth usage, LISTSERV does its rate limiting bandwidth calculations on the basis of the message size and the number of recipients, not on the amount of data actually transferred to the SMTP server. This is because the amount of data transferred to the SMTP server is different for a BSMTP (bulk SMTP) message than for the data transfer to the individual destination sites. For instance, a '.MAIL' file in LISTSERV's outbound spool containing a one kilobyte messages with 500 recipients may be only two kilobytes in total size in LISTSERV's spool, but it will be considered 500 kilobytes for the purposes of rate limiting (1k * 500 recipients). However, the LISTSERV SMTP workers will not actually separate a large '.MAIL' file for the purposes of rate limiting – either the whole file goes to the SMTP server, or none of it does. This means that if we have a large MAXBSMTP and/or a large message size, there may be observable period of inactivity between .MAIL files that individually exceed the SMTP_RATE_LIMIT setting. For smoother bandwidth usage between LISTSERV and the SMTP server, reduce the MAXBSMTP setting as to produce smaller '.MAIL' files in LISTSERV's spool.

For sites with multiple SMTP_FORWARD_n workers defined, the total rate limit is divided evenly among the SMTP workers. So with two SMTP workers defined and SMTP_RATE_LIMIT set to "12Mbps", each worker will individually try to use 6Mbps of bandwidth. Because the SMTP workers do not split individual '.MAIL' files, a single large '.MAIL' file will still go through a single worker. Again, reducing the MAXBSMTP setting will provide more output files and thus a more even distribution.

If LISTSERV is configured to use multiple SMTP worker pools, it is also possible to specify the rate limit per pool. For example:

In the example above, the "A" and "B" pools are limited to 1.5Mbps, while the C pool is limited to 150kbps. The entire output for the server is limited to 12Mbps (including the totals of the A, B, and C pools, as well as any other pools defined in LISTSERV's site configuration).

For more information on configuring delivery pools, contact L-Soft support. For more information on rate limiting, see the LISTSERV 15.0 release notes: www.lsoft.com/resources/manuals.asp.