LISTSERV Tech Tip: Why do my logs show my LISTSERV server sending mail to other LISTSERV sites?

Q: Why do my logs show my LISTSERV server sending mail to other LISTSERV sites?

New LISTSERV customers are often confused to look in their log files and find that their LISTSERV server is sending and receiving mail to and from other LISTSERV sites. For example:

The short answer is that by default, LISTSERV servers communicate with one another to share things like spam alerts, network routing updates and list mail distribution. This feature can be enabled or disabled by way of the RUNMODE site configuration variable. For the longer answer, we need to go back in time to the days when bandwidth was scarce, server time was expensive and network connectivity wasn't necessarily an "always-on" resource.

To understand the nature of LISTSERV's inter-server updates, it helps to understand a bit about the world into which LISTSERV was born. For most users today, the word "Internet" is nearly synonymous with the World Wide Web. Even for non-web services like email, a great many users access those services by way of a web browser. But in 1986, when Eric Thomas's "Revised LISTSERV" was released, the World Wide Web didn't yet exist, and the Internet itself was still in its infancy. It consisted of a number of disparate networks – ARPANET, BITNET, NSFNET, USENET, etc. – the nodes of which may or may not have had gateways to one another. Messaging between those nodes was neither direct nor instantaneous, and messages were usually sent or retrieved in batches as nodes connected to the network temporarily to communicate. For something like a mailing list, things were even more complicated because the sending server had to connect not just to one remote system to send a message to it, but to multiple recipient systems. Doing so was slow and costly – in terms of both money and network resources. Such was the electronic world into which Revised LISTSERV was set loose in 1986.

Let's start dreaming and consider the following situation. A revised LISTSERV
machine has been installed in the major hub nodes of all countries. It has been
granted enough  disk space to hold  the massive BITEARN NODES  file, and enough
CPU time to process it at  initialization to set various GLOBALV variables. One
of the network coordinators has volunteered to maintain a list of all the sites
who have installed revised LISTSERV, and  to generate a kind of "routing table"
for LISTSERV. All of this is wishful thinking, but what's wrong about dreaming?
:-) Now a user at  our node (a dead-end node indeed) wants  to send a 400-lines
file to a list of 500 persons all  over the network (let's say it's an electro-
nic magazine about computer networks, or something like that). Instead of doing
a SENDFILE, which would hog the network during days (128 link.Mbytes load, assu
ming an average distance of 8  links between two randomly selected BITNET nodes
-- see Hank's statistics), he issues a  LSEND command (same syntax). Instead of
going directly to RSCS,  the file goes to the nearest LISTSERV,  with a list of
recipients appended to it. The server will distribute the file to all the reci-
pients in its 'service  area' (ideally only the local node).  It will then scan
the list of remaining recipients, search its "routing table" for the correspon-
ding LISTSERV's userid@node, and build a  list of them. It will eventually ship
the file to all the selected LISTSERVs, after having edited the list of recipi-
ents so that only the nodes which  were routed through a given server appear on
the list of recipients  sent to that server. I can't find  a formula to compute
the resulting  network load,  because it  depends on  the configuration  of the
servers, but it would probably be 10 times smaller....

In short: what if each LISTSERV server knew about each of the others, and they used each other for shared mail distribution, passing messages only to and from their nearest neighbors, something like a fireman's bucket brigade? A crazy dream, indeed, but one that was to become a core feature of the LISTSERV service: the LISTSERV backbone.

As LISTSERV sites came online, they had (and still have) the opportunity to participate in the LISTSERV network, as well as the opportunity to join the LISTSERV backbone. LISTSERV nodes registered as backbone sites are sites that are assumed to be online 24 hours a day, 7 days a week and running the latest version of the LISTSERV software. As the name implies, LISTSERV backbone sites are the "spine" of the LISTSERV network – they pass mail along from one node to another along the backbone, and on to the local network nodes that are terminal points on the network but do not themselves relay mail to each other the way that the backbone sites do. The backbone model of LISTSERV mail distribution allowed for much more efficient use of network resources and greatly reduced the overhead involved in distributing list mail – a great benefit, indeed, in the early days of the Internet.

As the network grew and new challenges emerged, the LISTSERV network was put to purposes other than just mailing list distribution. For the network to operate effectively, it was important that all sites keep up-to-date copies of the network tables. Those tables themselves (contained in files like INTPEERS.NAMES and BITEARN.NODES) are pushed along the LISTSERV network via automatic updates. Networked LISTSERV sites also send along a list of their public email lists, all of which are added to CataList, the official catalog of LISTSERV lists. As spam became more of a problem and LISTSERV came to include spam-detection algorithms, the network was again leveraged. If one LISTSERV server detected a piece of spam, it could push a spam update including a message checksum across the network to the other LISTSERV nodes, so that if they received the same piece of spam, it could be immediately recognized and quarantined.

The most common types of LISTSERV backbone mail that you may see in your logs are as follows:

LISTSERV's participation in the LISTSERV network is voluntary and controlled by the RUNMODE site configuration variable. The possible settings are as follows:

LISTSERV Classic/HPO sites that wish to participate in the LISTSERV network should complete the registration form at: http://www.lsoft.com/products/regform.asp. Sites that do not wish to participate do not need to register and should set their RUNMODE to "Standalone".