Q: How can I prioritize my outbound delivery for different LISTSERV lists?

Answer by Liam Kelly
Senior Consulting Analyst, L-Soft

LISTSERV sites commonly have a need to assign a different priority to traffic for certain lists. For example, a university may have an emergency notification list, and messages for that emergency notification list shouldn't get stuck in the mail queue behind non-emergency messages. While the default LISTSERV queuing is first-in-first-out, that can be modified by way of the SMTP Worker Pool feature.

Introduction to SMTP Worker Pools

In the default LISTSERV delivery model, LISTSERV writes outbound messages to the spool folder as *.MAIL files, and the SMTP delivery workers process the *.MAIL files in sequential order. A simple SMTP_FORWARD configuration might look as follows:

In the example above we have two outbound mail servers: SCOOBY.EXAMPLE.ORG and SCRAPPY.EXAMPLE.ORG. Two SMTP worker processes grab mail from the spool and deliver it outbound through SCOOBY, and one SMTP worker process grabs and delivers mail through SCRAPPY. (SCOOBY also serves as our SMTP_FORWARD fallback if spawning the workers should fail for some reason.) There is no message prioritization or queue separation; all other things being equal, SCOOBY will end up handling roughly twice the mail of SCRAPPY, and all mail will be handled in a first-in-first-out order.

SMTP worker pools allow us to separate the queues. The idea is that we can have SCRAPPY handle mail only for certain lists, while the rest of the messages will go through SCOOBY. We may decide to reserve SCRAPPY for our emergency notification list, while SCOOBY does everything else. In that case, the configuration might look as follows:

Here, SCOOBY handles the 'A' pool, which is defined as the default pool for list messages. SCOOBY also handles the '-' pool, which is the default pool for things like command responses, message posting rejections and certain other administrative messages. SCRAPPY handles only the 'B' pool. To assign our emergency list to the 'B' pool, we would add the following to the list header:

We still get three SMTP worker processes, but only the two pointing to SCOOBY will be doing any work under normal circumstances. The worker assigned to the 'B' pool (and therefore SCRAPPY) will be idle until there is a message sent to the emergency notification list. At that point, the emergency list mail will start delivering through SCRAPPY, regardless of what might already be in the LISTSERV queue or the SMTP queue on SCOOBY.

Other Uses for SMTP Worker Pools

While SMTP worker pools are useful for creating high-priority queues with dedicated mail servers, they have other uses, as well. LISTSERV recognizes up to 27 pools: A-Z, plus the default '-' pool. One common use of pools is to do virtual domain hosting. LISTSERV can recognize any number of hostnames in the sitewide MYDOMAIN setting, and different lists can be assigned to different hostnames by way of the List-Address list header keyword. By using SMTP worker pools, it is possible for lists with different hostnames on the same LISTSERV server to deliver mail through different outbound SMTP servers. (This can be useful for SPF/SenderID verification, among other things.) So we could do:

In the sitewide MYDOMAIN setting, we would include both LISTSERV.EXAMPLE.ORG and LISTSERV.EXAMPLE.COM. Then we could set our list configurations as follows:

Mail sent to the first list would use the EXAMPLE.ORG mail server for its delivery; mail sent to the second would use the EXAMPLE.COM server. The hostname of the list address would match the hostname (and presumably the SPF record) of the appropriate SMTP server.

Another use of SMTP worker pools is to reduce the impact of backscatter. While there are ways to reduce the number of command responses and administrative mail that LISTSERV sends, there is still some risk of spam mail sent to LISTSERV command addresses generating backscatter responses that may negatively affect the reputation of the SMTP server. One way to mitigate that is to send command responses and administrative mail through a different pool than list messages:

In the example above, we've moved the '-' pool for command responses to SHAGGY. Our list messages go through SCOOBY by default, except for those messages in the 'B' pool that go through SCRAPPY. If we end up with any backscatter problems because of command responses, they should be limited to SHAGGY, without affecting list mail delivery for SCOOBY or SCRAPPY.

It is also possible to use the Delivery-Pool keyword to separate administrative mail (such as Daily Error Monitoring reports, welcome messages, bounce reports, etc.) from the list mail:

In the example above, messages posted to the list would go through the 'B' pool, while administrative mail would go through the 'A' pool.

Conclusion

The above serves as an introduction to SMTP worker pools. The LISTSERV Advanced Topics Manual covers worker pools in more detail, including setting pools for LISTSERV DISTRIBUTE jobs (including LISTSERV Maestro mailings), scheduling pools based on time-of-day, setting fallback servers, and rate-limiting delivery pools (LISTSERV HPO only). See the LISTSERV Advanced Topics Manual for details and additional examples.

References

• LISTSERV Advanced Topics Manual
• LISTSERV Site Configuration Keyword Reference
• LISTSERV List Header Keyword Reference

Subscribe to LISTSERV at Work.