Q: How do I use LISTSERV's Application Programming Interface (API) to allow an external application to interact directly with LISTSERV?

Answer by Tom McDaniel
Consulting Analyst, L-Soft

LISTSERV users are quite familiar with the traditional communication layers used for LISTSERV commands: email through the LISTSERV command address (listserv@lists.example.com) or via the LISTSERV web interface. While these interfaces are certainly adequate for the most common, ad-hoc commands issued by individual users, neither is the best solution for automated procedures. When developing an external application that needs to interact with LISTSERV, the TCPGUI interface is the ideal communication layer.

Often overlooked, TCPGUI is LISTSERV's Application Programming Interface (API). Documented in Section 10 of the LISTSERV Advanced Topics Manual, TCPGUI serves as a means to execute commands directly to LISTSERV and, with some exceptions, receive an immediate response through the same interface. This is particularly useful when attempting to automate LISTSERV management that is triggered by an external system.

As an example, let's assume that an organization has a web-based system for approving and creating project work groups. When a manager approves a new work group, the web application will create a LISTSERV discussion list for the group and subsequently update the new group's contact information page.

For the sake of brevity, this Tech Tip won't reiterate content covered in the Advanced Topics manual. Section 10 includes a thorough introduction to TCPGUI programming and sample code for the lcmdx client, which we'll employ in this exercise. The lcmdx program is a C-based command-line tool for sending commands to TCPGUI. While this could be easily implemented in the web programming language of choice (PHP is used in our examples), we'll utilize the version provided in Section 10.5 of the Advanced Topics guide to simplify the example code.

Included Files

The source for this example is available on the dropbox.lsoft.us FTP site. The bundle's contents are described below.

ftp://dropbox.lsoft.us/download/TCPGUI_TechTip.tar.gz

Configuration

The target LISTSERV instance will need to be configured to use TCPGUI, which is documented in the Advanced Topics Manual, Section 10.1. Most LISTSERV instances will be accessible via TCPGUI provided port 2306 of the host machine is reachable via TCP. In addition to the hostname and IP address of the target LISTSERV instance, we'll need a set of postmaster credentials to create the list, since that is a privileged operation. It is good practice to create a dedicated postmaster account for use by automated scripts such as this.

The configuration variables used in this example are captured in config.php, and would need to be modified for your installation.

main.php

At an abstract level, our script is fairly simple. Primary components of the automated list creation process are:

1. Error checking $_POST variables
2. Create the list header based on a template
3. Use TCPGUI to connect to the LISTSERV instance and send the list creation command
4. Check command result and perform subsequent processing

Let's examine these components one at a time, taking a look at the LCMDX class when appropriate.

1. Error Checking

In our example scenario, the script is processed upon submission of a web form. It is good practice to validate form submissions to allow the user to fix any mistakes. However, variable checks are performed here for redundancy. We'll assume that the requisite variables – list name, description and owner email addresses – are contained in the $_POST array. (For testing, the $_POST variables can be set explicitly – see inline comments – and the main.php script invoked via CLI.)

The list name needs to conform to LISTSERV's guidelines. The description is just a text string, and the owners should be a collection (array) of valid email addresses. We can ensure the variables are set with PHP's built-in isset() function and preg_match() to do some regular-expression validation. See the following functions for details:

2. Creating the List Header

Our assumption is that all automatically created lists will be based on the same template. The discussion.list file encapsulates the list header we'll use. It would be quite simple to create several template files for different headers and offer the user several options regarding "list type." Regardless, our approach is to read the header template from file and use variable substitution to set certain keywords.

3. Send Command to TCPGUI

Having defined our list header, the next step is to build the list creation command and send it via lcmdx. The LCMDX wrapper class encapsulates all the code that pertains to directly interacting with lcmdx, including a create_list($list, $header) method. Instantiate a LCMDX object, then invoke the method:

Let's take a closer look at some LCMDX method implementations.

3.1 LCMDX.create_list()

In LISTSERV, the commands to create a new list and to update an existing list are the same. To ensure that we don't overwrite an existing list, the create_list() function first checks for the existence of the specified list. The LCMDX class contains a separate method for this: list_exists().

If the list name is unique, then create_list() will invoke the replace_header() method (the command is the same, as mentioned, so reuse the code) and return its results.

3.2 LCMDX.list_exists()

While a generally simple operation, checking for a list's existence is a two-part operation. Because lists are not always publicly visible, the script user must first authenticate. Then, providing a valid login ticket (returned by LCMDX.login()), we employ the CKLIST() LISTSERV command which returns a 0 (if the list exists) or 1 (if it does not).

function list_exists($list) {
	$ticket = $this->login();
	$cmd = "X-LOGCK $ticket CKLIST($list) WM: NOP";
	$ret = $this->command($cmd,$result);
	list($response,$list_exists) = 
		self::parse_response_line(array_shift($result));
	if ($this->in_error_state()) {
		// error
		return $this->error_code;
	} else {
		// success
		return $list_exists == "0";
	}
}

3.3 LCMDX.replace_header()

This method actually builds the command to PUT the new list's header. There are four parts to the command, all of which is combined into one line before sending to LISTSERV. All of this is documented in Section 10.4.1 of the Advanced Topics Manual.

1. X-STL is a special command for sending a list header, accepting list name and compressed header data
2. MAKE_WA tells LISTSERV to create the list's web archive contents at the time of creation
3. MAKE_NOTEBOOK tells LISTSERV to create the physical directories used for list message archives, if specified in the header by the Notebook= keyword
4. The return value of LCMDX::compress_header($header) is the actual header data, each line of the original header being prepended by its character count and an underscore, then compressed onto a single line. This the required format per TCPGUI documentation.

function replace_header($list,$header) {
	$cmd = " X-STL $list ";

	// make the WWW files
	$cmd .= "7_MAKE_WA";

	// make the archive files
	$cmd .= "13_MAKE_NOTEBOOK";

	// compress template file for TCPGUI
	$cmd .= self::compress_header($header);

	// execute the command
	$result = $this->command($cmd,$response);

	// Snip...
}

4. Check Command Result

After execution, the list creation result or error code will be returned back up the stack to the top-level controller in main.php. In each LCMDX method that executes a LISTSERV command, the response is checked for success or error messages. Should an error be encountered, descriptive information (code and message) will be captured in the LCMDX.error_code property. The easiest way to check for errors is simply to check this property via the LCMDX.in_error_state() method.

If no errors are encountered, then the portion of our script that interacts with TCPGUI is complete and we can proceed to subsequent processing.

References

1. LISTSERV 16.0 Advanced Topics Manual, Section 10
http://www.lsoft.com/manuals/16.0/htmlhelp/advanced%20topics/TCPGUI.html

2. ftp://dropbox.lsoft.us/download/TCPGUI_TechTip.tar.gz
ftp://dropbox.lsoft.us/download/TCPGUI_TechTip.tar.gz

Subscribe to LISTSERV at Work (American Edition).