|
Tech Tip (LISTSERV) – Issue 2 – 2007
Q: What should I do in the unlikely event that LISTSERV crashes, and how can I prevent it?
Answer by Michael Shannon
Senior Consulting Analyst, L-Soft
L-Soft customer support sometimes gets email from site managers complaining that LISTSERV has suddenly crashed. On inspection of the system log, something similar to the following is often seen as the last few lines before the crash:
20 April 2007 10:00:00 Reindexing LISTNAME LOG0704...
20 April 2007 10:00:00 Reindexing MYLIST-L LOG0704...
20 April 2007 10:00:01 Reindexing LIST-L LOG0704...
20 April 2007 10:02:17 LISTSERV(R) for Windows version 15.0 starting...
20 April 2007 10:02:17 Copyright Eric Thomas 1986-2007
[...]
|
Note: All these examples use Windows notation. If you use another operating system, please adjust accordingly. Some Unix logs may even make reference to a HeapAlloc() error at some point.
Looking further into the log often reveals that LISTSERV was swapping a lot of data in and out of memory shortly before crashing:
20 April 2007 09:52:03 Reindexing LIST-L LOG0704...
*** FIO file cache now totals 41826k. A list of cached files follows. ***
File size Usage Flags File name
--------- ----- ----- ---------
39108k 1 U D:\LISTS\LIST-L\LIST-L.DBRINDEX
2718k 1 D:\LISTS\LIST-L\LIST-L.LOG0704
1k 1 U D:\LISTSERV\MAIN\DIGESTS.FILE
Load_index: V=672 T=922 size=165248k
*** FIO file cache now totals 40274k. A list of cached files follows. ***
File size Usage Flags File name
--------- ----- ----- ---------
26352k 1 D:\LISTS\MYLIST-L\MYLIST-L.DBRINDEX
13922k 1 U D:\LISTS\LISTNAME\LISTNAME.LOG0704
1k 1 U D:\LISTSERV\MAIN\DIGESTS.FILE
[...]
|
LISTSERV holds a lot of data in memory at one time, especially when indexing a list's archive files. Sometimes this can prove to be too much for the system and results in an unexpected crash.
But, I have plenty of memory! Why did it still happen?
The computer on which you have LISTSERV installed may very well have lots of memory in it. It's not uncommon to see installations where 2GB of RAM or more is present. However, just because you have 2GB installed doesn't necessarily mean that you have 2GB available.
The reason an otherwise high-spec computer may run out of memory is fragmentation. In other words, the operating system has allocated blocks of memory to all the applications running, but they aren't always blocks from the same, or adjacent, memory spaces. Thus, when LISTSERV requests a block of nn size, there may not be a contiguous block available, and a crash ensues.
A natural assumption is that the 'easy fix' will be to just install more RAM. This isn't always the case. Installing more RAM, or a memory management application, may help to relieve the situation but doesn't address the underlying problem.
What is the problem?
The most common reason for memory problems is the sheer size of list archives and indexes. More and more list archives are growing very large due to the list's being in existence for a long time, increases in traffic, or because the subscribers are sending large attachments in posts. LISTSERV needs to index these archives to facilitate searching – sometimes twice, which further compounds the problem (explained later).
Reducing the size of archive and index files is the most effective way to prevent LISTSERV from running out of memory. Limiting the number of indexes will also help in this respect, especially while they are being generated. During the indexing process, the actual amount of memory being consumed will be more than the size of the archive file itself, sometimes even by as much as a factor of 3 or more.
Reducing archive size #1
Check the logs for lists that regularly appear in the FIO warnings. This is a good indication that the archive files are getting overly large. Look at the list's Notebook= keyword to see how often they're generated. The aim is to reduce the size of an archive file so you need to increase the frequency that the notebooks are generated. So, for Yearly archives you will want to change them to Monthly. For Monthly archives you will want to change them to Weekly.
This will have the effect of splitting the archives into more 'chunks' than you previously had, but each one will be significantly smaller. It will also mean more entries on the list's Web archive interface, too. Note that it will only have an effect on all new archive files; older files will remain the same and will not be automatically split by LISTSERV.
If you want to go the extra step, you can manually split up the old notebook files into smaller files. Chapter 8.10 of the LISTSERV 15.0 Site Manager's Manual (www.lsoft.com/manuals/15.0/LISTSERV15.0_SiteManagersManual.pdf) describes their format in detail. Just remember that they must remain as plain-text, and to name them appropriately. Once you have removed the older, larger files and put in the newer, smaller files you need to delete the listname.DB* files (there's usually 3-4 of them). These are the old indexes and will be out-of-sync with the new files, causing errors in searches if they are left alone. They will be automatically regenerated the next time the list header is updated or a search is performed.
Reducing archive size #2
When an index is generated for a list, LISTSERV needs to parse all the notebook files that are present. So, if your list started in early 1997 and is archived monthly, that means you will have more than 120 individual files to index.
Another excellent method of reducing archive sizes is to split the list into one or more smaller lists that are used solely for the purpose of giving access to old posts. Using the example above you could have something like this:
MYLIST-L
MYLIST-L-2006
MYLIST-L-2005
MYLIST-L-2004
MYLIST-L-2003
[...]
|
You don't have to use yearly periods like this example does. You're free to divide them up into whatever size periods you like, keeping in mind not to let them become too big and to name each list appropriately.
For each period's new list, simply clone the original and set these keywords as follows:
Subscription=Closed
Send=Owner
Notebook=Yes, D:\LISTS\MYLIST-L-2006,Monthly,(MYLIST-L)
|
The important point here is to allow Notebook access to the original list by adding its name to the keyword's access-level. Move all the notebook files for 2006 from the original D:\LISTS\MYLIST-L directory and put them into D:\LISTS\MYLIST-L-2006, remembering not to copy over the .DB* files as well. Repeat this process for each period until you have as many lists with their own archive directories as required. (Don't forget to delete the .DB* files from the original list's archive directory.)
Now, LISTSERV will only have to index a fraction of the files for each time period. In fact, once the older periods have been done, they really shouldn't need indexing again. It will also have the effect of making the actual index files themselves correspondingly smaller. The subscribers of the original list will still have full access to all the archives; they just have to be told where to find the ones from previous periods and how to access them.
Reducing index size
As mentioned previously, LISTSERV will usually generate two indexes for a list. There's a forward-index (listname.DBINDEX) and a reverse-index (listname.DBRINDEX). The reason for this is that it increases the speed of searching, especially those searches done using the Web interface. You can reduce the strain on system resources by turning off reverse-indexing.
To do this, put the following keyword into the site configuration file:
Restart LISTSERV for the change to take effect. From this point on LISTSERV won't generate a reverse-index for any list. The tradeoff is that searches may take slightly longer to complete in the future, but the benefits will be less time spent indexing archives, less memory consumed, and fewer files for the operating system to keep track of.
For more details, including OS-specific notation and exporting, please see Appendix C of the LISTSERV 15.0 Site Manager's Manual: www.lsoft.com/manuals/15.0/LISTSERV15.0_SiteManagersManual.pdf.
|