Tech Tip (LISTSERV Maestro) – Issue 1 – 2006

Q: Sending a mailing to all of our recipients at once causes large spikes in web site access since most of them tend to respond at the same time. How can we spread the mailing out to ease the load on our web site?

Answer by Françoise Becker
VP Software Engineering, L-Soft

By combining auto-repeat jobs and target groups with formulas, LISTSERV Maestro makes it quite simple to ease the load on your web site when sending a mailing to a large number of recipients in a Hosted Recipient List (HRL). The example below illustrates one way to spread a mailing out over a specific period of time, sending one-fourth of the recipient list every hour over 4 hours. This example uses the following steps:

• Create a hidden dataset field called TIMESLOT containing the numbers 0 to 3 assigned evenly among all subscribers.
• Create a target group to check the timeslot against the current hour and select those recipients whose timeslot is equal to the hour modulo 4 (the remainder of the integer division of the hour by 4, resulting in an integer between 0 and 3).
• Create a job that uses this target group and auto-repeats four times, with an hour interval between each run.

The TIMESLOT Field

First, you'll need to create a dataset field called TIMESLOT. Define this field as a Number field and mark it as Hidden (see Figure 1), making it invisible to subscribers when they look at their data. (Hiding fields that are meaningless to subscribers prevents confusion.) Since it is hidden from the subscribers, the data administrator (or another team member who has permission to modify membership data) must assign values to each subscriber. This is easily done by uploading a file and mapping the TIMESLOT values to the email addresses. For more information on uploading a file, please see Section 9.3 in the Data Administrator's Manual.

Figure 1: TIMESLOT Field Definition

The Target Group

After creating the dataset and populating a hosted recipient list within it, you'll need to create a target group that will select all recipients sharing a single time slot. Because your auto-repeat jobs will always be spread over 4 hours, you can use the hour of sending modulo 4 as the time slot to use for each. If you sent one time slot each day, then you could use the day number modulo 4. Different formulas can be concocted for nearly any sending pattern.

To create the target group:

1. Go to the Recipient Warehouse.
2. Click on Recipient Target Groups.
3. Click the Create a new target group with recipients based on drop-down menu and select Hosted Recipient List. Click [Create].
4. Give the target group a name and description. Click [Next->].
5. Select the relevant dataset and hosted recipient list (see Figure 2). Click [Next->].

Figure 2: Select the Dataset and List

6. Click the Actions on Selected Node link, and then select Add Condition (see Figure 3).

Figure 3: Add Condition

7. Enter the condition (see Figure 4):

the field
TIMESLOT
=
the value of the formula
ToNum(ToDate(CurrentMillis, "HH")) %4

Click [OK].

Figure 4: Define the Condition

8. The Source Details screen shows the condition in text form as:

TIMESLOT=(ToNum(ToDate(CurrentMillis, "HH")) %4)

Click [Next->].

9. Check the Yes, allow the definition of recipients based on this target group option. Click [Finish].

The Auto-Repeat Job

When creating a mailing to be spread over 4 hours, select the target group defined above on the Source screen in the Define Recipients wizard (accessed by clicking Define Recipients from the Workflow diagram or by clicking the Edit link in the Recipients section of the Summary tab).

To make the job auto-repeat 4 times, click Delivery Settings from the Workflow diagram (or the Edit link in the Delivery section of the Summary tab), set the time for the first delivery, enable the advanced scheduling options, and then select the Auto-repeat delivery of this job option. Define the interval between repeated deliveries as 1 hour, and repeat until the threshold time is approximately three and a half hours after the time of the first delivery. This provides some leeway for a busy server, but not so much leeway that you end up sending the job 5 times instead of 4 (see Figure 5).

Figure 5: Define the Auto-Repeat Job

The Results

Figure 6 shows the total tracking data for the 4 auto-repeat jobs as well as the tracking data for each individual job. Figure 7 shows the tracking data for the same job sent to the same recipients (using a similar response pattern) all at once. Figure 8 shows the totals for both jobs side by side, clearly showing that the peak load was considerably reduced in the job spread over 4 mailings without significantly adding to the time elapsed before the bulk of the tracking events were received. The load could be reduced even further by waiting longer between runs, though the trade-off is that you'll have to wait that much longer for final tracking results.

Figure 6: Total and Per Instance Results for Repeating Job

Figure 7: Results for Mailing Sent to All Recipients At Once

Figure 8: Comparison of Spread Mailing and Single Mailing

Tips

• The results above were derived from test data using an automated "clicking" tool that assumes a particular statistical pattern for recipient responses. Your own results may differ, and you should analyze results from past jobs to determine the clicking patterns from your own recipients. This can help you decide the best way to spread the load to achieve the results you need.
• The hidden TIMESLOT field creates extra work for the data administrators, but gives them more control and makes it easy to acquire a list of all the subscribers in a particular time slot. However, it is not necessary for the TIMESLOT field to be hidden nor is it even necessary to have a separate "time slot" field. The above example uses a separate field to make it easier to explain the concepts. The TIMESLOT reference in the target group could be replaced by some formula based on an existing field, such as:
• The subscription date in milliseconds modulo 4: SubscribeTimeMillis % 4
• The length of the email address modulo 4: Length(&EMAIL;) % 4
• The customer account number modulo 4 (in this example, the customer account ID follows the pattern of some number of letters followed by a dash, followed by a number, such as "ABCD-123"): ToNum(Substring(&ID;,IndexOf(&ID;,"-")+1)) % 4
• Any formula that returns a number from 0 to 3, with values evenly spread out among recipients. Do not use the Random() function as that could result in duplicate messages to some recipients and no messages to others. The formula must produce consistent results for individual recipients.

For more information on formulas, please see Appendix A of the Data Administrator's Manual.

• Be careful when coming up with the target group formula and the auto-repeat times. Choose a combination that sends to each time slot once and only once. In the above example, if we had chosen 5 timeslots instead of 4, we would need to be careful not to cross a day boundary because 24 (hours in a day) is not evenly divisible by 5, therefore the hour modulo 5 does not result in consecutive time slots when crossing midnight.
• You can use target group parameters to build a more flexible target group that will work with any pattern for spreading the load. Figures 9 and 10 show an example of such a target group (requires version 2.1-3 or later). By using hours elapsed since a base date instead of hour of the day, this example avoids problems with crossing day boundaries. It can work with any HRL, at any time of day, with any number of runs, and with any number of hours between runs.

Figure 9: Parameterized Target Group to Spread Load

Figure 10: Defining Recipients Using the Target Group from Figure 9