Sending LOTS of emails (Hint: with Drush)
On my last blog post I talked about Message notify module, and mentioned we were using it to send digest emails. While Message notify really eases the pain of creating personalized emails, and sending them, we still needed to find a scalable solutions for sending lots of emails. LOTS of emails!
Doing it on hook_cron() is nice if you need to send few emails, but when you need to send thousands of emails a day, it won’t do. Cron will take too long, or might hang up if we try to process too many digest emails at once.
A nice and simple solution we came up with, was to write a Drush command that will process a small batch of users, and use Jenkins to fire up this command every two minutes.
This post will demonstrate the code, that can be used by others - however you will need to copy and change the command to fit your business logic. Assuming our module’s name is foo, you will need to place your code in foo.drush.inc:
<?php
/**
* @file
* Drush integration of Foo module.
*
*/
/**
* Implements hook_drush_help().
*/
function foo_drush_help($section) {
switch ($section) {
case 'drush:foo-digest':
return dt('Send digest emails.');
}
}
/**
* Implements hook_drush_command().
*/
function foo_drush_command() {
$items = array();
$items['foo-digest'] = array(
'callback' => 'drush_foo_digest',
'drupal dependencies' => array('foo'),
'description' => 'Send digest emails.',
'arguments' => array(
'range' => 'The number of users to be processed.',
),
'bootstrap' => DRUSH_BOOTSTRAP_DRUPAL_FULL,
'aliases' => array('fdg'),
'examples' => array(
'drush foo-digest' => 'Process 10 users.',
'drush foo-digest --range=50' => 'Process 50 users.',
),
);
return $items;
}
/**
* Callback function for sending digest email command.
*/
function drush_foo_digest() {
if (!lock_acquire('foo', 240.0)) {
// Digest is already processed.
drush_log(dt('Attempting to re-run foo digest command while it is already running.'), 'notice');
return;
}
$range = drush_get_option('range', 10);
// This function should be replaced by your function that gets all the
// users that need to get the digest email.
$uids = foo_get_digest_users($range);
if (!$uids) {
drush_log(dt('No users found for digest.'), 'ok');
}
// This function should be replaced by your function that will send the
// emails (maybe by using Message notify), and is expected to return
// the user IDs of the successful emails.
$return = foo_process_digest($uids);
$params = array(
'@uids' => !empty($return) ? implode(', ', $return) : 'none',
);
drush_log(dt('Users IDs that got email: @uids.', $params), 'ok');
}
?>Note the code comments - you are expected to replace foo_get_digest_users() and foo_process_digest(). Btw, in those functions, I’ve added debug messages like this.
<?php
if (function_exists('drush_log') && drush_get_option('verbose')) {
drush_log(dt('Digest email to user @uid could not be sent', array('@uid' => $account->uid)), 'ok');
}
?>After that, all you need to do is create a new Jenkins job.
Pantheon gives us this flexibility, here's the console log:
Don't forget to use Reroute email so your users don’t get dummy emails form your Dev and Test environments.
Comments
I don't see how this is any
I don't see how this is any different than using cron. Why not just fire your drush command with your crond instead of Jenkins? A lot less resource usage that way...
> I don't see how this is any
> I don't see how this is any different than using cron.
Cron is executed once an hour, and it has many tasks to deal with, not only with the digest task. By creating this Drush command, and invoking it more frequently (e.g. every two minutes), I'm able to process more digest emails than I was able to do with cron -- and prevent cron hanging and timeouts.
I mean, you could just add a
I mean, you could just add a line to your crontab that invokes your drush command every two minutes.
Oh, I see what you mean. What
Oh, I see what you mean. What I like about working with Jenkins is that it keeps a console log of the "builds", but I agree crontab is a good solution as-well -- thanks for pointing this out. As long as "something" is invoking my Drush command and making sure my emails are sent properly I'm happy :)
Link to "reroute email" is broken
Otherwise great post :)
10x. Fixed the link.
10x. Fixed the link.
HTTP Parallel Request Library
If your doing a lot of things over the network, a good option is to do it in parallel. That's where http://drupal.org/project/httprl comes into play. http://drupal.org/project/smtpua uses it to send out emails "instantly" instead of on cron. Just another way to do it.
@mikeytown2 , thanks for
@mikeytown2 ,
thanks for those links, I can think of other use cases for those (e.g. sending email notifications, upon comment submission).
Post new comment