Optimize exim performance

I experienced that exim was slow in sending outgoing mail, it was spending a lot of time in the queue, resulting in customer complains. So I started to google what could be done to increase performance in exim. I didn’t have much knowledge about exim, and I needed a quick fix to get the mail queue going. I googled, and even though there was a myriad of information about exim configuration options which were poorly explained, I didn’t find the golden answer I needed.

So I spent a week researching, reading and at some point I got it.

There are many ways to increase performance depending upon the servers need. I had a server and its sole purpose was to receive and send thousands of mails (news letters) during a short time period. I wanted as little mail to be stuck in the mail queue as possible.

Whenever I mention values on options, you have to choose the one that best suits your server. To figure this out, the best option is to make configuration changes, and monitor the performance impact it has on your server. How you can do that is written in the bottom of this article.

One of the performance options is related to how the mail is built up. Does the mail only have one recipient or multiple recipients?

Setting number of mail queue processes

By default exim spawned off one mail queue process (runner) every hour which was way to little. I needed multiple runners to receive and send the mail immediately without delay. This option is particularly useful when mails only have one recipient, but every exim server should configure this option anyway.

  1. Change how many queue runners that are spawned off. In /etc/init.d/exim I changed this option: QUEUE=5m. This ensured that a new queue runner was created every 5 minutes. How often a new queue runner needs to be configured on a server depends on its load.
  2. Change max number of queue runners that can exist simultaneously in /etc/exim.conf: queue_run_max = 50
  3. Since we made changes in /etc/init.d/exim, we need to restart the server to make the effects off number of queue runners: /etc/init.d/exim restart

Multiple recipients
If a message shall be delivered to multiple remote hosts (multiple recipients), exim can be configured to run several smtp deliveries simultaneously by setting the configuration option “remote_max_parallel” to a value greater than one. Otherwise a delivery is sent serially. Max number of simultaneous delivery processes is then “queue_run_max  * remote_max_parallel”.

In /etc/exim.conf change/add following: remote_max_parallel = 50

Whenever making a change to exim.conf, run the following command to ensure that exim reads the configuration changes and puts them in effect:
kill -HUP `cat /var/spool/exim/exim-daemon.pid`

Message management

There are multiple configuration options in how exim shall deal with the connections/mails itself, and these are the options I have played with.

#Simultaneous number of incoming connections to accept.
smtp_accept_max = 500

The server I was configuring had trusted servers that connected to exim and pushed thousands of mails per connection. So this option is a nice one:
#How many messages to accept per connection
smtp_accept_queue_per_connection = 2000

There is a point where the load gets so high that the server is unable to deal with what is coming. What that limit is depends on the hardware, software running on it, you name it. So the limit is different for every server. When the limit is reached, that’s when the server stops responding. You can set exim to have no queue deliveries if load is high in /etc/exim.conf: deliver_queue_load_max = 10

How to monitor the server during high load

  1. The top command can be used to view the current load on the server. In the picture you can see the load average in the top right corner (1, 5 and 15 minute average). I experienced that during high load, the server had an load average up to 30, but it barely registered in the cpu. That told me that I could safely configure the server to many more queue runners etc. because it was able to handle the load. I will not even try to enter the science of load average which I have yet to understand. That’s candy for another time.
  2. The sar command can be used to view historical load on the server by day, hour, minutes: /var/log/sa. Check the date on the files. sar04 is the 4th of the month in question. Command:
    sar -f sa09 | less

 

 

Haridasi

About Haridasi

integrity – the state of being whole, entire, or undiminished.

This entry was posted in English, Linux and tagged , . Bookmark the permalink.

19 Responses to Optimize exim performance

  1. Richard Wing says:

    Thank you for taking the time to write about this. Can you tell me if there is specific settings within WHM web interface to configure the options you mentioned?

    Thanks
    Richard Wing

  2. I didn’t do it through WHM, but logged into the server and changed the settings there. I’m not sure if WHM have an interface for it, but I doubt it.

  3. lester says:

    A little confused about the difference between queue runners and SMTP deliveries — are they the same thing? For example, you say: “Change max number of queue runners that can exist simultaneously in /etc/exim.conf: queue_run_max = 50” and above that, “In /etc/init.d/exim I changed this option: QUEUE=5m. This ensured that a new queue runner was created every 5 minutes. ”

    So one queue runner every 5 mintus equals 12 in an hour, yet you also say queue_run_max = 50 so it seems you would never hit that limit.

  4. A queue runner scans the mail pool and starts a delivery process on a single message. So a queue runner doesn’t do mail receivement/deliveries, but start a delivery process that deliver mails.

    A better explanation can be found here: http://www.datadisk.co.uk/html_docs/exim/processes.htm

    I interpret the queue_run_max as a failsafe or a way to ensure that your system isn’t maxed out. I have yet to find info on when queue runners die. I have seen many defunct queue runners that quickly dies, but I don’t know what makes the queue runner die in the first place. So what your limit should be on queue runner depends on your system and what it can handle.

  5. damian says:

    Helene how can I run more deliveries:

    13334 daemon: -q1h, listening for SMTP on port 25 (IPv6 and IPv4) port 26 (IPv6 and IPv4) port 587 (IPv6 and IPv4) and for SMTPS on port 465 (IPv6 and IPv4)
    13335 running queue: waiting for 1SbuG5-0002s7-9B (17661)
    17661 delivering 1SbuG5-0002s7-9B (queue run pid 13335)

    like 10 or more

    17661 delivering 1SbuG5-0002s7-9B (queue run pid 13335)

  6. Create more queue runners as I have mentioned in “Setting number of mail queue processes”

  7. JC says:

    Thank you for writing this. Most useful exim tweak of the month.

  8. Darby says:

    Wow that was strange. I just wrote an extremely long comment but after I clicked submit my comment didn’t show up. Grrrr… well I’m not writing all that over
    again. Anyway, just wanted to say fantastic blog!

  9. Haridasi haridasi says:

    haha, now I got curious about what you wrote of course 😀

    But thank you!

  10. imran says:

    HI
    i have a problem to sending mail we have dedicated server with below configuration

    processor:

    Intel Quad Xeon
    E3-1240
    4 Cores x 3.3 Ghz

    RAM

    16GB DDR3

    2 x 2TB SATA
    in RAID 1

    bandwidth
    100TB

    we only use this server to send mass mails,the server is not sending even 25000 mails per day what are the settings i have to change in WHM thanks in advance

  11. Haridasi haridasi says:

    Your server should be able to manage a lot more mail. Have you followed the tip about creating more queue runners?

    Unfortunately, you need command access to do it. You can’t do it through WHM.

  12. Bruno Wienandts says:

    Hello Helene,

    First of all, thank you for these tips, helped me a lot about exim!

    One question: you send all these emails by php script, or any PC software?

    Do you know any script that can be used for sending these emails? I am looking for someone who has smtp authentication and that can open more than just ONE simultaneous connections, like Interspire Email Marketer, for example. By the way, the IEM is a good script to send e-mails, but the problem is that only open 1 smtp connection, and it is very slow!

    Do you know any other good script?

    Thanks in advance!

  13. Haridasi haridasi says:

    Hi Bruno,

    Unfortunatly I can be of no help here. We have programmers at work who created scripts for sending out thousands of newsletters. I just made sure to make exim able to handle the load.

    My challenge is more in regards to keeping my mailservers away from spam lists.

  14. Paul says:

    Tusen takk ! Jeg er takknemlig.

    Sorry that is all I can remember from a Norwegian course I did 32 years ago.

    I spent hours trying to find the correct thing to change and then I found this web page

    > In /etc/init.d/exim I changed this option: QUEUE=5m.

    Bingo. Your web page solved my problem. I wanted to run queues every 15 minutes and not hourly but could not locate the parameter’s location.

    Best regards,

    Paul
    England
    EU.
    (Jeg liker Norge)

    Centos 5 and 6, Exim, Apache but no Micro$oft anywhere except in my recollection of distant nightmares 🙂

  15. Haridasi haridasi says:

    Your norwegian is still great 🙂

  16. I tried to add queue_run_max and when I restarted exim4 I got this error:
    root@ds6419:/etc/exim4# /etc/init.d/exim4 restart
    [....] Stopping MTA for restart:/usr/sbin/update-exim4.conf: 36: /etc/exim4/update-exim4.conf.conf: queue_run_max: not found

    Why would ‘queue_run_max’ not work?
    I run a split configuration file, so I add it to update-exim4.conf.conf with my other exim4 configuration settings.

    Cheers
    Murray

  17. Haridasi haridasi says:

    Unfortunately, I’m not able to test it longer or I would have helped you out (no access to a linux machine longer).

  18. @Murray

    I also have a split configuration. The file you need to edit in your case is most likely in /etc/exim4/exim.conf.template as well as /etc/exim4/exim.conf (if it exists). In my case lines such as queue_max_run and remote_max_parallel did not exist anywhere in the file so just add them and restart exim and you should be good to go. A tip to help you check if your changes have been made is run this command after restarting: exim -bP

    @haridasi

    Fantastic article. It really helped me. My config files didn’t have the lines you mention in your article so it took me ages to figure out I had to add them myself 😀

  19. Wallace Mwabini says:

    This is a very useful article. Ive done the tweaking myself. Let me see if I’ll get what I want. Also, for centos 7 I see that variable QUEUE is in /etc/sysconfig/exim as opposed to /etc/init.d/exim.

Leave a Reply

Your email address will not be published. Required fields are marked *