After setting up your AudioCodes OVOC server, you want to have it send alarms via emails to a designated mailbox.
The OVOC Installation Manual has a good guide for configuring this. You can find this on page 41 (56/306). This article would not be worth reading if all I am doing is redirecting you to the manual, yes, you guessed right, things did not go according to plan.
An overview of the steps as detailed in the document are as follows:
- Configure the Exim service on the OVOC server:
- SSH into OVOC as acems, then sudo root, and back up the exim configuration file.
- cp /etc/exim/exim.conf /etc/exim/exim.conf.bak
- SSH into OVOC as acems, then sudo root, and back up the exim configuration file.
Edit the exim.conf file using either vim or nano. The sections to edit are Routers, Transports, and Authentication.
Restart the exim service with:
systemctl restart exim
- If following the restart, alarm forwarding is still not working, edit /root/.muttrc and replace the default email address set from = OVOC@audiocodes.com with a proper email address of the owner of the OFFICE365_USERNAME account.
Finally log into the OVOC web portal and navigate to Alarm > Forwarding > New
No emails came through. Restarted the exim service a couple of time and still nothing.
Troubleshooting time.
The first thing I checked was to determine if the mailbox and the sending email address were valid. They were, I could send and receive emails from these accounts.
Afterwards, I needed to simulate the OVOC sending emails and to do this I did the following:
- Generate a tcpdump.
- Simulate sending emails.
- Have a look at the panic.log and the main.log files.
Generate tcpdump:
SSH into OVOC, sudo root, and type EmsServerManager. Select Diagnostics > Network Traffic Capture > Start tcpdump > Type “y”
ii. IPs > any; Port(s) > any; Capture time (in minutes) 10
iii. Proceed with the capture. Tcpdump is now running.
b. To simulate sending emails type: tail -f /var/log/maillog then duplicate the session and type the following: “echo “Report : Test ” | mutt -s “Scheduler: my date” -F /root/.muttrc your.email@domain.com” (without the first and last (“”) quote marks.
c. Once you have completed the tests, type the following: cd /var/log/ems/capture and locate the file generated by the tcpdump.
d. Change mode using chmod 777 <filename>, then use WinSCP to copy the file over to your local machine for analysis.
Check the content of the following files: “panic.log” and “main.log” both of which can be found at /var/log/exim/
What do we see in the logs?
User 0 set for local_delivery transport is on the never_users list
2021-08-07 18:37:21 1m179T-0005KQ-RS User 0 set for local_delivery transport is on the never_users list
2021-08-07 18:37:21 1m179T-0005KQ-RS == root@bshukipovoc01.bishopal.com R=localuser T=local_delivery defer (-29): User 0 set for local_delivery transport is on the never_users list
2021-08-07 18:37:21 1m179T-0005KQ-RS ** root@bshukipovoc01.bishopal.com: retry timeout exceeded
2021-08-07 18:37:21 1m179T-0005KQ-RS root@bshukipovoc01.bishopal.com: error ignored
2021-08-07 18:37:21 1m179T-0005KQ-RS Completed
Exim is a message transfer agent, so we need to check the exim.conf log and the maillog files.
User 0 is the root account and being on the never_users list means it has been blacklisted or prevented from sending email.
I replaced the exim.conf file with the exim.conf.bak file – restarted the exim service (after renaming back to exim.conf) and repeated the tcpdump and mail simulation process as detailed above.
Exim deamon started.
2021-07-08 08:43:11 exim 4.94 daemon started: pid=23238, -q1h, listening for SMTP on [127.0.0.1]:{25,587} and for SMTPS on [127.0.0.1]:465
Email started coming through 😊
2021-07-08 08:44:05 1m1OhY-0004wf-7X => ovoc-alert@bishopal.com R=dnslookup T=remote_smtp H=bishopal-com.mail.protection.outlook.com [109.56.2.16] X=TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256 CV=yes K C=”250 2.6.0 <20210708074404.GA19011@bshukipovoc01.bishopal.com> [InternalId=70261370001916, Hostname=DB7PR07MB5308.eurprd07.prod.outlook.com] 15074 bytes in 0.071, 204.944 KB/sec Queued mail for delivery”
2021-07-08 08:44:05 1m1OhY-0004wf-7X Completed
2021-07-08 08:44:05 1m1OhY-0004ws-Ds => ovoc-alert@bishopal.com R=dnslookup T=remote_smtp H= bishopal-com.mail.protection.outlook.com [109.56.2.16] X=TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256 CV=yes K C=”250 2.6.0 <20210708074404.GA19024@bshukipovoc01.bishopal.com > [InternalId=14839112017519, Hostname=AM9PR07MB7827.eurprd07.prod.outlook.com] 14610 bytes in 0.059, 240.188 KB/sec Queued mail for delivery”
2021-07-08 08:44:05 1m1OhY-0004ws-Ds Completed
2021-07-08 08:44:05 1m1OhY-0004wm-8z => ovoc-alert@bishopal.com R=dnslookup T=remote_smtp H= bishopal-com.mail.protection.outlook.com [109.56.2.16] X=TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256 CV=yes K C=”250 2.6.0 <20210708074404.GA19017@bshukipovoc01.bishopal.com > [InternalId=32968168972837, Hostname=DB6PR0701MB2455.eurprd07.prod.outlook.com] 14437 bytes in 0.149, 94.442 KB/sec Queued mail for delivery”
2021-07-08 08:44:05 1m1OhY-0004wm-8z Completed
2021-07-08 08:44:14 1m1Ohi-00050C-86 <= root@bshukipovoc01.bishopal.com U=root P=local S=6954 id=20210708074414.GA19230@bshukipovoc01.bishopal.com
2021-07-08 08:44:15 1m1Ohi-00050C-86 => ovoc-alert@bishopal.com R=dnslookup T=remote_smtp H= bishopal-com.mail.protection.outlook.com [109.56.2.16] X=TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256 CV=yes K C=”250 2.6.0 <20210708074414.GA19230@bshukipovoc01.bishopal.com > [InternalId=16578573779692, Hostname=PA4PR07MB7407.eurprd07.prod.outlook.com] 14993 bytes in 0.084, 174.141 KB/sec Queued mail for delivery”
2021-07-08 08:44:15 1m1Ohi-00050C-86 Completed
————–
At 08.56 the OVOC server started getting SMTP errors from the remote server,
2021-07-08 08:56:26 1m1OtV-0007bn-7q H= bishopal-com.mail.protection.outlook.com [109.56.2.16]: SMTP error from remote mail server after pipelined end of data: 451 4.7.500 Server busy. Please try again later from [20.90.96.59]. (S77714) [DB5EUR01FT062.eop-EUR01.prod.protection.outlook.com]
and these continued till the end of the log
Whitelisting (protection.office.com) the IP of the OVOC server in O365 took care of this.
