<< Back to Part 4 <<
5. Advanced configuration :
If you follow the previous part, you can now should have postfix log events displaying in Kibana, but there are some advanced trick that you may get interested in.
5.1 Include mail-from and subject :
To include mail-from and subject in event log, we will use action WARN in header_check part :
# vim /etc/postfix/header_checks
# vim /etc/postfix/main.cf
The log event then wil get like this :
# vim /etc/rsyslog.conf
Config td-agent to recieve this log stream :
# vim /etc/td-agent/td-agent.conf
### Listen on UDP port 5141
Set mail-from mapping to get term Top Sender :
5.1 Include mail-from and subject :
To include mail-from and subject in event log, we will use action WARN in header_check part :
# vim /etc/postfix/header_checks
/^(Subject|subject):/ WARN
# postmap /etc/postfix/header_checks
# vim /etc/postfix/main.cf
header_checks = regexp:/etc/postfix/header_checks
# /etc/init.d/postfix reload
The log event then wil get like this :Feb 27 15:24:23 postfixhost postfixOUTGOING/cleanup[7784]: 10EC911B8003: warning: header Subject: [subject here]; from=<someone@mydomain.com> to=<somebody@yahoo.com> proto=ESMTP helo=<Inside-BusL>
Config rsyslog to catch the string "header" and transmit to td-agent :
# vim /etc/rsyslog.conf
:msg, contains, "header" @10.90.7.195:5141
# /etc/init.d/rsyslog restart
Config td-agent to recieve this log stream :# vim /etc/td-agent/td-agent.conf
### Listen on UDP port 5141
<source>
type syslog
port 5141
bind 0.0.0.0
tag header
</source>
### Parser
<match header.mail.info>
type parser
remove_prefix header
format /^(?<queueid>[^ ]*): [^*]* from=<(?<mail-from>[^ ]*)> to=<(?<rcpt-to>[^ ]*)>/
key_name message
reserve_data yes
</match>
Set mail-from mapping to get term Top Sender :mail-from:{"type":"string","index":"not_analyzed"},
5.2 Split postfix in-comming and out-going in one dashboard
As usual we use one postfix instance to send and recive emal like this :
- To send : smtp.yourdomain.com -> [postfix IP server : 1.2.3.4] : DNS type A
- For receiving : mx.yourdomain.com -> [postfix IP server : 1.2.3.4] DNS type MX.
To separate incomming and outgoing postfix log stream in one dashboard as the image below, we need one more postfix instance (or server - is up to you) for examples with IP address 1.2.3.5.
To create another instance postfix on the same server we can use :
- postmulti : http://www.postfix.org/postmulti.1.html
- clone all the postfix files to a new dir ( /etc/init.d/postfix, /var/spool/postfix ...) For example :
# ls -al /var/spool/postfix*
drwxr-xr-x 16 root root 4096 Mar 13 2012 postfix1.2.3.4
drwxr-xr-x 16 root root 4096 Apr 21 2011 postfix1.2.3.5
# ls -al /etc/init.d/postfix*
-rwxr-xr-x 1 root root 2328 Jul 20 2011 /etc/init.d/postfix1.2.3.4
-rwxr-xr-x 1 root root 2328 Jul 20 2011 /etc/init.d/postfix1.2.3.5
Change the postfix syslog name :
# vim /etc/postfix1.2.3.4/main.cf
syslog_name = postfixOUTGOING
queue_directory = /var/spool/postfix1.2.3.4
command_directory = /usr/sbindaemon_directory = /usr/libexec/postfix
# /etc/init.d/postfix1.2.3.4 restart
# vim /etc/postfix1.2.3.5/main.cf
syslog_name = postfixINCOMMING
queue_directory = /var/spool/postfix1.2.3.5
command_directory = /usr/sbindaemon_directory = /usr/libexec/postfix
# /etc/init.d/postfix1.2.3.5 restart
The ident field now should include the direct :
{"host":"postfix","ident":"postfixOUTGOING/smtp","pid":"19190","message":"5136311B8003: to=<somebody@yahoo.com>, relay=mta6.am0.yahoodns.net[98.138.112.33]:25, delay=6, delays=0.1/0.03/1.8/4.1, dsn=5.0.0, status=bounced (host mta6.am0.yahoodns.net[98.138.112.33] said: 554 delivery error: dd This user doesn't have a yahoo.com account (somebody@yahoo.com) [0] - mta1170.mail.ne1.yahoo.com (in reply to end of DATA command))","queueid":"5136311B8003","rcpt-to":"somebody@yahoo.com","relay":"mta6.am0.yahoodns.net[98.138.112.33]:25","status":"bounced","@timestamp":"2014-01-23T00:44:41+07:00"}
Now config Kibana to catch the ident field, then we should have the dashboard like upper.
No comments:
Post a comment