30 Dec 2011

PyLogagent


I wrote a small python logging agent to work around with application log files which cannot be logged to central log server (Syslog-Ng)

Source code is available on Github
----
Some log files generated by application cannot be logged to syslog/syslog-ng

pylogagent will watch list of log files in "configuration" file, then push new lines to /dev/log socket with prefix.

USAGE:
1. Set up syslog-ng server to listen for log
----snip-----
source net {
udp(ip(ip_of_log_server) port(514));
};
destination nginx_error {
file("/data/log/$HOST_FROM/$YEAR/$MONTH/$DAY/nginx_error");
};
filter nginx_error {
match("^nginx_error");
};
log {source(net);filter(nginx_error);destination(nginx_error;};
-----snip-----

These line above will make syslog-ng server listen for log on udp port 514, whenever a log stream deliver to its port, syslog-ng will parse, look for regex ^nginx_error, if it is found, log to file nginx_error.


1. Configure which log files pylogagent will watch

#configuration
/data/log/nginx_error.log

2. Start logagent
# python agent 

3. Configure syslog-ng client

------snip--------
source s_local {
internal();
unix-stream("/dev/log");
};
destination d_loghost {
udp("ip_of_log_server" port(514));
};
filter nginx_error            {
match("^nginx_error");
};
log {
source(s_local);
filter(nginx_error);
destination(d_loghost);
};

------snip--------

Restart syslog-ng and you are done.
Tail log file on central log to see how it works ;)



--
More will be added later

No comments:

Disqus