Tag Archive: logging


No doubt that syslog-ng is cool when used to concentrate all logs from your IT infrastructure in the same place. But having just a bunch of ASCII files is not the most practical system.

I know what you are thinking right now: “It would be cool to log to a database and have some kind of front end to visualize it”. Yes, it would be… actually it IS.

Just a quick description of how to do it (the details are boring :) )

The trick is to modify the later configuration (check syslog-ng is my hero) and add a new destination clause:

# Destination MySQL

destination d_mysql {
pipe(“/tmp/mysql.syslog-ng.pipe”
template(“INSERT INTO logs
(host, facility, priority, level, tag, datetime, program, msg)
VALUES ( ‘$HOST’, ‘$FACILITY’, ‘$PRIORITY’, ‘$LEVEL’, ‘$TAG’, ‘$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC’,
‘$PROGRAM’, ‘$MSG’ );\n”) template-escape(yes));
};

This associates a named pipe, in our example located in /tmp/mysql.syslog-ng.pipe but the name and location are not important, to a MySQL statement. This SQL sentence has been crafted in this particular way for reasons that will be apparent later. This new destination must be used for every logging source we want to include in our database, modifying the configuration like this:

log{
source(src);
filter(f_MUNICH);
destination(d_mysql);
};

That says to syslog-ng to redirect the log stream to this pipe which, in turn, will insert the data into this MySQL table using the forementioned INSERT statement. Just a hint about this, we need to create the pipe under Linux with this command:

mkfifo /tmp/mysql.syslog-ng.pipe

and then connect it to the MySQL database as follows:

mysql –user=myuser –password=mypasswordhere logs < /tmp/mysql.syslog-ng.pipe

that is, just a regular flow redirection.
Even better, and recommended is to setup a script to do this everytime you boot your system, something like:

#!/bin/sh
#
# File: syslogng-mysql-pipe.sh
#
# Take input from a FIFO and run execute it as a query for
# a mysql database.
#
# IMPORTANT NOTE: This could potentially be a huge security hole.
# You should change permissions on the FIFO accordingly.
#

if [ -e /tmp/mysql.syslog-ng.pipe ]; then
while [ -e /tmp/mysql.syslog-ng.pipe ]
do
mysql -usyslog –password=mypasswordhere logs < /tmp/mysql.syslog-ng.pipe
done
else
mkfifo /tmp/mysql.syslog-ng.pipe
fi

After having this running we can begin to install the PHP frontend that will read and interpret the records in this database. After all, if plain text files weren’t very practical, a SQL database is even worse!

I won’t talk here about it because there’s an online guide much better than any explanation I could write here.
In case you want to check it out before going into all this process, there’s a live demo here. The user/pass are “demo/demo”.

Happy logging! ;)

Syslog-ng is my hero

When you have to administer (read painfully fix problems) in remote machines (mine are in Tokyo, for example) logfiles are a must-have feature. But what happens when you can’t access the remote host?

The problem is even worse in Cisco routers when you use logging buffered and this buffer is reset after a reboot. The usual question is “What made that machine reboot?“. Check the logs if you can.

One solution is, of course, to have a central log server to where they will be sent and stored. Following is a simple configuration file to use with syslog-ng.

At the Cisco Router (or switch)

service timestamps log datetime localtime

logging <ip syslog server>

logging trap notifications

This configuration will instruct the Cisco device to send the log files with a severity higher or equal to notifications (5) to the syslog servers. The less important notifications will be shown only through the console.

Now, at the syslog server side:

/etc/syslog-ng/syslog-ng.conf

options {

long_hostnames(off);

sync(0);

perm(0640);

stats(3600);

};

source src{

internal();

udp(ip(0.0.0.0) port(514));

};

# Filter for IP address (more options)

filter f_ROUTER01{ host(“10.0.0.1″); };

filter f_ROUTER02{ host(“10.0.0.2″); };

[...]

# Destination files

destination ROUTER01{ file(“/var/log/ROUTER01.log”); };

destination ROUTER02{ file(“/var/log/ROUTER02.log”); };

# The actual logging command(s)
# Note how they filter the input and connect
# the result with the destination (predefined)

log{
source(src);
filter(f_ROUTER01);
destination(ROUTER01);
};

[...]

After saving the configuration, the only tasks left are to create the logfiles (with touch) and restart the service.

Happy logging!

Powered by WordPress | Theme: Motion by 85ideas.