syslog-ng-users August 2011 archive
Main Archive Page > Month Archives  > syslog-ng-users archives
syslog-ng-users: Re: [syslog-ng] Parsing Question

Re: [syslog-ng] Parsing Question

From: Balazs Scheidler <bazsi_at_nospam>
Date: Mon Aug 01 2011 - 18:23:28 GMT
To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu>

On Fri, 2011-07-29 at 19:22 +0200, Jakub Jankowski wrote:
> On 2011-07-29, Brandon Phelps wrote:
>
> > Could anyone explain how I would parse a message that looks like this:
> > Jul 29 08:58:38 192.168.1.1 id=firewall sn=0017C5158708 time="2011-07-29
> > 08:58:38" fw=100.1.1.1 pri=6 c=262144 m=98 msg="Connection Opened" n=0
> > src=192.168.2.100:123:X0 dst=74.1.2.3:X1 proto=udp/ntp
> >
> > I am logging to mysql and would like to extract the 'src' and 'dst'
> > fields from the above message so that I can insert them into indexed
> > fields in my database.
> [...]
> > Is my only option in this case to write a perl script or something that
> > watches a named pipe and have syslog-ng log to the named pipe instead,
> > while my perl script does the actual parsing? Or can I do what I want
> > with syslog-ng alone?
>
> You seriously need to look at patterndb functionality.
> http://bazsi.blogs.balabit.com/2009/03/an-introduction-to-db-parser/
> http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-v3.2-guide-admin-en.html/index.html-single.html#chapter-patterndb

patterndb() would work if the order of the fields is definite. if they
are not, it's going to be ugly. I was pondering to write a welf parser
(which the above format is), that could be used to preprocess logs
prior to going to db-parser(), but that's something that you either have
to wait for, implement yourself or wait someone who has the same itch
and does it for you. :)

-- Bazsi ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq