RE: Log monitoring

Hi!

Sounds interesting. How I imagine this: you have set of rules for
matching log file records (or have no rules at all). If log record
matches one of the rules, it is being processed as described in policy.
If record is not matched, alarm is generated, and record placed to some
"unmatched records" store. Then, administartor can pick unmatched
records and (with some tool/wizard?) create new matching rule, which
will instruct agent either to ignore such kind of records or generate
specific event. Do I understand your idea correctly?

Best regards,
Victor

> -----Original Message-----
> From: Bostjan Golob [mailto:bostjan.golob_at_gmail.com]
> Sent: Tuesday, September 13, 2005 9:05 PM
> To: NetXMS Developers talks
> Subject: Re: [netxms-dev] Log monitoring
>
>
>
> Hello,
>
> I have thought about log monitoring a while ago, while
> looking at some other solutions for such monitoring. However,
> I have been disappointed by all the monitoring packages that
> I have looked for the same reason I dislike most IDS and
> antivirus systems: you need to write rules what normal
> behavior _doesn't_ look like, instead of describing what
> normal behavior is and then forgetting about the system,
> knowing that it will alert you when something doesn't go as planned.
>
> I have been doing some thinking and I think a possible way of
> solving this would be something like this:
>
> * event occurs: "check pass; user unknown"
> * alert is fired
> * administrator decides that any event that matches a regexp
> created from this rule isn't worthy of an alert
> * the next time such an event occurs, it's ignored
>
> another example:
>
> * event occurs: "foo is 100"
> * alert is fired
> * administrator creates a rule that any event matching "foo
> is $1" for 0 < foo < 150 is normal, creates a rule for
> critical alert when foo > 300...
> * the next time foo is over 150, an event occurs, as well as
> when foo is -1 (due to a bug perhaps?) or over 300 (a
> critical condition)
>
> after a while, the system would "learn" what normal behavior
> is and alert only when the log deviates from this behavior.
>
> I can give more thought to how this could be implemented if
> this idea is found acceptable.
>
> Bostjan
>
> On 9/13/05, Victor Kirhenshtein <victor_at_opticom.lv> wrote:
> >
> > Hi all!
> >
> > Below is a proposed scheme for log monitoring.
> >
> > I. Key points
> >
> > 1. Logs should be processed by agents.
> > 2. Agent should check each log record against log
> processing policy,
> > and geberate appropriate event if match detected. 3. Agent can have
> > more than one log processing policy installed. 4. Log processing
> > policies should be created and distributed from administrator's
> > console, allowing centralized management of log processing.
> >
> >
> > II. Log processing policy
> >
> > Log processing policy is a set of matching rules with the following
> > attributes:
> >
> > Matching criterias:
> > * Event id range (integer range)
> > * Event source (regexp)
> > * Log message (regexp)
> >
> > Action data:
> > * NetXMS event id
> >
> > Event ID range and event source matching used only for
> Windows Event
> > Log. All text log files should be matched only by message
> text. Also,
> > log processing policy should have version attribute, so agent after
> > restart could determine if he has latest policy installed
> and request
> > latest one from server if needed.
> >
> >
> > III. Additional questions to discuss
> >
> > 1. Should we implement log processing as a separate subagent or as
> > part of core agent? 2. How we can guarantee event delivery
> from agent
> > to server? 3. What to do on agent startup - parse already
> existing log
> > records or just wait for new records?
> >
> > Any comments?
> >
> > Best regards,
> > Victor
> >
> >
>
>
Received on Tue Sep 13 2005 - 22:48:46 EEST