Find text string within log file

Started by [email protected], November 08, 2016, 12:16:57 PM

Previous topic - Next topic

[email protected]

Hi there,

Can you please advise how I can configure netxms to monitor a folder on a node, this folder contains log files and if the log files contains a specific text string "Failed to connect to FX Server" an alert is generated?

I have looked at Logwatch but I have no idea how to make this work, or if its even the correct parser to use.

Can someone help me?

Many thanks, Damien

Victor Kirhenshtein

Hi,

yes, logwatch is a correct choice. Parser could be following:

<parser>
   <file>/path/to/log/file</file>
   <rules>
      <rule>
         <match>Failed to connect to FX Server</match>
         <event>FX_SERVER_CONNECT_FAILED</event>
      </rule>
   </rules>
</parser>


and add the following to nxagentd.conf:

# this is main section
SubAgent = logwatch.nsm

[LOGWATCH]
Parser = /path/to/parser.xml


you have to create event FX_SERVER_CONNECT_FAILED on server (of course you can name it differently) - it will be generated each time this string will be found in log file.

Best regards,
Victor

[email protected]

Hi there,

On the remote agent I have adjusted the conf file as described and in C:\NetXMS I have created a parser1.xml file

remote agent conf
<parser>
<file>C:\test\logs</file>
<rules>
<rule>
<match>TEST ERROR LOG</match>
<event>100000</event>
</rule>
</rules>
</parser>

parser1.xml
<parser>
<file>C:\test\logs</file>
<rules>
<rule>
<match>TEST ERROR LOG</match>
<event>100000</event>
</rule>
</rules>
</parser>

When you say I need to create an event , do you mean a DCI for the remote node?  Thanks

[email protected]

sorry I made an error in my copy and paste, the remote agent config is defined as:

#
# NetXMS agent configuration file
# Created by server installer at Tue Oct 11 15:24:54 2016
#

LogFile = {syslog}
MasterServers = 127.0.0.1, 192.168.11.207
FileStore = C:\NetXMS\var
RequireAuthentication = yes
SharedSecret = HIDDEN
SubAgent = winperf.nsm
SubAgent = portcheck.nsm
SubAgent = logwatch.nsm
# Below is log parsers definitions
*LOGWATCH
Parser = C:\NetXMS\parser1.xml

Dani@M3T

Hi Damien

Change in your nxagentd.conf '*LOGWATCH' to '[LOGWATCH]'.
With your parser xml the string 'TEXT ERROR LOG' in your Logfile 'C:\test\logs' would generate an NetXMS event 100000. So this event 100000 must exist in your event configuration in NetXMS. You don't need a DCI for that.

best regards
Dani

dmna007

Hi, just to let you know, I have this working if I point the parser directly at a file:

<parser>
   <file>C:\FX_CHUB\LOGS\test.txt</file>
   <rules>
      <rule>
         <match>TCP Command Socket Failed Error</match>
         <event>LOGWATCH_PLXCENTRALHUB_176.74.191.177</event>
      </rule>
   </rules>
</parser>

I have created the event in the main Netxms console and an email is sent when the match string is inserted into the txt file, so that's great. 

The problem I now have is the logs within this folder are all variable names, is it possible to point the parser at a folder opposed to a direct file?  This way any log file in this file will be parsed looking for the string to match.

I have tried <file>C:\FX_CHUB\LOGS\*</file>  and <file>C:\FX_CHUB\LOGS\</file>

But this does not seem to work.  Many thanks for your help

Victor Kirhenshtein

Hi!

You can use macros for forming date/time based file name (possible macros can be found here: http://www.cplusplus.com/reference/ctime/strftime/) or you can use output of shell script as file name using backticks, like this:


<file>C:\FX_CHUB\LOGS\`C:\gen_file_name.cmd`</file>


In this example file name will be taken from output of script C:\gen_file_name.cmd

Best regards,
Victor

tickett

#7
    I am trying this (dynamic filename) and struggling- I have tried;

    • using a wildcard * (didn't expect it to work, but worth a longshot)
  • adding a macro (see below)
  • using backticks with an echo command directly
  • using backticks with a batch file which executes an echo command
<macro name="timestamp">%Y_%m_%d</macro>

The log file seems to show whatever I enter and doesn't appear to show a "resolved" value. Is this normal? Regardless new log entries don't appear to be picked up.

2019.04.11 09:31:02.328 *D* [logwatch           ] Parser thread for file "C:\inetpub\wwwroot\website\logs\Log_@{timestamp}.txt" stopped
2019.04.11 16:33:00.121 *D* [logwatch           ] Parser thread for file "C:\inetpub\wwwroot\website\logs\Log_`echo %date:~6,4%_%date:~3,2%_%date:~0,2%`.txt" started
2019.04.11 16:38:34.340 *D* [logwatch           ] Parser thread for file "C:\inetpub\wwwroot\website\logs\Log_*.txt" stopped
2019.04.11 16:38:34.906 *D* [logwatch           ] Parser thread for file "C:\inetpub\wwwroot\website\logs\Log_`c:\netxms\etc\date.bat`.txt" started


I have the parser working fine for the Windows Event Log.

Any ideas? TIA

Victor Kirhenshtein

Yes, this is normal. On startup agent logs unexpanded file name. You should also see something like this when file is found:

2019.04.12 12:03:13.608 *D* [logwatch           ] File "/tmp/test-20190412.log" (pattern "/tmp/test-`date +%Y%m%d`.log") successfully opened

Are you sure your commands return correct values? You may also try to use cmd /c "command" within backticks (because echo for example is not an external executable but cmd's internal command).

Also, if you only need current date/time you may use date/time macros without external commands, like this:

<file>C:\inetpub\wwwroot\website\logs\Log_%Y_%m_%d.txt</file>

Best regards,
Victor

tickett

Thanks Victor, putting the wildcards straight in the filename as you suggested worked perfectly;

<file>C:\inetpub\wwwroot\website\logs\Log_%Y_%m_%d.txt</file>