Author Topic: Find text string within log file  (Read 2547 times)

dmna007@hotmail.com

  • Newbie
  • *
  • Posts: 6
    • View Profile
Find text string within log file
« on: November 08, 2016, 12:16:57 pm »
Hi there,

Can you please advise how I can configure netxms to monitor a folder on a node, this folder contains log files and if the log files contains a specific text string "Failed to connect to FX Server" an alert is generated?

I have looked at Logwatch but I have no idea how to make this work, or if its even the correct parser to use.

Can someone help me?

Many thanks, Damien

Victor Kirhenshtein

  • Lead Developer
  • Administrator
  • Hero Member
  • *****
  • Posts: 7036
    • View Profile
Re: Find text string within log file
« Reply #1 on: November 08, 2016, 12:47:15 pm »
Hi,

yes, logwatch is a correct choice. Parser could be following:
Code: [Select]
<parser>
   <file>/path/to/log/file</file>
   <rules>
      <rule>
         <match>Failed to connect to FX Server</match>
         <event>FX_SERVER_CONNECT_FAILED</event>
      </rule>
   </rules>
</parser>

and add the following to nxagentd.conf:
Code: [Select]
# this is main section
SubAgent = logwatch.nsm

[LOGWATCH]
Parser = /path/to/parser.xml

you have to create event FX_SERVER_CONNECT_FAILED on server (of course you can name it differently) - it will be generated each time this string will be found in log file.

Best regards,
Victor

dmna007@hotmail.com

  • Newbie
  • *
  • Posts: 6
    • View Profile
Re: Find text string within log file
« Reply #2 on: November 08, 2016, 01:10:27 pm »
Hi there,

On the remote agent I have adjusted the conf file as described and in C:\NetXMS I have created a parser1.xml file

remote agent conf
<parser>
<file>C:\test\logs</file>
<rules>
<rule>
<match>TEST ERROR LOG</match>
<event>100000</event>
</rule>
</rules>
</parser>

parser1.xml
<parser>
<file>C:\test\logs</file>
<rules>
<rule>
<match>TEST ERROR LOG</match>
<event>100000</event>
</rule>
</rules>
</parser>

When you say I need to create an event , do you mean a DCI for the remote node?  Thanks

dmna007@hotmail.com

  • Newbie
  • *
  • Posts: 6
    • View Profile
Re: Find text string within log file
« Reply #3 on: November 08, 2016, 01:36:35 pm »
sorry I made an error in my copy and paste, the remote agent config is defined as:

#
# NetXMS agent configuration file
# Created by server installer at Tue Oct 11 15:24:54 2016
#

LogFile = {syslog}
MasterServers = 127.0.0.1, 192.168.11.207
FileStore = C:\NetXMS\var
RequireAuthentication = yes
SharedSecret = HIDDEN
SubAgent = winperf.nsm
SubAgent = portcheck.nsm
SubAgent = logwatch.nsm
# Below is log parsers definitions
*LOGWATCH
Parser = C:\NetXMS\parser1.xml

Dani@M3T

  • Sr. Member
  • ****
  • Posts: 425
    • View Profile
Re: Find text string within log file
« Reply #4 on: November 08, 2016, 03:29:26 pm »
Hi Damien

Change in your nxagentd.conf '*LOGWATCH' to '[LOGWATCH]'.
With your parser xml the string 'TEXT ERROR LOG' in your Logfile 'C:\test\logs' would generate an NetXMS event 100000. So this event 100000 must exist in your event configuration in NetXMS. You don't need a DCI for that.

best regards
Dani

dmna007

  • Newbie
  • *
  • Posts: 8
    • View Profile
Re: Find text string within log file
« Reply #5 on: February 27, 2019, 04:14:59 pm »
Hi, just to let you know, I have this working if I point the parser directly at a file:

<parser>
   <file>C:\FX_CHUB\LOGS\test.txt</file>
   <rules>
      <rule>
         <match>TCP Command Socket Failed Error</match>
         <event>LOGWATCH_PLXCENTRALHUB_176.74.191.177</event>
      </rule>
   </rules>
</parser>

I have created the event in the main Netxms console and an email is sent when the match string is inserted into the txt file, so that's great. 

The problem I now have is the logs within this folder are all variable names, is it possible to point the parser at a folder opposed to a direct file?  This way any log file in this file will be parsed looking for the string to match.

I have tried <file>C:\FX_CHUB\LOGS\*</file>  and <file>C:\FX_CHUB\LOGS\</file>

But this does not seem to work.  Many thanks for your help

Victor Kirhenshtein

  • Lead Developer
  • Administrator
  • Hero Member
  • *****
  • Posts: 7036
    • View Profile
Re: Find text string within log file
« Reply #6 on: February 27, 2019, 10:01:27 pm »
Hi!

You can use macros for forming date/time based file name (possible macros can be found here: http://www.cplusplus.com/reference/ctime/strftime/) or you can use output of shell script as file name using backticks, like this:

Code: [Select]
<file>C:\FX_CHUB\LOGS\`C:\gen_file_name.cmd`</file>

In this example file name will be taken from output of script C:\gen_file_name.cmd

Best regards,
Victor

tickett

  • Newbie
  • *
  • Posts: 8
    • View Profile
Re: Find text string within log file
« Reply #7 on: April 12, 2019, 09:21:29 am »
    I am trying this (dynamic filename) and struggling- I have tried;
    • using a wildcard * (didn't expect it to work, but worth a longshot)
  • adding a macro (see below)
  • using backticks with an echo command directly
  • using backticks with a batch file which executes an echo command
Code: [Select]
<macro name="timestamp">%Y_%m_%d</macro>
The log file seems to show whatever I enter and doesn't appear to show a "resolved" value. Is this normal? Regardless new log entries don't appear to be picked up.

Code: [Select]
2019.04.11 09:31:02.328 *D* [logwatch           ] Parser thread for file "C:\inetpub\wwwroot\website\logs\Log_@{timestamp}.txt" stopped
2019.04.11 16:33:00.121 *D* [logwatch           ] Parser thread for file "C:\inetpub\wwwroot\website\logs\Log_`echo %date:~6,4%_%date:~3,2%_%date:~0,2%`.txt" started
2019.04.11 16:38:34.340 *D* [logwatch           ] Parser thread for file "C:\inetpub\wwwroot\website\logs\Log_*.txt" stopped
2019.04.11 16:38:34.906 *D* [logwatch           ] Parser thread for file "C:\inetpub\wwwroot\website\logs\Log_`c:\netxms\etc\date.bat`.txt" started

I have the parser working fine for the Windows Event Log.

Any ideas? TIA
« Last Edit: April 12, 2019, 09:27:26 am by tickett »

Victor Kirhenshtein

  • Lead Developer
  • Administrator
  • Hero Member
  • *****
  • Posts: 7036
    • View Profile
Re: Find text string within log file
« Reply #8 on: April 12, 2019, 12:07:22 pm »
Yes, this is normal. On startup agent logs unexpanded file name. You should also see something like this when file is found:

2019.04.12 12:03:13.608 *D* [logwatch           ] File "/tmp/test-20190412.log" (pattern "/tmp/test-`date +%Y%m%d`.log") successfully opened

Are you sure your commands return correct values? You may also try to use cmd /c "command" within backticks (because echo for example is not an external executable but cmd's internal command).

Also, if you only need current date/time you may use date/time macros without external commands, like this:

<file>C:\inetpub\wwwroot\website\logs\Log_%Y_%m_%d.txt</file>

Best regards,
Victor

tickett

  • Newbie
  • *
  • Posts: 8
    • View Profile
Re: Find text string within log file
« Reply #9 on: April 12, 2019, 05:44:19 pm »
Thanks Victor, putting the wildcards straight in the filename as you suggested worked perfectly;

<file>C:\inetpub\wwwroot\website\logs\Log_%Y_%m_%d.txt</file>