NetXMS Support Forum

After upgrading my agent from 1.2.10 to 1.2.12 I get a buffer overflow when trying to start the service. 

Here's the error...
[root@netxms ~]# service nxagentd start
Starting nxagentd: *** buffer overflow detected ***: nxagentd terminated
======= Backtrace: =========
/lib64/libc.so.6(__fortify_fail+0x37)[0x385cf02827]
/lib64/libc.so.6[0x385cf00710]
/lib64/libc.so.6[0x385cf00d5b]
nxagentd[0x40dff6]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x385ce1ed1d]
nxagentd[0x404cc9]
======= Memory map: ========
00400000-0041a000 r-xp 00000000 fd:00 392307                             /usr/bin/nxagentd
0061a000-00622000 rw-p 0001a000 fd:00 392307                             /usr/bin/nxagentd
01732000-01753000 rw-p 00000000 00:00 0                                  [heap]
3434400000-34345b5000 r-xp 00000000 fd:00 398524                         /usr/lib64/libcrypto.so.1.0.1e
34345b5000-34347b5000 ---p 001b5000 fd:00 398524                         /usr/lib64/libcrypto.so.1.0.1e
34347b5000-34347d0000 r--p 001b5000 fd:00 398524                         /usr/lib64/libcrypto.so.1.0.1e
34347d0000-34347dc000 rw-p 001d0000 fd:00 398524                         /usr/lib64/libcrypto.so.1.0.1e
34347dc000-34347e0000 rw-p 00000000 00:00 0
385ca00000-385ca20000 r-xp 00000000 fd:00 260791                         /lib64/ld-2.12.so
385cc1f000-385cc20000 r--p 0001f000 fd:00 260791                         /lib64/ld-2.12.so
385cc20000-385cc21000 rw-p 00020000 fd:00 260791                         /lib64/ld-2.12.so
385cc21000-385cc22000 rw-p 00000000 00:00 0
385ce00000-385cf8b000 r-xp 00000000 fd:00 260814                         /lib64/libc-2.12.so
385cf8b000-385d18a000 ---p 0018b000 fd:00 260814                         /lib64/libc-2.12.so
385d18a000-385d18e000 r--p 0018a000 fd:00 260814                         /lib64/libc-2.12.so
385d18e000-385d18f000 rw-p 0018e000 fd:00 260814                         /lib64/libc-2.12.so
385d18f000-385d194000 rw-p 00000000 00:00 0
385d200000-385d202000 r-xp 00000000 fd:00 260724                         /lib64/libdl-2.12.so
385d202000-385d402000 ---p 00002000 fd:00 260724                         /lib64/libdl-2.12.so
385d402000-385d403000 r--p 00002000 fd:00 260724                         /lib64/libdl-2.12.so
385d403000-385d404000 rw-p 00003000 fd:00 260724                         /lib64/libdl-2.12.so
385d600000-385d617000 r-xp 00000000 fd:00 260815                         /lib64/libpthread-2.12.so
385d617000-385d817000 ---p 00017000 fd:00 260815                         /lib64/libpthread-2.12.so
385d817000-385d818000 r--p 00017000 fd:00 260815                         /lib64/libpthread-2.12.so
385d818000-385d819000 rw-p 00018000 fd:00 260815                         /lib64/libpthread-2.12.so
385d819000-385d81d000 rw-p 00000000 00:00 0
385da00000-385da15000 r-xp 00000000 fd:00 260837                         /lib64/libz.so.1.2.3
385da15000-385dc14000 ---p 00015000 fd:00 260837                         /lib64/libz.so.1.2.3
385dc14000-385dc15000 r--p 00014000 fd:00 260837                         /lib64/libz.so.1.2.3
385dc15000-385dc16000 rw-p 00015000 fd:00 260837                         /lib64/libz.so.1.2.3
385de00000-385dee8000 r-xp 00000000 fd:00 397701                         /usr/lib64/libstdc++.so.6.0.13
385dee8000-385e0e8000 ---p 000e8000 fd:00 397701                         /usr/lib64/libstdc++.so.6.0.13
385e0e8000-385e0ef000 r--p 000e8000 fd:00 397701                         /usr/lib64/libstdc++.so.6.0.13
385e0ef000-385e0f1000 rw-p 000ef000 fd:00 397701                         /usr/lib64/libstdc++.so.6.0.13
385e0f1000-385e106000 rw-p 00000000 00:00 0
385e200000-385e283000 r-xp 00000000 fd:00 260821                         /lib64/libm-2.12.so
385e283000-385e482000 ---p 00083000 fd:00 260821                         /lib64/libm-2.12.so
385e482000-385e483000 r--p 00082000 fd:00 260821                         /lib64/libm-2.12.so
385e483000-385e484000 rw-p 00083000 fd:00 260821                         /lib64/libm-2.12.so
385ee00000-385ee16000 r-xp 00000000 fd:00 260722                         /lib64/libresolv-2.12.so
385ee16000-385f016000 ---p 00016000 fd:00 260722                         /lib64/libresolv-2.12.so
385f016000-385f017000 r--p 00016000 fd:00 260722                         /lib64/libresolv-2.12.so
385f017000-385f018000 rw-p 00017000 fd:00 260722                         /lib64/libresolv-2.12.so
385f018000-385f01a000 rw-p 00000000 00:00 0
385f600000-385f616000 r-xp 00000000 fd:00 260824                         /lib64/libgcc_s-4.4.7-20120601.so.1
385f616000-385f815000 ---p 00016000 fd:00 260824                         /lib64/libgcc_s-4.4.7-20120601.so.1
385f815000-385f816000 rw-p 00015000 fd:00 260824                         /lib64/libgcc_s-4.4.7-20120601.so.1
385fa00000-385fa26000 r-xp 00000000 fd:00 265021                         /lib64/libexpat.so.1.5.2
385fa26000-385fc25000 ---p 00026000 fd:00 265021                         /lib64/libexpat.so.1.5.2
385fc25000-385fc28000 rw-p 00025000 fd:00 265021                         /lib64/libexpat.so.1.5.2
3a92c00000-3a92c0f000 r-xp 00000000 fd:00 392585                         /usr/lib64/libnxtre.so.5.0.0
3a92c0f000-3a92e0e000 ---p 0000f000 fd:00 392585                         /usr/lib64/libnxtre.so.5.0.0
3a92e0e000-3a92e0f000 rw-p 0000e000 fd:00 392585                         /usr/lib64/libnxtre.so.5.0.0
3a93000000-3a93031000 r-xp 00000000 fd:00 392655                         /usr/lib64/libnetxms.so.1.0.0
3a93031000-3a93231000 ---p 00031000 fd:00 392655                         /usr/lib64/libnetxms.so.1.0.0
3a93231000-3a93233000 rw-p 00031000 fd:00 392655                         /usr/lib64/libnetxms.so.1.0.0
3a93233000-3a93237000 rw-p 00000000 00:00 0
3a93c00000-3a93c0b000 r-xp 00000000 fd:00 399141                         /usr/lib64/libnxdb.so.1.0.0
3a93c0b000-3a93e0b000 ---p 0000b000 fd:00 399141                         /usr/lib64/libnxdb.so.1.0.0
3a93e0b000-3a93e0c000 rw-p 0000b000 fd:00 399141                         /usr/lib64/libnxdb.so.1.0.0
7fc288935000-7fc28893e000 rw-p 00000000 00:00 0
7fc28893e000-7fc288941000 r-xp 00000000 fd:00 395787                     /usr/lib64/libappagent.so.1.0.0
7fc288941000-7fc288b40000 ---p 00003000 fd:00 395787                     /usr/lib64/libappagent.so.1.0.0
7fc288b40000-7fc288b41000 rw-p 00002000 fd:00 395787                     /usr/lib64/libappagent.so.1.0.0
7fc288b49000-7fc288b4b000 rw-p 00000000 00:00 0
7fff8b06e000-7fff8b083000 rw-p 00000000 00:00 0                          [stack]
7fff8b1b2000-7fff8b1b3000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
/bin/bash: line 1:  4241 Aborted                 (core dumped) nxagentd -d
                                                           [FAILED]
[root@netxms ~]#

And the config file
[root@netxms ~]# cat /etc/nxagentd.conf
#
# Sample configuration file for NetXMS agent
#

#
# Servers
#
# List of management servers which have read access to this agent.
# Both IP addresses and DNS names can be used. Multiple servers can be
# specified in one line, separated by commas. If this parameter used more
# than once, servers listed in all occurences will have access to agent.
#
# Examples:
# Servers = 10.0.0.1
# Servers = 127.0.0.1, srv1.domain.com, srv2.domain.com
Servers = localhost
#
# ControlServers
#
# List of management servers which can execute actions on agent. Hosts
# listed in this parameter also have read access to the agent. Both
# IP addresses and DNS names can be used. Multiple servers can be specified
# in one line, separated by commas. If this parameter used more than
# once, servers listed in all occurences will have access to agent.
#
# Examples:
# ControlServers = 10.0.0.1
# ControlServers = 127.0.0.1, srv1.domain.com, srv2.domain.com
ControlServers = localhost
#
# MasterServers
#
# List of management servers which have full access to agent. Hosts listed
# in this group can change agent's config, upload files to agent and initiate
# agent upgrade, as well as perform any task allowed for hosts listed in
# Servers and ControlServers. Both IP addresses and DNS names can be used.
# Multiple servers can be specified in one line, separated by commas.
# If this parameter used more than once, servers listed in all occurences
# will have access to agent.
#
# Examples:
# MasterServers = 10.0.0.1
# MasterServers = 127.0.0.1, srv1.domain.com, srv2.domain.com
MasterServers = localhost
#
# ListenPort
#
# Defines the port number for the agent to listen on incoming connections.
# Default values is 4700
#
# Examples:
# ListenPort = 4700

#
# LogFile
#
# Agent's log file. To write log to syslog (or Event Log on Windows),
# use {syslog} as file name.
#
# Examples:
LogFile = {syslog}
# LogFile = /var/log/nxagentd.log

#
# RequireAuthentication
#
# If set to "yes", host connected to agent have to provide correct shared
# secret before issuing any command. Default value is "no".
#
# Example:
# RequireAuthentication = no
RequireAuthentication = yes
#
# RequireEncryption
#
# If set to "yes", host connected to agent will be forced to use encryption,
# and if encryption is not supported by remote host, connection will be
# dropped. Default value is "no". This parameter has no effect if agent
# was compiled without encryption support.
#
# Example:
# RequireEncryption = no
RequireEncryption = yes
#
# EnabledCiphers
#
# Control what ciphers agent can use for connection encryption. Value for
# this parameter is a cipher code. To enable more than one cipher, they
# codes should be added. By default, all possible ciphers are enabled.
#
# Possible cipher codes:
#       1  =  AES-256
#       2  =  BLOWFISH
#       4  =  IDEA
#       8  =  Triple DES
#
# Example (enable AES-256 and IDEA):
# EnabledCiphers = 5

#
# SharedSecret
#
# Agent's shared secret. Used only if RequireAuthentication set to "yes".
#
# Example:
# SharedSecret = secret
SharedSecret = xxxxxxxx
#
# MaxSessions
#
# Maximum number of simultaneous communication sessions. Possible value can
# be in range from 2 to 1024. Default value is 32.
#
# Example:
# MaxSessions = 32

#
# SessionIdleTimeout
#
# Communication session's idle timeout in seconds. If agent will not receive
# any command from peer within specified timeout, session will be closed.
# Default value is 60 seconds.
#
# Example:
# SessionIdleTimeout = 60

#
# FileStore
#
# Directory to be used for storing files uploaded by installation server(s).
#
# Example:
# FileStore = /var/lib/netxms

#
# TimeOut
#
# GET request timeout in seconds. If GET request cannot be completed for
# specified amount of time, agent will return an error to server.
# Default value is 5 seconds.
#
# Example:
# TimeOut = 5

#
# StartupDelay
#
# Number of seconds agent should wait on startup before start servicing
# requests. This parameter can be useful to prevent false reports about
# missing processes or failed services just after monitored system startup.
# Default value is 0, i.e. no startup delay.
#
# Example:
# StartupDelay = 0

#
# PlatformSuffix
#
# String to be added as suffix to value of System.PlatformName parameter.
#
# Example:
# PlatformSuffix = dbg

#
# EnableSubagentAutoload
#
# Enable or disable autoloading of platform subagent(s).
# Default value is "yes".
#
# Example:
# EnableSubagentAutoload = yes

#
# EnableProxy
#
# Allow requests forwarding
# Default value is "no"
#
# Example:
# EnableProxy = no

#
# EnableSNMPProxy
#
# Allow SNMP requests forwarding
# Default value is "no"
#
# Example:
# EnableSNMPProxy = no

#
# SubAgent
#
# Subagent to load. To load multiple subagents, you should use multiple
# SubAgent parameters. Subagents will be loaded in the same order as they
# appears in configuration file.
#
# UNIX example:
# SubAgent = [path_to_netxms_install_dir]/lib/libnsm_linux.so
#
# Windows example:
# SubAgent = winperf.nsm
SubAgent = /usr/lib64/libnsm_ecs.so
SubAgent = /usr/lib64/libnsm_linux.so
SubAgent = /usr/lib64/libnsm_logwatch.so
SubAgent = /usr/lib64/libnsm_odbcquery.so
SubAgent = /usr/lib64/libnsm_ping.so
SubAgent = /usr/lib64/libnsm_portcheck.so
SubAgent = /usr/lib64/libnsm_sms.so
#SubAgent = /usr/lib64/libnsm_ups.so
#
# ExternalParameter
#
# Add parameter handled by external command. To add multiple parameters, you
# should use multiple ExternalParameter entries. Please note that on Windows
# agent uses system process execution API for execution of specified
# command, so you cannot use pipes or shell commands. If you need to
# execute command via shell, use ExternalParameterShellExec instead.
#
# Syntax:
# ExternalParameter = <parameter_name>:<command_line>
# If you specify parameter name as "name(*)", then you can use $1 .. $9 in
# command line to substitute actual arguments passed from server.
#
# Examples:
# ExternalParameter = Test:echo test
# ExternalParameter = LineCount(*):cat $1 | wc -l
# ExternalParameter = Test2(*):myprog $1 $2

#
# ExternalParameterShellExec
#
# Add parameter handled by external command. To add multiple parameters, you
# should use multiple ExternalParameterShellExec entries. This is similar
# to ExternalParameter with exception that agent will use shell to execute
# specified command instead of system process exeution API. This difference
# presented only on Windows system, on other systems ExternalParameter and
# ExternalParameterShellExec behaves identically.
#
# Syntax:
# ExternalParameterShellExec = <parameter_name>:<command_line>
# If you specify parameter name as "name(*)", then you can use $1 .. $9 in
# command line to substitute actual arguments passed from server.
#
# Examples:
# ExternalParameterShellExec = Test:dir c:\ | wc -l

ConfigIncludeDir = /etc/nxagentd.conf.d

*PING
Timeout = 1000
PacketRate = 30
Target = 24.96.198.137:Col-ISP
Target = 172.16.5.1:Atl-FW
Target = 172.16.30.1:QTS-FW

[root@netxms ~]#


Hi!

As it looks like you have core file from the crash, can you please send me backtrace obtained with gdb. Instruction is following:

1. Run gdb as

gdb /usr/bin/nxagentd core

(replace core with actual core file location)

2. At (gdb) prompt, enter command bt

Best regards,
Victor

[root@netxms ~]# cat /proc/sys/kernel/core_pattern
|/usr/libexec/abrt-hook-ccpp %s %c %p %u %g %t e
[root@netxms ~]# cd /var/cache/abrt-di/
[root@netxms abrt-di]# ls -la
total 8
drwxrwxr-x.  2 abrt abrt 4096 Nov 22 16:45 .
drwxr-xr-x. 11 root root 4096 Feb 23 13:07 ..
[root@netxms abrt-di]#


What's the best way to get the core file?  can i just echo "<file location>" > /proc/sys/kernel/core_pattern ?

Ok, i managed to get the core dump and ran it through gdb, output is attached

Can you please also run agent as

nxagentd -D9

and end me the output?

Best regards,
Victor

same result, output attached

EDIT: i guess it would help if i saved the file before attaching it

Looks like it's crashed even before agent completes initialization. Can you try to set log file to a file instead of syslog? Also, on what system you are running it?

Best regards,
Victor

I set logging to go to /var/log/nxagentd.log and tried running it with the -D9 option again, but i'm running it as root and the log file still doesn't exist

[root@netxms ~]# cat /etc/redhat-release
CentOS release 6.5 (Final)
[root@netxms ~]# uname -a
Linux netxms.x.x 2.6.32-431.5.1.el6.centos.plus.x86_64 #1 SMP Tue Feb 11 21:21:23 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
[root@netxms ~]#

Don't have CentOS 6.5 for testing at the moment, will try it tomorrow. How did you build agent (with what configure options)?

Best regards,
Victor

configure      \
  --with-server \
  --with-snmp   \
  --with-mysql  \
  --with-pgsql  \
  --with-sqlite \
  --with-odbc   \
  --with-client \
  --with-agent

Hi!

Try to add --disable-xmpp just in case.

Best regards,
Victor

Ok, i'm building now.  I was under the impression it was just the server that used the XMPP library?

EDIT: Same result

Same problem here. OS: Ubuntu 12.04

Hi,

Any news to this? Having the same problem on Ubuntu 12.04.

Nikk

Hi!

I've fixed the problem. Version 1.2.13 will work correctly.

Best regards,
Victor