NetXMS Support Forum

I'm not sure if this is related, but each time I restart my machine running the NetXMS agent, the server loses connectivity with it permanently until I manually restart the agent (sudo service nxagentd restart).

I am running v1.2.16 on both the server and agent.

More specifically, after I restart the machine running the agent (running Debian 6), the "Last Values" view for this agent's DCIs display "<<ERROR>>" in NetXMS Management Console.

I have tried polling configuration, status and the other polling options, but this does not help.

It seems as if the agent is not starting properly when the Debian 6 OS starts up, but then running "sudo service nxagentd restart" on the agent machine seems to clear things up.

I have looked at the agen'st log file and it indicates that the agent is started OK when the Debian OS machine boots up.

Any ideas? Should I report this as a separate issue, instead of coupling it with the original poster's report here?

Quote from: raypetter on April 03, 2014, 12:09:35 AM
Victor, thanks for fixing the debian package.

Jeff, my guess is that the issue is _not_ netxms related, but in fact postgresql related. I'd downgrade postgresql instead of running bleeding edge. I'm running the latest netxms release on postgresql 9.1 and having no issues.

I am coming to the same conclusion, but have not yet tested this hypothesis. But I am successfully running NetXMS 1.2.13 with PostgreSQL 9.3.4 on another machine (Ubuntu, not Debian), although in that case NetXMS was installed from Debian packages and here I am installing from sources.

Thanks for the input.

Quote from: Victor Kirhenshtein on March 31, 2014, 11:29:29 AM

Quote from: raypetter on March 29, 2014, 11:50:16 PM
Victor, please update the postgresql package so that it does NOT depend on postgresql 8.4

Most of us are running 9.1 and newer. For every update I need to extract the package and remove/update the package dependency (DEBIAN/control).

Thanks

I've changed dependency for Debian 7 package. Next release should be ok.

Best regards,
Victor

Is this change present in the hourly snapshot: http://git.netxms.org/netxms-snapshot-develop.tar.gz ?

I don't see it mentioned in the ChangeLog file.

Quote from: jeffreyz on March 23, 2014, 08:15:13 PM
I upgraded from 1.2.12 to 1.2.13 and now I cannot start the server because the server cannot log into the database.

I found that if I change "md5" to "trust" for 127.0.0.1 in pg_hba.conf, then NetXMS server starts up OK and can connect with the PostgreSQL database, i.e., if I change:

host    all             all             127.0.0.1/32            md5

to

host    all             all             127.0.0.1/32            trust

then the problem goes away, although so does the security :-)

To sum things up, nxdbmgr and psql connect flawlessly to the PostgreSQL database with auth-method = md5, but in order for NetXMS server to connect, I need to set auth-method = trust.  Strange.  It worked fine with v1.2.12.  Could it be that something did not get built properly (I installed and upgraded from sources)?

I will try upgrading to v1.2.14 when it appears. Fingers crossed.

I just performed this same test on a NetXMS v1.2.12 server and agent that were built with Unicode support.

I used the same bash script:

locale
echo 'asdf æøå ÆØÅ qwerty'


Here is the result of executing this script manually in a shell:

LANG=en_US.UTF-8
LANGUAGE=en_US:en
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
asdf æøå ÆØÅ qwerty


This is what I would hope to see. The Norwegian characters appear correctly.

Here is the result of the NetXMS agent executing these same commands on the same machine via the "ExternalList" mechanism (I generated this output with nxget -l ...)


LANG=en_US.UTF-8
LANGUAGE=en_US:en
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
asdf


This is bad.  The echo output is still truncated at the first non-ASCII character.

So just compiling NetXMS with Unicode support does not seem to fix this problem, at least according to my test here.

Is there anything else I can try?

I have bumped into a problem when trying to generate an agent list from a query on an SQL database. I am creating the agent list using the "ExternalList" agent configuration parameter.

This actually works very nicely - the "ExternalList" mechanism is really great and I am not saying that it has a problem, but I find that all non-ASCII characters are stripped out of the bash script that runs to define the list items. This bash script performs that SQL query, although this problem is not related to SQL in any way.

The best way to describe this problem is to show you a test script (nothing to do with SQL) and then show you the results of running this script manually on the agent machine (Debian Linux 7.4) in a bash shell, and then compare this to the list generated by executing exactly the same code by NetXMS via the "ExternalList" mechanism.

Here is the test script:


locale
echo 'asdf æøå ÆØÅ qwerty'


The "locale" command generates several lines of output (each line becomes a list element). The non-ASCII characters in the "echo" command are Norwegian letters that appear in SQL results in my actual case where I define the agent list via an SQL query. When I execute these commands manually from within a bash shell on the agent machine, I get the following output:


LANG=en_US.UTF-8
LANGUAGE=en_US:
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
asdf æøå ÆØÅ qwerty


However, when I let the NetXMS agent execute these same commands on the same machine via the "ExternalList" mechanism. Here is the list generated (using nxget -l ...):


LANG=C
LANGUAGE=C
LC_CTYPE="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_COLLATE="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_PAPER="C"
LC_NAME="C"
LC_ADDRESS="C"
LC_TELEPHONE="C"
LC_MEASUREMENT="C"
LC_IDENTIFICATION="C"
LC_ALL=C
asdf


Notice that the locale details are all different and that the command "echo 'asdf æøå ÆØÅ qwerty'" was truncated at the first non-ASCII character. It is not that the Norwegian characters were stripped from the output - the output was truncated at the first non-ASCII character because the "querty" string at the end does not appear.

Here are my questions:

• Is this the sign that the NetXMS agent and/or server were not built with Unicode support? If so, would the problem likely go away if I re-built the agent in Unicode support? Or would I have to rebuild the server as well? If not, then ...
• Is there another fix to this problem that you can suggest? If not, then ...
• Is this problem due to how the internals of NetXMS are implemented and out of my control? If so, is this something you can fix in a future release?

Thanks for the great support you provide on NetXMS.

Quote from: Alex Kirhenshtein on March 25, 2014, 07:59:04 PM

Quote from: jeffreyz on March 23, 2014, 08:15:13 PM
I upgraded from 1.2.12 to 1.2.13 and now I cannot start the server because the server cannot log into the database.  Here is the output from the server log file (DebugLevel = 9):

1. Try to change "localhost" to "127.0.0.1"
2. Please show your pg_hba.conf

Unfortunately, changing "localhost" to "127.0.0.1" did not work.  Same result. Here is the log file from that attempt:

[28-Mar-2014 16:29:45.942] Log file opened
[28-Mar-2014 16:29:45.942] [DEBUG] Code page set to ASCII
[28-Mar-2014 16:29:45.948] [INFO ] Platform subagent "/usr/local/lib/netxms/linux.nsm" successfully loaded
[28-Mar-2014 16:29:45.948] [DEBUG] Debug callback set for DB library
[28-Mar-2014 16:29:45.952] [INFO ] Database driver "pgsql.ddr" loaded and initialized successfully
[28-Mar-2014 16:29:45.952] [DEBUG] DBConnect: server=127.0.0.1 db=netxms_db login=netxms schema=netxms
[28-Mar-2014 16:29:50.966] [DEBUG] DBConnect: server=127.0.0.1 db=netxms_db login=netxms schema=netxms
[28-Mar-2014 16:29:55.979] [DEBUG] DBConnect: server=127.0.0.1 db=netxms_db login=netxms schema=netxms
[28-Mar-2014 16:30:00.992] [DEBUG] DBConnect: server=127.0.0.1 db=netxms_db login=netxms schema=netxms
[28-Mar-2014 16:30:06.003] [DEBUG] DBConnect: server=127.0.0.1 db=netxms_db login=netxms schema=netxms
[28-Mar-2014 16:30:11.015] [DEBUG] DBConnect: server=127.0.0.1 db=netxms_db login=netxms schema=netxms
[28-Mar-2014 16:30:11.023] [ERROR] Unable to establish connection with database (fe_sendauth: no password supplied)


Here is my pg_hba.conf:


# Database administrative login by Unix domain socket
local   all             postgres                                peer

# TYPE  DATABASE        USER            ADDRESS                 METHOD

# "local" is for Unix domain socket connections only
#local   all             all                                     peer
local   all             all                                     md5

# IPv4 local connections:
host    all             all             127.0.0.1/32            md5

# IPv4 connections from my local subnet
host    all             all             10.0.0.0/16             md5
host    all             all             ::1/128                 md5


Do you have any other suggestions?  The data is not very important since this is for a server that I am only using for testing.  Plus it is on a virtual machine so I am willing to try anything - I can always revert to a snapshot if things go south. But I do need to get it going somehow.

Quote from: Victor Kirhenshtein on March 24, 2014, 09:56:48 AM
Hi!

So nxdbmgr can login to database? That's very strange, because both nxdbmgr and netxmsd uses same library and same config. Just in case please show me your pg_hba.conf.

Yes, nxdbmgr can log in fine.  Here is some output I copied from a shell that also shows

• pg_hba.conf
• logging in to the database using psql (I specified the password from /etc/netxmsd.conf).

All commands are executed from the NetXMS server machine, of course:


root@netxms-debian-vm:~# nxdbmgr check
NetXMS Database Manager Version 1.2.13 Build 7864

Checking database (excluding collected data):
* Checking zone objects...                                            [PASSED]
* Checking node objects...                                            [PASSED]
* Checking interface objects...                                       [PASSED]
* Checking network service objects...                                 [PASSED]
* Checking cluster objects...                                         [PASSED]
* Checking template to node mapping...                                [PASSED]
* Checking object properties...                                       [PASSED]
* Checking event processing policy...                                 [PASSED]
* Checking network map links...                                       [PASSED]
Database doesn't contain any errors
Database check completed

root@netxms-debian-vm:~# nxdbmgr upgrade
NetXMS Database Manager Version 1.2.13 Build 7864

Upgrading database...
Your database format is up to date

root@netxms-debian-vm:~# tail -23 /etc/postgresql/9.3/main/pg_hba.conf
# Database administrative login by Unix domain socket
local   all             postgres                                peer

# TYPE  DATABASE        USER            ADDRESS                 METHOD

# "local" is for Unix domain socket connections only
#local   all             all                                     peer
local   all             all                                     md5

# IPv4 local connections:
host    all             all             127.0.0.1/32            md5

# IPv4 connections from my local subnet
# (so psql or pgAdmin can be used remotely, for example):
host    all             all             10.0.0.0/16             md5

# IPv6 local connections:
host    all             all             ::1/128                 md5
# Allow replication connections from localhost, by a user with the
# replication privilege.
#local   replication     postgres                                peer
#host    replication     postgres        127.0.0.1/32            md5
#host    replication     postgres        ::1/128                 md5

root@netxms-debian-vm:~# psql netxms_db netxms
Password for user netxms:
psql (9.3.4)
Type "help" for help.

netxms_db=> \q
root@netxms-debian-vm:~#


I am at a loss for how to debug this. Are there any other tests that you can suggest?

Quote from: Victor Kirhenshtein on March 24, 2014, 09:33:37 AM
You can create your own list on agent using ExternalList configuration option. Agent will execute command given and return all it's output as list elements. For example, to add list which will return all files in /tmp, add the following to nxagentd.conf:

Code Select Expand


ExternalList = TemporaryFiles:/bin/ls /tmp


Bingo! It works like a charm. Thank you very much.

Is "ExternalList" documented somewhere? It is not that I need more info, but I have not seen documentation that mentions this parameter and neither did a web search find anything. I would love to read more documentation, even if it were just source code.

I upgraded from 1.2.12 to 1.2.13 and now I cannot start the server because the server cannot log into the database.  Here is the output from the server log file (DebugLevel = 9):

[23-Mar-2014 18:58:33.677] Log file opened
[23-Mar-2014 18:58:33.677] [DEBUG] Code page set to ASCII
[23-Mar-2014 18:58:33.678] [INFO ] Platform subagent "/usr/local/lib/netxms/linux.nsm" successfully loaded
[23-Mar-2014 18:58:33.679] [DEBUG] Debug callback set for DB library
[23-Mar-2014 18:58:33.681] [INFO ] Database driver "pgsql.ddr" loaded and initialized successfully
[23-Mar-2014 18:58:33.683] [DEBUG] DBConnect: server=localhost db=netxms_db login=netxms schema=netxms
[23-Mar-2014 18:58:38.697] [DEBUG] DBConnect: server=localhost db=netxms_db login=netxms schema=netxms
[23-Mar-2014 18:58:43.710] [DEBUG] DBConnect: server=localhost db=netxms_db login=netxms schema=netxms
[23-Mar-2014 18:58:48.723] [DEBUG] DBConnect: server=localhost db=netxms_db login=netxms schema=netxms
[23-Mar-2014 18:58:53.736] [DEBUG] DBConnect: server=localhost db=netxms_db login=netxms schema=netxms
[23-Mar-2014 18:58:58.744] [DEBUG] DBConnect: server=localhost db=netxms_db login=netxms schema=netxms
[23-Mar-2014 18:58:58.751] [ERROR] Unable to establish connection with database (fe_sendauth: no password supplied)


The database is PostgreSQL 9.3.4 - it worked fine with NetXMS v1.2.12.  The log file mentions "no password supplied", but this password is *still* in /etc/netxms.conf. Here is that file (without the password). As I said,it worked fine with v1.2.12:

[netxms-debian-vm:/etc jeffreyz]$ cat netxmsd.conf
DebugLevel = 9
DBDriver = pgsql.ddr
DBServer = localhost
DBName = netxms_db
DBLogin = netxms
DBPassword = <removed>
DBSchema = netxms
LogFailedSQLQueries = yes
LogFile = /var/log/netxmsd
LogRotationMode = 1
[netxms-debian-vm:/etc jeffreyz]$

I ran both:

nxdbmgr check
nxdbmgr upgrade


and there were no problems reported.  I can log into the database fine using the PostgreSQL utility psql with the "netxms" login name and the database password from /etc/netxms.conf.

Here are a few more details:

OS:    Debian Linux 7.4

NetXMS installed from sources. Initially installed v1.2.12. I have just now upgraded from 1.2.12 to 1.2.13, also using sources (netxms-1.2.13.tar.gz).  Here is my configure command:


$ sh ./configure --with-server --with-pgsql --with-agent --enable-unicode

Any ideas?

I need to monitor a value that can be extracted from a database on a database server. This is easy to do with a DCI that uses a custom shell script that queries the database for the value to be monitored.

However, the database server has, in fact, not one but approximately 60 databases. Each database has the same schema, so the same script can be used to monitor the value in question for each database. But I do not want to create 60 different DCIs manually, one for each database to be monitored. I would rather use instance discovery so that they can be created automagically for me.

However, in order to use instance discovery for this case, it would appear to be necessary to do two things:

• Create a DCI based on my shell script that contains an {instance} parameter that accepts the name of a database on the server.
• Create an agent list that contains the names of all of the databases on the database server.

Are both of these things possible? I am new to NetXMS and so far I have only come across built-in DCIs that accept {instance} parameters, as well as only pre-defined agent lists. I would be willing to code something up in Java, but I would need a few pointers on how to get started.

I would appreciate any help to point me in the right direction, even if the answer is that this is not possible.