NetXMS Support Forum

On NetXMS server side.
Stopping NetXMS server service took around 15 minutes.
During that time amount of connections in CLOSE_WAIT state was slowly decreasing.

Hi,

NetXMS was not monitored on development system for several days. During that time oracle DB password has expired:
[04-Dec-2014 09:31:17.278] [ERROR] Unable to establish connection with database (ORA-28002: the password will expire within 5 days)
This issue was not noticed, but it was not the problem itself.

We had a problem with backups of development system as Networker client was failing.
After analysis I found over 32.000 connections from netxmsd towards DB in CLOSE_WAIT status:

153.98.100.55.36421  153.98.100.57.1525   49152      0 49152      0 CLOSE_WAIT
153.98.100.55.35894  153.98.100.57.1525   49152      0 49152      0 CLOSE_WAIT
153.98.100.55.33549  153.98.100.57.1525   49152      0 49152      0 CLOSE_WAIT
153.98.100.55.65128  153.98.100.57.1525   49152      0 49152      0 CLOSE_WAIT
153.98.100.55.60114  153.98.100.57.1525   49152      0 49152      0 CLOSE_WAIT
153.98.100.55.53602  153.98.100.57.1525   49152      0 49152      0 CLOSE_WAIT
153.98.100.55.57421  153.98.100.57.1525   49152      0 49152      0 CLOSE_WAIT
153.98.100.55.33275  153.98.100.57.1525   49152      0 49152      0 CLOSE_WAIT

NetXMS was opening new connection until Solaris 10 reached the limit of open connections.
As a consequence this caused issues with other applications.

I think that some internal limits of connections within NetXMS should be defined to avoid serious problems when DB is not available or not accessible for some time.

Best regards,
Marcin

Hello,

It seems that new version has different dependencies. I did an upgrade from (working) version 1.2.16 to 1.2.17 and I noticed following problem:
[root] /opt/netxms/bin $ ./nxagentd -D 9
ld.so.1: nxagentd: fatal: libxnet.so.1: version 'SUNW_1.3.1' not found (required by file /opt/netxms/lib/libnetxms.so.1)
ld.so.1: nxagentd: fatal: libxnet.so.1: open failed: No such file or directory
Killed

I noticed following changes:
RADnxagent.1.2.16.sparc.pkg:
[root] /opt/netxms/bin $  pvs /opt/netxms/lib/libnetxms.so.1
        libdl.so.1 (SUNW_0.7);
        libsocket.so.1 (SUNW_0.7);
        libxnet.so.1 (SUNW_1.1);
        libpthread.so.1 (SUNW_1.4);
        libc.so.1 (SUNW_1.22, SUNWprivate_1.1);
        libCstd.so.1 (SUNW_1.1.1);
        libCrun.so.1 (SUNW_1.2);

RADnxagent.1.2.17.sparc.pkg:
[root@] /opt/netxms/bin $ pvs /opt/netxms/lib/libnetxms.so.1
        libm.so.2 (SUNW_1.1);
        libresolv.so.2 (SUNW_2.2.2);
        libdl.so.1 (SUNW_0.7);
        libsocket.so.1 (SUNW_0.7);
        libxnet.so.1 (SUNW_1.3.1);
        libpthread.so.1 (SUNW_1.4);
        libc.so.1 (SUNW_1.22, SUNWprivate_1.1);
        libCstd.so.1 (SUNW_1.1.1);
        libCrun.so.1 (SUNW_1.2);

New version require new libraries:
libm.so.2 (SUNW_1.1); -> available
libresolv.so.2 (SUNW_2.2.2); -> available
and one library in never version:
libxnet.so.1 (SUNW_1.3.1); -> instead of libxnet.so.1 (SUNW_1.1);

This one is not available by default on Solaris SPARC 64bit.
[root] /lib $ pvs /lib/libxnet.so.1
        libxnet.so.1;
        SUNW_1.3;
        SUNW_1.2;
        SUNW_1.1;

It looks like this version of NetXMS is dependent on optional Solaris patch 147002-01 (3-years old one) that has to be installed in Solaris global zone.
Is this dependency really required?

Best regards,
Marcin

OK. Thank you for the information.
I misunderstood your previous post. I will wait for 1.2.17 then.

Hi Victor,

I created following filtering script:
sub main()
{
  s = GetEventParameter($event, "sourcePort");
  if (s == NULL)
  {
  return false; // Don't match rule
  }
  return true; // Match rule
}
but it always return true -> source port is NULL.

I tried to create an event using a macro with message:
"Test event, port: %<sourcePort>"
but it does not return any port for the SNMP trap in the Event Log.

I did something wrong?

Best regards,
Marcin

Hello,

We have three Weblogic domains configured on one system. In each domain there is AdminServer. When we shutdown one or all domains it's not clear which domain was stopped. In SNMP trap log all events looks exactly the same. In case stopping all domains:

23.09.2014 18:34:22   153.9.1.58   scrd   .1.3.6.1.4.1.140.625.0.70   .1.3.6.1.4.1.140.625.100.5 == 'Tue Sep 23 18:34:22 CEST 2014'; .1.3.6.1.4.1.140.625.100.10 == 'AdminServer'
23.09.2014 18:34:22   153.9.1.58   scrd   .1.3.6.1.4.1.140.625.0.70   .1.3.6.1.4.1.140.625.100.5 == 'Tue Sep 23 18:34:22 CEST 2014'; .1.3.6.1.4.1.140.625.100.10 == 'AdminServer'
23.09.2014 18:34:22   153.9.1.58   scrd   .1.3.6.1.4.1.140.625.0.70   .1.3.6.1.4.1.140.625.100.5 == 'Tue Sep 23 18:34:22 CEST 2014'; .1.3.6.1.4.1.140.625.100.10 == 'AdminServer'

Each Weblogic domain is configured separately and it's using it's own port to send SNMP events.
Is it possible to create an Event Processing Policy and use SNMP source port attribute to distinguish environments?

Best regards,
Marcin

It's Solaris 10.

Indeed. I see debug logs in the log file.
Thank you. This is very helpful.

Best regards,
Marcin

Is there any quick fix or workaround for this problem like sql query for user_groups table?

Best regards,
Marcin

Hi,

When server is running in daemon mode, I can't enable debug mode in the Server Console.
When I set "debug 9" nothing happen. There is no verbose logging.

When server is started without "-d" enabling debug mode in Server Console works.

Is it a bug or a feature of NetXMS?
I could not find an answer for this question in the documentation.

Best regards,
Marcin

Hi,

Alex, Victor, thank you for the prompt answer.
I will wait for 1.2.17. When this release will be available for the public?

Best regards,
Marcin

Hi,

I configured NetXMS with LDAP.
For the new user that was created and synced with LDAP, in the Properties -> System Rights I selected everything (for test purposes).
When I'm logging in as this user (LDAP password is accepted), when I click Alarm Log I get a message:
Cannot open server log "AlarmLog": Access denied
I have the same error for other views.

I don't know if this is related but while doing ldap sync for the first time I got many errors like this:
[19-Aug-2014 16:10:54.639] [ERROR] SQL query failed (Query = "INSERT INTO user_groups (name,system_access,flags,description,guid,ldap_dn,id) VALUES (?,?,?,?,?,?,?)"): ORA-01400: cannot insert NULL into ("NETXMSOWNER"."USER_GROUPS"."DESCRIPTION")
[19-Aug-2014 16:11:54.812] [ERROR] SQL query failed (Query = "INSERT INTO user_groups (name,system_access,flags,description,guid,ldap_dn,id) VALUES (?,?,?,?,?,?,?)"): ORA-01400: cannot insert NULL into ("NETXMSOWNER"."USER_GROUPS"."DESCRIPTION")

What could be the reason of access denied problem?

Best regards,
Marcin

Hi,

Without DBSchema it's working correctly.

Probably the same bug was causing problems with DBencryptedPassword:
https://www.netxms.org/forum/configuration/dbencryptedpassword-not-working-with-1-2-15/

Best regards,
Marcin

Problem solved by removing one line:
DBSchema = XXX
from netxsmd.conf configuration file.

Details are described in another thread:
https://www.netxms.org/forum/general-support/nxdbmgr-dumps-core-files/

Hi,

Through trial and error method I found that problematic line in the configuration file is:

DBSchema = XXX

After commenting this line tool seems to be working:
root$ /opt/netxms/bin/nxdbmgr check
NetXMS Database Manager Version 1.2.15 Build 7879

Checking database (excluding collected data):
Your database has format version 307, this tool is compiled for version 330.
Use "upgrade" command to upgrade your database first.
Database check aborted

I'm not sure if this line is necessary.

Best regards,
Marcin