NetXMS Support Forum

Thank you Victor.
Library was replaced and this solved the problem.

Best regards,
Marcin

Hi Victor,

Thank you very much for fixing the issue.
When release 2.0.3 will be available?
It would be nice to have a patched build for Solaris. We will replace binary file immediately.

As a workaround we disabled most of the DCI where thresholds were used and this slowed down the process of using cursors on dct_threshold_conditions table.
Now NetXMS server is usable for 6-8 hours after restart.

Best regards,
Marcin

Hello,

NetXMS server 2.0.2 is not usable. It's blocked few hours after every restart:

[04-Mar-2016 08:19:47.926] [ERROR] SQL query failed (Query = "INSERT INTO idata_280 (item_id,idata_timestamp,idata_value) VALUES (?,?,?)"): ORA-00604: error occurred at recursive SQL level 1
ORA-01000: maximum open cursors exceeded
ORA-01000: maximum open cursors exceeded

As a result of ORA-01000 it's not possible to login using NetXMS console or if this is possible, other actions are blocked, for example checking details of Alarm (see attached screenshot).

Increasing number open cursors from 300 to 400 does not help.

Best regards,
Marcin

Hi Victor,

Since upgrading NetXMS server on Solaris from RC version to 2.0.2 I observe following frequently problem:

[24-Feb-2016 10:33:53.520] [ERROR] SQL query failed (Query = "UPDATE raw_dci_values SET raw_value=?,transformed_value=?,last_poll_time=? WHERE item_id=?"): ORA-00604: error occurred at recursive SQL level 1
ORA-01000: maximum open cursors exceeded
ORA-00604: error occurred at recursive SQL level 1
ORA-01000: maximum open cursors exceeded
ORA-01000: maximum open cursors exceeded
[24-Feb-2016 10:33:53.520] [ERROR] SQL query failed (Query = "INSERT INTO idata_1899 (item_id,idata_timestamp,idata_value) VALUES (?,?,?)"): ORA-00604: error occurred at recursive SQL level 1
ORA-01000: maximum open cursors exceeded
ORA-00604: error occurred at recursive SQL level 1
ORA-01000: maximum open cursors exceeded
ORA-01000: maximum open cursors exceeded

On the Oracle Db side my observation is following:

       SID SQL_TEXT                                                     OPEN_CURSORS USER_NAME
---------- ------------------------------------------------------------ ------------ ------------------------------
       213 INSERT INTO dct_threshold_conditions (threshold_id,group_id,          299 NETXMSOWNER
       202 INSERT INTO dct_threshold_conditions (threshold_id,group_id,          299 NETXMSOWNER
       203 INSERT INTO dct_threshold_conditions (threshold_id,group_id,          299 NETXMSOWNER
       211 INSERT INTO dct_threshold_conditions (threshold_id,group_id,          299 NETXMSOWNER
       201 INSERT INTO dct_threshold_conditions (threshold_id,group_id,          299 NETXMSOWNER
       203 insert into sys.aud$( sessionid,entryid,statement,ntimestamp            1 NETXMSOWNER
       203 SELECT item_id FROM raw_dci_values WHERE item_id=223                    1 NETXMSOWNER
       203 SELECT threshold_id FROM thresholds WHERE item_id=126                   1 NETXMSOWNER
       202 SELECT threshold_id FROM thresholds WHERE item_id=21                    1 NETXMSOWNER
       202 INSERT INTO dct_thresholds (id,table_id,sequence_number,acti            1 NETXMSOWNER
       203 SELECT threshold_id FROM thresholds WHERE item_id=123                   1 NETXMSOWNER
       203 DELETE FROM dc_table_columns WHERE table_id=:1                          1 NETXMSOWNER
       203 SELECT item_id FROM raw_dci_values WHERE item_id=222                    1 NETXMSOWNER
       202 DELETE FROM dci_schedules WHERE item_id=9                               1 NETXMSOWNER
       202 DELETE FROM dci_schedules WHERE item_id=48                              1 NETXMSOWNER
       203 UPDATE dc_tables SET node_id=:1,template_id=:2,template_item            1 NETXMSOWNER
       203 DELETE FROM dci_schedules WHERE item_id=121                             1 NETXMSOWNER
       203 DELETE FROM dci_schedules WHERE item_id=218                             1 NETXMSOWNER
       203 UPDATE items SET node_id=:1,template_id=:2,name=:3,source=:4            1 NETXMSOWNER
       203 INSERT INTO dc_table_columns (table_id,sequence_number,colum            1 NETXMSOWNER
       203 SELECT threshold_id FROM thresholds WHERE item_id=219                   1 NETXMSOWNER
       202 SELECT item_id FROM raw_dci_values WHERE item_id=21                     1 NETXMSOWNER
       203 SELECT item_id FROM raw_dci_values WHERE item_id=117                    1 NETXMSOWNER
       202 DELETE FROM dci_schedules WHERE item_id=73                              1 NETXMSOWNER

SQL_FULLTEXT
--------------------------------------------------------------------------------
INSERT INTO dct_threshold_conditions (threshold_id,group_id,sequence_number,column_name,check_operation,check_value) VALUES (:1,:2,:3,:4,:5,:6)

"INSERT INTO dct_threshold_conditions" is opening maximum allowed number of cursors and blocking further operations.
This seems to be cursor memory leak. Application is opening a cursor and not closing it when it's done.

How to solve this issue? This problem is blocking connections (login-in) from Management Console few hours after restart of NetXMS server.

Regards,
Marcin

Hi,

Is it possible to use ServiceCheck.HTTPS to check status of the service behind normal proxy with login/password?

Best regards,
Marcin

Hi,

NetXMS version 2.0-RC2 solved the problem with memory leak.

Best regards,
Marcin

Moving netxms.mib to the correct location solved second problem.

Thank you,
Marcin

Hi Victor,

First problem:
I noticed a bug after upgrading NetXMS to version 2.0-RC2 on Solaris:

$/opt/netxms/bin $ ./netxmsd
ld.so.1: netxmsd: fatal: libnxagent.so.2: open failed: No such file or directory
Killed

Library is missing in the pkg file. As a workaround I copied this file from the NetXMS agent package.

Second problem:
Management console installed on Windows X86 is reporting following problem just after starting:
"Cannot load MIB file from server: File I/O operation failed"
How to solve this problem?

Best regards,
Marcin

Ok. I found. 1178 is node_id (object_id), not item_id.
And this helped me solving the problem (frequent fetching the list of open files) for node 1178.

Thank you,
Marcin

Hello,

I tried query database directly:
SQL> select op.name,d.name from dc_tables d join object_properties op on op.object_id=d.node_id where d.item_id=1178;

no rows selected

How can I check it in user interface? For which object I should look for DCI id in Data Collection Configuration or Last Values?
I don't have ID's higher than 220.

Best regards,
Marcin

Hi,

Could you help to identify why table TDATA_ROWS_1178 is growing very fast?

SQL> select count(*) from TDATA_ROWS_1178;

  COUNT(*)
----------
  18189554

Yesterday there was 10 million rows, today it is 18 millions rows.
This table grow by 300MB since yesterday and consumed all available space in tablespace.

SEGMENT_NAME                                                                      SEGMENT_TYPE       size_in_MB
--------------------------------------------------------------------------------- ------------------ ----------
SYS_C0034271                                                                      INDEX                     746
TDATA_ROWS_1178                                                                   TABLE                     624
SNMP_TRAP_LOG                                                                     TABLE                   511.1
SYS_C0034093                                                                      INDEX                     499
TDATA_ROWS_100                                                                    TABLE                     464

Best regards,
Marcin

Hi,

Today I installed officially released 2.0-RC2 agent. I can confirm there are no issues with libraries.
I will upgrade server package and check if still there are memory leaks as soon as Oracle NetXMS disk space consumption problem will be solved.

Best regards,
Marcin

Hi Victor,

How could I install NetXMS M5 release on Solaris?
I'm using binary packages.

Best regards,
Marcin

Hi Victor,

Is it a bug in the M5 release?
Should I create a symlink: libappagent.so.2 -> libappagent.so.1.0.0 or M5 should be delivered with new version of agent library?

Best regards,
Marcin

I install from binary package.
I did a rollback to M2 version.