NetXMS Support Forum

Please login or register.

Login with username, password and session length

Author Topic: DDCC client: Thread Item "Item poller" not responding  (Read 200 times)

isoftcom

  • Newbie
  • *
  • Posts: 7
    • View Profile
DDCC client: Thread Item "Item poller" not responding
« on: January 31, 2019, 03:00:27 pm »

Hello Devs

I have been seeing this popping almost every after 30 seconds

DDCC client: Thread Item "Item poller" not responding

This is where Netxms is installed and other servers and its currently having a red x.

How do i sort this problem out?
Logged

Victor Kirhenshtein

  • Lead Developer
  • Administrator
  • Hero Member
  • *****
  • Posts: 6538
    • View Profile
Re: DDCC client: Thread Item "Item poller" not responding
« Reply #1 on: January 31, 2019, 03:13:59 pm »

Hi,

this points to some internal locks being held for prolonged period. What version you are using, on what OS? Can you show event log for NetXMS server node?

Best regards,
Victor
 
Logged

isoftcom

  • Newbie
  • *
  • Posts: 7
    • View Profile
Re: DDCC client: Thread Item "Item poller" not responding
« Reply #2 on: January 31, 2019, 04:13:49 pm »

OS version - AIX 7.2
the version of netxms is 2.0.4
i cleared the logs to have fresh logs so i detect the problem. by event logs you mean netxmsd logs right?

some logs include the following

-----------------------------------------------------------------
[21-Jan-2019 07:24:50.357] [ERROR] SQL query failed (Query = "INSERT INTO tdata_155 (item_id,tdata_timestamp,record_id) VALUES (?,?,?)"): ORA-00001: unique constraint (NETXMS.SYS_C0012623) violated
[21-Jan-2019 07:24:50.909] [ERROR] SQL query failed (Query = "INSERT INTO tdata_155 (item_id,tdata_timestamp,record_id) VALUES (?,?,?)"): ORA-00001: unique constraint (NETXMS.SYS_C0012623) violated
[21-Jan-2019 07:24:56.374] [ERROR] SQL query failed (Query = "INSERT INTO tdata_151 (item_id,tdata_timestamp,record_id) VALUES (?,?,?)"): ORA-00001: unique constraint (NETXMS.SYS_C0012621) violated
[21-Jan-2019 07:30:01.469] [ERROR] Thread "Syncer Thread" does not respond to watchdog thread
[21-Jan-2019 07:30:01.473] [ERROR] Thread "Poll Manager" does not respond to watchdog thread
[21-Jan-2019 07:34:50.389] [ERROR] SQL query failed (Query = "INSERT INTO tdata_155 (item_id,tdata_timestamp,record_id) VALUES (?,?,?)"): ORA-00001: unique constraint (NETXMS.SYS_C0012623) violated
[21-Jan-2019 07:34:50.885] [ERROR] SQL query failed (Query = "INSERT INTO tdata_151 (item_id,tdata_timestamp,record_id) VALUES (?,?,?)"): ORA-00001: unique constraint (NETXMS.SYS_C0012621) violated
[21-Jan-2019 07:34:52.000] [ERROR] SQL query failed (Query = "INSERT INTO tdata_155 (item_id,tdata_timestamp,record_id) VALUES (?,?,?)"): ORA-00001: unique constraint (NETXMS.SYS_C0012623) violated
[21-Jan-2019 07:36:41.570] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 07:37:21.593] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 07:38:01.618] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 07:38:41.632] [ERROR] Thread "Syncer Thread" does not respond to watchdog thread
[21-Jan-2019 07:39:01.649] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 07:40:01.675] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 07:41:21.702] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 07:44:01.767] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 07:47:32.390] [ERROR] SQL query failed (Query = "INSERT INTO tdata_151 (item_id,tdata_timestamp,record_id) VALUES (?,?,?)"): ORA-00001: unique constraint (NETXMS.SYS_C0012621) violated
[21-Jan-2019 07:49:01.828] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 07:50:01.831] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 07:50:06.598] [ERROR] SQL query failed (Query = "INSERT INTO tdata_151 (item_id,tdata_timestamp,record_id) VALUES (?,?,?)"): ORA-00001: unique constraint (NETXMS.SYS_C0012621) violated
[21-Jan-2019 07:51:01.849] [ERROR] Thread "Syncer Thread" does not respond to watchdog thread
[21-Jan-2019 07:54:01.918] [ERROR] Thread "Syncer Thread" does not respond to watchdog thread
[21-Jan-2019 07:54:41.924] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 07:55:21.936] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 07:58:01.971] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 07:59:21.995] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:00:01.078] [DEBUG] IST clean syslg history called
[21-Jan-2019 08:00:02.000] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:01:02.018] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:02:42.077] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:03:42.096] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:04:22.101] [ERROR] Thread "Syncer Thread" does not respond to watchdog thread
[21-Jan-2019 08:05:22.109] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:07:02.148] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:14:02.234] [ERROR] Thread "Syncer Thread" does not respond to watchdog thread
[21-Jan-2019 08:14:42.260] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:15:42.292] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:16:22.307] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:18:02.327] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:19:02.345] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:20:22.362] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:20:54.132] [ERROR] SQL query failed (Query = "INSERT INTO tdata_155 (item_id,tdata_timestamp,record_id) VALUES (?,?,?)"): ORA-00001: unique constraint (NETXMS.SYS_C0012623) violated
[21-Jan-2019 08:20:54.137] [ERROR] SQL query failed (Query = "INSERT INTO tdata_155 (item_id,tdata_timestamp,record_id) VALUES (?,?,?)"): ORA-00001: unique constraint (NETXMS.SYS_C0012623) violated
[21-Jan-2019 08:22:42.407] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:24:22.453] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:26:42.511] [ERROR] Thread "Poll Manager" does not respond to watchdog thread
[21-Jan-2019 08:28:22.531] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:29:22.548] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:29:59.587] [ERROR] SQL query failed (Query = "INSERT INTO tdata_151 (item_id,tdata_timestamp,record_id) VALUES (?,?,?)"): ORA-00001: unique constraint (NETXMS.SYS_C0012621) violated
[21-Jan-2019 08:30:42.565] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:33:02.599] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:33:42.610] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:34:42.624] [ERROR] Thread "Syncer Thread" does not respond to watchdog thread
[21-Jan-2019 08:35:02.631] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:36:02.654] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:37:22.683] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:39:42.724] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:40:22.747] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:41:42.765] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:41:42.766] [ERROR] Thread "Syncer Thread" does not respond to watchdog thread
[21-Jan-2019 08:42:42.826] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:44:22.867] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:46:22.920] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:47:42.952] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:49:02.972] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:49:36.626] [ERROR] SQL query failed (Query = "INSERT INTO tdata_155 (item_id,tdata_timestamp,record_id) VALUES (?,?,?)"): ORA-00001: unique constraint (NETXMS.SYS_C0012623) violated
[21-Jan-2019 08:49:36.627] [ERROR] SQL query failed (Query = "INSERT INTO tdata_155 (item_id,tdata_timestamp,record_id) VALUES (?,?,?)"): ORA-00001: unique constraint (NETXMS.SYS_C0012623) violated
[21-Jan-2019 08:50:02.994] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:52:57.180] [DEBUG] Database Connection Pool: connection 1106bc590 reconnected
[21-Jan-2019 08:52:57.847] [DEBUG] Database Connection Pool: connection 11033b730 reconnected
[21-Jan-2019 08:52:58.788] [DEBUG] Database Connection Pool: connection 1103301b0 reconnected
[21-Jan-2019 08:52:59.681] [DEBUG] Database Connection Pool: connection 1100c8d10 reconnected
[21-Jan-2019 08:52:59.970] [DEBUG] Database Connection Pool: connection 1100eedd0 reconnected
[21-Jan-2019 08:53:23.051] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:54:23.062] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 08:54:37.828] [ERROR] SQL query failed (Query = "INSERT INTO tdata_151 (item_id,tdata_timestamp,record_id) VALUES (?,?,?)"): ORA-00001: unique constraint (NETXMS.SYS_C0012621) violated
[21-Jan-2019 08:54:40.867] [ERROR] SQL query failed (Query = "INSERT INTO tdata_151 (item_id,tdata_timestamp,record_id) VALUES (?,?,?)"): ORA-00001: unique constraint (NETXMS.SYS_C0012621) violated
[21-Jan-2019 09:00:23.134] [ERROR] Thread "Syncer Thread" does not respond to watchdog thread
[21-Jan-2019 09:03:03.190] [ERROR] Thread "Poll Manager" does not respond to watchdog thread
[21-Jan-2019 09:05:39.028] [DEBUG] IST clean syslg history called
[21-Jan-2019 09:05:40.744] [ERROR] SQL query failed (Query = "INSERT INTO tdata_151 (item_id,tdata_timestamp,record_id) VALUES (?,?,?)"): ORA-00001: unique constraint (NETXMS.SYS_C0012621) violated
[21-Jan-2019 09:05:41.990] [ERROR] SQL query failed (Query = "UPDATE raw_dci_values SET raw_value=?,transformed_value=?,last_poll_time=? WHERE item_id=?"): ORA-00060: deadlock detected while waiting for resource
[21-Jan-2019 09:05:42.309] [ERROR] SQL query failed (Query = "INSERT INTO tdata_151 (item_id,tdata_timestamp,record_id) VALUES (?,?,?)"): ORA-00001: unique constraint (NETXMS.SYS_C0012621) violated
[21-Jan-2019 09:10:43.309] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 09:11:52.536] [ERROR] SQL query failed (Query = "INSERT INTO tdata_151 (item_id,tdata_timestamp,record_id) VALUES (?,?,?)"): ORA-00001: unique constraint (NETXMS.SYS_C0012621) violated
[21-Jan-2019 09:11:52.650] [ERROR] SQL query failed (Query = "INSERT INTO tdata_155 (item_id,tdata_timestamp,record_id) VALUES (?,?,?)"): ORA-00001: unique constraint (NETXMS.SYS_C0012623) violated
[21-Jan-2019 09:11:52.786] [ERROR] SQL query failed (Query = "INSERT INTO tdata_151 (item_id,tdata_timestamp,record_id) VALUES (?,?,?)"): ORA-00001: unique constraint (NETXMS.SYS_C0012621) violated
[21-Jan-2019 09:12:23.353] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 09:13:43.374] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 09:14:43.393] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 09:18:43.447] [ERROR] Thread "Syncer Thread" does not respond to watchdog thread
[21-Jan-2019 09:19:43.459] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 09:22:03.488] [ERROR] Thread "Poll Manager" does not respond to watchdog thread
[21-Jan-2019 09:25:23.555] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 09:26:23.576] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 09:27:03.589] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 09:28:43.676] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 09:31:43.746] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 09:33:03.763] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 09:33:03.767] [ERROR] Thread "Syncer Thread" does not respond to watchdog thread
[21-Jan-2019 09:34:23.795] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 09:36:23.823] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 09:41:43.884] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 09:43:03.928] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 09:43:11.050] [ERROR] SQL query failed (Query = "INSERT INTO tdata_151 (item_id,tdata_timestamp,record_id) VALUES (?,?,?)"): ORA-00001: unique constraint (NETXMS.SYS_C0012621) violated
[21-Jan-2019 09:43:43.955] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 09:44:43.975] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 09:47:44.027] [ERROR] Thread "Item Poller" does not respond to watchdog thread
[21-Jan-2019 09:49:04
----------------------------------------------
« Last Edit: January 31, 2019, 04:19:34 pm by isoftcom »
Logged

Victor Kirhenshtein

  • Lead Developer
  • Administrator
  • Hero Member
  • *****
  • Posts: 6538
    • View Profile
Re: DDCC client: Thread Item "Item poller" not responding
« Reply #3 on: January 31, 2019, 05:25:05 pm »

This version is very old. I definitely recommend upgrade to 2.2.12 first. We have fixed lot of issues, including internal locks and performance issues.

Best regards,
Victor
Logged