Update from 2.2.10 to 2.2.11 - Log Flooding

Started by Spheron, January 09, 2019, 07:00:00 PM

Previous topic - Next topic

Spheron


Hello @all,

after Upgrade from 2.2.10 to 2.2.11 (Win7 64bit, MySQL 5.7.24) NetXMS Dashboard and Performance Tabs shows randomly "Cannot get Value for DCI XXX (Database Failure)"

If i take a look in de NetXMS Server Log, there are many many lines as the following:


2019.01.09 17:45:35.739 *E* SQL query failed (Query = "SELECT idata_timestamp,idata_value FROM idata_1561 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000"): Query was empty
2019.01.09 17:45:36.633 *E* SQL query failed (Query = "SELECT idata_timestamp,idata_value FROM idata_1524 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000"): Query was empty
2019.01.09 17:45:36.645 *E* SQL query failed (Query = "SELECT idata_timestamp,idata_value FROM idata_2405 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000"): Query was empty
2019.01.09 17:45:37.419 *E* SQL query failed (Query = "SELECT idata_timestamp,idata_value FROM idata_1561 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000"): Query was empty
2019.01.09 17:45:37.462 *E* SQL query failed (Query = "SELECT idata_timestamp,idata_value FROM idata_2405 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000"): Query was empty
2019.01.09 17:45:38.134 *E* SQL query failed (Query = "SELECT idata_timestamp,idata_value FROM idata_1561 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000"): Query was empty


I have run "nxdbmgr check" bevor and after update, but there are no errors...

Should i go back to 2.2.10. ??  Any help is welcome...


Greets
Marco


Victor Kirhenshtein

Hi,

this query is executed when DCI data is read for showing on graph or for export. Do you have any issues with displaying DCI data?

Best regards,
Victor

Spheron


Hello Victor,

as i wrote, all my Dashbords and Performance Tabs shows randomly DCI Data Errors.  "Cannot get Value for DCI XXX (Database Failure)". Every Time i hit the refresh button the error messages shows in a different field. See attachment please.

Greets
Marco

PS: Thanx for the fast response...  :-)


hayarm

#3
+UP, у меня тоже самое,  версия 2.2.11

Spheron


The log flooding only happens when i have a dashboard or a perfomance tab open...

In the NetXMS server log is one more line:


2019.01.10 08:58:14.609 *E* SQL query failed (Query = "SELECT numbering_scheme,row_count,layout_data FROM port_layouts WHERE device_oid=?"): Query was empty


Maybe this one might be help at searching for the root cause of this behaviour...

Greets
Marco

Victor Kirhenshtein

This looks like a bug in MySQL driver. I'm currently looking at it.

Best regards,
Victor

Victor Kirhenshtein

Please try replacing mysql.ddr with attached one. It should print additional diagnostic line when query fails.

Best regards,
Victor

Spheron


Hi Victor,

with your mysql.ddr i can't start the NetXMS Core service. There are follwoing error message in log:


2019.01.10 11:12:51.043 *E* Unable to load database driver module "mysql.ddr": Die angegebene Prozedur wurde nicht gefunden.


With the original mysql.ddr i can start the service..

Greets
Marco

Victor Kirhenshtein

Could you provide md5 sum of downloaded file and original file?

Spheron


MD5 File Hash:

Downloaded File:
D57024956EE56EC9948B5723AD38B23F   C:\test\mysql.ddr

Original File in NetXMS bin Directory:
8E4B91633D8B231C7EA9AFA6429C3C31   C:\NetXMS\bin\mysql.ddr

Victor Kirhenshtein

Did you install 2.2.11 from minimal or full installer?


Victor Kirhenshtein

Looks like something is wrong with binaries. Please try to update using this installer: https://cloud.netxms.org/index.php/s/gWYF0hHo9mFzLKU It also contains mysql.ddr with additional logging.

Best regards,
Victor

Spheron


Hi Victor,

i installed NetXMS via the provided installer.  (Installed via command:  netxms-2.2.11-7-x64.exe /silent )

After that, log file shows the following:



2019.01.10 17:16:45.714 *E* ** PREPARE ERROR ** query='SELECT idata_timestamp,idata_value FROM idata_4437 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000' converted='' ptr=0000000008D3DD40 local=0000000008D3DD40
2019.01.10 17:16:45.714 *E* SQL query failed (Query = "SELECT idata_timestamp,idata_value FROM idata_4437 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000"): Query was empty
2019.01.10 17:16:45.751 *E* ** PREPARE ERROR ** query='SELECT idata_timestamp,idata_value FROM idata_6326 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000' converted='' ptr=0000000008AFDE60 local=0000000008AFDE60
2019.01.10 17:16:45.751 *E* SQL query failed (Query = "SELECT idata_timestamp,idata_value FROM idata_6326 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000"): Query was empty
2019.01.10 17:16:45.831 *E* ** PREPARE ERROR ** query='SELECT idata_timestamp,idata_value FROM idata_4902 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000' converted='' ptr=00000000090ADD50 local=00000000090ADD50
2019.01.10 17:16:45.831 *E* SQL query failed (Query = "SELECT idata_timestamp,idata_value FROM idata_4902 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000"): Query was empty
2019.01.10 17:16:45.928 *E* ** PREPARE ERROR ** query='SELECT idata_timestamp,idata_value FROM idata_4426 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000' converted='' ptr=000000000885DD50 local=000000000885DD50
2019.01.10 17:16:45.928 *E* SQL query failed (Query = "SELECT idata_timestamp,idata_value FROM idata_4426 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000"): Query was empty
2019.01.10 17:16:45.948 *E* ** PREPARE ERROR ** query='SELECT idata_timestamp,idata_value FROM idata_6328 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000' converted='' ptr=000000000842DD80 local=000000000842DD80
2019.01.10 17:16:45.948 *E* SQL query failed (Query = "SELECT idata_timestamp,idata_value FROM idata_6328 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000"): Query was empty
2019.01.10 17:16:46.035 *E* ** PREPARE ERROR ** query='SELECT idata_timestamp,idata_value FROM idata_6326 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000' converted='' ptr=000000000871DE50 local=000000000871DE50
2019.01.10 17:16:46.035 *E* SQL query failed (Query = "SELECT idata_timestamp,idata_value FROM idata_6326 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000"): Query was empty
2019.01.10 17:16:51.510 *E* ** PREPARE ERROR ** query='SELECT idata_timestamp,idata_value FROM idata_1561 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000' converted='' ptr=0000000008D3DD40 local=0000000008D3DD40
2019.01.10 17:16:51.510 *E* SQL query failed (Query = "SELECT idata_timestamp,idata_value FROM idata_1561 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000"): Query was empty
2019.01.10 17:16:51.510 *E* ** PREPARE ERROR ** query='SELECT idata_timestamp,idata_value FROM idata_1524 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000' converted='' ptr=0000000008AFDE60 local=0000000008AFDE60
2019.01.10 17:16:51.511 *E* SQL query failed (Query = "SELECT idata_timestamp,idata_value FROM idata_1524 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000"): Query was empty
2019.01.10 17:17:06.381 *E* ** PREPARE ERROR ** query='SELECT idata_timestamp,idata_value FROM idata_6326 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000' converted='' ptr=0000000008D3DD40 local=0000000008D3DD40
2019.01.10 17:17:06.382 *E* SQL query failed (Query = "SELECT idata_timestamp,idata_value FROM idata_6326 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000"): Query was empty
2019.01.10 17:17:06.384 *E* ** PREPARE ERROR ** query='SELECT idata_timestamp,idata_value FROM idata_4902 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000' converted='' ptr=000000000885DD50 local=000000000885DD50
2019.01.10 17:17:06.384 *E* SQL query failed (Query = "SELECT idata_timestamp,idata_value FROM idata_4902 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000"): Query was empty
2019.01.10 17:17:06.429 *E* ** PREPARE ERROR ** query='SELECT idata_timestamp,idata_value FROM idata_6326 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000' converted='' ptr=000000000842DD80 local=000000000842DD80
2019.01.10 17:17:06.429 *E* SQL query failed (Query = "SELECT idata_timestamp,idata_value FROM idata_6326 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000"): Query was empty
2019.01.10 17:17:06.484 *E* ** PREPARE ERROR ** query='SELECT idata_timestamp,idata_value FROM idata_6328 WHERE item_id=? AND idata_timestamp>=? AND idata_timestamp<=? ORDER BY idata_timestamp DESC LIMIT 200000' converted='' ptr=00000000094CDFD0 local=00000000094CDFD0


...and so on...


Hope that helps...

Greets
Marco

Victor Kirhenshtein

Hi,

I have found the bug. Please try to replace libnetxms.dll to attached.

Best regards,
Victor