NetXMS core won't start after update to 3.7. Can't upgrade database.

Started by Appeh, February 04, 2021, 03:28:36 PM

Previous topic - Next topic

Appeh


After upgrading to the latest server version netxms-server-3.7.145-x64.exe  ( from a 3.x version beginning 2020, don't remember exactly which one ) NetXMS core service won't start.

I checked the log:

2021.02.04 10:22:19.526 *I* [logger             ] Log file opened (rotation policy 2, max size 16777216)
2021.02.04 10:22:19.527 *D* [                   ] LIB directory set to D:\NetXMS\lib
2021.02.04 10:22:19.532 *D* [dload              ] DLOpen: file="mysql.ddr", module=00007FFC5C8F0000
2021.02.04 10:22:19.532 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=drvAPIVersion, address=00007FFC5C8F8038
2021.02.04 10:22:19.532 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=drvName, address=00007FFC5C8F8040
2021.02.04 10:22:19.532 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvInit, address=00007FFC5C8F16F0
2021.02.04 10:22:19.532 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvConnect, address=00007FFC5C8F1720
2021.02.04 10:22:19.532 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvDisconnect, address=00007FFC5C8F18D0
2021.02.04 10:22:19.532 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvSetPrefetchLimit, address=0000000000000000
2021.02.04 10:22:19.532 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvPrepare, address=00007FFC5C8F1920
2021.02.04 10:22:19.532 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvFreeStatement, address=00007FFC5C8F1F00
2021.02.04 10:22:19.532 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvOpenBatch, address=0000000000000000
2021.02.04 10:22:19.532 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvNextBatchRow, address=0000000000000000
2021.02.04 10:22:19.532 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvBind, address=00007FFC5C8F1C10
2021.02.04 10:22:19.532 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvExecute, address=00007FFC5C8F1DB0
2021.02.04 10:22:19.532 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvQuery, address=00007FFC5C8F20A0
2021.02.04 10:22:19.532 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvSelect, address=00007FFC5C8F2330
2021.02.04 10:22:19.533 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvSelectUnbuffered, address=00007FFC5C8F2B40
2021.02.04 10:22:19.533 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvSelectPrepared, address=00007FFC5C8F2350
2021.02.04 10:22:19.533 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvSelectPreparedUnbuffered, address=00007FFC5C8F2D70
2021.02.04 10:22:19.533 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvFetch, address=00007FFC5C8F2FE0
2021.02.04 10:22:19.533 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvGetFieldLength, address=00007FFC5C8F2640
2021.02.04 10:22:19.533 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvGetFieldLengthUnbuffered, address=00007FFC5C8F30D0
2021.02.04 10:22:19.533 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvGetField, address=00007FFC5C8F2A60
2021.02.04 10:22:19.533 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvGetFieldUTF8, address=00007FFC5C8F2A80
2021.02.04 10:22:19.533 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvGetFieldUnbuffered, address=00007FFC5C8F3370
2021.02.04 10:22:19.533 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvGetFieldUnbufferedUTF8, address=00007FFC5C8F3390
2021.02.04 10:22:19.533 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvGetNumRows, address=00007FFC5C8F2AA0
2021.02.04 10:22:19.533 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvGetColumnCount, address=00007FFC5C8F2AB0
2021.02.04 10:22:19.533 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvGetColumnName, address=00007FFC5C8F2AC0
2021.02.04 10:22:19.533 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvGetColumnCountUnbuffered, address=00007FFC5C8F33B0
2021.02.04 10:22:19.533 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvGetColumnNameUnbuffered, address=00007FFC5C8F33C0
2021.02.04 10:22:19.533 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvFreeResult, address=00007FFC5C8F2AF0
2021.02.04 10:22:19.533 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvFreeUnbufferedResult, address=00007FFC5C8F33F0
2021.02.04 10:22:19.533 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvBegin, address=00007FFC5C8F3480
2021.02.04 10:22:19.533 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvCommit, address=00007FFC5C8F3530
2021.02.04 10:22:19.533 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvRollback, address=00007FFC5C8F35E0
2021.02.04 10:22:19.533 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvUnload, address=00007FFC5C8F1710
2021.02.04 10:22:19.533 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvPrepareStringA, address=00007FFC5C8F1500
2021.02.04 10:22:19.533 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvPrepareStringW, address=00007FFC5C8F11D0
2021.02.04 10:22:19.533 *D* [dload              ] DLGetSymbolAddr: module=00007FFC5C8F0000, symbol=DrvIsTableExist, address=00007FFC5C8F3690
2021.02.04 10:22:19.539 *I* [db.drv             ] Database driver "mysql.ddr" loaded and initialized successfully
2021.02.04 10:22:19.539 *D* [db.conn            ] DBConnect: server=localhost db=netxms_db login=netxms schema=
2021.02.04 10:22:19.540 *D* [                   ] Local administration interface listener started
2021.02.04 10:22:19.544 *D* [db.conn            ] New DB connection opened: handle=0000021B25513C80
2021.02.04 10:22:19.544 *D* [                   ] Successfully connected to database netxms_db@localhost
2021.02.04 10:22:19.544 *E* [                   ] Your database has format version 32.7, but server is compiled for version 37.6
2021.02.04 10:22:19.544 *D* [db.conn            ] DB connection 0000021B25513C80 closed

> I tried upgrading the database but received the following:

d:\NetXMS\bin>nxdbmgr upgrade
NetXMS Database Manager Version 3.7.145 Build 3.7-145-g26f1728e68 (UNICODE)

Upgrading database...
Upgrading from version 32.7 to 32.8
SQL query failed (Duplicate column name 'description'):
ALTER TABLE websvc_definitions ADD description varchar(2000)
Rolling back last stage due to upgrade errors...
Database upgrade failed

> After removing the description to see if that was the problem it went on to a new error:

d:\NetXMS\bin>nxdbmgr upgrade
NetXMS Database Manager Version 3.7.145 Build 3.7-145-g26f1728e68 (UNICODE)

Upgrading database...
Upgrading from version 32.7 to 32.8
SQL query failed (Table 'netxms_db.nodes' doesn't exist in engine):
ALTER TABLE nodes ADD vendor varchar(127)
Rolling back last stage due to upgrade errors...
Database upgrade failed


Help?

Filipp Sudanov

Can you check in MySQL if "nodes" table exists?

Did you also upgrade MySQL itself? Can you check, if there are any errors in MySQL log file?

Appeh

I checked via HeidiSQL, it's MariaDB. Logged in via 'root' account.

I can see a 'nodes' table but can't open it. The UI says : You need the SHOW VIEW privilege in order to edit a view.

and this is the output of the query tab when I click on the Nodes table:

"
SHOW CREATE VIEW `netxms_db`.`nodes`;
/* SQL Error (1932): Table 'netxms_db.nodes' doesn't exist in engine */
SELECT * FROM INFORMATION_SCHEMA.VIEWS WHERE TABLE_SCHEMA='netxms_db' AND TABLE_NAME='nodes';
/* Access violation at address 008C54E9 in module 'heidisql.exe'. Read of address 00000018 */
"

Shouldn't root have sufficient rights to do this?

Filipp Sudanov

It's recommended to connect using mysql client, not 3rd party software, as heidisql is crashing here.
https://dev.mysql.com/doc/refman/5.7/en/mysql.html

It all looks like some issue with MySQL, possibly the database is corrupted. May be you can find some useful information here: https://stackoverflow.com/questions/25039927/database-corruption-with-mariadb-table-doesnt-exist-in-engine

If you have backup, I would suggest to restore from it.
Also, we generally find Postgres more reliable (if you decide to go this way, nxdbmgr allows to migrate between database engines).

Appeh

Thanks Filipp. I don't have a backup of the database but I will restore to an earlier snapshot of the VM.