Migrate V2.2.17 on PostgreSQL to new server with V3.0 on Timescale-DB

Started by Dani@M3T, November 11, 2019, 10:38:26 AM

Previous topic - Next topic

Dani@M3T

Hi All
First: Thank's for the new V3 Release. We would like to migrate the current installation to a new server.
Old:openSUSE (old version)NetXMS 2.2.17 (from sources)PostgreSQL
New:openSUSE Leap 15.1NetXMS 3.0.x (compiled from sources)
Timescale DB
What would be the best migration path? On the old server we cannot update to NetXMS V3 because of missing libprce32. It's important that we can migrate the existing DB to the new one.
best regardsDani


Filipp Sudanov

Hi!

NetXMS has a tool called nxdbmgr that has an option for database migration. It is launched like this:
nxdbmgr migrate old.configuration.file
The idea is that this tool is launched on the system you are migrating to. In the regular netxms configuration file on that system you have all the configuration for the new database. And you have another configuration file that has connection parameters to the database you are migrating from - that file name is given to nxdbmgr as parameter.
Before migration you have to initialize the new database by running nxdbmgr init.
netxms server should be stopped so no new data appears in the database during migration.
Migration is only supported on the same version of the database. So you can install 2.2.17 on the new system, do the migration and the do the upgrade.
A more recommended way is to upgrade first and migrate then (in that case you would use newer nxdbmgr that possibly has some bugs fixed) - but for that you would need to compile libprce32, so I would try the above approach first.

Dani@M3T

Thanks for your information Filipp

But is Timescale DB really supported in V2.2.17 for migration purposes? Then I would install 2.2.17 on new server with Timescale directly. And upgrade after DB migration.

Or would it be better to install V2.2.17 on new server with DB on PostgreSQL, than update to V3.0.x and then Migration to Timescale?
What ist the current most recommendet DB engine for NetXMS? Still PostgreSQL oder already Timescale?Is performance in NetXMS really better on Timescale? (my main point to change db engine)

Victor Kirhenshtein

Hi,

recommended database engines are PostgreSQL and TimescaleDB, depending on deployment size. For small deployments (few thousands of DCIs) you most likely won't get any significant performance gains. The larger your deployment is the more performance improvement you'll see.

Support for TimescaleDB in 2.2.17 was still experimental, and I would not recommend to do migration to TimescaleDB using 2.2.17. I would do as following:
1. Install new server and initialize TimescaleDB database
2. Upgrade old database to 3.0 using new nxdbmgr
3. Migrate old database to new one

We are planning to add in-place conversion from plain PostgreSQL to TimescaleDB but I'm not sure when it will be ready.

Best regards,
Victor

Dani@M3T

thanks Victor for your information and suggestions.
Deployment is not so big. But in the past I saw the huge difference in speed between grafana graphs (on influx) and NetXMS graphs. So I hoped Timescale would speed up graphing in NetXMS. Or is this not the point where we can see more performance with Timescale in NetXMS?

Victor Kirhenshtein

Yes, TimescaleDB could increase graphing speed. You have to experiment. If you have enough disk space you can do migration from one to another quite fast and check the difference.

Best regards,
Victor

Dani@M3T

i tried to compile netxms-3.0.2357 on openSUSE LEAP 15.1 but got error from configure 'cannot find pcre.h'.
Then installed package 'pcre-devel' (version 8.41) from package-manager (YaST), configure and get 'error: libpcre32 is requred'.Then downloaded  https://ftp.pcre.org/pub/pcre/pcre-8.43.tar.gz
./configure --prefix=/opt/pcre --disable-static --disable-dependency-tracking --enable-pcre16 --enable-pcre32 --disable-cpp --enable-utf --enable-unicode-properties --disable-stack-for-recursionmakemake installthen configure for NetXMS with option --with-pcre=/opt/pcre but got again 'error: libpcre32 is requred

Victor Kirhenshtein


Dani@M3T


Victor Kirhenshtein

Very strange. Please show output of command

file /opt/pcre/lib/libpcre32.so*

and

ldd /opt/pcre/lib/libpcre32.so

Best regards,
Victor

Dani@M3T

netxms:~ # file /opt/pcre/lib/libpcre32.so*
/opt/pcre/lib/libpcre32.so*: cannot open `/opt/pcre/lib/libpcre32.so*' (No such file or directory)
netxms:~ # ldd /opt/pcre/lib/libpcre32.so
ldd: /opt/pcre/lib/libpcre32.so: No such file or directory

Victor Kirhenshtein

That means that libpcre was not properly built or installed. Are you sure that there were no compiltation or installation errors? What files are present in /opt/pcre/lib?

Best regards,
Victor

Dani@M3T

Thanks for your help Victor. Bu t I didnt's saw errors. Here all output of configure, make, make install: Maybee you see some problems in it.

Tursiops

Looks like compiling PCRE installed the libraries in /opt/pcre/lib64, while NetXMS is looking in /opt/pcre/lib.
Not sure if it would be enough to create a symbolic link lib64 -> lib?

Dani@M3T

I didn't knew NetXMS is looking in /opt/pcre/lib. Made a symlink and compiling NetXMS is fine now. I'm a step forward, thanks  :)