NetXMS Support Forum

Please login or register.

Login with username, password and session length

Author Topic: Timescale DB  (Read 694 times)

gmonk63

  • Full Member
  • ***
  • Posts: 127
    • View Profile
Timescale DB
« on: July 04, 2019, 07:51:28 pm »

If I want to migrate from Mysql to Timescale is there any way to import the data from mysql ?  By the way awesome choice :)  I always thought timeseries databases where better suited for NMS's   


Thanks
Logged

Tursiops

  • Sr. Member
  • ****
  • Posts: 462
    • View Profile
Re: Timescale DB
« Reply #1 on: July 05, 2019, 01:27:59 am »

The NetXMS database schema is different for TimescaleDB (even from Postgres -> TimescaleDB), so you'd have to use nxdbmgr for the migration. It's meant to support migration to TimescaleDB from 2.2.14 onwards (haven't tried it yet, we're still running load tests and comparisons between TimescaleDB and Postgres, feeding 50-60GB of live data into each system per day to compare performance).
One thing to note on TimescaleDB use, from what I can tell NetXMS configures the Hypertables with a 1 week chunk interval by default. Depending on your data ingestion rate, you may want to change that before you go live (e.g. for our test, I changed it to 1 day).
Logged

paul

  • Full Member
  • ***
  • Posts: 145
    • View Profile
Re: Timescale DB
« Reply #2 on: July 07, 2019, 03:25:46 am »

Out of curiosity - which version of Timescale are being considered being used - Open Source, community or Enterprise?
Logged

Tursiops

  • Sr. Member
  • ****
  • Posts: 462
    • View Profile
Re: Timescale DB
« Reply #3 on: July 07, 2019, 07:26:20 am »

We went with the packages for the community edition. Enterprise is not something we're considering at our end at this stage, but other NetXMS users might?
Logged

gmonk63

  • Full Member
  • ***
  • Posts: 127
    • View Profile
Re: Timescale DB
« Reply #4 on: July 09, 2019, 12:39:01 am »

Great  ;)  Thanks for the info
Logged

idaham

  • Newbie
  • *
  • Posts: 1
    • View Profile
Re: Timescale DB
« Reply #5 on: August 24, 2019, 05:09:30 pm »

The NetXMS database schema is different for TimescaleDB (even from Postgres -> TimescaleDB), so you'd have to use nxdbmgr for the migration. It's meant to support migration to TimescaleDB from 2.2.14 onwards (haven't tried it yet, we're still running load tests and comparisons between TimescaleDB and Postgres, feeding 50-60GB of live data into each system per day to compare performance).
One thing to note on TimescaleDB use, from what I can tell NetXMS configures the Hypertables with a 1 week chunk interval by default. Depending on your data ingestion rate, you may want to change that before you go live (e.g. for our test, I changed it to 1 day).

When feeding 50-60GB of data per day , that represent how many devices/instance ?
Logged

Victor Kirhenshtein

  • Lead Developer
  • Administrator
  • Hero Member
  • *****
  • Posts: 6746
    • View Profile
Re: Timescale DB
« Reply #6 on: August 24, 2019, 09:00:41 pm »

Just want to node that TimescaleDB support in 3.0 is much better then in 2.2.16, and schema is slightly different (including chunk intervals).

Best regards,
Victor
Logged

Tursiops

  • Sr. Member
  • ****
  • Posts: 462
    • View Profile
Re: Timescale DB
« Reply #7 on: August 25, 2019, 03:04:26 am »

The NetXMS database schema is different for TimescaleDB (even from Postgres -> TimescaleDB), so you'd have to use nxdbmgr for the migration. It's meant to support migration to TimescaleDB from 2.2.14 onwards (haven't tried it yet, we're still running load tests and comparisons between TimescaleDB and Postgres, feeding 50-60GB of live data into each system per day to compare performance).
One thing to note on TimescaleDB use, from what I can tell NetXMS configures the Hypertables with a 1 week chunk interval by default. Depending on your data ingestion rate, you may want to change that before you go live (e.g. for our test, I changed it to 1 day).

When feeding 50-60GB of data per day , that represent how many devices/instance ?


That was for testing purposes only. I configured a number of firewalls to generate verbose syslog messages on everything and had them send those to the test system, in addition to creating a range of "poll every 5 seconds" DCIs.
So it was live data, but not something I'd for a live system. Can't really compare that to a number of nodes.

As Victor said, TimescaleDB support is much improved in version 3. I'd suggest waiting for the release or upgrading to version 3 (beta version in 'unstable' repository) if you intend to use it now.
Logged