NetXMS Support Forum

English Support => General Support => Topic started by: agfjpcs on October 13, 2016, 09:26:59 AM

Title: How do I have DCI's added automatically?
Post by: agfjpcs on October 13, 2016, 09:26:59 AM
I've seen this mentioned in the manual under Instance Discovery
NetXMS can create parameters for Data Collection Item automatically

But with no mention of how to do it. So.... how do I have it so all interfaces are automatically added for monitoring. And perhaps additional intelligence so it knows if it's a MikroTik to add the MikroTik templates etc
Even better, can it try multiple templates and then automatically remove all the ones that are unsupported?
Title: Re: How do I have DCI's added automatically?
Post by: tomaskir on October 13, 2016, 12:42:57 PM
Look at auto-bind scripts for templates for automatic template binding/unbinding.

Then look at instance-discovery for automatic and dynamic DCI creation.

There is a bunch of topics on the forums and the wiki for each of those :)
If you want help, feel free to ask away.
Title: Re: How do I have DCI's added automatically?
Post by: agfjpcs on October 14, 2016, 06:59:35 AM
I'm still new to NetXMS and just trying to wrap my head around it. I don't even know where to go to find auto-bind scripts. The only templates I have installed are really basic ones that it comes installed with. And i've set up some for the equipment we use

Are you able to point me to more specific area's to look at to get my head around this? I.e. which wiki links specifically
Once it clicks i'll be fine, but i'm still trying to get to that stage. And for the moment i'd like to start adding a whole bunch of devices just to get some interface traffic logged. Not practical to add 2,000 devices and have to go and add interface stats individually

If I can get a how-to on just that, or some links to the topics to narrow my focus I should be fine from then on to fine tune
Title: Re: How do I have DCI's added automatically?
Post by: agfjpcs on October 14, 2016, 07:02:13 AM
Also while i'm at it (And I realise this is a different topic)
We have Ubiquiti devices to manage. This page here https://wiki.netxms.org/wiki/Ubiquiti_and_Ligowave_wireless_monitoring mentions an addon which works with the java plugin. But with no download link? Where do I get it?
Nor does it mention how to add the things I want to monitor. It shows pictures but not where to set it up
Title: Re: How do I have DCI's added automatically?
Post by: tomaskir on October 14, 2016, 12:43:49 PM
1) Example for dynamic interface monitoring by instance discovery:
https://www.netxms.org/forum/configuration/interface-names-in-instance-discovery/msg18102/#msg18102

2) Java SubAgent and all its plugins are included with the NetXMS agent.
You just need to install Java yourself, and set the appropriate configs (as per the wiki).
Then you will see the Agent Parameters as in the screenshots on the wiki, and you can create DCIs to monitor what you need on the UBNTs.
Title: Re: How do I have DCI's added automatically?
Post by: agfjpcs on October 17, 2016, 03:19:15 AM
Quote from: tomaskir on October 14, 2016, 12:43:49 PM
1) Example for dynamic interface monitoring by instance discovery:
https://www.netxms.org/forum/configuration/interface-names-in-instance-discovery/msg18102/#msg18102

2) Java SubAgent and all its plugins are included with the NetXMS agent.
You just need to install Java yourself, and set the appropriate configs (as per the wiki).
Then you will see the Agent Parameters as in the screenshots on the wiki, and you can create DCIs to monitor what you need on the UBNTs.

I had the agent installed automatically with the server binaries package. c:\netxms\ubntlw.jar wasn't there. The only file was netxms-agent.jar and 2 folders (ndd, sql)
I installed the agent on another computer, this included netxms-agent.jar and ubntlw.jar so I copied the ubntlw.jar file over
Modified c:\netxms\etc\nxagentd.conf to this

LogFile = {syslog}
MasterServers = 127.0.0.1
FileStore = C:\NetXMS\var
RequireAuthentication = yes
SharedSecret = **REMOVED**
SubAgent = winperf.nsm
SubAgent = portcheck.nsm
SubAgent = java.nsm

[JAVA]
Plugin = C:\NetXMS\lib\ubntlw.jar

[ubiquiti-ap]
10.255.5.1 = **REMOVED**

[ubiquiti-client]


Restarted the NetXMS agent. Opened management console and did poll->configuration and then poll->status on the netxms agent.
I open up DCI for the Ubiquiti AP. Add a new parameter, choose NetXMS Agent as the origin type, set source node as the NetXMS agent, click on 'Select...' and there's no Ubiquiti entries in there at all
Title: Re: How do I have DCI's added automatically?
Post by: tomaskir on October 17, 2016, 04:30:50 AM
I suggest setting

LogFile = C:\NetXMS\log\nxagentd.log


Then restarting agent and looking into the log.

Are there any errors or issues?

Another thing to check:
Right click on agent node > Tools > Agent > Subagent list

Is the Java SubAgent loaded?
Title: Re: How do I have DCI's added automatically?
Post by: agfjpcs on October 17, 2016, 06:15:56 AM
I think there's something wrong with the java.nsm file
I turned logging on and had no errors, however I then set debuglevel = 4 and get this

[17-Oct-2016 13:10:30.030] Log file opened
[17-Oct-2016 13:10:30.031] [INFO ] Additional configs was loaded from C:\nxagentd.conf.d
[17-Oct-2016 13:10:30.031] [INFO ] Debug level set to 4
[17-Oct-2016 13:10:30.031] [DEBUG] Data directory: C:\Windows\system32\config\systemprofile\AppData\Local\nxagentd
[17-Oct-2016 13:10:30.034] [DEBUG] Subagent API initialized
[17-Oct-2016 13:10:30.035] [DEBUG] Validating ciphers
[17-Oct-2016 13:10:30.564] [DEBUG]    AES-256 enabled
[17-Oct-2016 13:10:30.564] [DEBUG]    Blowfish-256 enabled
[17-Oct-2016 13:10:30.564] [DEBUG]    IDEA enabled
[17-Oct-2016 13:10:30.565] [DEBUG]    3DES enabled
[17-Oct-2016 13:10:30.565] [DEBUG]    AES-128 enabled
[17-Oct-2016 13:10:30.565] [DEBUG]    Blowfish-128 enabled
[17-Oct-2016 13:10:30.565] [DEBUG] Crypto library initialized
[17-Oct-2016 13:10:30.574] [DEBUG] ?????????
[17-Oct-2016 13:10:30.574] [INFO ] DB Library: Database driver "sqlite.ddr" loaded and initialized successfully
[17-Oct-2016 13:10:30.575] [DEBUG] New DB connection opened: handle=0000000001110220
[17-Oct-2016 13:10:30.587] [DEBUG] Local database opened successfully
[17-Oct-2016 13:10:30.587] [DEBUG] Thread pool SNMPPROXY initialized (min=2, max=128)
[17-Oct-2016 13:10:30.596] [INFO ] Subagent "WINNT.NSM" loaded successfully
[17-Oct-2016 13:10:30.766] [DEBUG] WinPerf: 3104 counter indexes read
[17-Oct-2016 13:10:30.793] [DEBUG] WinPerf: 3104 counter translations read
[17-Oct-2016 13:10:30.949] [DEBUG] WinPerf: "\Memory\Free & Zero Page List Bytes" is supported
[17-Oct-2016 13:10:30.957] [DEBUG] WINPERF: CollectorThread_B waiting for set change
[17-Oct-2016 13:10:30.957] [DEBUG] WINPERF: CollectorThread_A waiting for set change
[17-Oct-2016 13:10:30.958] [INFO ] Subagent "winperf.nsm" loaded successfully
[17-Oct-2016 13:10:30.958] [DEBUG] WINPERF: CollectorThread_C waiting for set change
[17-Oct-2016 13:10:30.958] [DEBUG] WINPERF: CollectorThread_A: set changed
[17-Oct-2016 13:10:30.960] [DEBUG] WINPERF: CollectorThread_A: Counter "\System\Processor Queue Length" added to query
[17-Oct-2016 13:10:30.960] [DEBUG] WINPERF: CollectorThread_A: Counter "\System\Processor Queue Length" added to query
[17-Oct-2016 13:10:30.960] [INFO ] Subagent "ping.nsm" loaded successfully
[17-Oct-2016 13:10:30.960] [DEBUG] WINPERF: CollectorThread_A: Counter "\System\Processor Queue Length" added to query
[17-Oct-2016 13:10:30.961] [DEBUG] WINPERF: CollectorThread_A: Counter "\Processor(_Total)\% Processor Time" added to query
[17-Oct-2016 13:10:30.961] [DEBUG] WINPERF: CollectorThread_A: Counter "\Processor(_Total)\% Processor Time" added to query
[17-Oct-2016 13:10:30.961] [DEBUG] WINPERF: CollectorThread_A: Counter "\Processor(_Total)\% Processor Time" added to query
[17-Oct-2016 13:10:30.962] [DEBUG] WINPERF: CollectorThread_A: Counter "\PhysicalDisk(_Total)\Avg. Disk Queue Length" added to query
[17-Oct-2016 13:10:30.962] [DEBUG] WINPERF: CollectorThread_A: Counter "\PhysicalDisk(_Total)\% Disk Time" added to query
[17-Oct-2016 13:10:30.962] [DEBUG] WINPERF: CollectorThread_A: Counter "\Processor(0)\% Processor Time" added to query
[17-Oct-2016 13:10:30.963] [DEBUG] WINPERF: CollectorThread_A: Counter "\Processor(0)\% Processor Time" added to query
[17-Oct-2016 13:10:30.963] [DEBUG] WINPERF: CollectorThread_A: Counter "\Processor(0)\% Processor Time" added to query
[17-Oct-2016 13:10:30.963] [DEBUG] WINPERF: CollectorThread_A: Counter "\Processor(1)\% Processor Time" added to query
[17-Oct-2016 13:10:30.963] [DEBUG] WINPERF: CollectorThread_A: Counter "\Processor(1)\% Processor Time" added to query
[17-Oct-2016 13:10:30.963] [DEBUG] WINPERF: CollectorThread_A: Counter "\Processor(1)\% Processor Time" added to query
[17-Oct-2016 13:10:30.963] [DEBUG] WINPERF: CollectorThread_A entered data collection loop
[17-Oct-2016 13:10:30.964] [INFO ] Subagent "portcheck.nsm" loaded successfully
[17-Oct-2016 13:10:30.965] [DEBUG] Initializing Java subagent
[17-Oct-2016 13:10:30.965] [DEBUG] JAVA: using JVM jvm.dll
[17-Oct-2016 13:10:30.965] [ERROR] JAVA: Unable to load JVM: The specified module could not be found.
[17-Oct-2016 13:10:30.965] [ERROR] Registration of subagent "java.nsm" failed
[17-Oct-2016 13:10:31.967] [DEBUG] External parameters providers poller thread will not start
[17-Oct-2016 13:10:31.969] [DEBUG] Data sender thread started
[17-Oct-2016 13:10:31.969] [DEBUG] Data collection scheduler thread started
[17-Oct-2016 13:10:31.969] [DEBUG] Thread pool DATACOLL initialized (min=1, max=64)
[17-Oct-2016 13:10:31.970] [DEBUG] Database writer thread started
[17-Oct-2016 13:10:31.970] [DEBUG] Data reconciliation thread started
[17-Oct-2016 13:10:31.971] [DEBUG] Trying to bind on 0.0.0.0:4700
[17-Oct-2016 13:10:31.971] [DEBUG] Trying to bind on [::]:4700
[17-Oct-2016 13:10:31.971] [INFO ] Listening on socket 0.0.0.0:4700
[17-Oct-2016 13:10:31.971] [INFO ] Listening on socket [::]:4700
[17-Oct-2016 13:10:31.972] [DEBUG] StorageDiscoveryConnector: named pipe created, waiting for connection
[17-Oct-2016 13:10:31.973] [DEBUG] PushConnector: named pipe created, waiting for connection
[17-Oct-2016 13:10:31.973] [DEBUG] Trying to bind on 127.0.0.1:28180
[17-Oct-2016 13:10:31.973] [DEBUG] Session agent connector listening on port 28180
[17-Oct-2016 13:10:32.972] [INFO ] NetXMS Agent started
[17-Oct-2016 13:10:37.483] [DEBUG] [session:0] Server ID set to 2BF9970B80001AD0
[17-Oct-2016 13:10:37.483] [DEBUG] 0 SNMP targets received from server 2BF9970B80001AD0
[17-Oct-2016 13:10:37.484] [DEBUG] 0 data collection elements received from server 2BF9970B80001AD0
[17-Oct-2016 13:10:37.484] [DEBUG] Data collection for server 2BF9970B80001AD0 reconfigured


And nope java does not appear in agent subagent list

I also tried starting the agent on a fresh computer and it has the same problems in the log file (fresh install, all i've done is edit the config file to include subagent = java.nsm, and added debug/logging commands)
Title: Re: How do I have DCI's added automatically?
Post by: agfjpcs on October 19, 2016, 01:43:44 AM
Is anyone able to fire up a fresh windows VM, install the latest version (2.0.6) of netxms agent and confirm it does indeed load java correctly?
Not sure if its a problem with the latest version of java or some netxms component. It's just not working for me, I get the same error messages as above
Title: Re: How do I have DCI's added automatically?
Post by: Victor Kirhenshtein on October 19, 2016, 03:39:21 PM
Hi,

By default agent looks for jvm.dll in your PATH or, of JAVA_HOME environment variable is set, in location %JAVA_HOME%\bin\server or %JAVA_HOME%\jre\bin\server. From your log file it seems that JAVA_HOME is not set correctly. Make sure that you either have jvm.dll in PATH, or that JAVA_HOME environment set correctly. Alternatively you can manually specify full path to jvm.dll using JVM parameter in subagent configuration.

Also, if you are on 64bit system make sure that you are using 64 bit version of both JVM and agent.

Best regards,
Victor
Title: Re: How do I have DCI's added automatically?
Post by: agfjpcs on October 20, 2016, 01:45:46 AM
The %PATH% variable contains this
C:\ProgramData\Oracle\Java\javapath;%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;%SYSTEMROOT%\System32\WindowsPowerShell\v1.0\

JAVA_HOME doesn't exist on any machine, so perhaps its not included in latest versions of Java?
Anyway I created it and entered C:\Program Files\Java\jre1.8.0_101
(Using both 32bit and 64bit java and 64bit agent)

Log file now shows
[20-Oct-2016 08:39:09.121] [DEBUG] Initializing Java subagent
[20-Oct-2016 08:39:09.121] [DEBUG] WINPERF: CollectorThread_A: Counter "\Processor(1)\% Processor Time" added to query
[20-Oct-2016 08:39:09.121] [DEBUG] JAVA: Default JVM set from JAVA_HOME: C:\Program Files\Java\jre1.8.0_101\bin\server\jvm.dll
[20-Oct-2016 08:39:09.121] [DEBUG] WINPERF: CollectorThread_A: Counter "\Processor(1)\% Processor Time" added to query
[20-Oct-2016 08:39:09.121] [DEBUG] JAVA: using JVM C:\Program Files\Java\jre1.8.0_101\bin\server\jvm.dll
[20-Oct-2016 08:39:09.121] [DEBUG] WINPERF: CollectorThread_A: Counter "\Processor(1)\% Processor Time" added to query
[20-Oct-2016 08:39:09.122] [DEBUG] WINPERF: CollectorThread_A entered data collection loop
[20-Oct-2016 08:39:09.127] [ERROR] JAVA: Unable to load JVM: The specified module could not be found.
[20-Oct-2016 08:39:09.127] [ERROR] Registration of subagent "java.nsm" failed



Also tried changing it to program files (x86). Same thing



[20-Oct-2016 08:44:33.058] [DEBUG] Initializing Java subagent
[20-Oct-2016 08:44:33.058] [DEBUG] WINPERF: CollectorThread_A: Counter "\Processor(0)\% Processor Time" added to query
[20-Oct-2016 08:44:33.058] [DEBUG] JAVA: Default JVM set from JAVA_HOME: C:\Program Files (x86)\Java\jre1.8.0_101\jre\bin\server\jvm.dll
[20-Oct-2016 08:44:33.058] [DEBUG] JAVA: using JVM C:\Program Files (x86)\Java\jre1.8.0_101\jre\bin\server\jvm.dll
[20-Oct-2016 08:44:33.058] [DEBUG] WINPERF: CollectorThread_A: Counter "\Processor(0)\% Processor Time" added to query
[20-Oct-2016 08:44:33.058] [DEBUG] WINPERF: CollectorThread_A: Counter "\Processor(0)\% Processor Time" added to query
[20-Oct-2016 08:44:33.058] [ERROR] JAVA: Unable to load JVM: The specified module could not be found.
[20-Oct-2016 08:44:33.058] [ERROR] Registration of subagent "java.nsm" failed
Title: Re: How do I have DCI's added automatically?
Post by: agfjpcs on October 20, 2016, 05:12:13 AM
Utterly baffled............................

I added the JAVA_HOME to my PC (which isn't the main NetXMS server) and hey presto java subagent loads just fine
However I just CANNOT get the damn thing to work on the main server. It throws the above error message. My PC runs Java 8 update 92, so I installed the same version on the server, updated the JAVA_HOME path to point to using that version - doesn't work
Tried a bunch of things including installing the standalone NetXMS agent on the server (64 bit). No difference
Deleted and copied the Java and NetXMS Agent folders across to the server so the files are identical. No difference
Gave full read/write permission to everyone on both folders just in case it was something silly like that. No difference

It just doesn't work
Title: Re: How do I have DCI's added automatically?
Post by: Victor Kirhenshtein on October 20, 2016, 09:05:16 AM
What is server OS version?

Best regards,
Victor
Title: Re: How do I have DCI's added automatically?
Post by: agfjpcs on October 24, 2016, 05:45:02 AM
2012 R2 Standard - build 9600
Title: Re: How do I have DCI's added automatically?
Post by: Victor Kirhenshtein on November 08, 2016, 11:40:21 AM
Hi,

just noticed from

JAVA: using JVM C:\Program Files (x86)\Java\jre1.8.0_101\jre\bin\server\jvm.dll

that you seems to be using 32 bit JVM, but most likely you have 64 bit agent. You should use 64 bit JVM with 64 bit agent and 32 bit JVM with 32 bit agent.

Best regards,
Victor
Title: Re: How do I have DCI's added automatically?
Post by: agfjpcs on November 23, 2016, 07:25:57 AM
That would have been an old config I tried. The server has both 32 and 64 bit JAVA installed
The current one is pointing to the 64 bit java jvm.dll and it doesn't work, same message as above
I tried uninstalling and reinstalling both 32 and 64 bit versions of Java, Tried manually adding
jvm = "C:\Program Files\Java\jre1.8.0_112\bin\server\jvm.dll" to nxagentd.cfg under [JAVA] and same problem

Still getting
[23-Nov-2016 15:24:33.026] [DEBUG] JAVA: using JVM C:\Program Files\Java\jre1.8.0_112\bin\server\jvm.dll
...
[23-Nov-2016 15:24:33.028] [ERROR] JAVA: Unable to load JVM: The specified module could not be found.
[23-Nov-2016 15:24:33.029] [ERROR] Registration of subagent "java.nsm" failed
Title: Re: How do I have DCI's added automatically?
Post by: agfjpcs on April 19, 2017, 09:51:24 AM
I'm reviving this thread because i've completely scrapped Windows as a host and gone to Ubuntu - various crashes and other such. Ignore the past few posts as they don't apply anymore. What is important is the title of this thread

I STILL can't work out how to get DCI's for interfaces added automatically because information isn't contained in a simple "here follow this from start to finish" manner, its "google it and work it out for yourself"

Can you let me know if i'm on the right track here and help me fill in the blanks...


1) Create a Template item
2) Right click the new template, choose properties, go to the 'Automatic Apply Rules' section
3) Fill in some sort of matching criteria, this alone is difficult enough to find, but for testing purposes it seems "return true;" will match everything
3a) In my case i'm using "return $node->driver == "UBNT";" to match Ubiquiti devices
4) Click ok, right click the Template and choose 'Data Collection Configuration' then create a new parameter
5) Fill in....
Description: "IN - {instance-name} (bytes/sec)"
Parameter: .1.3.6.1.2.1.2.2.1.10.{instance} (In this case this correlates to the inbound bytes counter of Ubiquiti devices)
Origin: SNMP
Data Type: Unsigned Integer
6) In the Instance Discovery section, choose 'SNMP Walk - OIDs'
7) Set Base SNMP OID as .1.3.6.1.2.1.2.2.1.2 (This should be the interface name/description value?)
8 ) In the script field type in "return $(true, $1, $2);"
9) Now whenever you do a Poll->Configuration (and this happens automatically when adding a new device) the template items should get added to anything that matches the template and each DCI

My understanding is that $1 effectively becomes {instance} and $2 becomes {instance-name}
However while this gets added to the node's as a DCI, those aren't replaced. The parameter still reads .1.3.6.1.2.1.2.2.1.10.{instance} and of course no data is collected
Can someone explain what i'm doing wrong and actually tell me the steps necessary instead of just asking me to go look on the forums. Have already done that...
Title: Re: How do I have DCI's added automatically?
Post by: Tursiops on April 19, 2017, 10:23:35 AM
Hi,

In step 5, change the data type to Unsigned Integer 64bit.
In step 7, set the Base SNMP OID as .1.3.6.1.2.1.2.2.1.1 (not .2).
In step 8, you will need this:
return %(true, $1, $2);
Note the % instead of the $ in your list.

Add a step after 8 to:
Click on Transformation and select Average delta per second. As you want Bytes per second, you don't need to add further any transformation (e.g. you could "return $1*8;" to return Bits per second instead. You may also want to implement some kind of transform set to avoid showing funky data caused by counter overflows (i.e. impossibly high values being shown). This part is something they are working on fixing with a new data type for this purpose, but for now, you'll simply have to filter the bad results out in one way or the other.

Poll Configuration will apply the template and Instance Discovery will create the required DCIs afterwards.
You need to run both.

Cheers
Title: Re: How do I have DCI's added automatically?
Post by: agfjpcs on April 21, 2017, 07:33:52 AM
Finally! Thank you! works fine now

And yes the $ instead of % was a typo above
I mostly had it working before but was unaware you needed to actually run 'instance discovery' after a config poll. Some of my old scripts magically started working