NetXMS Support Forum

Hi, I created a Cluster into All Services and created 3 nodes inside that Cluster object. I configured 2DCI with SNMP pool and custom port into cluster object. When I restart my computer then I see cluster node gets disappears. This has happened both time when I created cluster. In my first attempt on Saturday I configured 2 nodes inside and then on Monday I had configured 3 nodes inside cluster. Below is netxms.log which confirms some problem

[07-Dec-2010 10:18:57] Log file opened
[07-Dec-2010 10:18:58] Database driver "sqlite.ddr" loaded and initialized successfully
[07-Dec-2010 10:18:59] Failed to load node object with id 31 from database
[07-Dec-2010 10:18:59] Failed to load node object with id 32 from database
[07-Dec-2010 10:18:59] Failed to load node object with id 41 from database
[07-Dec-2010 10:18:59] Failed to load node object with id 42 from database
[07-Dec-2010 10:18:59] Failed to load node object with id 44 from database
[07-Dec-2010 10:18:59] Inconsistent database: cluster object 28 has reference to non-existing node object 31
[07-Dec-2010 10:18:59] Failed to load cluster object with id 28 from database
[07-Dec-2010 10:18:59] Inconsistent database: cluster object 40 has reference to non-existing node object 41
[07-Dec-2010 10:18:59] Failed to load cluster object with id 40 from database
[07-Dec-2010 10:18:59] Inconsistent database: ServiceRoot object has reference to non-existing child object 40
[07-Dec-2010 10:18:59] Inconsistent database: ServiceRoot object has reference to non-existing child object 44
[07-Dec-2010 10:18:59] NetXMS Server started
[07-Dec-2010 10:18:59] Listening for SNMP traps on UDP socket 0.0.0.0:162
[07-Dec-2010 10:18:59] Listening for client connections on TCP socket 0.0.0.0:4701

Dear Victor, The script you shared did not worked. I tried to "Test" above transformation with Test value = 2(random), I get error "Error 14 in line 2: Left argument of -> must be a reference to object". I further added a check
if (node2 == null)
      return 0;   // No such node or access denied

and surprisingly I get this "0" returned. Please note that my nodes are part of cluster. Is that could be limitation?

Using the approach you suggest, I may loose information of my one node. I just wonder if you can have some push method from scripting. Can I create a dummy node, define data collection with origion "Push Agent". Then define a schedule to run every minute "1 * * * *". Finally do something like below into Transformation?

node1 = FindNodeObject($node, "system-1");
value1 = GetDCIValue(node1, FindDCIByDescription(node1, "Auth-Req"));
node2 = FindNodeObject($node, "system-2");
value2 = GetDCIValue(node2, FindDCIByDescription(node2, "Auth-Req"));
return value1 + value2;

I'd even fine if database triger level of twick can be done as some very advance level.

Victor,
    Any though/suggestion for combined/sum graph?

In the last try, i recreated node with "Unmanaged object". I first configured everything and disabled NetXMS agent, ICMP ping, status poll, configuration poll, routing table poll, ifXTable for pooling interfacec into "Polling" tab of properties. Then I started "Manage" this node. I'm happy to see NetXMS working properly. Now SNMP-Query with my OID is only traffic b/w NetXMS and our RADIUS server.


Victor, for your kind attention I'd like to see a graph with combined (sum) value from 2 (or more) similar nodes. As you know I've 2 machines running same software and there is a load-balancer. There is no "official" clustering but it can be seen as server-farm. Please guide me how to achieve that?

I installed 1.0.8 and configure a node with custom port number. I see that NetXMS still not able to do data collection over SNMP custom ports. As soon as I add node NetXMS send SNMP-Query for 1.3.6.1.2.1.1.2.0 (4 retransmission with V1 and 4 retransmission with V2c) and 1.3.6.1.4.1.2620.1.1.10.0 (4 retransmission with V1 and no V2c)
Then I went into node-properties and selected SNMP UDP Port to 1161 under "Communication" tab. Further I added my RADIUS-OID i.e. .1.3.6.1.2.1.67.1.1.1.1.5 with proper custom SNMP port selection. Now I could see snmp requests (custom port) for OID .1.3.6.1.2.1.67.1.1.1.1.5  and 1.3.6.1.2.1.1.2.0 into Wireshark. I could see proper response from my agent as well.
Note that my SNMP Agent support only RADIUS-MIB and it sends "noSuchName" error-response to unsupported OIDs like 1.3.6.1.2.1.1.2.0 and 1.3.6.1.4.1.2620.1.1.10.0. My agent is behind firewall and ping is also blocked.
I think NetXMS is dependent on response for those 2 basic OIDs i.e. 1.3.6.1.2.1.1.2.0 and 1.3.6.1.4.1.2620.1.1.10.0.
I see some UDP packets for destination-port 260. Why that traffic is seen/originated?
Please advise on how to configure a node with custom SNMP Port.

Do you guys has any reluctance on my suggestion put about
1. Microsoft Loopback adapter
2. Registry values for MaxUserPort and TcpTimedWaitDelay
3. Using SO_REUSEADDR by setsockopt()
Please give them a try. I undestand that some setup have much higher load then current setup but you need to understand that every setup/network is different!!!

If there are some socket related limitation that can be overcome by below guidelines
Start Registry Editor. Browse to, and then click the following key in the registry:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters
On the Edit menu, click New, DWORD Value, and then add the following registry values
1. Reduce the client TCP/IP socket connection timeout value from the default value of 240 seconds
Value name: TcpTimedWaitDelay
Value data: <Enter a decimal value between 30 and 240 here>

2.  Increase the number of ephemeral ports that can by dynamically allocated to clients:
Value name: MaxUserPort
Value data: <Enter a decimal value between 5000 and 65534 here>

Victor, Please consider using "SO_REUSEADDR" socket setting, by calling "setsockopt()"

Reference for all registry is from http://support.microsoft.com/default.aspx?scid=kb;en-us;314053
http://technet.microsoft.com/en-us/library/bb726981.aspx#EDAA

Quote from: Victor Kirhenshtein on November 11, 2010, 07:36:09 PM
I'll try to implement it in 1.0.8 release.

Victor, When 1.0.8 will be made available with SNMP port option? Without that I cannot use NextXMS at all. I'm quite eager to monitor our software with NetXMS and I've to prepare my team as end user as well.
Thanks for your kind consideration.

I'd like to see some type of addition/sum of values collected (either by NetXMS or SNMP Agent) and then drawing a graph on that value.
This functionality will help to see overall through put for cluster, multi-link, etc kind of scenario. Example I've 2 Internet routers/links and I'd like to know overall Internet throughput. Similarly I've 4 HTTP server and I'd like to see overall request per second.

Quote from: Victor Kirhenshtein on November 12, 2010, 10:03:27 AM
2. is it possible that incoming SNMP packets blocked by firewall at NetXMS server machine?

Port was indeed blocked by Windows Firewall. I usually turn-off that, but Recently I've opted a new m/c which had Firewall turned on :-(
Trap messages now seen into both Event Browser and SNMP Trap Log. I see a problem where Event Browser shows SNMP_UNMATCHED_TRAP, while I've proper mapping of Trap OID with Event type. You already have PCAP so we can figure out who is doing what wrong ;-)
However Event Type is properly shown as SYS_SERVICE_UP/SYS_SERVICE_DOWN for SNMPv2 Trap.

Very random post... Please ensure that your network connectivity is stable. I experienced random NetXMS problem/behavior when network availability fluctuates.
Just give a try to install "Microsoft Loopback Adapter" from Add Hardware Wizard. Assign some Private IP (Other then your network). I'm sure there is no harm in installing loopback adapter

I can even provide you remove access via Team-Viewer like tool. I'm in India i.e. GMT+5:50 TZ.
I've taken various snapshots and created consolidated JPGs which can be found into attachment.
I've also attached RADIUS-MIBs which you can include into standard distribution.
Feel free to provide any of your/other personal contact details for quick coordination.

Quote from: Victor Kirhenshtein on November 11, 2010, 07:36:09 PM
Hi!

Am I understand your correctly - you have one three-node cluster, and each node has two SNMP agents?

Right now the only working solution I could imagine is to configure lot of external parameters on agent which will call nxsnmpget - but this cannot be considered as good solution. I think that easiest way is to allow change of SNMP port on node and DCI basis - this is relatively easy change and will solve this problem. I'll try to implement it in 1.0.8 release.

Best regards,
Victor

Yes you have rightly understtod my setup. Further I'm sure that having each SNMP-OID/DCI basis port is eagerly awaited by many of NetXMS users. Parallelly please look into possible BUG of "not using configured community-name while communicating with SNMP-Proxy".

My Agent software generate an SNMPv1 enterprise trap with OID .1.3.6.1.4.1.21067.1.3. I've properly compiled and loaded MIB and registered this SNMP Trap ID into SNMP Trap Configuration with SYS_SERVICE_DOWN Event. Furthere I do have "Generate new alarm" selected into Event Processing Policy Editor. In that policy I simply have "%m" message, "SERVICE_DOWN_%i_%2" in Alarm key, and Severity "From Event".

Now I cannot see my trap into either of "Event Browser", or "Syslog" or, "SNMP Traps log".

Please guide further and have a basic How-To on SNMP Trap Handling.

Hi,
   As we all know NetXMS doesnot support Port into SNMP and we end up setting NetSNMP as front-end. I spent almost 3 days to make a good NetSNMP setup for community based forwarding. For notes you must require 5.4 or higher for community based routing to remote.
   Now my real problem atart and before i'd like to explain my RADIUS (Remote Access Dial-In Server) cluster set-up

NetXMS-1.0.6 -> Net-SNMP-Proxy -> Firewall ---> RADIUS-1 (SNMP-Agent1-1:1161 and SNMP-Agent1-2:1162)
                                                                    RADIUS-2 (SNMP-Agent1-3:1161 and SNMP-Agent1-4:1162)
                                                                    RADIUS-3 (SNMP-Agent1-5:1161 and SNMP-Agent1-6:1162)

I created one node for "SNMP-Proxy" and a "RADIUS-cluster". I registered IP of all 3 RADIUS into "Object properties -> Resources" of RADIUS-Cluster. I created 3 nodes with-in RADIUS-cluster, where each node having "Community string" RADIUS-Auth-1, RADIUS-Auth-2, RADIUS-Auth-3 respectively. I disabled all pooling except "usage of SNMP for all pools" and "data collection" check-box.

Observation 1:
   Initially I defined SNMP OID at global RADIUS-cluster level and then from Net-SNMP log I see that request contains cluster level community anme "public

Observation 2:
   I removed cluster level SNMP setting and defined at individual node level only. Still i cannot see get-request with defined community names into net-snmp log.

   Please help me into setting up NetXMS.

P.S. When I start my WindowsXP without any network cable connected then NetXMS hangs my login/explorer. Once even I saw some "Illegal reference" related error as well. Please have a BUG-Fix.