Hi,
I've read a few posts on how to get this to work and nothing seems to work properly for discovery of a remote network through a proxy.
I have the proxy work, and am able to manually add remote nodes, but discovery doesn't seem to happen. I've tried active and passive discovery and they appear to get queued on the proxy agent but nothing gets discovered.
Any hints on where I should start looking first?
-Zeb
For active discovery do you specify particular proxy in "Active Discovery Targets"?
Do you have ping.nsm subagent enabled in proxy's configuration file?
Thanks Filipp,
Yes, I specify the specific proxy for the subnet I'm trying to scan in "Active Discovery Targets" and I do see a note on the agent log about a scan of the subnet being queued when I turn on more verbose logging.
I'm using the static agent, which seems to automatically enable a few subagents including ping. They are listed in the logs when nxagentd is loaded (running on CentOS 7):
2020.12.27 18:11:29.685 *I* [ ] Subagent "Linux" (static:LINUX) loaded successfully (version 3.6.254)
2020.12.27 18:11:29.685 *I* [ ] Subagent "DS18X20" (static:DS18X20) loaded successfully (version 3.6.254)
2020.12.27 18:11:29.685 *I* [ ] Subagent "ECS" (static:ECS) loaded successfully (version 3.6.254)
2020.12.27 18:11:29.685 *I* [ ] Subagent "FILEMGR" (static:FILEMGR) loaded successfully (version 3.6.254)
2020.12.27 18:11:29.686 *I* [ ] Subagent "LOGWATCH" (static:LOGWATCH) loaded successfully (version 3.6.254)
2020.12.27 18:11:29.686 *I* [ ] Subagent "PING" (static:PING) loaded successfully (version 3.6.254)
2020.12.27 18:11:29.686 *I* [ ] Subagent "PORTCHECK" (static:PORTCHECK) loaded successfully (version 3.6.254)
2020.12.27 18:11:29.686 *I* [ ] Subagent "UPS" (static:UPS) loaded successfully (version 3.6.254)
-Zeb
Can you please provide agent log on debug level 7 (after initiating active discovery range scan)?
Best regards,
Victor
I've attached a log for a 5-ish minute period and did manual scan for 2 separate single-ip ip ranges (192.168.223.1 and 192.168.223.106). Both of these devices have SNMP enabled and respond to pings.
The agent does seem to initiate a ping scan and does find these 2 devices responsive, for example the first IP:
2020.12.28 13:03:20.379 *D* [comm.cs.7 ] Requesting list "ICMP.ScanRange(192.168.223.1,192.168.223.1)"
2020.12.28 13:03:20.379 *D* [sa.ping ] ScanAddressRange: scanning 192.168.223.1 - 192.168.223.1
2020.12.28 13:03:20.380 *D* [sa.ping ] ScanAddressRange: got response from 192.168.223.1
Before I forget to ask, are passive discoveries possible through a proxy as well?
-Zeb
Quote from: Zebble on December 28, 2020, 08:14:28 PM
Before I forget to ask, are passive discoveries possible through a proxy as well?
Yes, but you have to enable SNMP proxy in agent (by adding EnableSNMPProxy = yes to core section) and you must use zoning, with this agent set as zone proxy.
Best regards,
Victor
Thanks Victor,
That's what I assumed and I had that already enabled, so that's great! Zoning is enabled and the proxy agent is in the new Zone.
For good measure, I just compiled the latest version of the agent instead of using the static agent. Seems to be loading fine, and I started another scan with the same results. It starts and gets ping responses, but nothing shows up on the NetXMS server and I don't see any SNMP queries happening (I'm watching snmp port 161 on the NetXMS proxy agent).
Here's my config of the proxy agent if it helps:
#
# NetXMS agent configuration file
# Created by agent installer at Sat Dec 26 14:47:06 2020
#
ServerConnection = <netxms server public IP>
MasterServers = <netxms server public IP>
ConfigIncludeDir = /etc/nxagentd.conf.d
LogFile = /var/log/nxagentd
FileStore = /opt/nxagentd
SubAgent = linux.nsm
SubAgent = ecs.nsm
SubAgent = filemgr.nsm
SubAgent = ping.nsm
SubAgent = logwatch.nsm
SubAgent = mqtt.nsm
SubAgent = netsvc.nsm
SubAgent = portcheck.nsm
SubAgent = ssh.nsm
SubAgent = mqtt.nsm
#SubAgent = winperf.nsm
#SubAgent = wmi.nsm
#SubAgent = ups.nsm
EnableProxy = yes
EnableSNMPProxy = yes
EnableSNMPTrapProxy = yes
EnableSyslogProxy = yes
EnableTCPProxy = yes
ZoneUIN = 1
ZoneId = 1
Thanks again!
-Zeb
Devices have started to appear! Not sure what I changed so I'm going to let it "settle", delete the devices and try again to make sure it wasn't a fluke.
Will keep this thread posted.
I started from fresh. Empty database on the server.
Only thing I did after startup was EnableZoning = 1 and then restarted the server.
Added a new Zone, restarted the agent in the new zone, re-bound and created the node and everything looks good.
I did a manual full configuration poll on the proxy agent for good measure.
I enabled passive discovery only, no filter, and the agent where the server is located started added nodes fairly quickly.
The proxy node didn't seem to do anything after an hour or so, so I enabled active and passive and let it sit for a couple more hours, nothing.
I added the subnet manually in the discovery for the proxy network, and did a manual scan. I see the ping attempts, but no SNMP and nothing gets added.
I then added the proxy subnet in the "Address Filters" and turn on all 3 options for the "Automatically generated script with following rules". Within seconds the proxy agent started doing a ping scan and SNMP connections. Within minutes there were nodes appearing on the server.
So, it seems like:
- A non-proxy node starts using passive discovery by simply enabling "Passive only" in the discovery settings. All other settings are default.
- A proxy node seems to only scan when there is a matching Address Filter in the discovery settings. I'm unsure if "No filtering" and/or removing the "Active Discovery Settings" and/or turning on Passive only again after adding the Address Filter will get things going. Will likely try that at some point.
I'm now building the latest version from source (3.7) to see if there is any difference.
Does any of this make sense?
-Zeb
Just installed the latest 3.7/4.0 build. No change in behavior.
I'm leaving it in Passive discovery mode and have added the main router at the proxied Zone manually in hopes the additional ARP tables might kick start a discovery. So far, nothing.
Let's see if anything changes in the morning, and if I have to force adding the proxied Zone's IP address space to Active discovery and address filters.
Anything here ringing any bells?
Full disclosure, we're an MSP and have been using Auvik. We find Auvik is a "one trick pony" with really good visuals and discovery but that's it. We don't really use it for any real day-to-day work other than figuring out what's at new clients. NetXMS discovery seems very similar but much more in-depth and not quite as automatic as Auvik. We would love to replace Auvik with NetXMS to give us something that looks like it will be a lot more useful in the long run.
Thanks for such a great tool!
-
Hi!
I will try to reproduce this on my test system tomorrow - but it definitely looks wrong. Passive discovery should go in a same way as for local zone. Can proxy agent read local ARP cache and is there anything to be used for discovery? You can check that by selecting "Execute server script" on proxy node in UI and running the following script:
for(line : $node->readAgentList("Net.ArpCache"))
println(line);
It should print some lines with ARP entries.
Am I understood correctly that proxy node is in zone 1, not in default zone (it is how it should be)?
Best regards,
Victor
Thanks Victor!
Running that script on the proxy node produces a long list of MACs with IP addresses in the correct subnet.
The proxy node for Zone 1 is in Zone 1. I assumed this was the correct way to configure it when the proxy uses a Tunnel with Certificates?
-Zeb
Hi,
I just created clean system, added new zone, single node as proxy in it, and passive discovery works as expected. Below are screenshots from my configuration (and you can see object tree in zone "Test" being populated with devices).
Could you please make clean system again, set debug level to 6 on both agent and server, add only proxy node, enable passive discovery, and send me server an agent logs?
Best regards,
Victor
Thanks Victor,
I will give this a try shortly. I'm using an agent to server tunnel with certificates for Zone 1. Do you think that might make a difference?
I've attached the requested logs.
I tried a non-tunnel connection to the proxy agent and still the same problems.
-Zeb
Hi,
from server log it looks like server considers all nodes discovered from ARP cache as unreachable. Just in case - are you running proxy agent as root? Also, are you sure that devices in remote zone accept SNMP requests from proxy agent IP address and that community strings are correct?
Best regards,
Victor
Thanks Victor,
- running proxy agent as root
- devices in remote zone are accepting SNMP requests for anything in the local subnet (public community)
What did you see in the log that indicated this was a problem? I didn't see anything that specific...
-Zeb
Actually I didn't immediately notice that those addresses are in zone 0. Lines like those:
2020.12.30 12:46:06.029 *D* [snmp.discovery ] SnmpCheckCommSettings(192.168.1.5): failed
2020.12.30 12:46:06.030 *D* [obj.poll.node ] AcceptNewNode(192.168.1.5): host is not reachable
I'll double check log tomorrow.
Best regards,
Victor
Just did a fresh install of the newly released 3.7.95 and am still seeing the same behaviour.
Is there any way to check what's in the node discovery queue? I've tried to use what I thought were relevant server commands but the lists were empty. "Show queues" does show that there are just under 300 items in the node poller queue...
Is there any way to prevent discovery in specific zones? I assume this can be done through a filter script. I'll have to dig, but would be good to solve the discovery issue first.
FYI, the server console from the Tools menu seems to have disappeared in 3.7.95. I'm using nxcmd instead.
Thanks again.
-Zeb
Regarding console from Tools menu - check "Access server console" user right.
Quote from: Zebble on January 05, 2021, 05:54:10 AM
FYI, the server console from the Tools menu seems to have disappeared in 3.7.95. I'm using nxcmd instead.
Make sure you are using Java 11 - it seems that this plugin is incompatible with Java 8.
Best regards,
Victor
Thanks to Victor's help, all this required was some tuning on the NetXMS server. Turning on parallel processing and increasing the Discovery.BaseSize from 1 to 8 seems to have helped immensely. The root cause seems to be the large number of subnets we had on our Default zone. It couldn't keep up with the discovery process for subsequent zones.
I am now having an issue with discovery filters. We want to only add devices that have SNMP enabled or the Agent installed. However, when we turn these filters on in 3.7.116, nothing gets added. If I restrict it to active scan only on a single test IP, and then turn off all filters, the device with agent gets added as expected.
If I check debug logs on the proxy agent, it does ping the device but never seems to try to connect via SNMP or Agent.
What it looks like is that if the filter is set to SNMP or Agent only, it only attempts a ping scan and since it hasn't seen it so far as an SNMP or Agent device, it doesn't bother scanning (a Catch 22).
-Zeb