NetXMS Support Forum

English Support => General Support => Topic started by: justrest on March 16, 2023, 09:32:16 AM

Title: How to discover all peer nodes completely?
Post by: justrest on March 16, 2023, 09:32:16 AM
Take an aggregation device as an example, it is down-connected to dozens of devices, all of which have LLDP and SNMP working normal and have added nodes in netxms, but regardless of the peer node column in the NIC list or in the network topology, only part of peer nodes can be found, and the debug log shows: ProcessLLDPConnectionEntry( :SNMP get failed for remote ID D494E8582820). Your help is urgently needed, thank you very much!
Title: Re: How to discover all peer nodes completely?
Post by: justrest on March 16, 2023, 09:48:08 AM
version 4.3.2 ,debian 11
Title: Re: How to discover all peer nodes completely?
Post by: justrest on March 16, 2023, 10:11:59 AM
2023.03.16 15:21:07.881 *D* [poll.topology      ] Started topology poll of node SW-HJ[241]
2023.03.16 15:21:07.942 *D* [poll.topology      ] Cannot retrieve VLAN list from node SW-HJ[241]
2023.03.16 15:21:10.998 *D* [topology.fdb      ] GetSwitchForwardingDatabase(SW-HJ[241]): 76 entries read from dot1qTpFdbTable
2023.03.16 15:21:11.001 *D* [topology.fdb      ] GetSwitchForwardingDatabase(SW-HJ[241]): 0 entries read from dot1dTpFdbTable
2023.03.16 15:21:11.002 *D* [poll.topology      ] Switch forwarding database retrieved for node SW-HJ[241]
2023.03.16 15:21:11.002 *D* [topology.lldp      ] Collecting LLDP topology information for node SW-HJ[241]
2023.03.16 15:21:25.630 *D* [topology.lldp      ] 550 entries in LLDP connection database for node SW-HJ[241]
2023.03.16 15:21:25.631 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID D494E8582820
2023.03.16 15:21:25.631 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 000E5EB9CF4F
2023.03.16 15:21:25.631 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 441AFACF4400
2023.03.16 15:21:25.631 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 441AFACF4400
2023.03.16 15:21:25.631 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 441AFACF4400
2023.03.16 15:21:25.631 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 441AFACF4400
2023.03.16 15:21:25.631 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID D494E8582820
2023.03.16 15:21:25.631 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 441AFACF9A00
2023.03.16 15:21:25.631 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 9CE895D9F420
2023.03.16 15:21:25.631 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID D494E8582820
2023.03.16 15:21:25.631 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID D494E8582820
2023.03.16 15:21:25.631 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 9CE895DA0B40
2023.03.16 15:21:25.631 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 9CE895DA0B40
2023.03.16 15:21:25.632 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 9CE895D9F420
2023.03.16 15:21:25.632 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 90E710C7DA68
2023.03.16 15:21:25.632 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 000E5EB9CF51
2023.03.16 15:21:25.632 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 74258ABEAA4D
2023.03.16 15:21:25.632 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 90E710C7C0A0
2023.03.16 15:21:25.632 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 90E710C7C0A0
2023.03.16 15:21:25.632 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 90E710C789A0
2023.03.16 15:21:25.632 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 3897D6B7DF24
2023.03.16 15:21:25.632 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 9CE895DA1A40
2023.03.16 15:21:25.632 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 441AFACF4200
2023.03.16 15:21:25.632 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 441AFACF4200
2023.03.16 15:21:25.632 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 7485C436A73C
2023.03.16 15:21:25.632 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 5CC999FFEEC3
2023.03.16 15:21:25.632 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 586AB153E7F6
2023.03.16 15:21:25.632 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 90E710C7F220
2023.03.16 15:21:25.632 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID B0F9634FA800
2023.03.16 15:21:25.632 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 703D155488DC
2023.03.16 15:21:25.632 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 48BD3D3B64F6
2023.03.16 15:21:25.632 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 703D155464EC
2023.03.16 15:21:25.633 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 70BAEFA16213
2023.03.16 15:21:25.633 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 441AFACF9600
2023.03.16 15:21:25.633 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 441AFACF9600
2023.03.16 15:21:25.633 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 9CE895DA0550
2023.03.16 15:21:25.633 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 04D7A51DF26A
2023.03.16 15:21:25.633 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 3897D6B7DF24
2023.03.16 15:21:25.633 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 9CE895DA1A40
2023.03.16 15:21:25.633 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 9CE895DA1A90
2023.03.16 15:21:25.633 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 90E710C7DA68
2023.03.16 15:21:25.633 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 9CE895D9F970
2023.03.16 15:21:25.633 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 5CC999FFEEC3
2023.03.16 15:21:25.633 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID B0F9634FA800
2023.03.16 15:21:25.633 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 90E710C79C30
2023.03.16 15:21:25.633 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 9CE895D9F330
2023.03.16 15:21:25.633 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 74258A14CE9F
2023.03.16 15:21:25.633 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 441AFACF9A00
2023.03.16 15:21:25.633 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID B0F9634BE400
2023.03.16 15:21:25.633 *D* [topology.lldp      ] ProcessLLDPConnectionEntry(SW-HJ[241]): SNMP get failed for remote ID 9CE895D9F330
2023.03.16 15:21:25.633 *D* [topology.lldp      ] Finished collecting LLDP topology information for node SW-HJ[241]
2023.03.16 15:21:25.634 *D* [topology.stp      ] Collecting STP topology information for node SW-HJ[241]
2023.03.16 15:21:26.248 *D* [topology.stp      ] Finished collecting STP topology information for node SW-HJ[241]
2023.03.16 15:21:26.248 *D* [poll.topology      ] Node::addHostConnections(SW-HJ[241]): FDB retrieved
2023.03.16 15:21:26.249 *D* [poll.topology      ] Link layer topology retrieved for node SW-HJ[241] (7 connections found)
2023.03.16 15:21:26.249 *D* [poll.topology      ] Link layer topology processed for node SW-HJ[241]
Title: Re: How to discover all peer nodes completely?
Post by: justrest on March 17, 2023, 10:18:12 AM
I compared the nodes with normal and abnormal LLDP discovery and found that the difference is whether the VLAN list can be obtained during topology discovery. To test, I installed a new netxms, added only test node, and set the value of custom attribute " snmp.driver " to "H3C". Although SNMP can get vlan information normally through either public OID or private OID of H3C, the topology discovery still prompts: unable to get vlan list, and there is no more information in the debug 5 log.

Very eager for your help, thank you!
Title: Re: How to discover all peer nodes completely?
Post by: justrest on March 19, 2023, 02:31:03 AM
I tried different versions from 4.2.461 to 4.3.2, both windows and debian systems. The result is that 4.2.461 does not have the above problem, 4.3.0-4.3.2 are unable to get the complete peer node list.
Title: Re: How to discover all peer nodes completely?
Post by: Victor Kirhenshtein on April 12, 2023, 10:15:44 AM
Hi!

It could be related to LLDPv2 MIB support that we have added in 4.3. Could you please check if device responds to OID .1.3.111.2.802.1.1.13.1.3.2.0? If yes, could you provide result of SNMP walk on .1.3.111.2.802.1.1.13.1.4.1.1 and .1.0.8802.1.1.2.1.4.1.1?

Best regards,
Victor
Title: Re: How to discover all peer nodes completely?
Post by: justrest on April 18, 2023, 09:13:21 AM
Thank you very much for your reply! 

I'm on a business trip and can't import the list, but I had a colleague try and the SNMP walk results for .1.3.111.2.802.1.1.13.1.4.1.1 and .1.0.8802.1.1.1.2.1.4.1.1 seems containing  all the neighbor information, just some information that shows up as garbled. The current codepage setting of snmp is CP936 (Chinese), could this problem be related to the codepage setting?
Title: Re: How to discover all peer nodes completely?
Post by: justrest on April 28, 2023, 03:20:06 AM
Quote from: Victor Kirhenshtein on April 12, 2023, 10:15:44 AMHi!
It could be related to LLDPv2 MIB support that we have added in 4.3. Could you please check if device responds to OID .1.3.111.2.802.1.1.13.1.3.2.0? If yes, could you provide result of SNMP walk on .1.3.111.2.802.1.1.13.1.4.1.1 and .1.0.8802.1.1.2.1.4.1.1?
Best regards,
Victor

Thank you very much for developing such an excellent network management software, could you please help to optimize the new version of LLDP protocol support, thank you very much!
Title: Re: How to discover all peer nodes completely?
Post by: Victor Kirhenshtein on April 28, 2023, 09:57:57 AM
Yes, it actually could be related to codepage. Depending on entry type, NetXMS server may require to search for peers using texts (interface descriptions for example), and if it is received incorrectly search will fail. To be able to diagnose it I likely will need exact walk outputs though. We can move discussion to private messages or Telegram if you don't like to share such data publicly.

Best regards,
Victor
Title: Re: How to discover all peer nodes completely?
Post by: Egert143 on April 28, 2023, 01:02:12 PM
I don't want to hijack a thread, but for example, Cisco C9200L-24P-4X switch peer discovery doesn't work 100%. Only other Cisco switches are discovered, but no other device directly connected to the switch. It broke with a newer version of NetXms, but I don't remember which version exactly.
Title: Re: How to discover all peer nodes completely?
Post by: justrest on May 02, 2023, 06:14:09 AM
Quote from: Victor Kirhenshtein on April 28, 2023, 09:57:57 AMYes, it actually could be related to codepage. Depending on entry type, NetXMS server may require to search for peers using texts (interface descriptions for example), and if it is received incorrectly search will fail. To be able to diagnose it I likely will need exact walk outputs though. We can move discussion to private messages or Telegram if you don't like to share such data publicly.
Best regards,
Victor
No, actually none of this information is considered private, it just seems like too much. Thank you very much for your reply, I've uploaded the relevant attachments, so please take the trouble to guide me, thank you very much!
Title: Re: How to discover all peer nodes completely?
Post by: justrest on May 02, 2023, 06:25:27 AM
Quote from: Victor Kirhenshtein on April 28, 2023, 09:57:57 AMYes, it actually could be related to codepage. Depending on entry type, NetXMS server may require to search for peers using texts (interface descriptions for example), and if it is received incorrectly search will fail. To be able to diagnose it I likely will need exact walk outputs though. We can move discussion to private messages or Telegram if you don't like to share such data publicly.
Best regards,
Victor
Currently the server configuration file netxmsd.conf as well as snmp.codepage and syslog.codepage are CP936, except that previously under version 4.2.461, the same configuration did not have this problem.
Title: Re: How to discover all peer nodes completely?
Post by: Victor Kirhenshtein on May 05, 2023, 05:03:08 PM
Hi,

I've found a bug introduced in 4.3 which may cause incorrect decoding of strings received via SNMP (SNMP codepage was ignored in some cases). Fix will be included into upcoming patch release 4.3.6.

Best regards,
Victor
Title: Re: How to discover all peer nodes completely?
Post by: justrest on May 06, 2023, 03:43:42 AM
Quote from: Victor Kirhenshtein on May 05, 2023, 05:03:08 PMHi,
I've found a bug introduced in 4.3 which may cause incorrect decoding of strings received via SNMP (SNMP codepage was ignored in some cases). Fix will be included into upcoming patch release 4.3.6.
Best regards,
Victor
thanks very much!!
Title: Re: How to discover all peer nodes completely?
Post by: justrest on May 09, 2023, 10:37:16 AM
Quote from: Victor Kirhenshtein on May 05, 2023, 05:03:08 PMHi,
I've found a bug introduced in 4.3 which may cause incorrect decoding of strings received via SNMP (SNMP codepage was ignored in some cases). Fix will be included into upcoming patch release 4.3.6.
Best regards,
Victor
I'm very excited to try the just released version 4.3.6, but the network device still can't seem to discover all neighboring devices ...... I uploaded the latest walk result of SNMP, please help me to see it, thank you very much!
Title: Re: How to discover all peer nodes completely?
Post by: Victor Kirhenshtein on May 09, 2023, 10:59:52 AM
Could you also provide server log file with debug level 6 for the duration of topology poll on that switch (i.e. set debug level to 6, go to Poll -> Topology on the switch node, set debug back)?
Title: Re: How to discover all peer nodes completely?
Post by: Victor Kirhenshtein on May 22, 2023, 07:01:04 PM
Hi,

I was able to reproduce and fix this issue (yet another bug in LLDPv2 MIB handling). Fix will be included into next patch release (we likely will publish it this week).

Best regards,
Victor
Title: Re: How to discover all peer nodes completely?
Post by: justrest on May 23, 2023, 05:17:23 AM
Thanks very much!
Title: Re: How to discover all peer nodes completely?
Post by: justrest on May 28, 2023, 07:57:39 AM
Quote from: justrest on May 09, 2023, 10:37:16 AM
Quote from: Victor Kirhenshtein on May 05, 2023, 05:03:08 PMHi,
I've found a bug introduced in 4.3 which may cause incorrect decoding of strings received via SNMP (SNMP codepage was ignored in some cases). Fix will be included into upcoming patch release 4.3.6.
Best regards,
Victor
I'm very excited to try the just released version 4.3.6, but the network device still can't seem to discover all neighboring devices ...... I uploaded the latest walk result of SNMP, please help me to see it, thank you very much!
Thank you so much for your great help! I tried the latest version 4.3.7 today and the problem with lldp v2 still seems to exist, please help me with the attached logs, much appreciated! Or is it possible to force the use of lldp v1 version? Looking forward to your reply!
Title: Re: How to discover all peer nodes completely?
Post by: justrest on July 11, 2023, 03:15:14 AM
Quote from: Victor Kirhenshtein on May 22, 2023, 07:01:04 PMHi,
I was able to reproduce and fix this issue (yet another bug in LLDPv2 MIB handling). Fix will be included into next patch release (we likely will publish it this week).
Best regards,
Victor
Since under 4.2.* version, all peer nodes can be discovered normally, and the topology is shown completely (more than 30 nodes), but after 4.3 version can only discover part of the nodes (about 7 nodes), and as the time becomes longer, only two nodes are left at last,  is  it possible to turn off the LLDP V2, and revert back to the recognition mechanism of 4.2.* version, or there is any way to solve the problem. Thank you very much!

Title: Re: How to discover all peer nodes completely?
Post by: Victor Kirhenshtein on July 14, 2023, 11:07:26 AM
I have made some changes in reading LLDP information in upcoming release 4.4.1. FIrst of all there were few bugs left in reading LLDP-V2 MIB. Also, now server will use both LLDP-MIB and LLDP-V2-MIB if supported by device, and LLDP-MIB takes precedence. This should solve discovery issues.

Best regards,
Victor
Title: Re: How to discover all peer nodes completely?
Post by: justrest on July 17, 2023, 06:00:34 AM
Quote from: Victor Kirhenshtein on July 14, 2023, 11:07:26 AMI have made some changes in reading LLDP information in upcoming release 4.4.1. FIrst of all there were few bugs left in reading LLDP-V2 MIB. Also, now server will use both LLDP-MIB and LLDP-V2-MIB if supported by device, and LLDP-MIB takes precedence. This should solve discovery issues.

Best regards,
Victor

Thank you very much!
Title: Re: How to discover all peer nodes completely?
Post by: justrest on July 19, 2023, 08:12:46 AM
Quote from: Victor Kirhenshtein on July 14, 2023, 11:07:26 AMI have made some changes in reading LLDP information in upcoming release 4.4.1. FIrst of all there were few bugs left in reading LLDP-V2 MIB. Also, now server will use both LLDP-MIB and LLDP-V2-MIB if supported by device, and LLDP-MIB takes precedence. This should solve discovery issues.

Best regards,
Victor

We've updated to version 4.4.1 and it looks like H3C's Neighborhood Discovery feature should be working fine, fantastic, thanks so much!