Syslog from source <unknown>

Started by Dani@M3T, July 07, 2014, 07:47:04 PM

Previous topic - Next topic

Dani@M3T

We just changed our internet gateway (same manufacturer but a bigger device model). In NetXMS I left the existing node, changed the node name and started configuration-, interface-name- and topology poll. All DCIs are still ok.
But the syslog messages from the new gateway arrive at the NetXMS-syslog-server with source <unknown>. The new gateway has the same IP address as the old. Maybe the gateway uses a wrong source interface to send the syslog messages. Is there an easy way to see the source IP address of the syslog messages in NetXMS server? As long as I know NetXMS is mapping incoming syslog messages by source IP adress to existing nodes. Right? On the other side SNMP Traps are working, source node is ok there. Or does anybody has an idea what could be wrong? Thank's in advance!

Addition:
I checked the source IP address of the syslog packets with wireshark. All syslog packets arrives with the primary IP address of this node. So the cause must be something else.

Victor Kirhenshtein

Hi!

Can you show me few examples of syslog packets? Server try to find source object using "host" part of syslog message first - maybe this is the problem.

Best regards,
Victor

Dani@M3T

Hi Victor

If the 'host' field is the part to map I see the problem. I have attached two printscreens. The old node name was 'fw1' the new name is 'gw1'. I can see the old device has 'fw1' in the host part, the new one has in the host part '2014' and the name is in 'tag'. Hmm looks like a bug from ZyXEL. I can choose 'CEF/Syslog' or 'VRPT/Syslog' format on the device but that makes no difference in the 'host'- and 'tag'-fields.

Victor Kirhenshtein

It could be. Can you provide full syslog packet dump (with original text)? Probably I can tweak syslog parser so it will understand Zyxel variant correctly (we already done that for Cisco for example).

Best regards,
Victor

Dani@M3T

Hi Victor

That behaviour seems to be new. I didn't had this on older ZyXEL devices. I have attached a tcpdump.

Thanks
Dani

Victor Kirhenshtein

Hi,

this message format dows not conform neither to BSD syslog format (RFC 3164) nor RFC 5424. The main problem is that 2014 as host name is a valid IP address (it's not widely known nor used, but it is valid to write IP addresses not only as n.n.n.n, but also as n, n.n, and n.n.n) so it resolves to 0.0.7.222, which of course is not found. I change node binding logic so that if server cannot find node by hostname/IP address provided in syslog message itself it still try source IP address of the message. This should solve your problem.

Best regards,
Victor

Dani@M3T

#6
Hi Victor

Thanks a lot for your help. That would be very good for me. But I will report this to ZyXEL as well. But I don't know with success or not ;-)
Should I stop the logs from this device for the moment. Can you integrate this change in V1.2.15?


Dani

Dani@M3T

Victor

Can you realize this change in V1.2.15?

Victor Kirhenshtein

Hi,

change already done, so it will be part of 1.2.15 release.

Best regards,
Victor

Dani@M3T

I just tested in V1.2.15, works perfect, thank you.

Dani@M3T

#10
New problem with this feature:

Here the situation:
NetXMS -- GW1.DOMAIN1.XX -- VPN -- GW1.DOMAIN2.XX

Both 'GW1' send syslog to NetXMS server. GW1.DOMAIN1.XX ist the ZyXEL device which sends the syslog messages with wrong 'host'-field (where the new feature in V1.2.15 matches by source IP address). GW1.DOMAIN2.XX is newly connected. This Device sends correctly 'gw1' in 'host'-filed. Now the NetXMS server shows GW1.DOMAIN1.XX as source!! So we have all syslog messages from both nodes on 1 node. Big ghetto.

By the way I opened a support case at ZyXEL for the wrong 'host'-field with no success yet :-(

Edit:
I have two other devices from the same type as GW1.DOMAIN2.XX: FW1.DOMAIN3.XX and FW1.DOMAIN4.XX. Here the syslog message source in NetXMS is correct. Thirst I thought the same host-name (but different domain) is the problem. But this is refute with this second test.


Victor Kirhenshtein

Maybe idea to match messages based on hostname field was initially wrong and we should just revert to matching by source IP (or at least give IP based matching higher priority)?

Best regards,
Victor

Dani@M3T

Thanks for your reply.
For me matching by source IP would be the most logical. But I don't know if there could be a problem with nodes with multiple IP addresses or in other cases I don't know. But when the 'host'-field has priority now, why is there no problem in my second case (see 'edit' in my posting)? There are two nodes with the same host name (but different domain of course).
How fast would a solution be possible?

Dani@M3T

Hi Victor

Sorry for my impatience. But would it be possible to fix/change this before V1.2.17?

thanks
Dani

Victor Kirhenshtein

Hi,

I think yes, it should be very simple change.

Best regards,
Victor