NetXMS Support Forum

Hi,

A small thing I've noticed in using network maps is that selecting a different label type via "Display objects as..." is stored locally, so a different user on a different machine may see objects on a map by their icon rather than e.g. small labels.

Problem with this is, in our setup, that the icons are a tad large, and the text below is unreadable due to the map background. Also the placement on our maps is done according to the small label display type.

Would it be possible to have a "Default label type" property or such on the map properties, where you could select how the objects are shown on the map for all users? To not break with current functionality, you could add a "let user select" option that uses the current locally stored setting.

Thanks in advance,

Kind regards,
bdefloo

Hi,

I noticed a small problem on a number of nodes that have the NetXMS agent v1.2.4 installed.
The Windows system event log shows messages like:
The description for Event ID 23 from source NetXMS Win32 Agent cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.

If the event originated on another computer, the display information had to be saved with the event.

The following information was included with the event:

Counter set B is empty, collector thread for that set will not start

the message resource is present but the message is not found in the string/message table
instead of
Counter set B is empty, collector thread for that set will not start

Some googling led me to the registry key
HKLM\System\CurrentControlSet\Services\Eventlog\system\NetXMS Win32 Agent\EventMessageFile
which I found set to "C:\NetXMS\bin" rather than "C:\NetXMS\bin\nxagentd.exe". Changing it and restarting Event Viewer solved the problem on that node.

Could this be a bug in the installer somewhere? I can't seem to pinpoint what version it exactly went wrong as I've upgraded through several versions of the agent, but I have just had the problem with a clean install of the x64 v1.2.4 agent on a Windows 2008 R2 server.

Kind regards,
bdefloo

Hi,

Number of database writers is already at 8.

I took a look in the history for the queue length DCI, and it appears it's often 0 with frequent spikes up to 20-40, which makes it look worse than it is in a line graph. There don't seem to be any more performance issues at the moment, so I'm going to let it run for a while to see if it's stable now.

Thanks for the help!

Kind regards,
Bastiaan

Hi,

Adding an index may indeed greatly reduce the issue. I'm not sure if using COUNT will make a difference, as it still needs to check all the records, instead of stopping after the first record found.

Just storing the last used object ID and always using a truly unique ID may be the safest approach, as they could also still be used in the event processing policy, any other log or even hardcoded in some random script, leading to unpredictable results for the user.

The high database writer queue length is also gone now, queue length is usually around 30-50 now with an occasional spike to 2000 which is gone by the next sample, when a large number of hourly DCI's come in.

Server also hasn't crashed since I reduced the event log size, so the memory access violations are probably somehow tied in with a large database writer queue.

Thanks for considering my suggestions and running an open source project, wouldn't have found this problem otherwise!

Hi,

In the progress of investigating why our NetXMS setup still crashes a few times a day I noticed the database writer queue got very high. It starts climbing about 30min after the crash, and goes up to about 400K to 750K before the server crashes due to a memory access violation in a random module.

Our NetXMS server is running on a Windows 2003 x86 server, with MSSQL 2008 Express. I noticed in the activity monitor that a particular query was taking over a minute to complete, and causing alot of disk read activity:
SELECT event_source FROM event_log WHERE event_source=50220

Searching in the code led me to the CleanDeletedObjects function in the housekeeping module of the server. The reason it's so expensive to run is that SQL Server has to run over all the records in the event log (in my case, about 8 million of them for the default 90 days) to check if that particular event_source is used somewhere, as its not in the index. Probably, some things can be optimized here.

First off, it's searching the object ID of an interface in the event log, while the event source is always a node ID, if I'm not mistaken. Could a filter be added based on the object_class field of the deleted_objects table?

Secondly, if a record does exist for that particular record, all the rest of the records are still processed. It would seem this can be resolved by using the EXISTS condition, which is triggered as soon as one record is found:
e.g. IF EXISTS (SELECT event_source FROM event_log WHERE event_source=50220) SELECT 1 ELSE SELECT 0
http://msdn.microsoft.com/en-us/library/ms188336.aspx
However, I'm not sure if this keyword is supported in all of NetXMS' supported DB environments.

Thirdly, I'm seeing the query repeating multiple times for the same object ID. Could it be timing out, and trying over and over to delete the same object? My deleted_objects table is at 2722 records, so it does seem like something's going wrong.

Whether this is related to our crashes I don't know, but it might be a good performance improvement for anyone with a sufficiently large environment for it to be a problem. I reduced my event log size to 14 days meanwhile to see if it alleviates some of the stress on the SQL server.

Hi,

You can check whether the policy was installed correctly by opening the file in the NetXMS\etc\nxagentd.conf.d folder on the side of the agent. You may need to restart the agent on the server before it is actually applied, though.

If there are errors in the policy, they will show up the agent's event log after the agent starts (on Windows, by default the system event log)

If you added new parameters or such, you can do a configuration poll on the node and see if they appear in the list of agent parameters.

As for reliability, I've been using policies for a month or two now, for Winperf counters, enabling log parsers and external parameters, and have had no real issues with them.

The only things I find lacking is that agents don't reload their configuration/restart automatically after installing policies, you can't move policies between policy groups yet, and that there's no policy for event logging yet.

Hi,

Just had another crash out of the blue. I disabled the large number of tests shortly after my last post, and NetXMS has been running stable since then, until now.

Again, same symptoms: First network services go to unknown and agents report "Communication session broken", and a while later, the netxmsd service crashes. Crash dump info is attached, netxmsd-9352-1351585156.mdmp is an empty file (0KB) so I didn't include it.

Hi,

Total number of objects: 14715
Number of monitored nodes: 848
Number of collectable DCIs: 7182

Config variables are attached.
Note that I did play around with the poller counts recently, but I can still reproduce the problem.

First, I do a walk on the desired OID in the MIB browser, this shows the results.

I then set debug to 9 and did a configuration poll:
[30-Oct-2012 10:14:53] [CLSN-0] Received message CMD_POLL_NODE
[30-Oct-2012 10:14:53] [CLSN-0] Sending message CMD_POLLING_INFO
[30-Oct-2012 10:14:53] [CLSN-0] Sending message CMD_POLLING_INFO
[30-Oct-2012 10:14:53] Starting configuration poll for node DCXAS041 (ID: 1225)
[30-Oct-2012 10:14:53] [CLSN-0] Sending message CMD_POLLING_INFO
[30-Oct-2012 10:14:53] ConfPoll(DCXAS041): checking for NetXMS agent Flags={00000010} DynamicFlags={00000400}
[30-Oct-2012 10:14:53] [CLSN-0] Sending message CMD_POLLING_INFO
[30-Oct-2012 10:14:53] ConfPoll(DCXAS041): checking for NetXMS agent - connecting
[30-Oct-2012 10:14:53] ConfPoll(DCXAS041): checking for NetXMS agent - connected
[30-Oct-2012 10:14:53] [CLSN-0] Sending message CMD_POLLING_INFO
[30-Oct-2012 10:14:53] [CLSN-0] Sending message CMD_POLLING_INFO
[30-Oct-2012 10:14:53] [CLSN-0] Sending message CMD_POLLING_INFO
[30-Oct-2012 10:14:53] [CLSN-0] Sending message CMD_POLLING_INFO
[30-Oct-2012 10:14:53] AgentConnection::getSupportedParameters(): RCC=0
[30-Oct-2012 10:14:53] AgentConnection::getSupportedParameters(): 151 parameters received from agent
[30-Oct-2012 10:14:53] AgentConnection::getSupportedParameters(): 2 tables received from agent
[30-Oct-2012 10:14:53] ConfPoll(DCXAS041): checking for NetXMS agent - finished
[30-Oct-2012 10:14:53] ConfPoll(DCXAS041): checking for CheckPoint SNMP on port 260
[30-Oct-2012 10:14:53] [CLSN-0] Sending message CMD_POLLING_INFO
[30-Oct-2012 10:14:53] Event::expandText(event=244C0F20 sourceObject=1225 template='Node capabilities changed (Old: %1; New: %2)' alarmMsg='(null)')
[30-Oct-2012 10:14:53] [CLSN-0] Sending message CMD_POLLING_INFO
[30-Oct-2012 10:14:53] EVENT 13 (ID:6396083 F:0x0000 S:0 TAG:"") FROM DCXAS041: Node capabilities changed (Old: 0x00002013; New: 0x00000012)
[30-Oct-2012 10:14:53] [CLSN-0] Sending message CMD_EVENTLOG_RECORDS
[30-Oct-2012 10:14:53] EPP: processing event 6396083
[30-Oct-2012 10:14:53] Node::updateInterfaceConfiguration(DCXAS041 [1225]): got 1 interfaces
[30-Oct-2012 10:14:53] Checking subnet bindings for node DCXAS041 [1225]
[30-Oct-2012 10:14:53] [CLSN-0] Sending message CMD_POLLING_INFO
[30-Oct-2012 10:14:53] [CLSN-0] Sending message CMD_POLLING_INFO
[30-Oct-2012 10:14:53] [CLSN-0] Sending message CMD_POLLING_INFO
[30-Oct-2012 10:14:53] [CLSN-0] Sending message CMD_POLLING_INFO
[30-Oct-2012 10:14:53] [CLSN-0] Sending message CMD_POLLING_INFO
[30-Oct-2012 10:14:53] Node::executeHookScript(DCXAS041 [1225]): hook script "Hook::ConfigurationPoll" not found
[30-Oct-2012 10:14:53] Finished configuration poll for node DCXAS041 (ID: 1225)
[30-Oct-2012 10:14:53] [CLSN-0] Sending message CMD_POLLING_INFO
[30-Oct-2012 10:14:53] [CLSN-0] Sending message CMD_OBJECT_UPDATE
Checked again with MIB browser, no results:
[30-Oct-2012 10:14:57] Node(DCXAS041)->GetItemFromSNMP(.1.3.6.1.4.1.232.6.2.6.8.1.4.0.2): dwResult=4

I can confirm this issue.

We were reading temperature via SNMP on the NetXMS server itsself since v1.0.x. After a number of upgrades up to 1.2.3, this still worked, until I added some unrelated external parameters to the agent configuration and did a configuration poll.

After a day or so it may get back on its feet, though, so it could be just manual polls.

Hi,

Seems like we're looking for the same thing:
https://www.netxms.org/forum/feature-requests/per-user-visibility-of-alarms/

The change of color/icon is due to an alarm being generated on the switch. If you make a category for those alarms, you could hide the other alarms for the network admin, so he only sees information useful to him.

Hi,

I've been having more crashes lately (4 in the last 24 hours).

Probably they're caused by a large amount of tests I added which poll an external parameter via the NetXMS server node. Maybe the crashes I've had before have the same root cause, maybe not. Either way, I sent the NetXMS crash dump files to [email protected]

Thanks in advance for any help you can offer!

Update:
I just checked the server logs. Only one of the crashes generated a crash dump, in the other 3 netxmsd just stopped working.
I also noticed the crash dump log refers to AgentPolicy::ModifyFromMessage, isn't this something that's called when changes are made to agent policies by users? Nobody was in NetXMS at the time.

Being able to select it would be the most flexible solution, I think. (without having to edit source code as user)

Useful options I can think of would be
1) Fold children open/closed
2) set focus to object details (and open it, if neccessary)
3) Open/set focus to data collection configuration
4) Open properties

Thanks!

Doesn't seem to be in the wiki yet, but there's a table at the end of the old user manual:
https://www.netxms.org/documentation/netxms_user_manual.pdf

Sounds great. By "extendable", you probably mean by editing the source code of the console?

The problem with Object Details is when a different window is in front of it; like a graph, configuration window or the alarm browser. The object browser is still visible to the left, but clicking in it won't seem to change anything.

I have the intuition that the object browser and object details pretty much belong together, similar to when you click a code file in the project explorer in Eclipse you expect the code editor for that file to come to the foreground.

One more small awkwardness I've noticed is working with alarm comments. Two ideas come to mind:
- a tooltip text when you hover over the comment field (or the entire row) in the alarm browser that show the last comment or such
- a single window for displaying the comments that goes along with the currently highlighted alarm, like object details with a selected object, instead of a tab for comments per alarm

Sorry for dumping all these ideas in one topic!

Sounds like a good suggestion.
I've found that there are a few things that could be more intuitive when it comes to (double)clicking in the console.
Most new users I've shown the console get stuck on these four:
1) doubleclicking containers to open them
2) doubleclicking nodes to go to the Object Details
3) doubleclicking network maps to open the map (rather than right click -> Open Map)
4) doubleclicking a DCI to go to a line graph

1 and 2 are a bit conflicting though, I guess, seeing nodes contain interfaces and containers have object details too.

While I'm at it, may I suggest something like control+double click to open Data Collection Configuration of a node? It would make my life a bit easier too