Hello
Currently running on version 4.2.395 and i noticed that some server performance metrics are high:
Server thread pool SYNCER: usage% - 100
Server thread pool POLLERS: usage% - 60
Server thread pool MAIN: usage% - 100*i looked at weekly history and values mostly didn't change, only pollers went to 100% at 2 times.
Show stats:
Objects: 42898
Monitored nodes: 6824
Collectible DCIs: 18999
Active alarms: 1673
Uptime: 23 days
Is this normal or should i adjust something?
Usage% is showing how many threads have been created in relation to the maximum number of threads. Maximum number of threads is configured in Configuration -> Server configuration, e.g. ThreadPool.Main.MaxSize.
So e.g. if you have 100 as ThreadPool.Main.MaxSize and at some point the system decided to launch all 100 threads, it does not necessarily mean that you have problems - may be these threads, while being at maximum number are still handling the job. Check statistics for "current load" and "average wait time" - this should give more information.
You can merely increase the MaxSize server configuration parameters in increase thread count.
POLLERS: current load (%): 44. Weekly average (min 6, max 202, avg 49).
DATACOLL: current load (%): 11. Weekly average (min 0, max 49, avg 9).
SYNCER: average wait time(ms): 117 Weekly average (min 39, max 6.419k, avg 198).
MAIN: average wait time(ms): 90.022k Weekly average (min 4.273k, max 117.547k, avg 59.684k).
config:
threadpool.poller.maxsize: 1200 (def 250)
threadpool.discovery.maxsize: 16 (def 64)
Threadpool.datacollector.maxsize: 1200 (def 250)
threadpool.syncer.maxsize: 4 (def 1)
How do these numbers look ?
Pollers ans Datacoll look ok, also it's worth to check the wait time for them.
Syncer is good.
Main is not good - it has average wait time of 59 seconds. Main thread is doing a lot of different things - executes commands on management server, communicates with proxies, calculates business service related stuff...
What are values of
ThreadPool.Main.MaxSize
ThreadPool.Main.BaseSize
?
Weekly:
Poller average wait: (min 0, max 3.677k, avg 6.427)
Datacoll average wait: (min 0, max 69, avg 0.011)
ThreadPool.Main
MaxSize: 256
BaseSize: 8
Egert
I'd suggest to upgrade to 4.3 if possible - it has improved algorithm for thread pool resizing. In any case you can try to increase min and max size for main thread pool, to say 64 (can be 16 for 4.3, it should expand it aggressively if needed) and 512.
Best regards,
Victor