Agent requirements.

Started by chris.petree, June 22, 2017, 10:42:08 PM

Previous topic - Next topic

chris.petree

So, the push agents installed on the clients establish a tunnel with the server. Is there a constant bandwidth demand on the server side per client? Does it make intermittent pushes? How exactly would I calculate the load on the firewall/server?

Tursiops

Are you asking about the push DCIs or the standard agents that establish active tunnels to the server?

The latter establishes an SSL tunnel on startup and from what I can tell whenever the tunnel breaks down. The agent does not push anything in this setup, the server initiates all queries, except it sends the requests via the established tunnel rather than directly to the agent's connection hostname/IP. So from a bandwidth perspective, it should be roughly the same as before + the SSL tunnel overhead, which I'd personally consider negligible. If the tunnel overhead brings your connection to its knees, it probably should have been upgraded a while ago.

I can't comment on the load too much, as we are currently only running around 100 active agent tunnels and the overall load doesn't really seem to have increased.

However, the load data is not very accurate, as our server has been crashing a lot since we upgraded to 2.1-RC1 and started using active agent tunnels. Apparently there was an issue here, which has been fixed in the development branch. With the full 2.1 release expected soon, I'd wait with setting up active agent tunnels until then.

chris.petree

We have a pfSense server as our firewall for our data center. That operates three data servers, two report servers, and three website servers with a total of seven websites. I think our bandwidth is 50mb up and 50mb down. I'm not planning on pushing this until the stable release. I was just doing prep work for it. We do have something like 120 PCs currently that we're wanting to set up these agent tunnels. By next year we'll probably have added over 300-400 more, and in two years, the ideal plan is probably several thousand.
If these tunnels are anything like IPSEC tunnels then each bit of communication has an added (more or less) 100 bytes of data. A standard of SNMP packet sizes are 1500 bytes. So if the agent matches these and is doing a constant push, opposed to a heartbeat, you can guess roughly 1600 bytes per unit. At 2000 units this would be a CONSTANT 32 megabyte draw on the bandwidth.
So, it all may sound negligible, but it adds up.

Tursiops

Hi,

As I said, the agent initiates the tunnel, but the server does the polling.
How often the server queries is up to you and your server (status/configuration/instance discovery polls) and DCI configuration.
If you decide to poll every item every second, your bandwidth (and server) requirements will indeed be very high.

The negligible point was in comparison to the bandwidth that would already be required for data collection (without the tunnel).

I am not quite sure in regards to your "constant 32 Megabyte draw" calculation, but our figures may be of help to you:
We have around 2500 nodes in our system (probably a quarter of which are agents, 100 of which are configured as agent tunnels; the rest is network equipment; once agent tunnels are stable the number of overall nodes will double) with over 60,000 DCIs. We do not poll all DCIs every second. Based on the expected change rate, we poll some things daily (e.g. serial numbers), others every hour, every five minutes or every minute. There are very few items we check every second (maybe one or two dozen) - we usually reserve this for live troubleshooting purposes.
The network gear sends through syslog data and SNMP traps (this is indeed pushed as it hits the agent, as opposed to DCI polls).
With the above setup we're at roughly 5 Megabit per second.

Cheers

Victor Kirhenshtein

Hi,

just to comment a bit on push/pull - you actually can switch agent into push mode by turning on agent cache. When cache is on, server will upload data collection configuration to the agent, and agent will do data collection according to schedule and push collected values to the server (or keep them in local cache if server is disconnected). This will not change bandwidth requirements though (besides few bytes on server poll requests).

Best regards,
Victor

chris.petree

Thanks! This all answers what I needed perfectly!