How does Silverline work on my server?

A key characteristic of Silverline is that it runs 100% in user space and doesn’t require any modification to the underlying operating system or the applications under management. Silverline in total requires less than 10MB of system RAM and typically introduces less than 1% performance overhead.

Silverline (SL) is subdivided in to two main components, SL Library and SL Agent, which both also operate in user space.

SL Library is a Linux shared library named liblm.so. As shown in Figure 1, SL Library sits between an application (and any associated middleware such as a JVM) and the C standard library (libc). SL Library intercepts all OS system calls that are related to resource consumption such as read/write and brk/mmap.

Figure 1: SL Library

SL Agent is a policy-based, fair share scheduler that can provide system resource quotas for large numbers of applications running on an individual host. There is only one SL Agent running on each managed host but it interacts with many SL Libraries (for each application that is being managed).

In order to provide fine grained application control, SL libraries periodically exchange data with SL Agent to determine the optimal resource quota to give each application. Figure 2 shows a step-by-step description of the interaction between one SL Library and SL Agent.

Figure 2: Silverline Service Interaction

(A) Intercept resource consumption related system calls such as read(), write(), mmap() or brk()

(B) SL Library periodically communicates application resource usage to SL Agent

(C) SL Agent's scheduling algorithm takes the usage statistics it receives from all applications and combines them with business policies to generate an optimal resource quota for each individual application.

(D) SL Agent periodically communicates application resource quotas to SL Library

(E) SL Library controls application behavior by, for example, breaking up a single system call in to multiple system calls

To better understand how Silverline controls system resource usage, let’s take an example and assume that a policy has been set which limits App1 to a maximum of 100Mbytes per second of file reading and App1 makes a request to read a 200Mbyte file via the read() system call. Without Silverline a single read system call requesting 200Mbytes of data would be issued to the OS and for a period of time App1 would overuse storage I/O resources resulting in degraded performance of other applications on the same server. With Silverline, instead of a single read() system call, SL Library would break up the 200Mbyte request in to a series of properly spaced, smaller read() system calls. For example, a read call of 1Mbyte every 10ms resulting in the allocated 100MBytes per second. This same concept can be extrapolated to CPU, memory and network I/O to better understand how Silverline can guarantee pre-defined usage policies or SLAs across all applications.

Communication between SL Library and SL Agent occurs as often as hundreds of times per second and as a result, resource allocation can be dynamically rebalanced in real time based upon actual application demand. For example, if a high priority application called App1 has been assigned a guarantee or allocation of 80% of the CPU but is currently not using that much, SL Agent will allow another application, App2, to utilize the spare capacity. However, if App1 suddenly demands 80% of the CPU it will receive that much instantaneously.

In the event that SL Agent fails load management will continue, however there will no longer be any dynamic rebalancing. Using the previous example, App1 would be statically assigned 80% of the CPU whether it used it or not and App2 would be statically assigned 20% and would not be able to use more even if spare capacity was available. This situation would only occur for a brief period of time because SL Agent will be automatically restarted by a watchdog process within Silverline that is itself started by the Linux init process.

Finally, SL Agent aggregates collected statistics from all applications and every 60 seconds pushes them to the Silverline SaaS portal.

Next...

Read more Knowledge Base articles
Go back to the Librato.com home page
Or just START YOUR FREE TRIAL