Goals and benefits
The availability of endless scalable resource at low cost is a dream come true. Instantly available these technologies help companies to get even more flexible by accelerating projects and time-to-market.
The concepts of shared resource and virtualization of servers offered by cloud providers are at the heart of this revolution.
However, sharing resources can have downsides as it creates interdependencies. Virtual machines share the resources of the hardware-server they are placed on. So typically there might be five or even ten virtual servers running simultaniously on the hardware.
If one of them demands a high processor performance, the performance of the other servers might drop even below what the customers thought to have purchased. While the effect is inherent, it is manageable and a high-quality provider would detect the variation and be able to minimize it.
So, how des your provider go about it?
Why provider performance should be monitored
Customers can not see beyond their own virtual machines, and the insight in the technical capabilities of a provider is limited.
This leaves open questions for companies wanting to use these services when it comes to purchase decisions or even daily operation. There is little evidence for fact based decisions on the quality of the service received or for comparison of different offers available.
We at ASCAMSO believe, that monitoring the performance is the only way to ensure optimized investment when switching to the pay-per-use models of cloud offerings.
The reason is simple: there is (yet) no fixed unit for computing power defined in virtual environments. To give some orientation to customers, providers use analogies like "number of cores" to vaguely describe the speed of a CPU in a VM.
But even identical specifications of CPU cores, RAM and storage might perform quite differently depending on the hardware, the configuration and the virtualization strategy of the provider.
As already discussed, these effects are beyond visibility of the customers, but their impact can be detected.
The ASCAMSO platform offers a powerful and easy testing method for virtual environments. The performance tests are designed in a way that even people with minor technical knowledge are enabled to do ad-hoc or permanent monitoring of virtual environments.
The monitoring provides transparency throughout the cloud-lifecycle:
- detect and understand variation in performance
- identify the best configuration for specific applications
- establish and monitor an effective cloud purchasing policy
- measure the performance against agreed SLAs
- compare provider performance / offers
Measuring performance in virtual environments is not trivial, especially as you do not want to interfere with the service provided while measuring the system. Read more on challenges, limitations and the solutions we found for measuring the performance of virtual resource.
Challenges to performance management in shared environments
The performance of complex computer systems depends highly on the interaction of the existing components. Changes in the state of the system will influence the performance parameters.
A high wokload for instance might bring certain system components to their limit, affecting the system as a whole.
It is therefore essential that systems are measured under exactly specified and reproducible loads and conditions. Tests with wavering system utilization are not reproducible and therefore not reliable.
Only a standardized test setting allows comparison between configurations or providers.
The real challenge is that the testload is put on the systems in parallel to the current workload, but must not interfere with it to truly reflect the usage and performance patterns of the cloud resource and avoid biases.
To create standardized and comparable tests for virtual systems, the ASCAMSO test suite uses load generators that apply a constant and defined load. The test load brought onto the system is known and can even be customized according to the use case.
It focusses on the hard- and software components that define virtual machines in Iaas environments:
- CPU-processing speed
- Memory usage
- Network throughput
The ASCAMSO platform allows to specify these tests in a few steps. The tests are carried out by the ASCAMSO Satellites which are software agents capable of performing defined tasks across operating systems, configurations and chipsets.
These are placed on the virtual machines and linked to so called scenarios, that manage runtime and duration of the satellites test runs according to your requirements. Scenarios keep connected to running satellites and receive live-data from them. The data can be viewed and analyzed right away. It is stored for future reference or comparsion of test runs.
Going further you can match the results monitored against ASCAMSOS provider & tariff database to instantly receive product and provider suggestions based on the performance measured. Suddenly the cloud market becomes transparent as you can compare your spendings against current market offers with low effort and on a regular basis.