NikCreate

As a DevOps engineer who started his career working for companies using AWS and ended up working across the three major cloud providers, it’s been interesting to realize how easily knowledge of one cloud translates to another and how most cloud services are the way they are today based on older technologies which came before them. Open source applications become managed AWS services and sometimes services originating in the cloud are released as open source tools later on. While just about every cloud service can be traced back to a pre-cloud predecessor, today I want to look at the origins of compute-related services and see how they’ve evolved.

What is Compute?

Compute is the backbone of any deployment or cloud service. It refers to the processing power required to run applications or workloads. It allows any operation or calculation to be performed and is necessary to complete any task or keep a process running. For a local app, it may be the power of your laptop and for a company, its machines in your data center. For cloud services, you’re viewing abstractions that provide access to resources in the provider’s data center, ranging from vitrtual machines, which act similar to a personal computer, to serverless functions where you provide the code and some capacity settings and the cloud provider handles the rest.

Starting from Bare Metal

For most purposes related to app deployment, compute will ultimately be provided by physical servers running processes. Physical servers are often referred to as “bare metal” and the amount of compute power provided is based on the specs of the computer. The CPU is most obvious as the processing unit but RAM and storage are important for being able to save information and network traffic can create its own bottleneck as well.

The processor, memory, and disk are all physical devices connected to the motherboard of the computer and these all play an important role in determining a server’s capabilities. Special software called firmware is installed on the hardware component and provides low level controls to the hardware along with the capacity to load an Operating System. If you’ve ever opened BIOS or EUFI to debug an issue with a personal computer, that is an example of firmware. The Operating System sits on top of the firmware and allows you to run applications and processes while controlling and accessing the low level resources under the hood. For a personal computer, your OS is likely Windows or Mac, though when it comes to web servers and cloud resources, it’s often Linux.

While the hardware sets limits on a machine’s compute power, the firmware and operating system are also important for being able to best utilize the physical resources. Using better algorithms can get the same processes done using less time and resources, and optimizations are important for both operating systems and the applications that run them.

And that’s the bare metal server, a physical machine running an operating system that users can run apps and processes on. Its physical resources create a limit, though the software should still run efficiently to best utilize it. With this you could run some processes or a web server with some limits as to its use, but to scale you would need to buy and manage more physical machines or add components to the one you have.

Virtual Machines: The First Abstraction

The truth of the matter is that ultimately, everything is run on a bare metal server. You just don’t necessarily have access to the servers themselves and are provided access through other apps and services. Large companies would just accrue lots of servers and create large data centers to host their services and virtualization tools were created in part to help them better manage their resources and give users and applications isolated environments in the same physical resources, a concept dating back to the 1960’s with the IBM CP-40.

Basically, a kind of software or firmware called a hypervisor allows virtual machines to be spun up, providing virtual resources to the VMs from the host’s physical resources. If installed directly on the physical hardware, it’s called a bare-metal hypervisor, if it runs on another OS, it’s called a hosted hypervisor. Virtual machines give the appearance of being a separate server while providing the ability to create abstractions from smaller portions of a host machine.

Virtual Machines also provide security and isolation and a user (or process in one) isn’t supposed to be able to break out of its host. Snapshots of a VM can also be created and spun back up in a specific state. This can help save a lot of setup work if creating multiple VMs from the same image for different users or make restoring a system easier and many of the practices from VMs would be copied over into later abstractions. Let’s take a look at notable Virtualization Software and Virtual Machines to see what the main VM-based compute services offered pre-cloud and on EC2, GCP and Azure are.

Virtual Machines Before Cloud – VMWare’s vSphere Suite

Before the Cloud, companies needed to manage VMs on servers in their own data center. One popular option for virtualization software was VMWare’s vSphere suite including their bare-metal hypervisor ESXi and controller vCenter. A company could buy their own servers and instead of installing Linux or Windows, they’d install something like VMWare ESXi as the base operating system on top of the server’s firmware. They would set up a separate vCenter and have it communicate with the ESXi server for visibility into all their servers. Once connected, they could use the vSphere client to manage their virtualized infrastructure in a convenient UI. Before cloud computing became the talk of most companies, VMWare and competitors like Microsoft Hyper-V for Windows Server, Oracle VM or Citrix XenServer were used for managing servers to create VMs and assist with clustering applications.

Cloud

Client’s like VSphere, which showed an appealing management tool to get a glimpse into their virtualized environment, probably sound familiar to anyone who has seen a view of their resources in a Cloud Platform’s console today. Simply go the the website of your Cloud Provider of choice (AWS, GCP, Azure, etc), go to their compute service and spin up a VM with your size and customization options. You can customize security, storage devices, permissions and more and simply delete the VM when you done. You don’t have to even think about physical servers, and this is before we get to what’s advertised as “serverless.”

Let’s take a quick look at the three major US-based Cloud Providers’ VM offerings. Amazon Web Services provides EC2 (Elastic Compute Cloud), Microsoft Azure’s Azure VMs and Google Cloud’s Compute Engine. These services all let you spin up a VM according to your desired configurations.

AWS EC2

Amazon’s VM service is called EC2 or Elastic Compute Cloud and offers the user the option to choose to run different operating systems on different instance types. There are different families of instances, like the C family which is optimized for compute, the R and X families optimized for memory, and D, H, and I which are optimized for storage. P and G instances are GPU based and useful for AI training or graphics intensive applications. H instances are optimized for high performance computing. M and T are the most commonly used, being general purpose and new instances types are announced fairly regularly.

Within an instance family, there are a variety of different instance types and sizes. For instance, within the T family, there are T2, T3 and T3a instance types as of this writing. Each of these instance types comes in different sizes. For instance, you can choose t3.nano with 2 vCPU and 0.5 GiB, t3.2xlarge with 8 vCPU and 32 GiB and various iterations in between. Different instance types have different specs for their varying sizes, such as a t2.nano only having 1 vCPU.

Different instance types have different features, such as the processor they use, whether they provide instance store or require you to use Elastic Block Storage (EBS) volumes for long term storage, custom networking enhancements and thresholds and scalability. T2 and T3 instance utilize various Intel CPUs such ask Sky Lake, Cascade Lake and Sapphire Rapids, T3a use AMD EPYC 7571 and T4g makes use of AWS’s Graviton 2 processors. Generally the newer instance types tend to make more use use of custom chips AWS has recently announced more of their own chips such as Graviton, Trainium and Inferentia which are AI accelerators (similar to GPUs) and likely more by the time your reading this. The T family also requires using EBS which separates storage from instance type and AWS offers them with “burstable” CPU capacity, where they allowed to access more resources from the host than allotted to the VM if needed based on a credit system.

For this article I won’t go too in depth on the different instance types, but beyond this, they offer support for attaching IAM roles to allow processes running on the instance access to specified AWS services without configuring credentials on the instance. There’s support for selecting your operating system and architecture, specifying a specific VM image called an AMI to load a custom VM (your own or bought from AWS’s marketplace) and for creating AMI’s and snapshots for your instances. You can also launch them in specific network configurations, such as public subnets for easy accessibility, private for internal services not open to the internet, or placement groups if multiple resources need to be in certain proximity in physical data center (with certain instance types offering enhanced networking for those configuration). All in all, you have a lot of options to choose from for your virtual servers.

Azure VMs
Azure’s Virtual Machine service is just called Azure VMs and there are a lot of similarities. Choose between Windows or Linux on a variety of instance types divided into instance series. A and B series are for dev and test workloads with B being burstable and based on Intel’s Ice Lake or AMD EPYC 7763. D is general purpose production work loads, E is for in memory workloads like SAP HANA, F is compute-optimized, G is is for memory and storage (database, SAP, ERP), H is high performance compute, L is storage optimized, M is memory optimized, and N is GPU focused using a variety of NVIDIA chips for GPUs. The instances make use of a variety of Intel and AMD chips as the MAIA AI accelerator and Azure Cobalt chips announced this year at Microsoft Ignite are Microsoft’s first entry into the custom chip space.

Ultimately, it’s quite similar. You’ll find a lot of predefined instance sizes to choose from in different regions at different costs and can save and deploy your own images as well.

Google Compute Engine

Google doesn’t change things up too much. Its predefined instance’s are divided into 4 families, general purpose, compute optimized, memory optimized and GPU. General purpose has E, N, C, and Tau instances increasing in scale which they can service with E being dev/test, N low traffic, C high performance and Tau to scale. Compute optimized has H and C series for high performance computing. Memory optimized has M1 to M3 based on different Intel chips. Accelerator Optimized chips largely use Cascade Lakes with NVIDIA GPUs attached. Closer to Amazon, Google also offers their custom Intel Infrastructure Processing Unit on some of their general purpose and compute optimized work loads, offering the same AMD and Intel chips as before as alternatives as well.

GCE has basically the same options as the other clouds with the key distinguishing factor being that for any of the families, you can choose to use a custom machine type, where you can choose the number of vCPUs and how much memory to assign to the instance per CPU which can help to create the exact size of instance you want rather than looking for a matching instance type.

In Summary

The different Cloud Providers have similar offerings when it comes to VMs. All offer Virtual Machines based on the same basic Intel and AMD chips though at the time of writing this, there seems to be a growing focus on using custom hardware to differentiate. Perhaps one of the biggest distinguishers would be additional support for instances to have permissions to access services in their cloud providers using their respective IAM services. Besides that, all of them offer options for different operating systems, architectures and networking capabilities as well as options for burstable instances and cost discounts for running spot/pre-emptable VMs which can be bought at a discount but interrupted if needed by another user, or reserving instances and committing to long term use. As prices can easily change, I won’t get into a price comparison here. Burstable is a neat example here as it’s automating the ability for the VM to gain access to resources on the underlying hardware that isn’t originally allowed for the VM. This is a feature that a hypervisor helps to provide and a clue why the Cloud options and services provided by hypervisors such as VMWare seem so similar.

Cloud Hypervisors

AWS, Azure and GCP all run on their company’s own hypervisors and the basic service they offer is an interface which lets users around the world allocate resources on them for their own workloads. While new services appear more abstracted, this is the basic premise of what Cloud is. The companies manage their server resources using hypervisors and provide VM offerings by running virtual machines on those Hypervisors, with container services and “serverless” offerings being more abstracted from those base machines.

None of this is really secret either as all three of those platforms have written about them on their platform’s respective websites. In Amazon’s own documentation, they define Cloud Hypervisors and mention the ones EC2 run on are based on Xen, a popular open source hypervisor. More recently, AWS has attempted to improve their utilization of server resources with their Nitro System helping to securely deliver server resources to instance with custom hardware and hypervisor software. Similarly, Google Cloud uses another open source hypervisor, KVM and, unsurprisingly, Azure’s Cloud Hypervisor is based on Hyper-V.

Conclusion

I hope this provides a good overview of what virtualization is, its evolution and how it’s used in popular Clouds todays. There’s a lot more to dig into here in many directions, from the specifics of different hardware, hypervisors and detailed comparisons of cloud providers’ custom chips and prices. At some point, I’d love to study and test more hardware and learn more about chip design and optimizing firmware and operating systems to best utilize resources. But, for the next time I continue this “Linux to Lambda” series, I’ll do a deep dive into containerization, a lightweight file-system based alternative for creating isolated environments for services to run that’s recently become a popular alternative to virtual machines and the start of the shift to “serverless” offerings.