Let’s start with some basic questions, what is a GPU?
How GPU and CPU different from each other? And then, lastly, why use a GPU? And, is it even important to use a GPU on the cloud?
Difference between GPU and CPU
Let’s start first with, what is a GPU?
GPU stands for “Graphics Processing Unit“. But, oftentimes people are more familiar with CPUs. So, CPUs are actually made up of just a few cores.
You can think of these cores as the power, the ability of a CPU to do certain calculations or computations.
On the other hand, GPUs are made up of hundreds of cores. But, what difference does it make?
The thing with the CPU is that when it does a computation it does so in a serial form, one computation at a time whereas GPU, does all computations in a parallel way.
The importance of these two differences is that with a GPU, you’re able to do computations all at once, and very intense computations at that.
Oftentimes when you have app codes, a lot of it’s going to the CPU. But then you’re going to have an application that’s going to require quite a bit of compute-intensive support that the CPU just can’t do, so it’s going to be offloaded to the GPU.
So, you can say that GPU is an extra muscle or that extra brainpower that the CPU just can’t do on its own.
There are two main providers of GPUs in the industry: NVIDIA and AMD. Both of these providers manufacture GPUs that are optimized for certain use cases. So, let’s jump into that because a big question I get is:
Why do I even need a GPU? In what companies and in what use cases?
Before to start further, first we’ll talk about VDI.
Virtual Desktop Infrastructure (VDI)
VDI stands for “Virtual Desktop Infrastructure“. GPUs are designed to support highly intensive graphic applications. Think about if you are a construction worker, And you’re out in the field, and you need to access a very high-graphic-intensive 3D CAD program.
So, rather than having the server right next to you or right in the field with you, you can have a server that’s a country away in a cloud data centre and you will able to see that 3D graphic as it was right in front of you. This one is supported by the Graphics Processing Unit (GPU) because the GPU supports these types of graphic-intensive applications.
Another example of this Technology is movie animation or rendering.
GPUs got their name mainly with the gaming Technology or industry.
Oftentimes they were referred to as “gaming processing units” because of this ability to provide end-users with low-latency graphics. But gaming is no longer the focus in the industry anymore.
It’s a big piece of it, but now financial services, life sciences, and even healthcare are starting to get into it with artificial intelligence.
Artificial intelligence has two big pieces to it -there’s machine learning and there’s deep learning. Now, there are some GPUs that are specially optimized and created for these types of applications.
There are some that are created for inferencing for machine learning purposes, and there are some that are created to help data scientists create and train neural networks.
In other words, they’re trying to create these algorithms that can think like a human brain. This is a task where a CPU can simply not do on its own, and it requires GPU capabilities.
Lastly, let’s talk about HPC.
High-Performance Computing (HPC)
HPC is a buzz word, that’s been going around, it stands for “High-Performance Computing“. By the way, a GPU is not necessary for the HPC, it is an important part of it.
You can simply say that High-Performance Computing (HPC) is the company’s ability to spread out their compute-intensive workloads amongst multiple compute nodes (or, in the case of cloud, servers).
Since these applications are very intense in computing. It could include rendering or artificial intelligence (AI) and that’s where a GPU comes in.
It’s up to you, you can add or connect a GPU to these servers that are spread out amongst an HPC application and use those in that manner. So, this is a nice little segue into why should use GPUs on the cloud.
GPU On Cloud
If HPC is such a big piece of that, what else is important about the cloud?
The first part of that is you get high performance, you need the cloud for that. The GPUs are great but not on their own. So, back in the day (and even still today), there are companies that use a lot of on-prem infrastructures, and they utilize that infrastructure for any of their compute-intensive applications.
However, especially in the case of GPUs, the technology is ever-changing.
In fact, every single year there are new technologies in GPU is coming out.
So, it’s actually very expensive and nearly impractical for companies to keep up with the latest technology at this point.
Whereas, Cloud providers continuously update their technology and provide GPUs to these companies to utilize them when they need them.
So, generally on the basis of infrastructure, cloud technology can be subdivided into Bare Metal and Virtual Servers.
Bare Metal and Virtual Servers
Both of these technologies have advantages as well as disadvantages of using GPU on the basis of their infrastructure.
If you are using a Graphics Processing Unit (GPU) on a bare-metal infrastructure, the companies oftentimes have access to the entire server itself, and they can customize the configuration.
So, this is a great deal for the companies that are going to be really utilizing that server and that GPU-intense application on a pretty consistent basis. On the other hand, for the companies that need a GPU maybe just on a burst-workload scenario, the virtual server option might be even better.
And the good thing about virtual technology is that there are often different pricing models available as well, including hourly. And the cool thing about the cloud is that you only pay for what you use.
If a company is operating on-prem technology or infrastructure but they’re not utilizing it at the time, that technology is depreciating, and it’s essentially a waste of money for that company. When they utilize the cloud, they only pay for what they’re using. And, so, it just makes a lot more sense from a cost perspective; and then, because the GPU is so great at performance, it just makes sense for a performance perspective as well.
That’s why companies are able to focus more on output and working techniques than they are on keeping up with the latest technology.