Azure Managed vs Unmanaged disks : The choice

Hi folks,

Recently (few months) , a new feature was announced to bring a new capability to Azure Virtual Machines : Azure Managed Disks.

Many blog posts explain well the purpose of managed disks, and how they bring enhancements to Azure IaaS virtual machines. I recommend the following readings :

• Introduction to Azure Managed Disks : https://www.petri.com/introduction-azure-vm-managed-disks
• How to convert a VM to a Managed Disks VM : https://www.petri.com/converting-azure-vm-disks-managed-disks
• Advantages of Managed Disks : http://www.danielstechblog.info/use-managed-disks-microsoft-azure/

The latter post shows the advantages of using Azure Managed disks, which I agree and confirm. But on the meanwhile, there is some ‘inflexible’ properties of managed disks, that may not be suitable for your or for your expectations. This is the purpose of this post : What is the model that fits my requirements , Managed or unmanaged disks.

1- The main difference between Managed disks and Storage Accounts based disks

There are some main differences between managed and unmanaged disks :

Category	Managed disks	Unmanaged disks
Management	Is an ARM (Azure Resource Manager) object (resource)	Is not an ARM resource, but a file (.vhd) residing on a Azure Storage Account. The latter is an ARM  object
Size	The managed disks sizes are fixed (and can be resized). Which means that you cannot choose a custom size. You will need to pick up from a list. See (1)	You can choose the disk size during the provisioning (and can be resized) when using Standard Storage. See (2)
Cost	You will pay : ·       Standard Storage : o   A fixed price per disk size (Per month), whatever the disk usage is o   Operations cost* ·       Premium Storage o   A fixed price per disk size whatever the disk usage is See (1)	You will pay : ·       Standard Storage : o   The GB / month disk usage. You pay only what you consume o   Operations cost* ·       Premium Storage o   A fixed price per disk size whatever the disk usage is See (3) and (4)
Performance	A managed disk have a predictable performance, with Standard storage (500 IOPS) or Premium storage (Depends on the disk).	Only premium storage disks have a predictable performance (depends on the disk). Standard storage have a predictable performance (500 IOPS) unless they are impacted by the Storage Account performance limits (A maximum of 40 disks per standard storage account is recommended, otherwise disks can be throttled). See (5)
Availability	When placing VMs using managed disks under an Availability Set, disks are placed on different fault domains in order to achieve the better SLA (The Availability Set SLA is only for compute)	When placing VMs using unmanaged disks under an Availability Set, there is no guarantee that the disks are placed on different fault domains, even if they are on different Storage Accounts.
Redundancy	LRS	LRS, GRS
Encryption	ADE, SSE (Coming soon)	ADE, SSE

* Operations cost means : Replication data transfer cost (In case of GRS) + Storage operations costs

 

2- Are managed disks more expensive that unmanaged disks ?

The answer is : It depends, but except in some cases, managed disks are always more expensive than unmanaged disks. Let’s prove it :

Standard Storage managed disk cost per month

• Managed disk cost = Fixed Cost (Per disk size) + Operations cost
• Unmanaged disk cost = Storage_Usage_In_GB * CostperGB + Operations cost

Because the Operations cost is the same for both models, we will omit them during calculation. Because the managed disk pricing model is not per usage, we will calculate the Disk size equity* value, to be able to compare with unmanaged disk :

Managed disk type	Size (GB)	Price	Cost per GB	Standard Storage price per GB	Disk size equity*
S4	32	1.3	0.040625	0.0422	31
S6	64	2.54	0.0396875		60
S10	128	4.98	0.03890625		118
S20	512	18.36	0.035859375		435
S30	1024	34.56	0.03375		818

* If you use less than the given size, then Unmanaged disks will cost less than managed disks. If you use more, then managed disks cost will be less than unmanaged disks. The €/GB will be greater as long you consume less storage space.

NB : New disk sizes have been announced (6) that finally make and end for the 1 TB disk size limit, with 2 and 4 TB for managed disks, and up to 4 TB for unmanaged disks. The service started on the West US Central region and will be generalized for the remaining regions during the coming months

3- Do I really need managed disks ?

This is a good question, but the answer is very relative to your needs. As you probably have read on the posts I mentioned earlier in this post, there are many benefits of using managed disks:

• Disk snapshots
• Predictable performance
• Distribution in different fault domains when associated with Availability Sets
• ARM object

Some workarounds may be used to have similar properties with unmanaged disks:

Properties	Unmanaged disk workaround
Disk snapshots	No workarounds
Predictable performance	Place less than 40 disks per Storage Account
Distribution in different fault domains when associated with Availability Sets	No Workaround. There is no way to know if the disks are place on different pools even if they are on different Storage Accounts
ARM object	Place each disk on its own Storage Account. Look if this will fit your needs  (Do not forget quotas)

 

4- Verdict

As you can see, managed disks brought new experience and features to Azure VM storage that permits better controlling the VM storage. Personally, I would recommend using managed disks, even if the ‘Pay as you consume’ model is not adopted there. But the features and the simplicity is worth the ‘little’ difference we can see with pricing. 

Azure : Premium Storage empty disks are also billed

Hi all,

We all know that among the  Azure advantages is to pay what you consume. In storage for example, if you create a 500 GB VHD and there is just 100 GB used, you will be billed just for 100 GB instead of 500 GB. But this is not true for the Azure Premium Storage:

For Azure Premium Storage, you will be billed for the number of P disks you are using, and here some examples to better understand:

If you create a 50 GB VHD, you will be billed for a P10 disk (128 GB)

If you create a 150 GB GB VHD, you will be billed for a P20 disk (512 GB)

If you create a 600 GB VHD, you will be billed for a P30 disk (1024 GB)

I think that this billing model is very expensive and will not be appreciated by customers, I wish Microsoft will rethink again about it.

The following table resumes the billing possibilities depending on VHD sizes

Source : Microsoft (See the FAQ)

Understand how Azure Premium Storage works : Watch HERE

Azure Virtual Machines : Storage IOPS and Throughput

Hi All,

This is a fast post about something I’m encountering when trying to understand the data disks performance in Azure virtual machines.

If you read the official TechNet article where the different Azure virtual machines series are exposed and their configuration detailed, maybe (like me) you will be confused about the data disks performances between the (A-Series, D-Series and G-Series) vs (DS-Series).

In the A-Series, D-Series and G-series  virtual machines, the data disks performances are described using the metric: Max IOPS

In the DS-Series virtual machines, the data disks performances are described using the metric: Max. disk IOPS and bandwidth

The questions that I was asking myself are:

1. What this IOPS means, how much data per IOPS ?
2. Why there is an additional bandwidth metric in the DS  Series ?

And here, I will answer based on information I got:

  What this IOPS means, how much data per IOPS ? 

IOPS means Input/Output Per Second, and it’s absolutely not referring to bandwidth. There is a tight relation between bandwidth and IOPS, but we need another parameter to do that. The Input/output size. What that means ?

Let’s imagine when you were kids, you are playing a game. You need to fill a bucket with water, cross the road, empty the bucket, re-cross the road, and do that for 1 minute, and finally we will see, who is the winner ? The winner is the one who collects more water, yes you say it, more water, not more back and forth. Yes, it depends of how the bucket is filled. If my bucket is completely  filled and I made 5 turns, and my friend’s bucket is half-filled and he made 5 turns, I will win !! This is exactly the same logic, you have to know first, what is the size of the Input/output unit used here, and you win

So in the case of the A, D and G Series, using a IO unit of 8KB for the 500 IOPS per disk, will result in approximately 8*500 = 4000KB/s = 3,9 MB/s

This guy makes a nice test to verify that, thanks to him : LINK HERE

In fact, the guy making the tests found that with a 4 disks strip, only 1130 IOPS was achieved (We expect 4*500 = 2000). So it’s a real best effort and storage performances are not guaranteed or that the test is not relevant, or and i guess it is the more likely, the total throughout is throttled.

In addition, I wonder why Microsoft did not show the maximum bandwidth, because based on my calculation : The maximum bandwidth for a 8KB IO unit is, and for a maximum size of 1 TB is : (Number of disks*500*8)/(1024). Based on this I can achieve a 62 MB/s on a A11 VM, and 250 MB/s on a G5 VM. Why Microsoft did not make things more clear, because this can’t be true !!!! There is certainly something I missed

  Why there is an additional bandwidth metric in the DS Series ? 

DS-Series can use the Azure Premium Storage. Azure Premium Storage is a new storage service that gives you high throughput capacity, low latency and  maximum performances. PS is based on SSD disks. To acquire Premium Storage, you will need a premium storage account.

To confirm the high performances of its Premium Storage service, Microsoft added an additional metric to describe the storage throughput.

But what this means ?

First, it’s a little complicated (Just a little) but I will explain it here, the easiest way (My explanations are based on this Microsoft article : http://azure.microsoft.com/en-us/documentation/articles/storage-premium-storage-preview-portal/)

• With Azure Premium Storage you can achieve a 50,000 IOPS and 32 TB of storage. The throughput is not mentioned here, because it has to be calculated
• Premium Storage is based on three Azure Storage Disk types. P10, P20 and P30. That means that your VHDs will be stored on disks of that type. You need to know that the IOPS&Throughput are dependent of the disk size (therefore on the VDH size)

The table above shows the P10, P20 and P30  specifications:

P10 disk is 128 GB, it can achieves 500 IOPS and up to 100 MB/s.

P20 disk is 512 GB, it can achieves 2300 IOPS and up to 150 MB/s.

P30 disk is 1024 GB, it can achieves 5000 IOPS and up to 200 MB/s.

You can see that the specifications are not linear. That means that Capacity, IOPS and throughput metrics are not linearly dependent. For example, the P20 disk size is 4 times greater than the P10 disk size, but the IOPS is a little greater than that, and for the throughput, it’s only 1.5 greater. So we need to be sharp when we create our VHDs.

Let’s see some example to better assimilate the facts

Example 1 :  I want to create a 200 GB VHD (Option1)

Azure will roundup your choice, so the more accurate disk to use is a P20, because the P10 disks are 128 GB only, and VHD will not fit in. You will benefit then, of 2300 IOPS and up to 150 MB/s

Example 2 :  I want to create a 200 GB VHD (Option2)

You will create 2 VHDs with 100 GB each. Azure will create these two VHDs using P10 disks (smaller than 128 GB). Then you will use Windows (Or Linux) to create a strip (Storage spaces for example) using these 2 VHDs. You will have as a result a 200 GB VHD with 1000 IOPS and 200 MB/s

Example 3: I want to create a VHD with 600 GB and 400 MB/s of throughput

You will not obtain such throughput if you just create a 600 GB VHD, because Azure will create a 600 GB VHD on a P30 Disk, and then you will have only 200 MB/s.

To achieve that, you should use stripping, and to do that, you can proceed with different ways:

Way1 : You create two 600 GB VHDs. Azure will create them using P30 disks. Then you use your stripping tool (Storage spaces) to create a 1200 GB volume. This volume will permit 400 MB/s and 10000 IOPS. But in this case you will have 600 un-nedded GB

Way2 : You create 3 VHDs with 200 GB each. Azure will create them using P20 disks (Example 1, Option1). Then you use your stripping tool (Storage spaces) to create a 600 GB volume. This volume will permit 450 MB/s (150 MB/s *3) and 6900 IOPS (2300 IOPS *3).

Example 4: I want to create a VHD with 600 GB and 600 MB/s of throughput

Unfortunately, we can’t just dream, and ask Azure to do it, not till today. In fact, the maximum throughput possible is 512 MB/s, we can’t do better.

  IMPORTANT   

• The total data storage, the IOPS and the throughput are limited by the VM series and size. Each Azure Virtual Machine type is limited by a number of disks (total storage size), a maximum IOPS (IOPS) and a maximum throughput (Throughput). For example, you  may achieve a 400 MB/s (Example 3) only in a Standard_DS14 VM.  All the other VM types will throttle your IOPS or throughput when you reach the threshold. The following picture (Microsoft credit) shows the DS-Series maximum storage performances

Follow this link for all the Azure Virtual machines types, sizes and specifications : https://msdn.microsoft.com/en-us/library/azure/dn197896.aspx

• You should now that Azure will throttle storage whenever one of the two metrics threshold is reached: IOPS or Throughput. Let’s take a DS2 VM, If you run 2000 IOPS workload with a 100 KB IO unit, you will reach a 195 MB/s throughput. This value exceeds the 64 MB/s threshold (The DS2 threshold) , and Azure will throttle your disk access, this will result on a huge IO queue, and you will suffer from performances degradation.
• When you choose your storage, you will choose your VM too. And when you choose your VM, you choose your storage too. So make a reasonable calculation before spending money

  Conclusion 

Azure is relatively new, new features and terms are coming everyday, and lack of information and ambiguity is a real headache, specially for persons that will make a decision. Storage for virtual machines in Azure a bit tricky, and Microsoft did not provide any tool where we put our need to obtain a list of possible configurations. But till this day, I invite you to proceed like the following:

1. Determine your storage needs : 1- How much data disks 2- How much IOPS, Throughput for each disk
2. Calculate the options when converting to Azure storage (P10, P20, P30), it’s like the Examples 1, 2, 3 and 4 we discussed above
3. Match the results to Azure Virtual machines Series to find the best suitable size for you : For each Option found in step 2, find the VM sizes that fit
4. Create a table with all these information exposing all the scenarios
5. Calculate the cost for each scenario
6. Choose the best Spec/Cost VM