Azure Managed vs Unmanaged disks : The choice

Hi folks,

Recently (few months) , a new feature was announced to bring a new capability to Azure Virtual Machines : Azure Managed Disks.

Many blog posts explain well the purpose of managed disks, and how they bring enhancements to Azure IaaS virtual machines. I recommend the following readings :

The latter post shows the advantages of using Azure Managed disks, which I agree and confirm. But on the meanwhile, there is some ‘inflexible’ properties of managed disks, that may not be suitable for your or for your expectations. This is the purpose of this post : What is the model that fits my requirements , Managed or unmanaged disks.

1- The main difference between Managed disks and Storage Accounts based disks

There are some main differences between managed and unmanaged disks :

Category

Managed disks

Unmanaged disks

Management

Is an ARM (Azure Resource Manager) object (resource) Is not an ARM resource, but a file (.vhd) residing on a Azure Storage Account. The latter is an ARM  object

Size

The managed disks sizes are fixed (and can be resized). Which means that you cannot choose a custom size. You will need to pick up from a list. See (1) You can choose the disk size during the provisioning (and can be resized) when using Standard Storage. See (2)

Cost

You will pay :

·       Standard Storage :

o   A fixed price per disk size (Per month), whatever the disk usage is

o   Operations cost*

·       Premium Storage

o   A fixed price per disk size whatever the disk usage is

See (1)

You will pay :

·       Standard Storage :

o   The GB / month disk usage. You pay only what you consume

o   Operations cost*

·       Premium Storage

o   A fixed price per disk size whatever the disk usage is

See (3) and (4)

Performance

A managed disk have a predictable performance, with Standard storage (500 IOPS) or Premium storage (Depends on the disk). Only premium storage disks have a predictable performance (depends on the disk). Standard storage have a predictable performance (500 IOPS) unless they are impacted by the Storage Account performance limits (A maximum of 40 disks per standard storage account is recommended, otherwise disks can be throttled). See (5)

Availability

When placing VMs using managed disks under an Availability Set, disks are placed on different fault domains in order to achieve the better SLA (The Availability Set SLA is only for compute) When placing VMs using unmanaged disks under an Availability Set, there is no guarantee that the disks are placed on different fault domains, even if they are on different Storage Accounts.

Redundancy

LRS LRS, GRS

Encryption

ADE, SSE (Coming soon) ADE, SSE

* Operations cost means : Replication data transfer cost (In case of GRS) + Storage operations costs

 

2- Are managed disks more expensive that unmanaged disks ?

The answer is : It depends, but except in some cases, managed disks are always more expensive than unmanaged disks. Let’s prove it :

Standard Storage managed disk cost per month

  • Managed disk cost = Fixed Cost (Per disk size) + Operations cost
  • Unmanaged disk cost = Storage_Usage_In_GB * CostperGB + Operations cost

Because the Operations cost is the same for both models, we will omit them during calculation. Because the managed disk pricing model is not per usage, we will calculate the Disk size equity* value, to be able to compare with unmanaged disk :

Managed disk type

Size (GB) Price Cost per GB Standard Storage price per GB Disk size equity*

S4

32 1.3 0.040625 0.0422 31

S6

64

2.54

0.0396875

60

S10

128

4.98

0.03890625

118

S20

512

18.36

0.035859375

435

S30 1024 34.56

0.03375

818

* If you use less than the given size, then Unmanaged disks will cost less than managed disks. If you use more, then managed disks cost will be less than unmanaged disks. The €/GB will be greater as long you consume less storage space.

NB : New disk sizes have been announced (6) that finally make and end for the 1 TB disk size limit, with 2 and 4 TB for managed disks, and up to 4 TB for unmanaged disks. The service started on the West US Central region and will be generalized for the remaining regions during the coming months

3- Do I really need managed disks ?

This is a good question, but the answer is very relative to your needs. As you probably have read on the posts I mentioned earlier in this post, there are many benefits of using managed disks:

  • Disk snapshots
  • Predictable performance
  • Distribution in different fault domains when associated with Availability Sets
  • ARM object

Some workarounds may be used to have similar properties with unmanaged disks:

Properties

Unmanaged disk workaround

Disk snapshots

 

No workarounds

Predictable performance

 

Place less than 40 disks per Storage Account

Distribution in different fault domains when associated with Availability Sets

 

No Workaround. There is no way to know if the disks are place on different pools even if they are on different Storage Accounts

ARM object

 

Place each disk on its own Storage Account. Look if this will fit your needs  (Do not forget quotas)

 

4- Verdict

As you can see, managed disks brought new experience and features to Azure VM storage that permits better controlling the VM storage. Personally, I would recommend using managed disks, even if the ‘Pay as you consume’ model is not adopted there. But the features and the simplicity is worth the ‘little’ difference we can see with pricing.  Continue reading

Advertisements

How to protect and backup your Branch and Remote offices data (Files and Folders) ?

Hi everyone,

Since the first days of the adoption of an Information System by companies, backing up the workloads was crucial and a production blocker : No production without backup, no backup,  no business.

Today, companies are better mastering and understanding their backup needs, solutions and they are continually seeking for better, simple and cost effective backup software.

One of the ‘headache’ subjects that bother the majority of the backup admins and decision makers is the Remote Offices / Branch Offices (ROBO) ‘Files and Folders’ data backup.

During this post, I will show why Azure Backup via the MARS agent is your best choice to get rid of the ROBO workloads backup problematic. I will present :

  • Use cases for using Azure Backup via MARS agent
  • What do you need to know in order to be comfortable with this solution
  • What are the steps to plan and start using Azure Backup via MARS agent for your ROBO

1- Use cases for using Azure Backup via MARS agent

Azure Backup is the name of a complete enterprise backup solution allowing several backup scenarios and using the last technologies, specially the ability to back up to the cloud and to benefit from a GFS model (a model allowing  efficient Long term retention policies).

What is interesting about Azure Backup via MARS agent is that it allows you to backup your files and folders without the need to deploy a Backup Infrastructure or a Storage infrastructure. This opens up a lot of use cases :

Backup without backup infrastructure

The following picture shows the end to end data journey from your Windows Server or Workstation to the cloud storage (More details about the components later on this post). As you can note, the backup will needs only the installation of the Azure Backup Agent (MARS agent : Microsoft Azure Recovery Services agent) and to configure it to
backup data to a cloud location (Recovery Services Vault)

This is fantastic since it removes the classic requirements to enable workloads backup :

  • Backup software infrastructure (Backup server, Backup Proxy…)
  • Local storage : No need for a SAN or a NAS. Azure backup will directly send data to the cloud using an internet connection

Short and Long term retention without backup infrastructure

In addition to the great value from the first discussed statement, Azure Backup provides in the same time, Short and Long term retention within the same policies. No need for tapes, no need for external provider to handle it. Azure Backup use a GFS model to allow  Long Term retentions without any additional configuration. You can reach up to 99 years of retention period for up to 9999 recovery points (These values can change on the future).

Low bandwidth/Latency ROBO locations

The Azure Backup agent supports throttling (2) the data transfer to the cloud location (Not for all OSs). This is very important for ROBO location with limited bandwidth that prevent you from using your central backup infrastructure (Backup to a central backup repository)

 

2- What do you need to know

In this section, I will resume the important information that you need to know about the Azure Backup (Specially with the MARS agent). These information will give you the ability to decide, design and implement Azure backup into your information system.

2.1- Pricing

Fortunately, the Azure Backup pricing is very simple. It’s well explicated on the official documentation (1) but to resume:

When you backup a workload, you pay for :

  • An Azure Backup fixed cost for each backed up instance (The cost depends on the size of the data being backed up)

 

  • The storage used by the recovery points:
    • You can choose between LRS or GRS storage (3). To resume, LRS (Locally redundant storage) is a storage only available with the region where you create the Recovery Vault. GRS is a storage replicated asynchronously to another paired region providing hence, a protection against region failure, but more expensive (4) (~ * 2)
    • The redundancy cannot be changed after the first workload backup, so be sure of your decision before going forward

 

For example, if you backup 4 windows servers, you will pay:

  • 4* Azure Backup fixed cost
  • The cost of the Azure storage (cloud storage) used by the recovery points

2.2- Requirements

In this section, I will resume what do you need to technically be ready to use Azure Backup (via the MARS agent)

 

2.2.1-  Azure Level

As discussed earlier in this post, you need the location where you will send and store backups. This is called Recovery Services Vault (RSV). An RSV is a Microsoft Azure resource, which means that you need to subscribe to Azure in order to deploy it. Subscribing to Microsoft Azure is very simple, there are many ways to achieve it, depending on your needs and the billing/relation model that you want. In order to use Azure, you need to create an Azure subscription (5). After creating it, you can directly without any requirement create an Azure Recovery Vault, ready to host your backups (within minutes).

You will then need access* to the Recovery Vault in order to begin. You can benefit from the Azure RBAC roles (6) in order to have or give required permissions.

In order to backup Files and Folders via the MARS agent, you will just need:

  • The MARS agent installation file : Allowing you to install the agent on the required servers
  • The Vault credentials : Allowing the MARS agent to find and authenticate to the Azure Recovery Vault.

Both of them can be downloaded via the Azure portal via the Azure Recovery Services resource blades.

* Technically, you don’t need access to the Recovery Vault to enable backups. An Admin can send you the required information instead.

2.2.2- Local level

I mean by local level, what do you need at the server level (The server where the folders and files to be backed up) in order to start backing up :

  • A supported Operating system : Only Windows is supported, Linux is not yet supported.
  • A internet connectivity : The agent needs outbound internet connection to the Azure services in order to send data. Using a Proxy is supported. You can in addition limit the outbound flows to only Azure services public IPs (7) (And even more, only the IPs belonging to the RSV region)

 

There are limitations regarding the supported operating systems, what can you backup, how often you can backup and more. Please refer to the Azure Backup FAQ for complete information

 

2.3- Security and data confidentiality

Azure backup via the MARS agent provides many precious security aspects, let me enumerate some of them:

  • You will need a Vault credentials file in order to register an agent to a vault. Only backup admins can download such file from the Azure portal
  • Before enabling the backup, you will be prompted to provide a ‘passphrase’. A passphrase is a ‘complex password’ used to encrypt data before sending it to the RSV. Data is encrypted and send via HTTPS to the RSV where it remains encrypted. Note that without this passphrase, you will not be able to restore data in case you lose the original server (Or its configuration), the passphrase must be kept securely somewhere (You can use Azure Key Vault to store your secrets)
  • In case your server is compromised, the compromiser (Hacker, malicious admin) cannot delete you recovery points. Azure backup provides a security setting (enabled by default) that requires the ‘remover’ to login to the Azure Portal and generate a PIN code. The probability that the ‘compromiser’ owns the credentials to login to the Azure portal is small. In addition, you can benefit from the ‘MFA’ feature of Azure portal in order to more secure the portal access.
  • In case of ransomware/crypto-locker attack or infection, your backup data is protected, since the backup media is totally independent of the server.
  • Other security prevention feature are also available (8) :
    • Retention of deleted backup data: Backup data retained for 14 days after delete operation
    • Minimum retention range checks: Ensures more than one recovery point in case of attacks
    • Alerts and notifications: For critical operations like Stop backup with delete data
    • Multiple layers of security: Security PIN required for critical operations (Already mentioned)

2.4- Monitoring and reporting

Like you noticed, there is no server nor a console to install, monitor or see what is happening. All is done via the Azure Portal. You can use the Azure portal to :

  • Backup Items : View the backed up items (Server name, volume…)
  • Backup Status : You can view and show the status of the backups, with ‘filtering’ options
  • Backup jobs: You can see the backup jobs and their status. You can see the duration and the size of the backups and restore operations
  • Notifications : You can configure and see the notifications related to the jobs. Currently, you can only configure notifications based on the jobs status (Critical, Warning, Information)

Currently, there is no ‘Reporting’ feature with Azure backup via the portal. But this feature is coming very soon.

3- How to start : The plan

In this third and final section, I will present the planning steps in order to successfully plan and implement your ‘Folders and Files’ backup. The main steps are :

  1. Create a Recovery Services Vault
  2. Configure the vault
  3. Download the Recovery Vault credentials
  4. Install the MARS Agent on the server
  5. Create a backup policy and a schedule

This link shows the detailed steps to achieve the above steps : https://docs.microsoft.com/en-us/azure/backup/backup-configure-vault

The Azure Backup FAQ contains the most answers to your questions :

https://docs.microsoft.com/en-us/azure/backup/backup-azure-backup-faq

To finish, the following are my recommendations when planning to implement Azure Backup via the MARS agent:

Question / Constraint

Choice

Are my source servers located on the same region ? It’s recommended to backup data to the nearest location in order to benefit from a better performance / Latency during backup and restore operations.
Do I need to back up to the same RSV ? No, but to have a simple design, it’s better to minimize the number of RSV for the a similar servers group.
When do I need to backup to different RSV What can differentiate two Recovery Services Vault  :

–         The redundancy of the Storage (LRS or GRS)

–         The user rights on the RSV

–         The vault credentials

So :

–               If you have different ‘data’ importance, and you want to optimize the costs, you can create ‘LRS’ RSVs for less important data, and ‘GRS’ RSVs for more important and critical data

–               You can give permissions to access or manage the Recovery Service Vault. If you want different security levels for your Vault, you can create multiple RSV

–               The Vault Credentials are unique for an RSV. A user with a valid Vault credentials file (expires after 2 days) can backup data to the vault

Use the same passphrase for each server ? No. This is absolutely not recommended for the unique reason is that someone compromises the passphrase, he can access you all your server’s restore points (He will need a valid Vault credentials file)

 

Useful Links:

 

(1) Azure Backup pricing : https://azure.microsoft.com/en-us/pricing/details/backup/

(2) Azure Backup agent network throttling : https://docs.microsoft.com/en-us/azure/backup/backup-configure-vault

(3) Azure Storage redundancy : https://docs.microsoft.com/en-us/azure/storage/storage-redundancy

(4) Azure Storage pricing : https://azure.microsoft.com/en-us/pricing/details/storage/blobs-general/

(5) Designing Azure Subscriptions : https://buildwindows.wordpress.com/2016/03/30/azure-iaas-arm-architecting-and-design-series-azure-subscriptions/

(6)Azure Backup Roles : Backup Contributor, Backup Operator, Backup Reader

(7) Azure Public IP ranges : https://www.microsoft.com/en-us/download/details.aspx?id=41653

(8) Azure-backup-security-feature : https://azure.microsoft.com/en-us/blog/azure-backup-security-feature/

(9) Azure subscription and service limits, quotas, and constraints : https://docs.microsoft.com/en-us/azure/azure-subscription-service-limits

Azure TCO calculator Public Preview

Hi,

Microsoft just annouced the public preview of its TCO calculator tool.

This tool will help you see if Azure will reduce or not the TCO of your Virtualization or Physical on-premises platform. It will give you a cost forcast on a 3 years period. This is a valuable tool for organisations planning to renew their on-premises infrastructure or a part of it. 

This tool is a must for Cloud Architects, consultants or IT managers.

Give it a try, i will update the article with the feedback links.

https://www.tco.microsoft.com 

Azure Virtual Machines single instance SLA, a big step!

Hi all,

Early this week, Microsoft made an exciting announcement with its SLA for a single virtual machine of 99.9 ℅ : https://azure.microsoft.com/en-us/blog/announcing-4-tb-for-sap-hana-single-instance-sla-and-hybrid-use-benefit-images/

Before this announcement, single instance VMs (which are not part of an Availability Set) were not covered by an SLA. This was unattractive for workloads which do not support, afford or need a multi-instance deployment. This was generally applicable for legacy workloads and to be honest, to the majority of non-critical workloads, and for SMB workloads which do not afford investing in redundancy and HA.

Many of my customers avoided migrating workloads to Azure, just because of this, which was offered by AWS since a while.

With this announcement, customers have the warranty that an SLA of 99,9 % provided to their VMs, which means a maximum downtime of 8.76 hours per 356 days –> 44 minutes per month

Do not forget that this is only applicable for virtual machines with all disks stored on Premium Storage.

What do we need to know about Azure Stack : The Q & A

Hi all,

It has been a long time since I didn’t blogged anything. A lot of news actually happened on the last few months, and in this post I will explain one of the most existing for me : Azure Stack

Azure Stack was introduced earlier this year (January) with a first Proof of Concept named TP1 (Technical preview 1). The Technical Preview goal was to give customers, consultants and early adopters a view of what Microsoft baptized as the future of Private and Hybrid cloud. But, really, what is Azure Stack ?

The modest definition 

Azure Stack is a platform (Software) that you can deploy on-premises to have similar Microsoft Azure services,  features and user experience. If you are using Microsoft Azure (The new portal, known as Ibiza portal portal.azure.com), than this is what you will get when you deploy Azure Stack on your datacenter. You will be able to leverage the Azure underlying technology on-premises, to deploy, manage and benefit from the Azure services like virtual machines, web apps, virtual networks and the list evolves. Think just that instead of typing portal.azure.com on your browser, you will type a custom URL on your domain that will land you on an Azure portal, but on your datacenter.

Is Azure Stack suitable for my company or business ?

Azure Stack brings advanced cloud technologies to your datacenter from the virtualization platform (a simple virtual machine) to the Azure App Services (A PaaS model to rapidly deploy Web Applications). So technically, Azure Stack can be used by any company aiming at least to use virtualization, but this is not enough to adopt it. As a consultant and  an Azure Architect, I think that Azure Stack is suitable for you if :

  • You are using or at least experimented the user experience, concept and different services provided by Microsoft Azure. If you have validated that Azure is suitable for your company, and you are looking for the same experience on-premises (for any reason) then Azure Stack may be a good choice (Azure Stack is consistent with Azure)
  • You are looking for a  private cloud platform which can provide the last cloud technologies and concepts. Azure Stack is born from Azure and will continually benefit from the last enhancements made and tested on the Azure public cloud platform
  • You are looking for a modern way to faster build your applications and services, which is the model based on PaaS and micro services. Azure Stack in its first version (mid 2017) will support Azure Web Apps and maybe Azure Fabric if they decide to bring it.
  • The constraints  I will mention next do not bother you

How Azure Stack will be delivered to customers ?

This is the actual debate, but Microsoft elected the winner, with a sort of inflexibility. Azure Stack will only be provided via System Integrated platforms with the freedom to choose between 3 different Hardware providers : HPE, DELL EMC and Lenovo (formerly x86 IBM servers). This means that you cannot deploy an Azure Stack platform on top of your hardware, but you will need to acquire the hardware with Azure Stack pre-packaged, and just plug it to your datacenter.

This last statement created a rage from the community, and we got two visions :

  • Microsoft is affirming that this model is the only possible way to achieve the wished Enterprise level private/hybrid cloud platform. Microsoft is stating that the integration with the hardware is a very heavy task and it prefers validating the platform and then provide a ‘ready engine’ to the customer.
  • The community is surprised that Microsoft is, first locking-out its solution for a set of non-affordable hardware providers, and secondly, not following the original virtualization and cloud ideology, which is the reuse and the optimization of the existing resources, and even, use affordable and commodity hardware.

Azure Stack licensing and prices

This is what I call the mystery question for the public because I have an early information, but due to NDA, I’m not allowed to publish it. What I can say, is that whatever the licensing model is, I think that it will be expensive, and I wonder if it will reach the SMB market. Remember, there are 3 parties involved : The hardware provider, the software provider and the integrator (which is Microsoft anyway but should be counted as a third party IMHO)

 

What if I acquire Azure Stack ? What about the test platform ?

This is a question I have asked before, and the answer was quite not sufficient. This is the summary :

  • Microsoft will provide the one Node PoC which is a one node Azure Stack platform. It’s what delivered today on TP1 and TP2. You can install Azure Stack on one node to be able to PoC, discover and make the tests you want. But, on the meanwhile we are not certain (no information) of the accuracy of the one node PoC with the System Integrated Azure Stack platform in terms of minor updates and bug fixes, and more important features.
  • You can do what you actually doing on Azure : Create a Test subscription with limited quotas, where you can make deployment tests  –> This still depends on the licensing model as we don’t want that a test platform be costly

 

What are the actual sizes of the Azure Stack Integrated System offers ?

It’s too early to speak about the capacity (CPU, Memory, Storage) of the Azure Stack platforms provided via the Integrated Systems. Things can change and I think that the final sizes will be revealed during the GA. Anyway the minimum published size today is  4 hosts with 256 GB and double sockets with 8 cores each constituting a single scale unit (Hyper-V cluster) and which will contain both the management infrastructure (VMs needed for Azure Stack to work) and your workloads (IaaS VMs, App Service Plan VMs…). Do not forget that the Azure consistency implies that the virtual machine sizes that you will be able to deploy are the same than Azure VM sizes. Hans Vredevoort has thorough articles about the Azure Stack Architecture :

Where System Integrators are placed on this whole thing ?

This is one of the questions I asked to myself. On standard products  like System Center and Windows Azure Pack, system integrators were almost mandatory to successfully deploy these products within a customer site. But the question raised up with the decision to only provide Azure Stack via System Integrated : No more need for system integrators to deploy the solution, a sort of plug and play.

This is true, but this isn’t so bad (No the case  for geeks, unfortunately Smile )

In fact, what are you integrators doing today with your customers when they call you to help them design and deploy their workloads on Azure? you are certainly happy, and this is why we should be optimistic when speaking about Azure Stack.  Because Azure Stack is Azure on your datacenter (An Azure sealed black box), customers will need you to help them first choose which Azure Stack offer (tier) to purchase, and then help them use Azure Stack, the same what you are doing  today on Azure. The consistency will make your Azure expertise vey valuable on the on-premise field.

What we will miss, is having our hands on an Azure Stack real platform to make some practice. But, theoretically, this will not be the biggest problem since we can use the one node PoC to achieve such goals. What is causing the headache for the com munity, is the near-impossibility of deploying the 1 Node PoC on our LAB at home. The minimum RAM requirement for the TP2 is 96 GB of RAM, and this is the minimum, expect up to 128 GB to start enjoying the LAB and the deployment of all the services. I don’ t know many having a real 128 GB RAM server at home.

 

Can CSP benefit from Azure Stack ?

Things are not yet dry, and Microsoft did not yet publish a clear view of the Cloud Service provider interaction with Azure Stack. But, it’s clear that that through the CSP program, CSP will be able to use Azure Stack to deliver sophisticated Azure like features. The biggest factors that may slow down CSP form using Azure Stack are :

  • Locked-down hardware providers : Cloud Service Providers are certainly partnering with hardware providers to acquire discounts and advantages when buying hardware. I’m very pessimistic regarding this factor. CSP may look to other cloud platform solution or continue build their own
  • Licensing and pricing : The introduction of  locked-hardware model may impact the margin the CSP can generate. No comment on the Software part licensing.

 

What do I think of Azure Stack and the implementation model ?

Microsoft is a great Enterprise, master minds are working there trying every day to enhance their products, creating new technologies and approaches, pivoting and changing their business model to impact the market. But, no human is bullet-proof, Microsoft can make mistakes and failures (The case for Windows phone, which is terribly not progressing ), it’s dramatically changing its business model with Azure Stack : Cloud Appliances. I’m really waiting for the licensing announcement to see which customer segment it’s targeting. But what I’m waiting for is the customer reaction. I have no idea how they will react. My first impression is that this model is not appreciated by the community and by me. I’m not against System Integrated platforms, but I’m with the virtualization and cloud early goals : Hardware reuse and cost reduction. Azure Stack is not fitting there, in addition of making restrictions on the hardware providers we can choose from, I have a bad feeling melted with a great excitement to this Azure Stack era, hoping it will find its way.

Based on my opinion, the following are the Pro and Cons of Azure Stack :

The GOOD

    • Azure services on your datacenter : This is the most exiting about Azure Stack, you can bring Azure features (IaaS, PaaS…) on your datacenter. You have the last tested*  cloud technologies on your hand. If you are avoiding using the public cloud platforms for any reason (Privacy, Compliance, trust, network connectivity) but on the same time wishing using the provided features, then Azure Stack is for you
    • Plug-and-Play model : The System Integrated model will reduce the TCO, by bringing a ready-to-use private cloud platform
    • A true Consistent and Hybrid cloud platform : Azure Stack is a real advantage for customers using or planning to use Azure services, since the consistency between the platform is guaranteed. You can use the same approaches, design decision factors, tools, scripts and features. You no longer need two different models to manage your cloud and on-premise platform, thus reducing significantly the IT efforts which can be spent on real business concerns (Deploying apps, migrating, enhancing…)

The Bad

    • The other side of System Integrated model : Hardware locking was never a good idea, depriving the customer from freely choosing  its hardware provider, and hence better control the costs. This model will reduce the early adopters market and hence can slow down this beautiful product from being used widely.

* Azure Stack will bring features already used on Azure, so we are sure that the features were widely tested on the public cloud

If you think the System Integrated model is not suitable for you, support this idea, you can participate and influence changing the Azure Stack future : Provide an installable version of Azure Stack

How can we use Azure Cool Storage for Archiving data ?

Azure Cool Storage was introduced early this summer as a new Azure Storage Account tier type. The story is that before this time, the Azure Storage provides only one kind of Storage Accounts, which is for general purpose, and which includes two performance tiers, this makes using a Cloud Storage Account for storing daily data (Virtual Machines disks, File shares…) the same than archival and backup data.

A new type of Storage Accounts was introduced which brought two different tiers of Storage Accounts : Hot and Cool. The following picture shows the Azure Storage Account types :

SA-Types

This article explains how you can use the Azure Cool Storage tier for Archival purpose.

1- Scenarios proposal

1.1- Use StorSimple physical Array

Azure StorSipmle is a hybrid storage array which provides features to store data locally and on the cloud. StorSimple has many advantageous and offers a lot of interesting features like :

  • Tiering: As your data ages, automatically tier it and move it from solid-state disk to spinning hard drive to the cloud
  • Local volume: Ensure any primary data that you specify as local will stay on the physical StorSimple device
  • Cloud snapshot: Enable rapid movement of volume snapshots to the cloud to support integrated disaster, as well as dev/test and business analytic environments
  • Data deduplication: Deduplicate data inline to reduce your storage growth rates

The following capabilities will make the proposed design successful:

  • Azure Storsimple supports tiering on the Cloud –> Cold data will be moved from the local storage to the Cloud Storage
  • Azure StorSimple supports Azure Cool Storage Accounts as a Cloud Storage
  • Azure StorSimple can be exposed via iSCSI

The scenario is like the following :

  • A Cool Azure Storage Account (or more)  will be created an linked to the StorSimple.
  • A Volume (or more) will be created on the StorSimple
  • The volume (or volumes) will be presented to a Windows File Server
  • File Shares will be accessible for the Admins or the tools which will move data to be archived to these shares

–> Thanks to the Tiering feature, data which is Cold will be automatically tiered to Azure Storage Account. Because this data is not willing to change; we expect 100 % of cold data and hence tiered data

NB : Keep in mind the different StorSimple limits : https://azure.microsoft.com/en-us/documentation/articles/storsimple-limits/

1.2- Use StorSimple virtual array

If you don’t or are not planning to use the StorSimple physical array, you can use the StorSimple Virtual Array which was introduced early this year. The Virtual Array can be hosted by a Hyper-V or VMware virtualization platform. The StorSimple virtual array provides similar features than the physical array except the local storage, which should be provisioned from the virtualization platform storage. Keep in mind that the virtual Array provides less capacity than the physical one (Look to the first table on https://azure.microsoft.com/en-us/documentation/articles/storsimple-ova-overview/).

The Virtual Array has an additional feature compared to the physical one, which is the File Share feature. In fact, it can exposes directly SMB shares to the users.

The design is like the following :

Variant 1

  • A Cool Azure Storage Account (or more)  will be created an linked to the StorSimple.
  • A Volume (or more) will be created on the StorSimple
  • The volume (or volumes) will be presented to a Windows File Server
  • File Shares will be accessible for the Admins or the tools which will move data to be archived to these shares

Variant 2

  • A Cool Azure Storage Account (or more)  will be created an linked to the StorSimple
  • A file share (or more) will be created
  • File Shares will be accessible for the Admins or the tools which will move data to be archived to these shares

–> Thanks to the Tiering feature, data which is Cold will be automatically tiered to Azure Storage Account. Because this data is not willing to change, we expect 100 % of cold data and hence tiered data

1.3- Use Microsoft Azure Storage Explorer

Microsoft Azure Storage Explorer is a tool developed and maintained by Microsoft which allow managing and making different operations on the Azure Storage accounts.

The tool can be downloaded here : http://storageexplorer.com/

NB : Please, do not confuse Microsoft Azure Storage Explorer and Azure Storage Explorer. This latter is an open sourced project which is not very well maintained (But was here first)

The scenario is like the following :

  • Install MASE on a server (We recommend a dedicated server)
  • Use AZcopy (The MASE Command line) to copy the data to be archived to the Azure Cool Storage Account. You can use the Storage Explorer UI either. The AZCopy tool is good if you want to schedule data movement or create scripts.
  • Use MASE to explorer and download the Azure Storage Account data

NB: MASE does not provide mounting a Storage Account Container, and managing the content (Browser, Copy, Delete…) via a drive letter.

1.4- Use a third party tool

There are different Third Party Tools which allow managing and accessing Azure Storage Accounts. The difference regarding MASE is that these tools allow mounting the Storage Account containers to virtual drives. This way they can be accessed directly via the windows explorer.

The scenario is like the following :

  • Install the tool on a server (We recommend a dedicated server)
  • Attach a Cool Azure Storage Account using the tool and create a mount point (Or a virtual drive)
  • Admins can move data to be archived directly to the mount points

The following are two tools which can be used for this purpose:

2- Scenarios Comparison

Scenario

+

My Rating

StorSimple Physical Array Best capacity and performance Expensive or needs minimum credit if purchased under EA

No SMB file shares support like the SVA

Use it if you already have a physical Storsimple

Enterprise solution

StorSimple Virtual Array Support SMB File Shares (Windows File Share + Share Permissions)

Free

Needs a virtualization platform and a local storage capacity  (10 % of the total storage)

Limited capacity if exposed via iSCSI (5 TB per Volume)

Use it if you don’t have a physical Storsimple or you don’t want to deploy a File Server

Enterprise solution

Microsoft Azure Storage Explorer Free

Simple to use

Storage Accounts are only exposed via the tool UI and command line Personal or occasional use
Third Party Tools Simple to use May not be scalable for Free editions Personal or occasional use

3- Azure cool storage costs

Assumptions :

  • I do not count blobs operations which are billed too. We estimate that the cost of these operations will not pring impact on the total cost. In addition, it’s not predictable
  • I assume that there is no change on the already written data (archived data does not change over time). The change will inquire write costs
  • I’m using Public prices North Europe region (Euro)
  • Do not forget that the pricing is per Month for the Storage and per Operation (per GB) for Write, change and retrieval
 

Storage per Month

Data write per Operation

Data retrieval per operation

LRS

GRS

LRS

GRS

LRS/GRS + outbound data transfers

1 GB

0.0084 €

0.0169 €

0,0021 €

0,0042 €

0,0818 € = 0,0084 + 0.0734

10 GB

0.084 €

0.169 €

0,021 €

0,042 €

0,818 € = 0,084 + 0.734

100 GB

0.84 €

1,69 €

0,21 €

0,42 €

8,18 € = 0,84 + 7,34

1 TB

8,4 €

16,9 €

2,1 €

4,2 €

81,8 € = 8,4 + 73,4

Costs example with a Cool storage account in GRS redundancy:

  • If we store 1 TB of data we’ll pay: 16,9 € / month + 4,2 € once
  • If we download 1 GB we’ll pay: 0.0818 € once
  • If we download 100 GB we’ll pay: 8,18 € once

Change Azure ARM VM size

Hi all,

Microsoft has published an article describing how to resize an Azure Virtual Machine, and shown that on the major cases, using the Portal or PowerShell is enough to resize the VM within few minutes with a short downtime (Reboot). I have discussed that on a previous post. You can see that on same cases, the host where the VM is deployed does not support the target size, so you will not be able to resize your VM without redeploying it.

I thought that the ‘Redeploy’ feature will redeploy your VM on another host supporting more VM sizes, but I think that the it redeploys it on the same ‘Pool’ with same characteristics.

So I decided to write a quick script (you can use the functions or adapt it to your context) to easily resize the VM, in few minutes.

The script will do the following :

1- Remove the VM

2- Construct the new VM object with the new Size

3- Create the new VM

You can download the script from the following link: https://gallery.technet.microsoft.com/Change-Azure-ARM-VM-size-039cda22