Is Microsoft Azure ready for IaaS ?

Update 

It’s almost one year since Azure Resource Manager was announced as Generally Available. This was a great day for me, since as per my point of view, this was the beginning of Azure IaaS as an Enterprise Public cloud Platform.

Since that day, a great customer I’m working for decided to accept the challenge and implement Azure IaaS on to of Azure Resource Manager. A decision which  was mixed with some fear and perplexity. And today here we are : What happens after One year ? Is Azure IaaS Resource Manager is ready for Enterprise ?

I will need tens of pages to describe all the story : Decisions, Challenges, missing features, bad and good designs, workarounds, patience. But to resume these are the main conclusions I may sate:

1- Building blocks of IaaS

IaaS (Infrastructure as a Service) can be ‘jargon’ speaking replaced  by Virtual Machines on the Cloud (Private or Public). To be able to build a complete platform using Iaas, the next build blocks must be present :

  • Compute : compute is the heart of IaaS, since it’s the heart of a Virtual Machine. It’s basically defined by : CPU and Memory (RAM) capabilities ie the principal Hardware Configuration. Compute decides of the performance of your workloads
  • Storage : Storage is  where all data reside and to which data is written and read. The quality of the Storage will decide of the quality of your workloads
  • Network : Network is the veins of the infrastructure, through it all information and data transit. It impacts directly the performance and the quality of  your workloads.

In addition to the above key building blocks, there are other factors which we judge ‘mandatory’ for an Infrastructure as a Service platform

  • Security : It’s the tools, features and functionalities controlling the security of the platform
  • Operations and Management : It’s the tools and the functionalities provided to manage and operate your platform
  • Backup : Mandatory and not an option, backup features are a key point to choose which platform and solution to use

 

2- Where is Microsoft Azure in all of this ?

The following are my conclusions and my thoughts about how Azure (my focus is on Resource Manager) performs regarding IaaS building blocks:

2.1- Compute

Microsoft Azure provides a rich offer on the Compute part. It’s continually evolving and providing new configurations. The offer difference is mainly about Compute Hardware (CPU type, Memory type) or the configuration itself (Core/Memory ratio, supported disk count). With the Public Cloud platforms, we may first be uncomfortable with the hardware configurations which we may find not standard (3.5 GB of RAM in of 4 GB!), but this is it, and our vision and ‘on-prem’ habits should change. Azure provides today different levels of compute configurations with different pricing variations, and I estimate the offer rich till today. Microsoft is progressively adding new configurations and compute offers.

Azure Virtual Machine series and sizes: https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-windows-sizes/

Azure Virtual Machine Pricing : https://azure.microsoft.com/en-us/pricing/details/virtual-machines/

Positive points

Negative points

Rich configurations offer
Continuous evolution
No SLA for Single Virtual Machine (You need to split your service on at least two VMs to achieve SLA)

No intermediate hardware configurations (From 7 to 14 GB RAM for example)

Positive points : Rich configurations offer, Continuous evolution

Negative Points : No SLA for Single Virtual Machine (You need to split your service on at least two VMs to achieve SLA)

2.2- Storage

Microsoft Azure currently provides two kind of Storage types : Standard and Premium. The difference is mainly about Performance, and consequently Pricing. While Premium Storage provides interesting and high performance, it’s relatively expensive and does not follow the ‘Pay as you Consume’ model, an extremely negative point. Standard Storage is not expensive, but we can’t say the same thing about Performance. Capped at 500 IOPS/disk, you cannot expect  results for your workloads requiring on storage performance.

Positive points

Negative points

Two Storage classes
Pricing for Standard Storage
Standard Storage performance is poor
Premium Storage pricing model : Expensive + ‘No pay as you go’
Should see more Storage offers (Cold Storage, Archive Storage…)

2.3- Network

Microsoft Azure Networking concept is very interesting. You can ‘almost’ translate you on-prem networking configuration on Azure. The concept of Virtual Networks and Subnets is easy to adopt. Still, passing from the VLAN based networks to this type of flat networks may be cumbersome for some people, but to be fair : It’s the same concept. Interconnections with on-premises networks or other virtual networks on other Azure regions is also provided by Azure via two models : VPN and Express Route. VPN is the standard way to connect two sites using commodity hardware and the internet network. Express Route is a Microsoft offer which uses a Private circuit (provided of course by a Network provider) to interconnect your on-premises sites to Azure datacenters with high bandwidth / Low latency properties. But on the meanwhile there still some missing features which may stop you or slowing you down indeed. Many customers want to use their standard products ad tools for Network Routing/Firewalling purposes (VPN, Firewall, IDS, IPS…) and to avoid using the Azure Network Security Groups feature (NSG –> Many drawbacks), for that many MS partners provide products adapted for Azure : Virtual Appliances (Fortinet, baracuda…). There are two big limitations blocking customers using Virtual Appliances:

  • VA high Availability : Since VA is an Azure VM, and since there is no SLA for single VM, you need to deploy at least two VAs and load balance traffic between them. But with 2 VAs we hit the load balancing limit. Azure Load balancer (which is the only way to achieve Lad balancing on Azure, since VIP are not supported) does not support balancing traffic at the IP level, protocol and a port is mandatory, hence no HA for virtual appliances.
  • UDR on Gateway Subnet : In order to use the native VPN and Express Route offer from Microsoft, you need to deploy a Gateway at the Virtual Network level. The issue today is that if you want to use a VA associated with the Azure Gateway to connect to other sites, it will be impossible. UDR (User Defined Routing) is a feature permitting changing the routing for your Azure subnets ie you can control where the next hops of your packets will land. This is very interesting since you can send all your Azure traffic to the Virtual Appliance for filtering. Unfortunately, since UDR cannot be applied to the Gateway Subnet, if you use a VA and the Azure Gateway to connect Azure to other sites, then all the inbound traffic to Azure via the Gateway will land directly on the subnets and not the VA : The combination of a VA + Gateway is impossible. In the other hand, you can use the VA to create a VPN with your sites, but as stated on the first limitation, there is no SLA for VA (Azure gateway is under an SLA of 99.9)

 

Positive points

Negative points

Simple to understand
Simple to design and deploy
Network Security Group is not a scalable Security solution

No HA virtual Appliances (No Load Balancing at the IP level)

No UDR on the Gateway Subnet

2.4- Security

Security is a huge topic, since Security can touch all the fabric level : Access Security, Network Security, Confidentiality, Integrity… and so on. But this post is intended to see if globally, Azure provides features and requirement for Security. Let’s talk about some Security aspects at different levels:

  • RBAC : Role Based Access Control was introduced last year with the new Portal to add the capacity to use pre-defined roles to limit the Access to the Azure resources. This was an achievement and gave customers more control about how their users and admin can manage the resources. The introduction of ‘Custom roles’ at the end of 2015 was just amazing.
  • Storage : Usually when we say Storage and Security, we say Encryption. Azure Storage does not support Encryption at Rest, unfortunately. This feature was introduced on Preview few weeks ago and supports only Microsoft managed keys and no Customers keys (Encryption at Rest first goal is to avoid MS access users data, so …), support for Customers keys is in progress. Today, if you want to achieve encryption for your IaaS VMs storage, you can enable Bitlocker on your Windows VMs as a solution. But I don’t think that managing bitlocker is appreciated by customers.
  • Network : This is actually a hard point on Azure. The feature provided by Microsoft for Network filtering and ACLing is Network Security Groups (NSG). NSG are  very difficult to managed and to implement, in addition to the limitations. The alternative is to use Virtual Appliances ie leaders products. But as stated on 2.3- Network, there are many limitations so far
  • Compliance and Trust : Microsoft Azure is adding day after day Compliance and attestation about the compliancy of its Cloud Platform. Nothing to say about it. https://www.microsoft.com/en-us/trustcenter/Compliance

 

Positive points

Negative points

RBAC
Compliance and trust
Partner Solutions for Security
Storage Encryption at Rest (Only on Preview + No customer managed keys)

Network Security features (weak points)

2.5- Operations and Management 

Imagine you got a Mercedes S Class with the best options and engine, but without a dash (This is unlikely but just image it), you will drive like a blind, no Speed, no signals, nothing. I’m sure you will not buy this car and prefer a bicycle instead. But this is not the case for Microsoft Azure and specially IaaS. A bunch of tools and solutions are provided to manage and operate the platform. I can enumerate the Azure Portal itself, Azure Powerhell, CLI as the basic management ways. Azure provide Audit Logs for all the operations logs, notifications, dashboards and diagnostic logs too. Azure also provides OMS (Operations Management System) which include a monitoring feature, and the catalog is growing. The least and not the last, Azure Security center which gives you a Security view of your IaaS platform.

Positive points

Negative points

A bunch of tools for management and monitoring OMS is cool, but not all the features are yet included

A lot of tools means a lot of needed knowledge

2.6- Backup

The last point of this post is Backup. After one year of Struggle, Microsoft announced on Mars the Public Preview of Azure Backup for IaaS V2, finally. Azure Backup for IaaS V1 is GA from long time ago, but before I’m mainly talk about Azure ARM, this was my struggle. This capability prevents us from using Azure for Production since the completion of the Pilot (End 2015). No Backup means No Production means nothing.  Fortunately, GA is expected at Q2, so starting from July, Azure IaaS V2 customers will be able to backup their Virtual Machines and hence, use them for production. Customers can from now, PoC the solution, and create their policies, since no major changes on operations or design will affect it (https://buildwindows.wordpress.com/2016/04/13/azure-backup-with-azure-recovery-services-features-and-limitations/)

Positive points

Negative points

GA expected before July No supported Backup solution for IaaS V2 today

3- The Verdict

I’m an Azure lover, I trust and believe in this solution as a believed on Hyper-V, since 2008 R2 SP1 vs the giant VMware. The question is : Is Azure ready for IaaS today ?

The answer : Yes for Envisioning, Design and PoC, and starting from this summer* for Production

* In this Summer, many missing features are expected to see birth like Azure Backup GA, UDR on Subnet Gateway, Storage Encryption GA and VA high Availability. This will be enough to finally benefit for the Power of Azure

Azure Backup with Azure Recovery Services : Features and limitations

Hi all,

It has been  days since Microsoft announced the Public Preview of Azure Backup via Azure Recovery Services. In this post I will enumerate the different features and limitations of the service, to help you decide if it fits your needs.

NB : This post is only related to IaaS part of Azure Backup

The following is the agenda of this post :

Introduction to Azure Backup via Recovery Services

Azure Backup for Azure IaaS features (Current and Coming)

Azure Backup for Azure IaaS  limitations

1- Introduction to Azure Backup via Recovery Services

Azure Backup was released first time under Azure Backup vaults, and it was only supporting classic Azure IaaS (Azure Service Management ie IaaS v1). With the GA of the Azure Resource Manager stack on summer 2015, IaaS V2 users were not able to use Azure Backup to protect their V2 virtual machines. This was the first blocker of the ARM stack adoption and one of the most wanted feature regarding the ARM platform.

0960

https://feedback.azure.com/forums/258995-azure-backup-and-scdpm/suggestions/8369907-azure-backup-to-support-iaas-vm-v2

After 10 months of struggle, Microsoft announced the Public Preview of Azure Backup supporting IaaS V2 virtual machines. It’s a real alleviation for Azure IaaS V2 users, but also for all Azure users planning to use Azure backup features. The main difference is that Azure Backup is now part of Azure Recovery Services vaults, and no longer Azure Backup vaults. Azure Backup vaults still exist under the ASM stack, but it’s clear that sooner or later, all will be integrated to Azure Recovery Services.

Azure Recovery Services include both Azure Backup and Azure Site Recovery supporting both ASM and ARM stacks. This is what we call great news:

  • Azure Recovery Services is integrated to the new portal (Ibiza portal)
  • Azure Backup and ASR under Recovery Services vaults support both ASM and ARM stacks

Azure Backup under Recovery Services vaults support the 4 backup scenarios:

  • Azure Backup Server or Agent based:
    • Azure Backup Agent to Azure –> Backup files and foders to Azure Storage
    • Azure Backup with System Center Data Protection Manager –> Backup Hyper-V VMs, SQL server, SharePoint, files and folders to Azure Storage
    • Azure Backup with Azure Backup Server (MABS, code name Venus) –> Backup Hyper-V VMs, SQL server, SharePoint, files and folders to Azure Storage
  • Azure Backup on the Azure Service Fabric :
    • Azure Backup for IaaS VMs –> Backup Classic and ARM Azure Virtual Machines

0962

This post will only detail Azure Backup for IaaS virtual machines

2- Azure Backup for Azure IaaS features (Current and Coming)

Azure Recovery Services is currently under Public Preview. The following are the features of Azure Backup and the expected features that will come with GA:

  • Backup and Restore ARM and ASM Azure virtual machines (V1 and V2)
  • Based on backup policies : Two backup schedules exist : Daily and Weekly. This way you can define backups which occur daily or weekly
  • Azure Backup provides different retention periods possibility : Daily, Weekly, Monthly and yearly. Microsoft officially stated a maximum retention period of 99 years, however, thanks to Azure Backup flexibility, you can have unlimited retention period, up to 9999 years. This way, you can achieve long term retention using the same policy and mechanism (9999 days for daily backups, 9999 weeks for weekly backups,9999 months for monthly retentions ,9999 years for yearly retention)
  • Azure Backup provides 3 recovery point consistency types : Application, File and Crash consistent recovery points. You can consult the documentation to get the requirements and prerequisites for each type
  • The Backup Vault’s Storage redundancy can be GRS or LRS. GRS is more secure (Data is replicated between two regions) but more expensive (LRS *2), LRS is less secure (Locally Redundant) but cheaper. As per my experience, because the Azure Backup pricing is per protected instance (And the price is relatively high), you will notice that the Storage cost is a small fraction of the Azure Backup instances cost, so using GRS will not really impact the bill.
  • Azure Backup use incremental backups : The first recovery point is a full backup, the next ones are incremental backups : This reduce the consumed backup storage. Due to the Azure Backup design and mechanism, incremental backups will not impact the restore time.
  • Simple pricing model : The cost of Azure Backup is like the following : Total Cost = Instance Cost + Consumed Storage. If you know the daily change or growth of your data, than you can easily predict the backup cost. See this link for Azure Backup pricing :  https://azure.microsoft.com/en-us/pricing/details/backup/
  • A backup operation consist of two phases : Snapshot phase and Data transfer phase. The snapshot phase occur when the scheduled moment comes. The data transfer he backup vault begins just after the snapshot completion. This operation lay take up to 8 hours during rush hours but will always completes before 24 hours.
  • Azure Backup provides 99,99 availability SLA for Backup and Restore, monthly based. This is only applicable for the GA product.
  • Currently, two restore options are available
    • To a Virtual Machine : A new Virtual Machine is created
    • To a Storage Account : VHDs can be restored to a Storage Account
  • I expect some features to come with and post GA, but this my own thoughts, since this is what actually implemented with DPM and MABS :
    • Backup/Restore of Files and folders from a VM recovery point
    • Backup/Restore SQL or/and MySQL databases directly from a VM

3- Azure Backup for Azure IaaS limitations

  • Azure Backup does not currently support Premium Storage virtual machines. This feature will released probably during the GA
  • Currently, the daily backup supports 1 recovery point per day ie you cannot backup a Virtual Machine more than once time a day. To achieve this, use the ‘manual backup’ to schedule more than one backup a day. Keep in mind that two simultaneous backups are not supported, so you will need to wait for the first once to compete before triggering the next one.
  • The Azure VM agent and the Backup extension are required to achieve Application or File consistent recovery points. Otherwise, the recovery point will be crash consistent. Be careful of the Azure VM and Backup agents network requirements 
  • The ‘Backup now’ operation does not replace a ‘Snapshot’ mechanism if you want to rapidly restore a VM (The recovery point may take up to 8 hours to be available)
  • Currently, the Restore to a VM is not very customizable : You cannot choose a number of properties like Storage Container, VHDs names, NIC names … To have control of the created VM, you can restore the VHDs to a storage account and use a script or template to create a VM with the configuration of your choice.
  • There is no notification system built-in with Azure backup. So you can’t at this stage configure notifications for the backup jobs statuses. However, there possible alternate methods to do it : When Powershell will be supported, you can create automation scripts which get the Backup jobs statuses and make the notifications. You can also use the Azure Audit logs since the Backup operations are logged within them
  • No Powershell support, but will be released with GA
  • You cannot edit en exiting policy. If you want to change a policy, you will need to create a new one and change the VM’s assignment. Things will change by GA, so no worry
  • You cannot change the vault Redundancy type once you configured at least one backup. You need to change the redundancy  before any data is being transferred to the vault
  • There some limitations about the backup / restore possibilities, I will rephrase here the documentation
    • Backing up virtual machines with more than 16 data disks is not supported.Backing up virtual machines with a reserved IP address and no defined endpoint is not supported.
    • Backing up virtual machines by using the Azure Backup service is supported only for select operating system versions:
      • Linux: See the list of distributions that are endorsed by Azure. Other Bring-Your-Own-Linux distributions also should work as long as the VM agent is available on the virtual machine.
      • Windows Server: Versions older than Windows Server 2008 R2 are not supported.
    • Restoring a domain controller (DC) VM that is part of a multi-DC configuration is not supported.
    • For classic VMs, restore is supported to only new cloud services.
    • Restoring virtual machines that have the following special network configurations is supported through restoring disks to a desired storage account and using PowerShell to attach restored disks to VM configuration of choice. To learn more, see Restoring VMs with special network configurations.
      • Virtual machines under load balancer configuration (internal and external)
      • Virtual machines with multiple reserved IP addresses
      • Virtual machines with multiple network adapters