How to protect and backup your Branch and Remote offices data (Files and Folders) ?

Hi everyone,

Since the first days of the adoption of an Information System by companies, backing up the workloads was crucial and a production blocker : No production without backup, no backup,  no business.

Today, companies are better mastering and understanding their backup needs, solutions and they are continually seeking for better, simple and cost effective backup software.

One of the ‘headache’ subjects that bother the majority of the backup admins and decision makers is the Remote Offices / Branch Offices (ROBO) ‘Files and Folders’ data backup.

During this post, I will show why Azure Backup via the MARS agent is your best choice to get rid of the ROBO workloads backup problematic. I will present :

  • Use cases for using Azure Backup via MARS agent
  • What do you need to know in order to be comfortable with this solution
  • What are the steps to plan and start using Azure Backup via MARS agent for your ROBO

1- Use cases for using Azure Backup via MARS agent

Azure Backup is the name of a complete enterprise backup solution allowing several backup scenarios and using the last technologies, specially the ability to back up to the cloud and to benefit from a GFS model (a model allowing  efficient Long term retention policies).

What is interesting about Azure Backup via MARS agent is that it allows you to backup your files and folders without the need to deploy a Backup Infrastructure or a Storage infrastructure. This opens up a lot of use cases :

Backup without backup infrastructure

The following picture shows the end to end data journey from your Windows Server or Workstation to the cloud storage (More details about the components later on this post). As you can note, the backup will needs only the installation of the Azure Backup Agent (MARS agent : Microsoft Azure Recovery Services agent) and to configure it to
backup data to a cloud location (Recovery Services Vault)

This is fantastic since it removes the classic requirements to enable workloads backup :

  • Backup software infrastructure (Backup server, Backup Proxy…)
  • Local storage : No need for a SAN or a NAS. Azure backup will directly send data to the cloud using an internet connection

Short and Long term retention without backup infrastructure

In addition to the great value from the first discussed statement, Azure Backup provides in the same time, Short and Long term retention within the same policies. No need for tapes, no need for external provider to handle it. Azure Backup use a GFS model to allow  Long Term retentions without any additional configuration. You can reach up to 99 years of retention period for up to 9999 recovery points (These values can change on the future).

Low bandwidth/Latency ROBO locations

The Azure Backup agent supports throttling (2) the data transfer to the cloud location (Not for all OSs). This is very important for ROBO location with limited bandwidth that prevent you from using your central backup infrastructure (Backup to a central backup repository)

 

2- What do you need to know

In this section, I will resume the important information that you need to know about the Azure Backup (Specially with the MARS agent). These information will give you the ability to decide, design and implement Azure backup into your information system.

2.1- Pricing

Fortunately, the Azure Backup pricing is very simple. It’s well explicated on the official documentation (1) but to resume:

When you backup a workload, you pay for :

  • An Azure Backup fixed cost for each backed up instance (The cost depends on the size of the data being backed up)

 

  • The storage used by the recovery points:
    • You can choose between LRS or GRS storage (3). To resume, LRS (Locally redundant storage) is a storage only available with the region where you create the Recovery Vault. GRS is a storage replicated asynchronously to another paired region providing hence, a protection against region failure, but more expensive (4) (~ * 2)
    • The redundancy cannot be changed after the first workload backup, so be sure of your decision before going forward

 

For example, if you backup 4 windows servers, you will pay:

  • 4* Azure Backup fixed cost
  • The cost of the Azure storage (cloud storage) used by the recovery points

2.2- Requirements

In this section, I will resume what do you need to technically be ready to use Azure Backup (via the MARS agent)

 

2.2.1-  Azure Level

As discussed earlier in this post, you need the location where you will send and store backups. This is called Recovery Services Vault (RSV). An RSV is a Microsoft Azure resource, which means that you need to subscribe to Azure in order to deploy it. Subscribing to Microsoft Azure is very simple, there are many ways to achieve it, depending on your needs and the billing/relation model that you want. In order to use Azure, you need to create an Azure subscription (5). After creating it, you can directly without any requirement create an Azure Recovery Vault, ready to host your backups (within minutes).

You will then need access* to the Recovery Vault in order to begin. You can benefit from the Azure RBAC roles (6) in order to have or give required permissions.

In order to backup Files and Folders via the MARS agent, you will just need:

  • The MARS agent installation file : Allowing you to install the agent on the required servers
  • The Vault credentials : Allowing the MARS agent to find and authenticate to the Azure Recovery Vault.

Both of them can be downloaded via the Azure portal via the Azure Recovery Services resource blades.

* Technically, you don’t need access to the Recovery Vault to enable backups. An Admin can send you the required information instead.

2.2.2- Local level

I mean by local level, what do you need at the server level (The server where the folders and files to be backed up) in order to start backing up :

  • A supported Operating system : Only Windows is supported, Linux is not yet supported.
  • A internet connectivity : The agent needs outbound internet connection to the Azure services in order to send data. Using a Proxy is supported. You can in addition limit the outbound flows to only Azure services public IPs (7) (And even more, only the IPs belonging to the RSV region)

 

There are limitations regarding the supported operating systems, what can you backup, how often you can backup and more. Please refer to the Azure Backup FAQ for complete information

 

2.3- Security and data confidentiality

Azure backup via the MARS agent provides many precious security aspects, let me enumerate some of them:

  • You will need a Vault credentials file in order to register an agent to a vault. Only backup admins can download such file from the Azure portal
  • Before enabling the backup, you will be prompted to provide a ‘passphrase’. A passphrase is a ‘complex password’ used to encrypt data before sending it to the RSV. Data is encrypted and send via HTTPS to the RSV where it remains encrypted. Note that without this passphrase, you will not be able to restore data in case you lose the original server (Or its configuration), the passphrase must be kept securely somewhere (You can use Azure Key Vault to store your secrets)
  • In case your server is compromised, the compromiser (Hacker, malicious admin) cannot delete you recovery points. Azure backup provides a security setting (enabled by default) that requires the ‘remover’ to login to the Azure Portal and generate a PIN code. The probability that the ‘compromiser’ owns the credentials to login to the Azure portal is small. In addition, you can benefit from the ‘MFA’ feature of Azure portal in order to more secure the portal access.
  • In case of ransomware/crypto-locker attack or infection, your backup data is protected, since the backup media is totally independent of the server.
  • Other security prevention feature are also available (8) :
    • Retention of deleted backup data: Backup data retained for 14 days after delete operation
    • Minimum retention range checks: Ensures more than one recovery point in case of attacks
    • Alerts and notifications: For critical operations like Stop backup with delete data
    • Multiple layers of security: Security PIN required for critical operations (Already mentioned)

2.4- Monitoring and reporting

Like you noticed, there is no server nor a console to install, monitor or see what is happening. All is done via the Azure Portal. You can use the Azure portal to :

  • Backup Items : View the backed up items (Server name, volume…)
  • Backup Status : You can view and show the status of the backups, with ‘filtering’ options
  • Backup jobs: You can see the backup jobs and their status. You can see the duration and the size of the backups and restore operations
  • Notifications : You can configure and see the notifications related to the jobs. Currently, you can only configure notifications based on the jobs status (Critical, Warning, Information)

Currently, there is no ‘Reporting’ feature with Azure backup via the portal. But this feature is coming very soon.

3- How to start : The plan

In this third and final section, I will present the planning steps in order to successfully plan and implement your ‘Folders and Files’ backup. The main steps are :

  1. Create a Recovery Services Vault
  2. Configure the vault
  3. Download the Recovery Vault credentials
  4. Install the MARS Agent on the server
  5. Create a backup policy and a schedule

This link shows the detailed steps to achieve the above steps : https://docs.microsoft.com/en-us/azure/backup/backup-configure-vault

The Azure Backup FAQ contains the most answers to your questions :

https://docs.microsoft.com/en-us/azure/backup/backup-azure-backup-faq

To finish, the following are my recommendations when planning to implement Azure Backup via the MARS agent:

Question / Constraint

Choice

Are my source servers located on the same region ? It’s recommended to backup data to the nearest location in order to benefit from a better performance / Latency during backup and restore operations.
Do I need to back up to the same RSV ? No, but to have a simple design, it’s better to minimize the number of RSV for the a similar servers group.
When do I need to backup to different RSV What can differentiate two Recovery Services Vault  :

–         The redundancy of the Storage (LRS or GRS)

–         The user rights on the RSV

–         The vault credentials

So :

–               If you have different ‘data’ importance, and you want to optimize the costs, you can create ‘LRS’ RSVs for less important data, and ‘GRS’ RSVs for more important and critical data

–               You can give permissions to access or manage the Recovery Service Vault. If you want different security levels for your Vault, you can create multiple RSV

–               The Vault Credentials are unique for an RSV. A user with a valid Vault credentials file (expires after 2 days) can backup data to the vault

Use the same passphrase for each server ? No. This is absolutely not recommended for the unique reason is that someone compromises the passphrase, he can access you all your server’s restore points (He will need a valid Vault credentials file)

 

Useful Links:

 

(1) Azure Backup pricing : https://azure.microsoft.com/en-us/pricing/details/backup/

(2) Azure Backup agent network throttling : https://docs.microsoft.com/en-us/azure/backup/backup-configure-vault

(3) Azure Storage redundancy : https://docs.microsoft.com/en-us/azure/storage/storage-redundancy

(4) Azure Storage pricing : https://azure.microsoft.com/en-us/pricing/details/storage/blobs-general/

(5) Designing Azure Subscriptions : https://buildwindows.wordpress.com/2016/03/30/azure-iaas-arm-architecting-and-design-series-azure-subscriptions/

(6)Azure Backup Roles : Backup Contributor, Backup Operator, Backup Reader

(7) Azure Public IP ranges : https://www.microsoft.com/en-us/download/details.aspx?id=41653

(8) Azure-backup-security-feature : https://azure.microsoft.com/en-us/blog/azure-backup-security-feature/

(9) Azure subscription and service limits, quotas, and constraints : https://docs.microsoft.com/en-us/azure/azure-subscription-service-limits

Azure TCO calculator Public Preview

Hi,

Microsoft just annouced the public preview of its TCO calculator tool.

This tool will help you see if Azure will reduce or not the TCO of your Virtualization or Physical on-premises platform. It will give you a cost forcast on a 3 years period. This is a valuable tool for organisations planning to renew their on-premises infrastructure or a part of it. 

This tool is a must for Cloud Architects, consultants or IT managers.

Give it a try, i will update the article with the feedback links.

https://www.tco.microsoft.com 

Azure Virtual Machines single instance SLA, a big step!

Hi all,

Early this week, Microsoft made an exciting announcement with its SLA for a single virtual machine of 99.9 ℅ : https://azure.microsoft.com/en-us/blog/announcing-4-tb-for-sap-hana-single-instance-sla-and-hybrid-use-benefit-images/

Before this announcement, single instance VMs (which are not part of an Availability Set) were not covered by an SLA. This was unattractive for workloads which do not support, afford or need a multi-instance deployment. This was generally applicable for legacy workloads and to be honest, to the majority of non-critical workloads, and for SMB workloads which do not afford investing in redundancy and HA.

Many of my customers avoided migrating workloads to Azure, just because of this, which was offered by AWS since a while.

With this announcement, customers have the warranty that an SLA of 99,9 % provided to their VMs, which means a maximum downtime of 8.76 hours per 356 days –> 44 minutes per month

Do not forget that this is only applicable for virtual machines with all disks stored on Premium Storage.

What do we need to know about Azure Stack : The Q & A

Hi all,

It has been a long time since I didn’t blogged anything. A lot of news actually happened on the last few months, and in this post I will explain one of the most existing for me : Azure Stack

Azure Stack was introduced earlier this year (January) with a first Proof of Concept named TP1 (Technical preview 1). The Technical Preview goal was to give customers, consultants and early adopters a view of what Microsoft baptized as the future of Private and Hybrid cloud. But, really, what is Azure Stack ?

The modest definition 

Azure Stack is a platform (Software) that you can deploy on-premises to have similar Microsoft Azure services,  features and user experience. If you are using Microsoft Azure (The new portal, known as Ibiza portal portal.azure.com), than this is what you will get when you deploy Azure Stack on your datacenter. You will be able to leverage the Azure underlying technology on-premises, to deploy, manage and benefit from the Azure services like virtual machines, web apps, virtual networks and the list evolves. Think just that instead of typing portal.azure.com on your browser, you will type a custom URL on your domain that will land you on an Azure portal, but on your datacenter.

Is Azure Stack suitable for my company or business ?

Azure Stack brings advanced cloud technologies to your datacenter from the virtualization platform (a simple virtual machine) to the Azure App Services (A PaaS model to rapidly deploy Web Applications). So technically, Azure Stack can be used by any company aiming at least to use virtualization, but this is not enough to adopt it. As a consultant and  an Azure Architect, I think that Azure Stack is suitable for you if :

  • You are using or at least experimented the user experience, concept and different services provided by Microsoft Azure. If you have validated that Azure is suitable for your company, and you are looking for the same experience on-premises (for any reason) then Azure Stack may be a good choice (Azure Stack is consistent with Azure)
  • You are looking for a  private cloud platform which can provide the last cloud technologies and concepts. Azure Stack is born from Azure and will continually benefit from the last enhancements made and tested on the Azure public cloud platform
  • You are looking for a modern way to faster build your applications and services, which is the model based on PaaS and micro services. Azure Stack in its first version (mid 2017) will support Azure Web Apps and maybe Azure Fabric if they decide to bring it.
  • The constraints  I will mention next do not bother you

How Azure Stack will be delivered to customers ?

This is the actual debate, but Microsoft elected the winner, with a sort of inflexibility. Azure Stack will only be provided via System Integrated platforms with the freedom to choose between 3 different Hardware providers : HPE, DELL EMC and Lenovo (formerly x86 IBM servers). This means that you cannot deploy an Azure Stack platform on top of your hardware, but you will need to acquire the hardware with Azure Stack pre-packaged, and just plug it to your datacenter.

This last statement created a rage from the community, and we got two visions :

  • Microsoft is affirming that this model is the only possible way to achieve the wished Enterprise level private/hybrid cloud platform. Microsoft is stating that the integration with the hardware is a very heavy task and it prefers validating the platform and then provide a ‘ready engine’ to the customer.
  • The community is surprised that Microsoft is, first locking-out its solution for a set of non-affordable hardware providers, and secondly, not following the original virtualization and cloud ideology, which is the reuse and the optimization of the existing resources, and even, use affordable and commodity hardware.

Azure Stack licensing and prices

This is what I call the mystery question for the public because I have an early information, but due to NDA, I’m not allowed to publish it. What I can say, is that whatever the licensing model is, I think that it will be expensive, and I wonder if it will reach the SMB market. Remember, there are 3 parties involved : The hardware provider, the software provider and the integrator (which is Microsoft anyway but should be counted as a third party IMHO)

 

What if I acquire Azure Stack ? What about the test platform ?

This is a question I have asked before, and the answer was quite not sufficient. This is the summary :

  • Microsoft will provide the one Node PoC which is a one node Azure Stack platform. It’s what delivered today on TP1 and TP2. You can install Azure Stack on one node to be able to PoC, discover and make the tests you want. But, on the meanwhile we are not certain (no information) of the accuracy of the one node PoC with the System Integrated Azure Stack platform in terms of minor updates and bug fixes, and more important features.
  • You can do what you actually doing on Azure : Create a Test subscription with limited quotas, where you can make deployment tests  –> This still depends on the licensing model as we don’t want that a test platform be costly

 

What are the actual sizes of the Azure Stack Integrated System offers ?

It’s too early to speak about the capacity (CPU, Memory, Storage) of the Azure Stack platforms provided via the Integrated Systems. Things can change and I think that the final sizes will be revealed during the GA. Anyway the minimum published size today is  4 hosts with 256 GB and double sockets with 8 cores each constituting a single scale unit (Hyper-V cluster) and which will contain both the management infrastructure (VMs needed for Azure Stack to work) and your workloads (IaaS VMs, App Service Plan VMs…). Do not forget that the Azure consistency implies that the virtual machine sizes that you will be able to deploy are the same than Azure VM sizes. Hans Vredevoort has thorough articles about the Azure Stack Architecture :

Where System Integrators are placed on this whole thing ?

This is one of the questions I asked to myself. On standard products  like System Center and Windows Azure Pack, system integrators were almost mandatory to successfully deploy these products within a customer site. But the question raised up with the decision to only provide Azure Stack via System Integrated : No more need for system integrators to deploy the solution, a sort of plug and play.

This is true, but this isn’t so bad (No the case  for geeks, unfortunately Smile )

In fact, what are you integrators doing today with your customers when they call you to help them design and deploy their workloads on Azure? you are certainly happy, and this is why we should be optimistic when speaking about Azure Stack.  Because Azure Stack is Azure on your datacenter (An Azure sealed black box), customers will need you to help them first choose which Azure Stack offer (tier) to purchase, and then help them use Azure Stack, the same what you are doing  today on Azure. The consistency will make your Azure expertise vey valuable on the on-premise field.

What we will miss, is having our hands on an Azure Stack real platform to make some practice. But, theoretically, this will not be the biggest problem since we can use the one node PoC to achieve such goals. What is causing the headache for the com munity, is the near-impossibility of deploying the 1 Node PoC on our LAB at home. The minimum RAM requirement for the TP2 is 96 GB of RAM, and this is the minimum, expect up to 128 GB to start enjoying the LAB and the deployment of all the services. I don’ t know many having a real 128 GB RAM server at home.

 

Can CSP benefit from Azure Stack ?

Things are not yet dry, and Microsoft did not yet publish a clear view of the Cloud Service provider interaction with Azure Stack. But, it’s clear that that through the CSP program, CSP will be able to use Azure Stack to deliver sophisticated Azure like features. The biggest factors that may slow down CSP form using Azure Stack are :

  • Locked-down hardware providers : Cloud Service Providers are certainly partnering with hardware providers to acquire discounts and advantages when buying hardware. I’m very pessimistic regarding this factor. CSP may look to other cloud platform solution or continue build their own
  • Licensing and pricing : The introduction of  locked-hardware model may impact the margin the CSP can generate. No comment on the Software part licensing.

 

What do I think of Azure Stack and the implementation model ?

Microsoft is a great Enterprise, master minds are working there trying every day to enhance their products, creating new technologies and approaches, pivoting and changing their business model to impact the market. But, no human is bullet-proof, Microsoft can make mistakes and failures (The case for Windows phone, which is terribly not progressing ), it’s dramatically changing its business model with Azure Stack : Cloud Appliances. I’m really waiting for the licensing announcement to see which customer segment it’s targeting. But what I’m waiting for is the customer reaction. I have no idea how they will react. My first impression is that this model is not appreciated by the community and by me. I’m not against System Integrated platforms, but I’m with the virtualization and cloud early goals : Hardware reuse and cost reduction. Azure Stack is not fitting there, in addition of making restrictions on the hardware providers we can choose from, I have a bad feeling melted with a great excitement to this Azure Stack era, hoping it will find its way.

Based on my opinion, the following are the Pro and Cons of Azure Stack :

The GOOD

    • Azure services on your datacenter : This is the most exiting about Azure Stack, you can bring Azure features (IaaS, PaaS…) on your datacenter. You have the last tested*  cloud technologies on your hand. If you are avoiding using the public cloud platforms for any reason (Privacy, Compliance, trust, network connectivity) but on the same time wishing using the provided features, then Azure Stack is for you
    • Plug-and-Play model : The System Integrated model will reduce the TCO, by bringing a ready-to-use private cloud platform
    • A true Consistent and Hybrid cloud platform : Azure Stack is a real advantage for customers using or planning to use Azure services, since the consistency between the platform is guaranteed. You can use the same approaches, design decision factors, tools, scripts and features. You no longer need two different models to manage your cloud and on-premise platform, thus reducing significantly the IT efforts which can be spent on real business concerns (Deploying apps, migrating, enhancing…)

The Bad

    • The other side of System Integrated model : Hardware locking was never a good idea, depriving the customer from freely choosing  its hardware provider, and hence better control the costs. This model will reduce the early adopters market and hence can slow down this beautiful product from being used widely.

* Azure Stack will bring features already used on Azure, so we are sure that the features were widely tested on the public cloud

If you think the System Integrated model is not suitable for you, support this idea, you can participate and influence changing the Azure Stack future : Provide an installable version of Azure Stack

How can we use Azure Cool Storage for Archiving data ?

Azure Cool Storage was introduced early this summer as a new Azure Storage Account tier type. The story is that before this time, the Azure Storage provides only one kind of Storage Accounts, which is for general purpose, and which includes two performance tiers, this makes using a Cloud Storage Account for storing daily data (Virtual Machines disks, File shares…) the same than archival and backup data.

A new type of Storage Accounts was introduced which brought two different tiers of Storage Accounts : Hot and Cool. The following picture shows the Azure Storage Account types :

SA-Types

This article explains how you can use the Azure Cool Storage tier for Archival purpose.

1- Scenarios proposal

1.1- Use StorSimple physical Array

Azure StorSipmle is a hybrid storage array which provides features to store data locally and on the cloud. StorSimple has many advantageous and offers a lot of interesting features like :

  • Tiering: As your data ages, automatically tier it and move it from solid-state disk to spinning hard drive to the cloud
  • Local volume: Ensure any primary data that you specify as local will stay on the physical StorSimple device
  • Cloud snapshot: Enable rapid movement of volume snapshots to the cloud to support integrated disaster, as well as dev/test and business analytic environments
  • Data deduplication: Deduplicate data inline to reduce your storage growth rates

The following capabilities will make the proposed design successful:

  • Azure Storsimple supports tiering on the Cloud –> Cold data will be moved from the local storage to the Cloud Storage
  • Azure StorSimple supports Azure Cool Storage Accounts as a Cloud Storage
  • Azure StorSimple can be exposed via iSCSI

The scenario is like the following :

  • A Cool Azure Storage Account (or more)  will be created an linked to the StorSimple.
  • A Volume (or more) will be created on the StorSimple
  • The volume (or volumes) will be presented to a Windows File Server
  • File Shares will be accessible for the Admins or the tools which will move data to be archived to these shares

–> Thanks to the Tiering feature, data which is Cold will be automatically tiered to Azure Storage Account. Because this data is not willing to change; we expect 100 % of cold data and hence tiered data

NB : Keep in mind the different StorSimple limits : https://azure.microsoft.com/en-us/documentation/articles/storsimple-limits/

1.2- Use StorSimple virtual array

If you don’t or are not planning to use the StorSimple physical array, you can use the StorSimple Virtual Array which was introduced early this year. The Virtual Array can be hosted by a Hyper-V or VMware virtualization platform. The StorSimple virtual array provides similar features than the physical array except the local storage, which should be provisioned from the virtualization platform storage. Keep in mind that the virtual Array provides less capacity than the physical one (Look to the first table on https://azure.microsoft.com/en-us/documentation/articles/storsimple-ova-overview/).

The Virtual Array has an additional feature compared to the physical one, which is the File Share feature. In fact, it can exposes directly SMB shares to the users.

The design is like the following :

Variant 1

  • A Cool Azure Storage Account (or more)  will be created an linked to the StorSimple.
  • A Volume (or more) will be created on the StorSimple
  • The volume (or volumes) will be presented to a Windows File Server
  • File Shares will be accessible for the Admins or the tools which will move data to be archived to these shares

Variant 2

  • A Cool Azure Storage Account (or more)  will be created an linked to the StorSimple
  • A file share (or more) will be created
  • File Shares will be accessible for the Admins or the tools which will move data to be archived to these shares

–> Thanks to the Tiering feature, data which is Cold will be automatically tiered to Azure Storage Account. Because this data is not willing to change, we expect 100 % of cold data and hence tiered data

1.3- Use Microsoft Azure Storage Explorer

Microsoft Azure Storage Explorer is a tool developed and maintained by Microsoft which allow managing and making different operations on the Azure Storage accounts.

The tool can be downloaded here : http://storageexplorer.com/

NB : Please, do not confuse Microsoft Azure Storage Explorer and Azure Storage Explorer. This latter is an open sourced project which is not very well maintained (But was here first)

The scenario is like the following :

  • Install MASE on a server (We recommend a dedicated server)
  • Use AZcopy (The MASE Command line) to copy the data to be archived to the Azure Cool Storage Account. You can use the Storage Explorer UI either. The AZCopy tool is good if you want to schedule data movement or create scripts.
  • Use MASE to explorer and download the Azure Storage Account data

NB: MASE does not provide mounting a Storage Account Container, and managing the content (Browser, Copy, Delete…) via a drive letter.

1.4- Use a third party tool

There are different Third Party Tools which allow managing and accessing Azure Storage Accounts. The difference regarding MASE is that these tools allow mounting the Storage Account containers to virtual drives. This way they can be accessed directly via the windows explorer.

The scenario is like the following :

  • Install the tool on a server (We recommend a dedicated server)
  • Attach a Cool Azure Storage Account using the tool and create a mount point (Or a virtual drive)
  • Admins can move data to be archived directly to the mount points

The following are two tools which can be used for this purpose:

2- Scenarios Comparison

Scenario

+

My Rating

StorSimple Physical Array Best capacity and performance Expensive or needs minimum credit if purchased under EA

No SMB file shares support like the SVA

Use it if you already have a physical Storsimple

Enterprise solution

StorSimple Virtual Array Support SMB File Shares (Windows File Share + Share Permissions)

Free

Needs a virtualization platform and a local storage capacity  (10 % of the total storage)

Limited capacity if exposed via iSCSI (5 TB per Volume)

Use it if you don’t have a physical Storsimple or you don’t want to deploy a File Server

Enterprise solution

Microsoft Azure Storage Explorer Free

Simple to use

Storage Accounts are only exposed via the tool UI and command line Personal or occasional use
Third Party Tools Simple to use May not be scalable for Free editions Personal or occasional use

3- Azure cool storage costs

Assumptions :

  • I do not count blobs operations which are billed too. We estimate that the cost of these operations will not pring impact on the total cost. In addition, it’s not predictable
  • I assume that there is no change on the already written data (archived data does not change over time). The change will inquire write costs
  • I’m using Public prices North Europe region (Euro)
  • Do not forget that the pricing is per Month for the Storage and per Operation (per GB) for Write, change and retrieval
 

Storage per Month

Data write per Operation

Data retrieval per operation

LRS

GRS

LRS

GRS

LRS/GRS + outbound data transfers

1 GB

0.0084 €

0.0169 €

0,0021 €

0,0042 €

0,0818 € = 0,0084 + 0.0734

10 GB

0.084 €

0.169 €

0,021 €

0,042 €

0,818 € = 0,084 + 0.734

100 GB

0.84 €

1,69 €

0,21 €

0,42 €

8,18 € = 0,84 + 7,34

1 TB

8,4 €

16,9 €

2,1 €

4,2 €

81,8 € = 8,4 + 73,4

Costs example with a Cool storage account in GRS redundancy:

  • If we store 1 TB of data we’ll pay: 16,9 € / month + 4,2 € once
  • If we download 1 GB we’ll pay: 0.0818 € once
  • If we download 100 GB we’ll pay: 8,18 € once

Change Azure ARM VM size

Hi all,

Microsoft has published an article describing how to resize an Azure Virtual Machine, and shown that on the major cases, using the Portal or PowerShell is enough to resize the VM within few minutes with a short downtime (Reboot). I have discussed that on a previous post. You can see that on same cases, the host where the VM is deployed does not support the target size, so you will not be able to resize your VM without redeploying it.

I thought that the ‘Redeploy’ feature will redeploy your VM on another host supporting more VM sizes, but I think that the it redeploys it on the same ‘Pool’ with same characteristics.

So I decided to write a quick script (you can use the functions or adapt it to your context) to easily resize the VM, in few minutes.

The script will do the following :

1- Remove the VM

2- Construct the new VM object with the new Size

3- Create the new VM

You can download the script from the following link: https://gallery.technet.microsoft.com/Change-Azure-ARM-VM-size-039cda22

Azure IaaS (ARM) : Architecting and Design Series | Azure Subscriptions

Hi All,

This is the first post of the Azure IaaS (ARM) : Architecting and Design Series, and today the topic is : Azure Subscriptions

This post will detail  the following points :

1- What is an Azure Subscription

2- How to get an Azure Subscription

3- Designing the Azure Subscriptions

 

1- What is an Azure Subscription

It’s not very complicated to understand what is an Azure Subscription, since the name  holds half the meaning. An Azure Subscription is simply a container and a boundary for all the operations, services and objects you make and create within it. An Azure subscription will be the common parent for anything you deploy and do within it. If you ask a physician about Earth, it will say : The Earth is part of the Solar System, which is part of the Milky Way, which is a Galaxy among billions of galaxies, which are part of the Universe. The Universe is the limit, it’s the final boundary (Based on the single Universe theory), it’s the parent of all the child objects. This is the same thing when you define a Virtual Machine or a Web Site deployed on Azure. You will say, this Web Site is part of the App Service Plan, which is part of the Resource Group, part of this Subscription : This identifies this Web Site. An Azure Subscription is the first door to use Azure Services : You cannot use or deploy any Azure Service without a Subscription. You will need at least one subscription to use Azure services. The Azure Subscription is your ID regards Microsoft to use its Azure services.

A Subscription has the following properties :

  • Identified by a Unique ID : Each subscription has a Subscription ID which identifies it across the world. Two customers can have Subscriptions with the same name but no with the same ID
  • Billing Boundary : When you get billed, MS will charge you per Subscription and for the Subscription usage. You still can get the billing details (consumption per service or per resource)
  • Management Boundary : The Subscription Admin has the total control of everything deployed within the subscription. Thanks to RBAC, the Subscription admin can assign users roles within the Subscription to give them specific permissions.

So what to keep in mind before continuing :

  • A subscription is the top entry point to use Azure Services
  • Is a management and billing boundary

 

2- How to get an Azure Subscription

Before starting using Azure services, the first step is to get an Azure Subscription. Getting a Subscription depends on your Contract type with Microsoft and how you want to proceed. I added a comparison table at the end of this section to explain which Subscription  type is more suitable for your usage

2.1- Subscriptions types

2.1.1- Trial Subscriptions

For people whom want to discover and test Azure, Microsoft provides today a Free trial Subscription, which will allow you to have some credit (170 $ the time of this post writing) which allow you to create some Azure Resources.

The Trial account expires after one month and cannot be renewed. If you want to keep using the resources you created, you can convert the Subscription to a Pay-As-You-Go Subscription, otherwise your resources will be suspended and then deleted.

The following are the requirements to get a trial subscription:

  • A valid Credit Card : Must be registered on the same country you entered when applying to the trial account
  • A valid phone number : Must be from the same country when applying to the trial account
  • A Microsoft  or an Organizational Account

The Trial Subscription is only available on some countries, but if you know some one from a supported country, you know what to do (It’s free anyway!)

More information ad details on this Link : https://azure.microsoft.com/en-us/pricing/free-trial-faq/ and https://azure.microsoft.com/en-us/free/

2.1.2- Pas-As-You-Go Subscriptions

This is the first type of Subscription dedicated for real work or business. As you can notice, it’s a Pay as you Go contract which means, you will pay what you consume, but How ?

  • You will need to subscribe to the offer first, just like the Trial Subscription you will need
      • A valid Credit Card : Must be registered on the same country you entered when applying to the trial account
      • A valid phone number : Must be from the same country when applying to the trial account
      • A Microsoft  or an Organizational Account 
  • There is no Commitment, which means, you can cancel (end) the subscription at any time
  • The consumed services will get paid automatically by the end of the month. You can be billed by invoice too
  • You will be rated at the standard Azure services prices

For more information, visit the following link : https://azure.microsoft.com/en-us/offers/ms-azr-0003p/

 

2.1.3- Prepaid Subscriptions

As indicated by the name of this offer, you will give a amount of money before using the Azure services within the Subscription. The prepaid Subscription works like the following :

  • It’s a commitment for 12 months automatically renewable
  • You can pay using Credit card or via Invoice
  • The minimum prepaid amount is  6000 $ without tax. Tax may vary by Country. For example, In France we are taxed at 23 % (Huge!!!)
  • You will earn 5 % discount compared to public Azure services prices
  • When you reach the amount of credit you have paid, you can make an additional prepayment or use a Pay-As-You-Go model.
    • If you use the Pas-as-you-Go model, you will not benefit from the 5 % Discount
    • If you add a Prepaid amount, you should consume it before the End of the first 12 months since you billing period will always start since your first engagement.
  • If you reach the 12 months, you will loose all the non-consumed credits
  • You cannot use this credit to purchase some items like third party solutions on the Market place or Azure services sold separately like Azure AD Premium
  • Items with prices depending on the volume will not be covered by the discount (Azure Storage, Data transfer…)
  • You can cancel the Subscription ant any time but keep in mind:
    • No refund
    • If your usage exceeds the prepaid amount, you will have to pay it before ending the Subscription (Since billing for extra usage is at each end of month)

For more information, visit the following link  : https://azure.microsoft.com/en-us/offers/ms-azr-0026p/

 

2.1.4- Subscriptions under the OPEN Volume License Program

Many customers use today the Open Volume license program to purchase their Microsoft licenses like Windows, RDS or Microsoft Office licenses. You can use the same approach to get Azure Subscriptions.

To get a new subscription, you can process like the following :

  • Contact your preferred reseller. Ask them to purchase Azure Credits depending of the Open Program you are following. If you are not familiar wit Open licensing program, go HERE
  • The credits are purchased on form of Online Service Activation Keys (OSAK). After that, you have five years to enable those OSAKs. Each enabled key is valid for 12 months, after that, the credit will be forfeited
  • Not all Azure Services can be consumed via activated OSA keys. There are some services like Azure AD Basic and Premium, Market place items or Support Plans which need to be purchased separately.
  • Currently, there are no discounts when using Azure via the Open program

2.1.5- Subscriptions under the Enterprise Agreement

Customers under an Enterprise Agreement contract with Microsoft can make a monetary commitment to benefit from the Azure Services. EA provides the best benefits for Azure customers:

  • Best Prices : Azure services prices under EA are discount prices compared to public prices. Prices can be 30 % lower than Public. Also Microsoft is committed to always provide lower prices than its big competitor AWS
  • Purchase simplicity : Customers under EA can rapidly and simply purchase additional credit faced to an unplanned growth, and benefit from some great rates
  • Annual Payment : Faced to a unplanned growth and usage, customers can pay Microsoft later on the year for the additional  consumed resources, and this under the same rates (under conditions)
  • Enterprise Portal : Customers can benefit from the access to the Enterprise Portal which provide many features
    • Manage and Create Subscriptions
    • Manage Accounts
    • Billing

How and when to start :

  • Contact Microsoft (generally through the TAM or SDM) and ask them for an Azure commitment
  • Choose the amount of credit to pay for a Period of 12 months. If it’s the first enrollment, It’s recommended to ask a Specialist like an Azure architect to predict your usage during the next 12 months. For your next enrolments, it will be easier to predict the usage by using forecast tools (Based on the previous consumption, growth rates and planned growth)
  • Once activated, start using the Enterprise Portal to create Subscriptions

2.1.6- Other Azure Subscriptions types

Microsoft provides other subscription types, mainly for dev/test purposes, we can find :

  • Microsoft Azure for DreamSpark : This is an Azure subscription for students. This subscription type provides only a set of Azure services, which are generally deployable via Visual Studio Community. Teachers, Educators and Researchers can also benefit from this subscription type by applying and requesting an access to it. Of course, in all cases, enrollers should prove that they are from the education circle

  • Azure Subscriptions for Visual Studio subscribers (MSDN) : This is a benefit for the Visual Studio subscribers. Microsoft offers an amount of credit per month to allow these users to deploy and test several Azure services for free. A monthly Azure credit of 150$ (The time of writing this article) is provided and can also be credited when needed (users can add credit to their balance). This type of subscription is the only Microsoft subscription type which allow to deploy Client Operating Systems images from the Market Place like Windows 10, 8.1…

 

2.2- Subscriptions types Comparison

Subscription type

For whom

Advantages

Links

Trial

  • First Azure experience
  • Test basic features
  • Discover the Azure Services (Portal, User Experience…)
  • Free
https://azure.microsoft.com/en-us/free/

Pay-As-You-Go

  • Infrequent usage
  • Point in time usage (Fixed period)
  • Personal usage (Teaching, demo)
  • No monetary commitment
https://azure.microsoft.com/en-us/offers/ms-azr-0003p/

Prepaid

  • SMB market (Small and Middle business)
  • Mid term usage (Al least one year)
  • Predictable usage (Good for fixed budget)
  • Constant usage over the year
  • 5 % discount
  • Continuous usage over the year
https://azure.microsoft.com/en-us/offers/ms-azr-0026p/

Open Program

  • Customers already familiar/Using with the Open Licensing Program
  • SMB market
  • Small usage variation and growth
  • Under the Open program
  • OSA keys can be bought and activated within 5 years
https://azure.microsoft.com/en-us/offers/ms-azr-0111p/

Enterprise Agreement

  • For customers already part of an Enterprise Agreement
  • For customers with big annual usage
  • For customers with huge, unpredictable and unplanned usage growth
  • For customers whom want to benefit from the best Azure contract type which includes completive discount rates
  • Attractive discount rates
  • Face to unplanned growth
https://azure.microsoft.com/en-us/pricing/enterprise-agreement/

Other types

  • Students, educators
  • Dev/Test purposes
  • Limited usage per month
  • Free
https://azure.microsoft.com/en-us/offers/ms-azr-0064p/

https://azure.microsoft.com/en-us/pricing/member-offers/msdn-benefits-details/

https://www.dreamspark.com/Product/Product.aspx?productid=99

 

3- Designing the Azure Subscriptions

After presenting the different Azure Subscription types and their specifications, here the most important section of this post, as I will present guidance to better design the first build block of an Azure infrastructure.

3.1- Subscription type

This is the first thing you need to think about : What is the subscription type I will choose ?

This is what was detailed on the first section, you will need to choose which type to use depending on your needs and budget. Read the ‘For whom’ information to look if you are part of the right population. It’s clear that this section concerns only non dev/test scenario :

  • Pay-As-You-Go
  • Prepaid
  • Open Program
  • Enterprise Agreement

3.2- Subscriptions count

After soundly choosing the commitment and subscription type, comes an obvious question on mind ? How many subscription will you need ? The answer can be predicted by asking it otherwise ? Why do you need 2 subscriptions instead of 1, or three instead two ? what will push us to create This is the first X subscriptions instead Y ?

Let me remember you the 3 basics about Azure Subscriptions :

  • Billing boundary
  • Management boundary
  • Quotas and limits constraint*

* You can notice that during Section 1, this information was not mentioned. I intentionally added it in this section because this constraint may change over the time, and hence, no longer be a constraint

 

3.2.1- Billing boundary

The first characteristic of an Azure subscription is that it’s the top level of the billing detail. Depending of the type of enrollment, you will pay for the usage within a single subscription. The following table shows how the Subscriptions get billed:

Enrollment type

Billing

Pay-As-You-Go

  • The Subscription bill gets paid at the end of the month depending of the payment method (Credit card or Invoice)

Prepaid

  • The Subscription bill gets subtracted from the total ‘prepaid’ amount at the End of each month. You can at any time consult your account to check the balance

Open Program

  • Same statement than Prepaid subscriptions

EA

  • Same statement than Prepaid subscriptions

Now, you can choose how many Subscriptions you will create based on your constraints. The following table shows examples of constraints :

Constraint

What to do

You have different independent (or just depending a few core services) IT departments and you want to keep independent cost management Create a Subscription per department
You are a Holding Company with a centralized IT service which manage the IT service for all the child companies Create a Subscription for each company. You can go deeper by creating several Subscriptions per department per child company. It depends on the model
You are following a Charge-Back model for you departments or services. You wan to keep using this model when moving to the Azure cloud Create an Azure Subscription per entity to easily track the expenses and keep the charge-back model
You are a Cloud Service Provider managing and charging-back to your customers Create one or more Azure Subscriptions per customer. It’s evident that 2 customers can’t share the same Subscription

Just to note that you can get the billing details ( Per Resource Group…) and can extract usage based on these information, which can be used to charge-back or to get detailed cost spends. As mentioned, an Azure Subscription is the top of the bill, but not the unique billing information.

3.2.2- Management boundary

The choice of how many Subscriptions you will create may be impacted by this second factor which is the management rights. In fact, a Subscription is the top level of the rights assignment. Of course, thanks to the RBAC model which will be seen on a coming post, you can assign rights based on pre-defined or created user roles, but the Subscription stills  the starting point of rights management.

A– Account types and roles

Let me first refresh you memory about the different default account types and roles which apply to Azure and Azure Subscriptions :

Enterprise Administrator

This account type is only applicable for Enterprise Agreement customers. The Enterprise Administrator has the ability to :

  • Add Accounts to the Enrollment
  • Associate existing Accounts to the Enrollment
  • View usage data across all the Accounts
  • Consult the monetary balance of the enrollment and provide the Account’s Owners the rights to view charges

An Enrolment can have many Enterprise Administrators

Account Owner

This is the Principal Important account. This Account can be a Microsoft Account (Live ID) or an Azure Active Directory Account. It has the Ability to create Azure Subscriptions, update the Service Administrator account, and add co-administrators to the Subscriptions. It is financially responsible for the Azure monetary commitment. Microsoft will send invoices to the email associated with the Account Owners. In case of an EA , the Account owner has less privileges since it cannot by default view the Subscriptions balance, unless authorized by the Enterprise Administrators. Account Owners are created by the EA’s Enterprise Administrator. In case of other enrollments, it’s the first created account when enrolling.

Service Administrator (ASM)

The Account owner can create Subscriptions. Each Subscription has a ‘Super Admin’ account which has the total rights across the Subscription. By default, the Account owner is the Service Administrator. The Service Administrator has the ability to add co-administrators to the Subscription.

Co-Administrators (ASM)

Co-Administrators are accounts with the same ‘Service Administrator’ privileges except changing the Service Administrator account. A maximum of 200 Co-admin accounts can be associated to a Subscription (Unlimited with RBAC since it’s the Owner role)

NB : The Service Administrator and the Co-Administrators are part of the ‘Subscription Admins’ group. This is very important to know since the ‘Service Administrator’ and the ‘Co-admins’  are only valid for Azure Service Management model (IaaS V1). In fact, with Azure Resource Manager model, these accounts are no longer used natively. The ‘Super Administrators’ are part of the ‘Owner’ RBAC role. For compatibility reasons between ASM and ARM, the  Subscription Admins group is by default part of the Owner role. This lets the Subscription Admins be ‘Administrators’ at the ARM level

Owner (New Portal)

This is the most powerful RBAC role. Users part of the Owner role are Co-admins of the Subscription. But unlike Co-admins, they cannot use the ‘manage.windows.azure.com’ portal, since the RBAC model is not supported on the the old portal. More information will come on the RBAC post.

RBAC Roles (New Portal)

With the new Portal, Azure adds the possibility to use RBAC roles. There are built-in RBAC roles, but you can create your custom RBAC roles for more granular permissions. This allows you to control management rights over a Subscription. More will come on the RBAC post (The last of this Series)

NB : You need to keep in mind that the Enterprise Administrator and the Account Owner are different account types than the others. In fact, they are used to manage the Subscriptions (Create, Manage…), while other account types are ‘roles’ used to operate within the Subscriptions.

 

B- Design considerations

After presenting the different account types and roles which can be used  in Azure, you may now make decisions about how much subscription you want to create. ’The following are some examples of cases you may be encountering :

Scenario

Possible decision

You have different departments with an IT team for each one. You want that the IT’s head have total management rights on their respective platform. You can create different subscriptions and make the IT’s head the Owner of the subscription. This way, each one have the full ability to manage its Subscription and assign users the required roles
You are a Cloud Service Provider managing and charging-back to your customers

Create one or more Azure Subscriptions per customer. It’s evident that 2 customers can’t share the same Subscription, even by using the right RBAC roles

You are a Company with a centralized IT service which manage the IT service for all the child companies. Depending on the hierarchy you can :
– Create different Enterprise Administrators and make each company’s IT head an Enterprise Admin
– Create different Account owners and make each company’s IT head an Account Owner
– Create different Subscriptions and make each company’s IT head an owner

It always depends on your needs and requirements, so just keep in mind the management boundary property of a Subscription

 

3.2.1- Quotas and limits constraint

This the third constraint which will influence you design, and this is us much important us the other factors. Let me explain the meaning of the title.

Azure does not let you create us much resource as you want, it applies limits! For example, as mentioned above you can have a Maximum of 200 co-admin per Subscription (ASM), this is a limit.

The following link provides all the quotas and limits applied to Azure. https://azure.microsoft.com/en-us/documentation/articles/azure-subscription-service-limits/#comments

NB : It’s important to differentiate the limits applied to Azure Service Management (Classic model) and Azure Resource Manager model, in few words:

  • For Classic model : Limits are applied per Subscription (For example, the default limit of the virtual networks count is 50 per Subscription across all the regions)
  • For ARM model : Limits are applied per Subscription per Region, for simplicity we can say per Region (For example, the default limit of the virtual networks count is 50 per region)

When mentioned and supported, you can ask the Azure Support to increase the limit applied to you since there are default limits and maximum limits. The question is what happens If you plan to exceed these limits ?

Let me write about a real life scenario :  NSG (Network Security Groups). A NSG is a Set of network security rules which can be applied on a NIC (on a VM) to control the network traffic (It’s the Firewall implementation on Azure). If you look to the quotas and limits values, you will see that there is a limit of 400 NSG per region and 500 rules per NSG. Based on the following NSG facts:

  • You can apply an NSG only on Subnet or a VM NIC
  • You cannot apply multiple NSG on a single resource (VM or Subnet)
  • You cannot create composed rules with NSG

What happens then : You will be hit by the limits! Suppose you use the model of 1 NSG per VM  –> You can deploy a maximum of 400 VMs per region, so if you decide to create different VNETs on the same region (VNET Prod, VNET Dev, VNET Int…), this maybe will not be suitable since you can deploy a maximum of 400 VMs across all VNETs. The only solution is to create multiple subscriptions and place a VNET per subscription (Anyway this is not scalable for enterprises creating more than 400 VMs per VNET)

To resume, the Azure maximum quotas and limits is a factor and constraint when designing your Subscriptions.

 

3.3- Naming the Subscriptions

This is the final section of this post, and I will discuss how you can name your Azure subscriptions. Like explained on the first section, a Subscription is identified via its ID, unique across all Azure customers. But to simplify things, a Subscription has a Name, which can be edited by the Account owner. Choosing a good naming convention is the key of easily manage, identify and recognize the subscriptions.

3.3.1- Questions to ask when making the Naming Convention

The basic thoughts when preparing a naming convention are merely tied to how on the future, we can easily identify a Subscription. Technically, what the Subscription name should contain which satisfies my needs.

Q: What are the common information between all the Subscriptions

A: Generally, this the Enterprise name or the Subscription’s Owner name or identifier. This information should be present on all the Subscription names to identify you

Q: What are the information which differs a Subscription from another

A: This is generally the key points on which you made your Subscription design, for example, you decided to create a Subscription for each Environment (Prod, Dev, UAT…), so these information are worth to be seen on the Subscription name. It can be 1 or more info depending on you deign

Q: Do you plan to add a Subscription of the same type on the future (to scale up)

A: If yes (And I recommend doing it anyway), add a  count suffix to easily identify the Subscription rank

 

3.3.2- Naming Convention examples

The following table details some examples of naming conventions, and what are the factors leading to choosing them:

Scenario

Naming Convention

We are a holding company named SamirHolding
We have different child companies
Each child company have 3 different environments
The decision was to create a Subscription for each Environment

HoldingCompany-ChildCompany-Env-xx (where xx is between 01 and 99)
Examples:
The first company name is Excelium, the subscription names will be:
– SamirHolding-Excelium-Prod-01
– SamirHolding-Excelium-Dev-01
– SamirHolding-Excelium-UAT-01
The second company name is Hydra, the subscription names will be:
SamirHolding-Hydra-Prod-01
SamirHolding-Hydra-Dev-01
SamirHolding-Hydra-UAT-01

We are a company (SamirCorp) with different departments, we wan to create a Subscription for each department. Each department have two environments : Prod and Test CompanyName-Departement-Env-xx
SamirCorp-IT-Prod-01
SamirCorp-IT-Test-01

SamirCorp-RH-Prod-01
SamirCorp-RH-Test-01

SamirCorp-RD-Prod-01
SamirCorp-RD-Test-01