Introduction to Azure Private Endpoints

Azure Private Links and Endpoints have been recently announced in Public Preview after months of Private Preview and testing. Private Link/Endpoint is a huge step in Azure Networking as it allows to make private any internet facing public service (Like PaaS services: Azure SQL, Azure Storage…), and provides a unified way to expose and consume services between tenants, partners or even within the same customer.

This post is an introduction of Private Endpoints

1- Concept

There are two terms that should be memorized:

• Private Link: See my next blog post
• Private Endpoint: Is an association (couple) of a supported resource (ex: azure sql server) and a Subnet in your virtual network so it can be assigned a private IP address

2- Example Scenario

2.1- Need

One of the easiest/most requested scenario is the ability to each an Azure SQL Database privately from a Virtual Network, without exposing it to Internet, or using its Public IP address.

The following picture shows a Virtual Machine (SQLClient) deployed to a VNET/Subnet (VNETEastUS/VM), and an Azure SQL DB (azuresqlprivatedb01) deployed within an Azure SQL server (azuresqlpriavte).

The goal is that the VM (SQLClient) reaches the azuresqlprivate over a private connection, and not via internet (we will not allow any IP to consume the Azure SQL server through the firewall): The dotted line in orange is the goal

2.2- Solution via Private Endpoints

The solution is to create a Private Endpoint that will expose the Azure SQL server via a Private IP address on a Subnet. The following picture shows a Private Endpoint (PE) that is using an IP address from the Subnet Privatesubnet and connected to azuresqlprivate

2.3- Deploy the Solution via the Azure Portal

1- Go to the Private Link Center –> Private endpoints –> Add

2- Type the required information like the Subscription, RG, Location…

3- Choose the resource that you want to create a Private Endpoint for (In my case, it’s the azuresqlprivate Azure SQL server

4- Choose a VNET/Subnet that will enable the access to your resource. Under the hood, a Network Interface (NIC) will be created, assigned an IP address and assigned to your resource. You can optionally integrate the resource with an Azure Private DNS Zone in order that you can call the private endpoint using a DNS name. This is not required as you can create your own DNS record on your own DNS service (A record)

5- After the deployment, you will notice the following:

• The Private Endpoint is created an the Connection State is Approved*

* Approved means that the Azure SQL party has approved the Private Endpoint, this is useful when both parties are not from the same Team/Tenant, where the requester can ask for the Private Endpoint connection, and waits for the owner to approve it. In addition, you can Reject the connection at any time

• A NIC has been created and deployed to the Subnet

• A DNS record has been created (in case you have enabled private DNS option)

2.4- Test the Private Endpoint

On the SQLClient Virtual Machine, you can install SQL Server Management Studio and test the access to the DB

Note that the Private IP is resloved

2.5- Secure the access to the Private Endpoint

Now that we have private access to a PaaS Service, there amy ways to secure the access to it:

• Use Network Security Groups for the PE Inbound: You can create an NSG and apply it to the Subnet NIC or the Private Endpoint NIC, and filter Inbound rules like any other NSG –> Looks like this not yet supported on the Public Preview
  • Applying a NSG to the NIC is not supported
  • Applying an NSG to the Subnet is without any affect on Private ENdpoints

• Use Use Network Security Groups for the Outbound: You can filter outbound traffic from your sources to your Private Endpoints: This is supported but not convenient, since it’s better to filter at the destination and not a the source, when securing access from a Destination standpoint (The picture shows a rule that blocks access to the private IP of the private endpoint, and applied to the SQLClient VM)

• Since the IP address of the Private Endpoint is within your VNET, you can filter access to it on your perimeter Firewalls, like Azure Firewall or your own Firewall

3- Private Links

The next blog post will introduce the Private Link concept, which is a service that allows your external customers/partners to securely access your services via a private connection, and using an Approval Process.

LINKS

• What is Azure Private Link
• Concept of Azure Private Endpoint
• Concept of Azure Private Link Service

Get Azure Datacenter IP ranges via API V2

Hi,

In my previous post, I showed how to create a light-weight Azure function that allows you to request the Azure Datacenter IP ranges via API. You can rapidly test it by following the instructions on the section 3- Try it before deploying
it here : https://buildwindows.wordpress.com/2017/11/19/get-azure-datacenter-ip-ranges-via-api/

The feedback was positive, but a lot have asked for a way to see if there were updates compared to the last version, and what the updates are if any.

In this post, I will publish the second version of the API (with the how-to), that allows you to :

• Get the current Azure Datacenter IP ranges
• Get the current Azure Datacenter IP ranges for a specific region
• Get region names (since, unfortunately, the region names published by Microsoft are not exactly the same used by the Microsoft Azure services)
• New : Get the current version ID of the Azure Datacenter IP ranges
• New : Get the previous version ID of the Azure Datacenter IP ranges
• New : Get the difference between the current and the previous version for all regions
• New : Get the difference between the current and the previous version for a specific region

The new features will allow you an easy integration with your environment, and simplify the update of your Firewall rules within your infrastructure.

• 1- How to request the API ?

The API supports only POST requests. You can make the following API requests using the following body construction.

Here the examples using Powershell, but you can use any tool to request the API using the same body content

#V1

#Get the current Azure IP address ranges of all region

$body = @{“region”=“all”;“request”=“dcip”} | ConvertTo-Json

#Get the current Azure IP address ranges of a specific region, example europewest

$body = @{“region”=“europewest”;“request”=“dcip”} |ConvertTo-Json

#Get the azure regions names, that we can request IPs for

$body= @{“request”=“dcnames”} |ConvertTo-Json

#Post the request

$webrequest=Invoke-WebRequest -Method “POST” -uri ` https://azuredcip.azurewebsites.net/getazuredcipranges -Body $body

ConvertFrom-Json -InputObject $webrequest.Content

#New in V2

#Get the (added and/or removed) IP address ranges updates of a specific region

$body = @{“request”=“getupdates”;“region”=“asiaeast”} | ConvertTo-Json

#Get the (added and/or removed) IP address ranges updates of all regions

$body = @{“request”=“getupdates”;“region”=“all”} | ConvertTo-Json

#Get the current Azure DC IP ranges version ID

$body = @{“request”=“currentversion”} | ConvertTo-Json

#Get the previous Azure DC IP ranges version ID

$body = @{“request”=“previousversion”} | ConvertTo-Json

#Post the request

$webrequest = Invoke-WebRequest -Method “POST” -uri ` https://azuredcip.azurewebsites.net/getazuredcipupdates -Body $body

ConvertFrom-Json -InputObject $webrequest.Content

• 2- How to build the solution ?

2.1- Solutions components

The V2 version is still using only Azure Functions, but unlike V1, it uses multiple functions within the Function App:

• 1 Function App
  • Function 1
  • Function 2
  • Function 3
  • Proxy1
  • Proxy2
  • Storage Account

The following table details each component configuration. If you want to create the solution within your environment, create the same components using the given configuration:

Function App

Name	Description	App Service Plan
azuredcip	This Function App will host the entire solution. It will include 3 functions, 1 Storage Account and two Proxies	Shared or greater. Use at least a Basic Tier to benefit from SSL, Custom Names and backup

Function1

Name	Description	Language	Integrate	Manage
azuredcipranges	This function will return you the V1 information. https://buildwindows.wordpress.com/2017/11/19/get-azure-datacenter-ip-ranges-via-api/	HttpTrigger – Powershell	Allowed HTTP Methods : POST Authorization level : Anonymous	You can add Function Keys if you want to secure the API Access. In my case, my API still Public (anonymous) to continue support my V1

Function2

Name	Description	Language	Integrate	Manage
azuredciprangesupdater	This function will do the following : – Get the current Azure DC IP ranges version and store it to the storage account – Always keep the previous Azure DC IP ranges version file in the storage account – Create and store a file containing the current and previous files difference and store it to the storage account – Return the mentioned information based on the API request body	HttpTrigger – Powershell	Inputs (Type : Azure Blob Storage) Name Path SA connection vnowinblob azuredcipfiles/vnow.json AzureWebJobDashboard vpreviousinblob azuredcipfiles/vprevious.json AzureWebJobDashboard vcompareinblob azuredcipfiles/vcompare.json AzureWebJobDashboard Outputs (Type : Azure Blob Storage) Name Path SA connection vnowoutblob azuredcipfiles/vnow.json AzureWebJobDashboard vpreviousoutblob azuredcipfiles/vprevious.json AzureWebJobDashboard vcompareoutblob azuredcipfiles/vcompare.json AzureWebJobDashboard Keep the default http output Allowed HTTP Methods : POST Authorization level : Function	You can use the default function key or generate an new key. This API will not be directly exposed, so you can protect it with a key

Function3

Name	Description	Language
triggerazdciprangesupdate	This function will trigger the azuredciprangesupdater weekly to update the current and previous version	TimerTrigger – Powershell Schedule : 0 0 0 * * 3 (Each Wednesday, but you can choose any day of the week, as Microsoft will not apply the updates before one week of their publication)

Proxy 1

Name	Description	Root template	Allowed HTTP methods	Backend URL
getazuredcipranges	This proxy will relay requests to azuredcipranges	/getazuredcipranges	POST	https://azuredcip.azurewebsites.net/api/azuredcipranges (add the key if you have secured it)

Proxy 2

Name	Description	Root template	Allowed HTTP methods	Backend URL
getazuredcipupdates	This proxy will relay requests to azuredcipranges	/getazuredcipupdates	POST	https://azuredcip.azurewebsites.net/api/azuredciprangesupdater?code=typethekeyhere

Storage Account

Name	Description	container	Files
azuredcipsa	This is the storage account automatically created with the Function app (it can have any other name)	azuredcipfiles	Upload the following files: – vnow.json – vprevious.json NB : These files are fake files. During the first API update request, the vnow content will be copied to the vprevious file. The vnow content will be then replaced by the real current version. At this moment, you can only request the current version. After one week, a new version of the file will be published by MS, so another cycle will set the vnow and the vprevious to real 2 consecutive versions, so you can benefit from the update and comparison feature.

2.2- Download the files

You can download the needed files here, you will find the powershell code for each function (function1, function2, function3) and the two json files (vnow.json and vprevious.json).

NB : As mentioned before, after the first request to the API (getupdates), you will have a valid vnow version, but the previous version will be the version uploaded now. You need to wait at least 1 week to have the valid version.

Enjoy!!

Get Azure Datacenter IP address ranges via API

Hi folks,

One of the struggles that we may face when dealing with Azure services, is the network filtering. Today, we consume a lot of services provided by Azure, and some Azure services can consume services from our infrastructure. Microsoft is publishing an ‘xml’ file that contains the exhaustive list of the Azure Datacenter IP ranges for all its public regions (No gov). This ‘xml’ file is regularly updated by Microsoft to reflect the list updates as they can add or remove ranges.

The problem is that consuming an ‘xml’ file is not very convenient as we need to make many transformations to consume the content. Many of you have requested that Microsoft at least publishes this list via an API so it can be consumed by many sources, using Rest requests. Until Microsoft make it, I will show you today how to create a very lightweight web app on Azure (using Azure Functions) that will ‘magically’ do the job.

NB : The Azure Datacenter IP ranges include all the address space used by the Azure datacenters, including the customers address space

1- Why do we need these address ranges ?

If you want to consume Azure services or some Azure services want to consume your services, and you don’t want to allow all the “Internet” space, you can ‘reduce’ the allowed ranges to only the Azure DC address space. In addition, you can go further and select the address ranges by Azure region in case the Azure services are from a particular region.

Examples :

• Some applications in your servers need to consume Azure Web Apps hosted on Azure. In a world without the Azure DC address space, you should allow them to access internet, which is a bad idea. You can configure your firewalls to only permit access to the Azure DC IP ranges
• If you are using Network Virtual Appliances on Azure, and you want to allow the VM’s Azure agent to access the Azure services (storage accounts) in order to function properly, you can allow access only to the Azure DC IPs instead of internet.

2- The Solution

In order to consume the Azure Datacenter IPs via an API, I used the powerful and simple Azure functions to provide a very light weight ‘File’ to ‘JSON’ converter. The Azure function will do the following:

• Accept only a POST request
• Download the Azure Datacenter IP ranges xml file
• Convert it to a JSON format
• Return an output based on the request:
  • A POST request can accept a Body of the following format : { “region”: “regionname”, “request”: “requesttype” }.
    • The “request” “parameter can have the value of :
      • dcip : This will return the list of the Azure Datacenter IP ranges, depending on the “regionname”  parameter. “regionname” can be :
        • all : This will return a JSON output of all the Azure Datacenter IP ranges of all regions
        • regionname : This will return a JSON output of the regionname’s Azure Datacenter IP ranges
      • dcnames : This will return al list of the Azure Datacenter region’s names. The “regionname” parameter will be ignored in this case
    • In case of a bad region name or request value, an error will be returned

3- Try it before deploying it

If you want to see the result, you can make the following requests using your favorite tool, against an Azure function hosted on my platform. In my case, I’m using powershell and my function Uri is https://azuredcip.azurewebsites.net/api/azuredcipranges

3.1- Get the address ranges of all the Azure DCs regions

#Powershell code

$body = @{“region”=“all”;“request”=“dcip”} | ConvertTo-Json

$webrequest= Invoke-WebRequest -Method “POST” -uri ` https://azuredcip.azurewebsites.net/api/azuredcipranges -Body $body

ConvertFrom-Json -InputObject $webrequest.Content

 3.2- Get the address ranges of the North Europe region

In this case, note that we must use europnorth
instead of northeurope

#Powershell code

$body = @{“region”=“europenorth”;“request”=“dcip”} | ConvertTo-Json

$webrequest = Invoke-WebRequest -Method “POST” -uri `
https://azuredcip.azurewebsites.net/api/azuredcipranges -Body $body

ConvertFrom-Json -InputObject $webrequest.Content

3.3- Get the region names

$body = @{“request”=“dcnames”} | ConvertTo-Json

#or #$body = @{“region”=”anything”;”request”=”dcip”} | ConvertTo-Json 

$webrequest = Invoke-WebRequest -Method “POST” -uri `
https://azuredcip.azurewebsites.net/api/azuredcipranges -Body $body

ConvertFrom-Json -InputObject $webrequest.Content

4- How to deploy it to your system ?

In order to deploy this function within your infrastructure, you will need to create a Azure function (Section : Create a function app) within your infrastructure. You can use an existing Function App or App Service Plan.

NB : The App Service Plan OS must be Windows

After creating the Function App, do the following:

Step	Screenshot
Go to your Function App and click the Create new (+)
In Language chose Powersell then select HttpTrigger – Powershell
Give a Name to your function and choose an Authorization level. In my case, i set the Authorization to anonymous in order to make the steps simpler. We will see later how to secure the access to the function http trigger
Copy paste the following code on the function tab, then click Save # POST method: $req $requestBody = Get-Content $req -Raw | ConvertFrom-Json $region = $requestBody.region $request = $requestBody.request #Main if (-not$region) {$region=‘all’} $URi = “https://www.microsoft.com/en-us/download/confirmation.aspx?id=41653” $downloadPage = Invoke-WebRequest -Uri $URi -usebasicparsing $xmlFileUri = ($downloadPage.RawContent.Split(‘”‘) -like “https://*PublicIps*”)[0] $response = Invoke-WebRequest -Uri $xmlFileUri -usebasicparsing [xml]$xmlResponse = [System.Text.Encoding]::UTF8.GetString($response.Content) $AzDcIpTab = @{} if ($request -eq ‘dcip’) { foreach ($location in $xmlResponse.AzurePublicIpAddresses.Region) { if ($region -eq ‘all’) {$AzDcIpTab.Add($location.Name,$location.IpRange.Subnet)} elseif ($region -eq $location.Name) {$AzDcIpTab.Add($location.Name,$location.IpRange.Subnet)} } if ($AzDcIpTab.Count -eq ‘0’) {$AzDcIpTab.Add(“error”,“the requested region does not exist”)} } elseif ($request -eq ‘dcnames’) { $AzDcIpTab = $xmlResponse.AzurePublicIpAddresses.Region.name } else {$AzDcIpTab.Add(“error”,“the request parameter is not valid”)} $AzDcIpJson = $AzDcIpTab | ConvertTo-Json Out-File -Encoding Ascii -FilePath $res -inputObject $AzDcIpJson
Go to the integrate tab, and choose the following: Allowed HTTP methods : Selected methods Selected HTTP methods : Keep only POST Click Save
It’s done ! To test the function, go back to the main blade and develop the Test tab
On the request body, type : { “region” : “europeewest”, “request”: “dcnames” } Then click Run You should see the results on the output, and the http Status 200 OK
Click Get function URL to get the URL of your function in order to query it via an external tool

 

5- Securing the access to the function URL

There are 3 options that let you secure the access to the Function URL:

5.1- Network IP Restrictions

I personally think that this is best option to secure the access to Function URL. IP Restrictions allows you allow only a set of Public IP addresses to access the Function URL. For example, if you have an automation script that requests the API and update a Firewall object or a database, you can whitelist only the Public IP address used by this automation workflow, which is the outbound IP address. This feature is available for Basic Tier App Service Plans and greater. It’s not supported for free and shared sku. Start using it by following this tutorial : https://docs.microsoft.com/en-us/azure/app-service/app-service-ip-restrictions.

NB : The Networking blade for a Function App can be found by clicking on the Function Name à Platform features à Networking

5.2- Function Key and Host Key

You can restrict the access to the Function URL by leveraging an Authorization feature, by protecting querying the URL  via a ‘Key’. There are two Key types : The Function Key which is defined per Function and a Host key which is defined and the same for all the functions within a Function App. In order to protect a function via a Function key, do the following :

Step	Screenshot
Go to the Integrate blade, and change the Authorization level to Function
Go to the Manage blade. You can use the default generated key or Add a new function key. Generating a new key is provided for keys rotation
Now, you can access the ‘protected’ URL by clicking on the Get function URL on the main blade. You can select which key to use.

 

5.3- Authentication and authorization feature in Azure App Service

This feature allows you to secure your Function App by requesting the caller to authenticate to an Identity Provider and provide a token. You can use it against Azure Active Directory or Facebook for example. I will not detail the steps in this post, but here’s some materials :

Overview : https://github.com/MicrosoftDocs/azure-docs/blob/master/articles/app-service/app-service-authentication-overview.md

How to configure your App Service application to use Azure Active Directory login : https://github.com/MicrosoftDocs/azure-docs/blob/master/articles/app-service/app-service-mobile-how-to-configure-active-directory-authentication.md

 

Use the new Azure Services Tags preview

Hi all,

At Ignite 2017, MS announced a very so waited feature, which is Azure Services Endpoints Tags for Network Security Groups. The new additional Service Tags will permit you to allow/Deny access to and from Azure Datacenter IP addresses.

Why is this important ?

A lot of Azure Services, specially Services related to Azure IaaS rely on network access to Azure endpoints located on the Internet address space. For example, you cannot use Azure Backup for IaaS virtual machines if your virtual machines do not have network access to the Azure storage endpoints of the same region. This causes a lot of frustration, since if you are using NSGs, you cannot easily create and maintain rules for only the Azure IP addresses since the list is huge and dynamic (Azure Datacenter IP addresses). This results on all VMs to have access to Internet using HTTPS.

What is new ?

Additional Service tags have been added to some regions to allow filtering access to :

• Azure Storage endpoints
• Azure SQL

The feature is now in Preview, and other regions will be added on the future. Make a look to this article to have the last information : Azure Services Tags

How to use the feature ?

You just need to register to the new preview feature. Use the following powershell code against your subscription :

Register-AzureRmProviderFeature -FeatureName AllowAccessRuleExtendedProperties -ProviderNamespace Microsoft.Network
Register-AzureRmResourceProvider -ProviderNamespace Microsoft.Network

How to add a NIC to an Azure Virtual Machine (ARM)

Hi all,

In this short post, I will show you how to add a  NIC (third, fourth…) to an Azure virtual machine under Azure Resource Manager and using Azure Powershell.

Before continuing, keep in mind:

– Not all the virtual machines sizes support multiple NICs. Check if your VM size is supported ( https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-size-specs/)

– The VM should already have at least two NICs. It’s not supported to pass from a single NIC VM to multiple NIC VM and vice versa

– If you are creating the VM with multiple NICs (Use this post to do it), the resource group to which you are deploying the VM must contain only [multiple NIC]’s VM types. In other words, you cannot create a VM with multiple NICs in a resource group containing [single NIC]’s VMs It seems that this limitation was removed

– The VM will be rebooted when adding a virtual NIC

– You can use Azure CLI, Azure API or Powershell to make this operation. The portal does not provide a way to add a  NIC.

– In this post I’m using Azure Powershell 1.0. If you are using the 0.9.8 or prior, remove the ‘Rm’ suffix from your commands and change the mode to ResourceManager

Walkthrough

This is easy, only two steps:

1. Create a new Virtual NIC
2. Attach the VNIC to the virtual machine

1- Create a new Virtual NIC

The first step is to create a Virtual NIC to associate it later to to the virtual machine. You will need to provide the following information:

• The  Name of the virtual NIC to be created : $NICName
• The Resource Group Name where the VNIC will be created : $NICResourceGroup
• The location/region where the VNIC will be created, of course it has to match the VNET/Subnet location : $Location
• The subnet ID to which you will connect the VNIC, you can get it via Powershell (See my example below) : $SubnetID
• The IP address you want to assign to the NIC : $IPAddress

Powershell

# Get the VNET to which to connect the NIC
$VNET = Get-AzureRmVirtualNetwork -Name ‘VnetV2’ -ResourceGroupName ‘RGNetworkV2’
# Get the Subnet ID to which to connect the NIC
$SubnetID = (Get-AzureRmVirtualNetworkSubnetConfig -Name ‘VNETV2Subnet01’ -VirtualNetwork $VNET).Id
# NIC Name
$NICName = ‘A3VM-NIC3’
#NIC Resource Group
$NICResourceGroup = ‘RG’
#NIC creation location
$Location = ‘north europe’
#Enter the IP address
$IPAddress = ‘10.20.30.21’

#–> Create now the NIC Interface

New-AzureRmNetworkInterface -Name $NICName -ResourceGroupName $NICResourceGroup -Location $Location -SubnetId $SubnetID -PrivateIpAddress $IPAddress

 

2-  Attach the VNIC to the Virtual Machine

The second and last step is to associate the created VNIC to to the virtual machine. You will need to provide the following information:

• The virtual machine Name : $VMname
• The Virtual machine resource Group : $VMRG

Powershell

$VMname = ‘VMname’
$VMRG =  ‘VM-RG’

#Get the VM
$VM = Get-AzureRmVM -Name $VMname -ResourceGroupName $VMRG

#Add the second NIC
$NewNIC =  Get-AzureRmNetworkInterface -Name $NICName -ResourceGroupName $NICResourceGroup
$VM = Add-AzureRmVMNetworkInterface -VM $VM -Id $NewNIC.Id

# Show the Network interfaces
$VM.NetworkProfile.NetworkInterfaces

#we have to set one of the NICs to Primary, i will set the first NIC in this example
$VM.NetworkProfile.NetworkInterfaces.Item(0).Primary = $true

#Update the VM configuration (The VM will be restarted)
Update-AzureRmVM -VM $VM -ResourceGroupName $VMRG