From Public to Private: Azure Private Endpoints and Links


I was pleased to be a speaker in the Azure Virtual Community Day, where i presented how to use Private Links and Endpoints, and how important they are. If you want the slide deck, please message me.

Session Description

In this session will will show how to turn Public Azure PaaS services into Private using Private endpoints and links.

After this demo you will learn to reach services such like Azure Storage, Azure SQL or Azure Key Vault through a private IP address on your network


Design Azure Private DNS Zones when using Private Links or any other scenario


Private Endpoints and Private Links are powerful services provided by Azure to allow you access PaaS services (Azure SQL, Azure Storage including Data Lake Gen2, Azure Web App…) via an IP address in your own network (your own Virtual Network).

See here an introduction of Private Endpoints / Private Links:

Supported Azure PaaS Services:

1- Is using Azure Private DNS a requirement ?

Unlike many are thinking, using Azure Private DNS is not required if you want to use PE/PL, but it’s recommended as it provides first party integration allowing your PE Private IP record to be automatically created in Private DNS zone, for future DNS resolution

2- What Zone name do i need to use for PE/PL ?

Microsoft is ‘weirdly’ recommending using a specific Zone name for PE/PL (The full list can be found here). The following table shows some DNS zones for some services:

Private Link resource typeSubresourceZone name
SQL DB (Microsoft.Sql/servers)Sql Server (sqlServer)
Azure Synapse Analytics (Microsoft.Sql/servers)Sql Server (sqlServer)
Storage Account (Microsoft.Storage/storageAccounts)Blob (blob, blob_secondary)
Storage Account (Microsoft.Storage/storageAccounts)Table (table, table_secondary)
Storage Account (Microsoft.Storage/storageAccounts)Queue (queue, queue_secondary)
Source: Microsoft

I’m saying ‘weirdly’ because we have found a scenario where this is unsupported.

The following picture shows:

  • Project Source: A VM deployed to VNET Source
  • Project A: A Storage Account SA1 with a PE/PL in VNET A and registered in the Private DNS Zone
  • Project B: A Storage Account SA2 with a PE/PL in VNET B and registered in another Private DNS Zone (In fact Project A and Project B are independent, and it’s supported to create multiple private dns zones with the same name


  • If VM in VNET source wants to solve SA1 PL, the solution is to link VNET Source to the prive DNS Project A –, so any resource in VNET Source can solve records
  • If VM in VNET source wants to solve SA2 PL, the solution is to link VNET Source to the prive DNS Project B –, so any resource in VNET Source can solve records –> This is not supported since a VNET cannot be linked to 2 Private Zones with the same name

-> Microsoft recommendation is not recommended in fact, because this will create future issues. It’s like creating VNETs with the same address space, and then to peer them : Unsupported

3- Design Azure Private DNS Zones

The design recommendation is simple: Each Project (The scope of how resources are managed by the same team) must have its own DNS Zone.

In order to achieve this, you can use the following rules:

  • Choose a base domain (root domain) to be used by all the company’s projects. For example, your company came is CLOUDCASE, your root domain can be
  • Ask each project to create a Private DNS Zone using the format {project name or project code}.RootDomain or *.{project name or project code}.RootDomain
    • Project Avalon (code AVAL) can only create Private DNS zones like
    • Project DEVON(code DEVO) can only create Private DNS zones like

That’s it. Now you are sure that all your proejcts have unique private dns zones. I highly recommend that you create Azure Policies that enforce your naming conventions


Creating a PE/PL via the Azure Portal currently only shows the Private DNS zones with the recommended names, which makes you believe that a different naming is not supported.

In order to ‘hijack’ this behavior, use different deployment method than the Portal (ARM templates), or continue with the wizard and stop in the last step (do not deploy).

  • Click Download a template for automation
  • Click Deploy
  • Go to Private Dns Name and type a existing Private DNS Zone and the Resource Group. Then click Purchase

Azure VNET integration finally hits GA, but very quitely


It has been almost one year since Azure VNET integration v2 has been announced in preview.

Azure Vnet Integration v2 allows a direct peering between your App Service Plan (web apps) and your VNET without using p2s connections like the v1 integration.

You can now consume vnet and on-premises resources from your web apps and functions.

Announcement Link

How to deploy a Resource via ARM Templates even if it’s not documented ?


Recently, i tried to enable the Update Management Service in Azure Automation via ARM template. Not very complicated, many blog posts and examples can be found over the www.

But what i have not found anywhere is how to to create an Update Deployment Configuration via ARM template (what you see in the picture)

After about 10 minutes of searching the net, i decided to to it myself. But the question is : if it’s undocumented, how can i guess the ARM template json definition?

Two questions arose:

1- Is this supported via ARM templates?

2- How to do it in case it’s supported?

Since no documentation can be found, question 1 answer can be hard to find, but if i answer question 2 and test it, i can answer my question 1.

1- What are ARM templates after all?

Long story short, all ARM deployments are basically Rest API calls, so any ‘operation’ is supported through Rest API. ARM templates are just the Rest API content (the Body of the Rest call, for POST Requests).

So when you write an ARM template, you create the BODY of the Rest call and you send it to Azure. Azure will use it under the hood to create the Rest calls and bingo.

NB: i’m not sure whether the Template to Rest is made at client side (powershell, cli) or at server side, but this is not important in our case.

So the KEY here is that: If you can get the Rest Call Body, you can ‘probably’ get the ARM template JSON definition.

NB: Before continuing, try first to export the ARM template of your resource or resource group and look if you find the JSON definition of your resource. If you don't find it, go to section 2

2- How to get the Rest API Body content?

There are many ways to get this:

  • You can use ARMclient
  • You can use dev tools in the Azure Portal
  • You can use any tool that can unwrap a http request and let you see inside (your favorite tool)

3- Getting the Body Content using the Azure Portal

In my example above, my goal is to create an Update Management Deployment via ARM template, so i started like the following:

1- Go to the Azure Portal and use the wizard to create the UMD, but before submitting the creation b(Create Button) , make step 2

2- Use the Dev Tools (F12 in chrome and Firefox) and start recording a session

3- Submit the creation (Click Create in my case)

4- Stop the recording

5- Find the request that contains the Rest call for your operation

6- Take the following information:

  • Resource Type: It’s contained in the Request URL. In my example it’s Microsoft.Automation/automationAccounts/softwareUpdateConfigurations
  • api-version
  • Body: It’s contained in the Request Paylaod. You can use the ‘view source’ to get the Json definition directly. It contains the name of the resource and the Properties

4- Authoring your ARM template

I will not go through the ARM template Authoring, but you know, prepare a basic ARM Template with parameters and Resources at least (resources is the only needed part actually)

A resource json definition in an ARM template has usually the following format

	"$schema" : "",
	"contentVersion" : "",
	"parameters" : {},
	"variables" : {},
	"resources" : [
			"type" : "",
			"dependsOn" : [],
			"apiVersion" : "",
			"name" : "",
			"location" : "",
			"properties" : {}

Here the view with Json Edit (

Now, you guess it, fill the blanks:

That’s it, you achieved 95% of the ARM template, now you need to adapt it to your context (add parameters, variables, concat, functions if needed…)

5- Deploying the ARM template

Now, you can deploy your ARM template and have the answer to question 1 (in fact, we are not yet sure that this is supported through ARM templates, even if it’s supported via Rest API)

I don’t have the screenshot, but the deployment was successful, and i was able to author an ARM template even if no documentation is provided. This is great!

Introduction to Azure Private Endpoints

Azure Private Links and Endpoints have been recently announced in Public Preview after months of Private Preview and testing. Private Link/Endpoint is a huge step in Azure Networking as it allows to make private any internet facing public service (Like PaaS services: Azure SQL, Azure Storage…), and provides a unified way to expose and consume services between tenants, partners or even within the same customer.

This post is an introduction of Private Endpoints

1- Concept

There are two terms that should be memorized:

  • Private Link: See my next blog post
  • Private Endpoint: Is an association (couple) of a supported resource (ex: azure sql server) and a Subnet in your virtual network so it can be assigned a private IP address

2- Example Scenario

2.1- Need

One of the easiest/most requested scenario is the ability to each an Azure SQL Database privately from a Virtual Network, without exposing it to Internet, or using its Public IP address.

The following picture shows a Virtual Machine (SQLClient) deployed to a VNET/Subnet (VNETEastUS/VM), and an Azure SQL DB (azuresqlprivatedb01) deployed within an Azure SQL server (azuresqlpriavte).

The goal is that the VM (SQLClient) reaches the azuresqlprivate over a private connection, and not via internet (we will not allow any IP to consume the Azure SQL server through the firewall): The dotted line in orange is the goal

2.2- Solution via Private Endpoints

The solution is to create a Private Endpoint that will expose the Azure SQL server via a Private IP address on a Subnet. The following picture shows a Private Endpoint (PE) that is using an IP address from the Subnet Privatesubnet and connected to azuresqlprivate

2.3- Deploy the Solution via the Azure Portal

1- Go to the Private Link Center –> Private endpoints –> Add

2- Type the required information like the Subscription, RG, Location…

3- Choose the resource that you want to create a Private Endpoint for (In my case, it’s the azuresqlprivate Azure SQL server

4- Choose a VNET/Subnet that will enable the access to your resource. Under the hood, a Network Interface (NIC) will be created, assigned an IP address and assigned to your resource. You can optionally integrate the resource with an Azure Private DNS Zone in order that you can call the private endpoint using a DNS name. This is not required as you can create your own DNS record on your own DNS service (A record)

5- After the deployment, you will notice the following:

  • The Private Endpoint is created an the Connection State is Approved*

* Approved means that the Azure SQL party has approved the Private Endpoint, this is useful when both parties are not from the same Team/Tenant, where the requester can ask for the Private Endpoint connection, and waits for the owner to approve it. In addition, you can Reject the connection at any time

  • A NIC has been created and deployed to the Subnet
  • A DNS record has been created (in case you have enabled private DNS option)

2.4- Test the Private Endpoint

On the SQLClient Virtual Machine, you can install SQL Server Management Studio and test the access to the DB

Note that the Private IP is resloved

2.5- Secure the access to the Private Endpoint

Now that we have private access to a PaaS Service, there amy ways to secure the access to it:

  • Use Network Security Groups for the PE Inbound: You can create an NSG and apply it to the Subnet NIC or the Private Endpoint NIC, and filter Inbound rules like any other NSG –> Looks like this not yet supported on the Public Preview
    • Applying a NSG to the NIC is not supported
    • Applying an NSG to the Subnet is without any affect on Private ENdpoints
  • Use Use Network Security Groups for the Outbound: You can filter outbound traffic from your sources to your Private Endpoints: This is supported but not convenient, since it’s better to filter at the destination and not a the source, when securing access from a Destination standpoint (The picture shows a rule that blocks access to the private IP of the private endpoint, and applied to the SQLClient VM)
  • Since the IP address of the Private Endpoint is within your VNET, you can filter access to it on your perimeter Firewalls, like Azure Firewall or your own Firewall

3- Private Links

The next blog post will introduce the Private Link concept, which is a service that allows your external customers/partners to securely access your services via a private connection, and using an Approval Process.


Error deleting an App Service Plan, ghost function


You may not be able to remove an App Service Plan and you hit the following error

Succeeded: 0, Failed: 1, Canceled: 0.Error details {app service plan name}: Server farm ‘ app service plan name ‘ cannot be deleted because it has web app(s) setproperties assigned to it. (Code: Conflict) Server farm ‘ app service plan name ‘ cannot be deleted because it has web app(s) setproperties assigned to it. undefined (Code: Conflict) undefined

Looks like a bug related to the Web App or the Function not ‘correctly’ removed by ARM. You can verify this by looking to the Apps deployed to that App Service plan.

The solution is to force delete this ghost app , and this can easily done through powershell

  • Go to Sites and copy the Resource ID of the troublsome site
  • Open an Powershell window (you need az module) or use the Azure Shell in the Azure Portal
  • Use the Remove-azresource cmdlet to remove that ghost resource
Remove-AzResource -ResourceId "/subscriptions/qsqsqsqsqsqsqs/resourceGroups/qsqsqsqsqsq/providers/Microsoft.Web/sites/setproperties"

Now, the resource has been completely cleaned, you can delete your app service plan

The request to create role assignment ”xxx” is not valid. Role assignment scope. … must match the scope specified on the URI …


If you trying to make a role assignment via ARM template and encountered this error, then please note that this something annoying and not very clear even on MS docs.

The solution:

1- Either notice that there is a small difference in how ResourceGroups is written: resourceGroups vs resourcegroups –> You need to use lowercase or use a replace to replace resourceGroups by resourcegroups

In my case, i have created a variable where i replace resourceGroups by resourcegroups

"variables": {
      "factoryId": "[replace(resourceid('Microsoft.DataFactory/factories/', parameters('DataFactoryObject').name),'resourceGroups','resourcegroups')]",

2- If this does not then it means that you are likely using “type”: “Microsoft.Authorization/roleAssignments” for your role assignment. Unfortunately this does not work if you are making an assignment to anything else the resource group you are deploying in.

To solve this, you can something like:

            "type": "/Microsoft.DataFactory/factories/providers/roleAssignments",
            "name": "[variables('adfroleassignmentname')]",
            "apiVersion": "2015-07-01",
            "dependsOn": [
            "properties": {
              "roleDefinitionId": "[variables('ADFRunpipelineRoleDefId')]",
              "principalId": "[parameters('AzDcrbObjectId')]"

In orther words you will make a sub-resource deployment “ResourceType/providers/roleAssignments”

Azure ARM Template: “ObjectID” with Azure Key vault policy assignment

Hi all,

When you author ARM Templates, and you are deploying a Key Vault and setting the Access Policies via the template, be careful about the content of the objectID.

  "accessPolicies" : [                    {                        "tenantId": "xxxxx-30d9-xxxxx-8015-ddddddd",                        "objectId": "rrrrr-tttt-rrrr-rrrr-tttttt",                        "permissions": {
"keys": ["all"],
"secrets": ["all"]

If you are assigning the policy to a user account, use the objectId value found on Azure AD:

If you are assigning the policy to a Service Principal, use the ObjectID of the Application that you can get from the Enterprise Application blade, and not the App Registration blade.



Delete Azure Backup Restore Points collections error : InternalOperationError goal seeking tasks failed


During an operation to move Azure resources between Subscriptions (Or resource groups), we were obliged to delete the “Microsoft.Compute/restorePointCollections” in order to be able to move VMs protected by a Backup policy, as described here

Unfortunately, when deleting the
“Microsoft.Compute/restorePointCollections” resources, we were hit by the following error.

 {X} goal seeking tasks failed. 

It took us time to figure out that trying to delete the same resources multiple times ends by a successful operations. But because each operation took about 1 minute, it will be a waste of time of doing it by hand.

So today, i’m sharing with you a Powershell script that will allow you to make all the deletion operations, in parallel!!

Go here