How to access Azure PostgreSQL Flex Server from ADF Managed VNet using a Private End Point

Currently, a PostgresSQL flexible server doesn't support Azure Private Link. Instead, it uses virtual network injection to make the flexible server available within a virtual network.

While Azure Private Link is on the road map, today, to access Azure PostgreSQL flexible server from an Azure Data Factory VNET using a private endpoint, the following architecture and steps can be adopted.

Note:

• Similar setup can be done for accessing PostgreSQL Flexible Server from an Azure Synapse Analytics Managed Vnet.
  
• The architecture and steps follow the tutorial provided in the documentation link below with minor changes specific to PostgreSQL Flexible server.
  Access Microsoft Azure SQL Managed Instance from Data Factory Managed VNET using Private Endpoint - Azure Data Factory | Microsoft Learn
  

Prerequisites

• Virtual Network. Virtual Network with Azure PostgreSQL Flexible server deployed to it, create one following Create Virtual Network.
  
• Data Factory with Managed VNET enabled. If you don’t have a Data Factory or Managed VNET is not enabled, create one following Create Data Factory with Managed VNET.
  

Create subnets for resources

• Use the portal to create subnets in your virtual network.
  
   
  
  

Create a standard load balancer

Use the portal to create a standard internal load balancer.

1. In the search bar at the top of the portal, search for and select Load Balancers in the Services section of the search pane.
   
2. On the Load balancing services page, Select Create to create a new load balancer.
   
3. On the Basics tab of the Create load balancer page, enter, or select the following details:
    
    
   
4. On the Frontend IP configuration tab of the Create load balancer page, select Add a frontend IP configuration, and then enter, or select the following details on the Add frontend IP address configuration pane:
    
    
   
5. Accept the defaults for the remaining settings, and then select Review + create.
   
6. In the Review + create tab, select Create.
   

Create load balancer resources

Create a backend pool

A backend address pool contains the IP addresses of the virtual (NICs) connected to the load balancer.

Create the backend address pool myBackendPool to include virtual machines for load-balancing internet traffic.

1. Select All services in the left-hand menu, select All resources, and then select myLoadBalancer from the resources list.
   
2. Under Settings, select Backend pools, then select Add.
   
3. On the Add a backend pool page, for name, type myBackendPool, as the name for your backend pool, and then select Add.
   

Create a health probe

The load balancer monitors the status of your app with a health probe.

The health probe adds or removes VMs from the load balancer based on their response to health checks.

Create a health probe named myHealthProbe to monitor the health of the VMs.

1. Select All services in the left-hand menu, select All resources, and then select myLoadBalancer from the resources list.
   
2. Under Settings, select Health probes, then select Add.
    
    
   
    
    
   
3. Leave the rest the defaults and select OK.
   

Create a load balancer rule

A load balancer rule is used to define how traffic is distributed to the VMs. You define the frontend IP configuration for the incoming traffic and the backend IP pool to receive the traffic. The source and destination port are defined in the rule.

In this section, you'll create a load balancer rule:

1. Select All services in the left-hand menu, select All resources, and then select myLoadBalancer from the resources list.
   
2. Under Settings, select Load-balancing rules, then select Add.
   
3. Use these values to configure the load-balancing rule:
   
    
    
   
4. Leave the rest of the defaults and then select OK.
   

Create a private link service

In this section, you'll create a Private Link service behind a standard load balancer.

1. On the upper-left part of the page in the Azure portal, select Create a resource.
   
2. Search for Private Link in the Search the Marketplace box.
   
3. Select Create.
   
4. In Overview under Private Link Center, select the blue Create private link service button.
   
5. In the Basics tab under Create private link service, enter, or select the following information:
   
    
    
   
6. Select the Outbound settings tab or select Next: Outbound settings at the bottom of the page.
   
7. In the Outbound settings tab, enter or select the following information:
   
   
8. Select the Access security tab or select Next: Access security at the bottom of the page.
   
9. Leave the default of Role-based access control only in the Access security tab.
   
10. Select the Tags tab or select Next: Tags at the bottom of the page.
    
11. Select the Review + create tab or select Next: Review + create at the bottom of the page.
    
12. Select Create in the Review + create tab.
    

Create backend servers

1. On the upper-left side of the portal, select Create a resource > Compute > Virtual machine.
   
2. In Create a virtual machine, type or select the values in the Basics tab:
   
    
    
    
   
3. Select the Networking tab, or select Next: Disks, then Next: Networking.
   
4. In the Networking tab, select or enter:
   
    
    
    
   
5. Select Review + create.
   
6. Review the settings, and then select Create.
   
7. You can repeat step 1 to 6 to have more than 1 backend server VM for HA.
   

Creating Forwarding Rule to Endpoint

1. Login and copy script ip_fwd.sh to your backend server VMs.
   
2. Use the nslookup command to get the canonical name for the PostgreSQL Flexible Server.
   

nslookup -type=ptr <FQDN/IP>

<FQDN/IP> is the host/server name of your Postgres Flexible Server Instance.

3. Run the script on with the following options:

sudo ./ip_fwd.sh -i eth0 -f 5432 -a <Canonical Name> -b 5432

<Canonical Name> is the name retrieved for your Postgres Flexible Server Instance in step 2.

4. Run below command and check the iptables in your backend server VMs. You can see one record in your iptables with your target IP.

sudo iptables -t nat -v -L PREROUTING -n --line-number

Note: Run the script again every time you restart the VMs behind the load balancer.

Create a Private Endpoint to Private Link Service

1. Select All services in the left-hand menu, select All resources, and then select your data factory from the resources list.
   
2. Select Author & Monitor to launch the Data Factory UI in a separate tab.
   
3. Go to the Manage tab and then go to the Managed private endpoints section.
   
4. Select + New under Managed private endpoints.
   
5. Select the Private Link Service tile from the list and select Continue.
   
6. Enter the name of private endpoint and select myPrivateLinkService in private link service list.
   
7. Add FQDN of your target PostgreSQL Flexible server. This value is the <Canonical Name> retrieved through running the nslookup command on the backend VM in the “Creating forwarding rule to endpoint” step.
   
    
    
    
   
8. After creating the end point, go to the private link service and approve the private endpoint request.
   
   
9. Under Managed private endpoints, ensure the provisioning and approval state show the end point creation has completed.
    
    
    
    
   

Create a linked service and test the connection

1. Go to the Manage tab and then go to the Managed private endpoints section.
   
2. Select + New under Linked Service.
   
3. Select the Azure Database for PostgreSQL tile from the list and select Continue.
   
4. Input the FQDN of your Postgres Flexible server Instance, user name and password. FQDN will be the server name listed in the Azure portal for the Postgres flexible server.
    
    
   
    
   

Continue reading...