Standalone Service Fabric Cluster secured with Windows gMSA

[Guest VladSterian]

Background Information

 

 

 

 

To prevent unauthorized access to a Service Fabric cluster, you must secure the cluster. Security is especially important when the cluster runs production workloads. This article describes how to configure node-to-node and client-to-node security by using Windows security, where gMSA is the preferred security model.

 

 

 

A standalone Managed Service Account (sMSA) is a managed domain account that provides automatic password management, simplified service principal name (SPN) management and the ability to delegate the management to other administrators.

 

 

The group Managed Service Account (gMSA) provides the same functionality within the domain but also extends that functionality over multiple servers. When connecting to a service hosted on a server farm, such as Network Load Balanced solution, the authentication protocols supporting mutual authentication require that all instances of the services use the same principal. When a gMSA is used as service principals, the Windows operating system manages the password for the account instead of relying on the administrator to manage the password.

 

 

 

 

 

Scenario:

 

• SFC Running on Azure VMSS - 4 Nodes behind an Azure Load Balancer
  
• Standalone Service Fabric Cluster secured by Windows gMSA
  
• VM's running on Windows Server 2022 (same exact scenario is fully working Windows Server 2019, while on other OS needs to be tested)
  
• This is just a demo setup! We strongly recommend to follow all the planning, security and all other guidelines per each product.
   
  

 

 

 

Table of Content

 

 

Step 1 - Create the infrastructure to host Service Fabric cluster

 

Step 2 - Setup Node0 as Active Directory - Domain Controller

 

Step 3 - Join servers to newly created AD Domain

 

Step 4 - Create gMSA Service Account & Join nodes

 

Step 5 - Prepare nodes for SFC and deploy Service Fabric cluster secured with gMSA Service Account

 

 

 

 

 

 

 

 

 

Step 1 - Create the infrastructure to host Service Fabric cluster

 

 

 

 

Since this step depends on the business needs, we will skip it. Standalone Service Fabric Cluster can run on any environment (On-Prem, Azure VMSS, AWS or any other cloud platform that complies with the minimum requirement of SF and gMSA).

 

Find below article on how to setup the infrastructure on Azure & AWS:

 

Create infrastructure for a cluster on Azure VMs - Azure Service Fabric | Microsoft Learn

 

Create infrastructure for a cluster on AWS - Azure Service Fabric | Microsoft Learn

 

 

 

Keep in mind that for SFC secured with Windows Security, gMSA in particular, we will need at least 4 nodes: 1 AD Domain Controller + 3 Cluster Nodes.

 

 

 

As always, be mindful about the cluster capacity planning. You can decide upon your environment that will host SFC by following the guidelines described in the following article: Service Fabric cluster capacity planning considerations - Azure Service Fabric | Microsoft Learn

 

 

Step 2 - Setup Node0 as Active Directory - Domain Controller

 

 

 

 

2.1 Setup Active Directory

 

• As soon as we have access to node0, we should go to Server Manager:
  -> Add Roles and Features -> Role-based or feature-based installation -> Select : "Active Directory Domain Services" & DNS Server and install these features:
   
   
  

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

2.2 Setup node0 as Active Directory Domain Controller

 

 

 

 

• Just after the feature installation is finished, we can go ahead and promote this node as Domain Controller (DC):
  
  • In Server Manager open the Notification Flag -> Post Deployment configuration -> "Promote this server to a domain controller" :
     
    

 

 

 

 

 

 

• From the Deployment Configuration tab we will select the "Add a new forest" option and we will provide a Root Domain name which can be anything you want and then click on Next to proceed:
   
  

 

 

 

 

• Select a functional level for your forest and domain based on your environment in the Domain Controller Option tab. Please tick the DNS Server and the Global Catalog check boxes because this is the first domain controller inside the forest. Then, for retrieval/restoration of Active Directory data, input the DSRM password. Then, to proceed, click Next:
  

 

 

 

 

• After opening the tab named Additional Options, we will enter a NetBIOS name for our domain which is suggested to be the same as our root domain name entered in previous step.Click Next to proceed.
  
• After you confirm and acknowledge the Prerequisites Check you hit Install . Installation should take a couple of minutes and it will require a node Restart.
  
• When the Restart is done, please check in Active Directory Users and Computers -> Domain Controllers that your node is now a AD DC:
  

 

 

 

 

Step 3 - Join servers to newly created AD Domain

 

 

 

 

Now that we have a functional Domain Controller, we can go ahead and setup the other nodes to join our newly created AD Domain:

 

 

 

• We need to grab the Private IP Address of the DC : we open cmd on DC and quickly run "ipconfig". We can see that the IP is 10.17.0.4 in our case.
  This will be our gateway IP Address for all the other nodes that will join wilddomain.local domain:
  

 

 

 

 

 

 

• Now we move to our second node and we set this IP as "Preferred DNS Server":
  

 

 

 

 

• Now, that we have made the required network settings, we can go ahead and join this server to our wilddomain.local Domain.
  You will be asked to provide the DC Credentials in order to successfully join. After you get the pop-out that Computer successfully joined domain, you will need to restart the machine to have the setting applied.
  

 

 

 

 

 

 

• Repeat previous step for each node that will be part of your domain and finally part of your cluster.
  
• After you have joined all your Servers, please double check in Active Directory Users and Computers-> Computers that you can see your nodes:
  

 

 

 

 

Step 4 - Create gMSA Service Account & Join nodes to the newly created Service Account

 

 

 

 

Now that we have a fully functional AD Domain and all of our servers are already joined our Domain, we can proceed with the gMSA Service Account Creation. We will first deploy our gMSA Service Account on the DC and then move, node by node to join this newly created Service Account.

 

 

 

4.1 Create gMSA Service Account on DC

 

 

 

 

Links: Create the Key Distribution Services KDS Root Key | Microsoft Learn

 

Getting Started with Group Managed Service Accounts | Microsoft Learn

 

Run an Azure Service Fabric service under a gMSA account - Azure Service Fabric | Microsoft Learn

 

 

• Firstly, we need to create one KDS Key and make it available immediately for use. Domain Controllers (DC) require a root key to begin generating gMSA passwords.
  We will run the following PS Commands:
  

 

 

 

Add-KdsRootKey -EffectiveImmediately

Add-KdsRootKey -EffectiveTime ((get-date).addhours(-10))

 

 

 

 

 

• Then, we can tweak the following PS Command based on our needs, to Create and Install the gMSA Service Account on our DC:
  

 

 

 

 

 

 

 

New-ADServiceAccount -name wild-gmsa$ -DnsHostName wild-gmsa.wilddomain.local -ServicePrincipalNames http/wild-gmsa.wilddomain.local -PrincipalsAllowedToRetrieveManagedPassword EWILDVMSS000001$,EWILDVMSS000002$,EWILDVMSS000003$

 

 

 

 

 

Please be aware that we must use "$" after each machine name in order to work

 

 

 

 

 

 

 

 

• We can verify that the creation of gMSA Service Account is done by going to Active Directory Users and Computers-> Managed Service Accounts and check it:
  

 

 

 

 

 

 

 

 

 

 

4.2 Install gMSA Service Account on each Server

 

 

 

 

Links : Run an Azure Service Fabric service under a gMSA account - Azure Service Fabric | Microsoft Learn

 

 

 

• Log-in to second machine, now with your domain username (domain\user). In Server Manager, go ahead and click the Add Roles and Features -> Role-based or feature-based installation -> Select : "Active Directory Domain Services" and install this feature. This will allow us to perform AD specific commands from Powershell or cmd.
   
  
• After installation is done, perform the following PS Commands to Install and Test the gMSA Service Account:
  

 

 

 

Install-ADServiceAccount wild-gmsa$

Test-AdServiceAccount wild-gmsa$

 

 

 

 

 

 

 

 

 

 

 

 

• Repeat previous step for each node that will be part of your domain and finally part of your cluster.
  

Step 5 - Prepare nodes for SFC and deploy Service Fabric cluster secured with gMSA Service Account

 

 

 

 

Links: Create infrastructure for a cluster on Azure VMs - Azure Service Fabric | Microsoft Learn

 

Install Service Fabric standalone client - Azure Service Fabric | Microsoft Learn

 

Secure a cluster running on Windows by using Windows security - Azure Service Fabric | Microsoft Learn

 

 

 

 

 

5.1 Prepare nodes for SFC Deployment

 

• Login using your domain account to the future first node of your cluster.
  

 

 

 

• We need to validate that remote registry was running, enable SMB, and open the requisite ports for SMB and remote registry.
  

 

 

 

To enable SMB, run this PowerShell command:

 

 

 

 

 

netsh advfirewall firewall set rule group="File and Printer Sharing" new enable=Yes

 

 

 

 

 

To open the ports in the firewall run this PowerShell command:

 

 

 

 

 

New-NetFirewallRule -DisplayName "Service Fabric Ports" -Direction Inbound -Action Allow -RemoteAddress LocalSubnet -Protocol TCP -LocalPort 135, 137-139, 445

 

 

 

 

 

 

 

 

• Repeat previous step for each node of your future cluster
   
  

5.2 Modify the ClusterConfiguration template based on our node setup

 

• While connected to the first node of your future cluster, run through following steps.
   
  
• Download the Standalone Setup Package from this link: Standalone Service Fabric .
  Extract it to a specific folder. In my case, I have created new folder called "standalone" under C:\ .
   
  
• Since we are planning to deploy SFC secured with gMSA , we will modify "ClusterConfig.gMSA.Windows.MultiMachine.json" file accordingly.
  I will mention only the minimal settings to be made in order for us to deploy the desired scenario. Feel free to modify it based on your needs.
  
  • Modify the IP Addresses of your machines with the Private IP's of each node
     
    

 

 

 

{

 

"nodeName": "vm0",

 

"iPAddress": "10.17.0.5",

 

"nodeTypeRef": "NodeType0",

 

"faultDomain": "fd:/dc1/r0",

 

"upgradeDomain": "UD0"

 

},

 

 

 

 

 

 

 

• Modify the Security parameters with our newly created gMSA Service Account details
   
   
  

 

 

 

"security": {

 

"ClusterCredentialType": "Windows",

 

"ServerCredentialType": "Windows",

 

"WindowsIdentities": {

 

"ClustergMSAIdentity": "wild-gmsa$@wilddomain.local",

 

"ClusterSPN": "http/wild-gmsa.wilddomain.local",

 

"ClientIdentities": [

 

{

 

"Identity": "wilddomain\\localuser",

 

"IsAdmin": true

 

}

 

]

 

}

 

},

 

 

 

 

 

Be mindful of the format to be used. Not following this format will result in failed SFC Deployment. For example, ClustergMSAIdentity must be in format mysfgmsa@mydomain .

 

I have found out that these formats may be different from one OS to another. If SFC deployment fails, I would recommend trying these formats as well:

 

 

 

"ClustergMSAIdentity": "gMSAname.domain"

 

"ClustergMSAIdentity": "gMSAname"

 

 

 

 

 

If none of these option is working, feel free to contact Microsoft Support.

 

 

 

• Save the ClusterConfig.gMSA.Windows.MultiMachine.json file .
  

 

 

 

5.3 Deploy SFC Standalone secured with gMSA Service Account

 

 

 

 

• Now we simply need to run the following PowerShell command to start the deployment from within the extracted folder:
   
  

 

 

 

.\CreateServiceFabricCluster.ps1 -ClusterConfigFilePath .\ClusterConfig.gMSA.Windows.MultiMachine.json -AcceptEULA

 

 

 

 

 

 

 

 

 

 

• Cluster deployment will kick-off and after couple of minutes we should see the confirmation message that SFC is now accessible via its endpoints.
  

 

 

 

 

 

 

• After deployment of SFC is done, we can check that it's fully working by accessing SFX and verifying that the Fabric processes are running under gMSA Service Account:
  
  • SFX can be accessed at http://localhost:19080/Explorer/
    
  • From PowerShell we can connect to SFC using Connect-ServiceFabricCluster command
    
  • From TaskManager - > Detail tab : We can see that Fabric.exe is running under gMSA account, which is the expected behavior
     
    

 

 

Continue reading...