Replicating VMware NSX-T Services with REST API and PowerShell -

Introduction to NSX

NSX is a network virtualization VMware product. Originally, VMware came up with NSX-V, which worked exclusively with vSphere. Then, they released NSX-T which supports third-party cloud solutions like AWS. There is an unofficial PowerShell module for NSX-V, PowerNSX, an open-source project created by VMware employees.

On the other hand, PowerCLI comes with a NSX-T module that includes five low-level cmdlets. These are not the typical vSphere cmdlets, like Get-VM. You would need to connect to the NSX-T server first and then create an instance of the service corresponding to the objects that you want to work with (that is what the Get-NsxtService cmdlet is for) and then use methods to execute actions on them. There is a learning curve and I personally find this approach not so convenient.

The alternative to the official NSX-T PowerCLI module is using the native PowerShell REST API cmdlets, and this is what we are going to talk about today.

Some Background Information

I have worked on several NSX-V and NSX-T implementations that required the creation of an important number of objects, in the order of the thousands. Therefore, automation has been a must, so I have gained experience with PowerNSX, the PowerCLI NSX-T Module and REST API calls. Recently, I was working on getting a new NSX-T environment configured exactly the same as an existing one. When I was working on setting up Services, I decided to look for a way to take advantage of the configuration that was already in place on one of the data centers and replicate it on the new one. That is how I ended up writing this script.

This is my first post about VMware automation that does not involve PowerCLI. We will use native PowerShell cmdlets exclusively.

Important Notes and Assumptions

The recommended version of PowerShell is 6 or higher, preferably 7. The SkipCertificateCheck parameter of the 'Invoke-RestMethod' cmdlet may be required if the certificates on any of the NSX-T Managers are self-signed.
Ideally, both Managers should be running the same NSX-T version, 3.1.0.
Both Managers must be accessible when running the script.
The script expects only Service Entries of type NestedServiceServiceEntry and L4PortSetServiceEntry.

NSX-T Services and Service Entries

Before getting into the script code, it is important to explain the process to create Services. Be aware that NSX-T has many built-in Services, 409 to be precise; as of now, in version 3.1.0 and purely based on observation, this should not be considered as an indisputable fact.

Services group together protocols and other services (nested services). Then firewall rules can utilize Services to filter traffic. As mentioned before, I will only be working with TCP / UDP ports and nested services.

NSX-T Services Output — NSX-T Service and Service Entry Output. Notice how properties have to be expanded to get to the actual Service and Service Entry objects.

Replicating NSX-T Services with REST API

Time for some coding. The first step is creating the variables for the NSX Manager addresses and building a credential object. Later, we will use this object to authenticate REST API calls against the NSX-T Manager. I am using the abbreviations Src for source and Dst for destination.

$DstNSXTServer = 'https://mynsxtsrc.org'
$SrcNSXTServer = 'https://mynsxtdst.org'

$Username = 'admin'
$SrcPass = 'MySrcPass' | ConvertTo-SecureString -AsPlainText -Force
$DstPass = 'MyDstPass' | ConvertTo-SecureString -AsPlainText -Force

$DstCred = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $Username,$DstPass
$SrcCred = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $Username,$SrcPass

Then, we retrieve the Service objects from both source and destination data centers. These are the first API requests in the script. Right after that, we compare both lists of services to find the ones not present on the source. The result of the difference between both object collections will be our new services.

#Get services in src site
$SrcResults = Invoke-RestMethod -Authentication Basic -SkipCertificateCheck -Method Get -Uri "$SrcNSXTServer/policy/api/v1/infra/services" -Credential $SrcCred |
Select-Object -ExpandProperty results

#Get services in dst site
$DstResults = Invoke-RestMethod -Authentication Basic -SkipCertificateCheck -Method Get -Uri "$DstNSXTServer/policy/api/v1/infra/services" -Credential $DstCred |
Select-Object -ExpandProperty results

#Compare dst with src
$ServiceDiff = Compare-Object -ReferenceObject $SrcResults -DifferenceObject $DstResults -Property id | Select-Object -ExpandProperty id

#Extract from src services ($SrcResults) the ones not present in dst
$ServicesToAdd = $SrcResults | Where-Object id -in $ServiceDiff

#Set counter to 0. It will be used to show progress
$Counter = 0

Before going any further, let's have a closer look at the REST API request syntax. First, there is the Invoke-RestMethod cmdlet. Then, the Authentication parameter tells PowerShell to look for the credential object after the Credential paramater and use basic authentication. SkipCertificateCheck ignores the fact that the NSX Manager's certificate is self-signed (if it is). The Get method, means retrieve/read. Finally, Uri specifies the Manager's FQDN or IP address, followed by the API location for the object we are working with, Services in this case.

IMPORTANT: There is a main foreach loop which I will omit for now since it is a large block of code. It is probably better to break it down in smaller chunks. So, please keep in mind that the rest of the script is inside a loop that processes each Service object ($Service) in the $ServicesToAdd variable. This will become evident at the end of this post, in the full script code.

NSX-T REST API Body: Service

When we send the final request to create the Service, one of the critical components will be the body. The body tells the API endpoint the actions that it must execute. The body structures for NSX-T are formatted as JSON. That said, the next few lines will start building the body with basic Service information from the $Service variable, which at the same time is an item inside the $ServicesToAdd variable (remember the main foreach loop). Service entries have a '[' as opening character. In JSON format, the characters '[' and ']' delimit an array, meaning that there might be more than one service_entry in our body. We are not closing the array with the ']' character right away. We are going to start adding the corresponding Service's service entries from the source.

#Add the first 4 lines of the request body to the $Body variable , these are service properties
$Body = "
{
    ""description"": ""$($Service.description)"",
    ""display_name"": ""$($Service.display_name)"",
    ""_revision"": 0,
    ""service_entries"": ["

NSX-T REST API Body: Nested Service Entries

Now it's time to start adding the Service's service entries to the body. To accomplish this, we need a nested foreach loop that processes each Service Entry object for each Service. First of all, we evaluate the resource_type property and check if it matches the NestedServiceServiceEntry value. If so, we add to the body the lines corresponding to this type of entry using the Insert method.

#Nested loop to iterate through each service entry
foreach ($Service_entry in $Service.service_entries) {
    <#If the service entry type is 'NestedServiceServiceEntry' add the lines in $Service_entry_text
      to the request body ($Body variable). These are service entry properties#>
    If ($Service_entry.resource_type -eq 'NestedServiceServiceEntry') {
        $Service_entry_text = "
        {
            ""resource_type"": ""NestedServiceServiceEntry"",
            ""display_name"": ""$($Service_entry.display_name)"",
            ""nested_service_path"": ""$($Service_entry.nested_service_path)""
        },"
        $Body = $Body.Insert($Body.Length,"`n$Service_entry_text")
    }
}

NSX-T REST API Body: Ports

Next, we do the same for the resource_type L4PortSetServiceEntry but, in this scenario, there could be three possible combinations:

Source and destination ports are specified.
Only source ports are specified.
Only destination ports are specified.

First we add an ElseIf statement that is linked to the If statement for the NestedServiceServiceEntry resource type, and then we add three nested If statements, one for each scenario described above. These nested If structures could be an If/ElseIf construct as well, either way is fine. So, if the resource type is L4PortSetServiceEntry, the required lines are added to the body by one of the nested If statements.

The next step is adding the closing characters to the body. First ']' to close the service entry array and then '}' to close the JSON body structure. Then, the Insert method adds those two characters in two different lines. Lastly, the Remove method takes out the last comma. Each array item is followed by a comma but the last item does not need one, in fact, in that position it is a syntax error.

#Nested loop to iterate through each service entry
foreach ($Service_entry in $Service.service_entries) {
    <#If the service entry type is 'NestedServiceServiceEntry' add the lines in $Service_entry_text
      to the request body ($Body variable). These are service entry properties#>
    If ($Service_entry.resource_type -eq 'NestedServiceServiceEntry') {
        $Service_entry_text = "
        {
            ""resource_type"": ""NestedServiceServiceEntry"",
            ""display_name"": ""$($Service_entry.display_name)"",
            ""nested_service_path"": ""$($Service_entry.nested_service_path)""
        },"
        $Body = $Body.Insert($Body.Length,"`n$Service_entry_text")
    }

    <#If the service entry type is 'L4PortSetServiceEntry' check if both source and destination ports
      are specified and, if true, add the lines in $Service_entry_text to the request body ($Body variable).
      These are service entry properties#>
    ElseIf ($Service_entry.resource_type -eq 'L4PortSetServiceEntry') {
        If ($null -ne $Service_entry.source_ports -and $null -ne $Service_entry.destination_ports) {
            $Service_entry_text = "
            {
                ""resource_type"": ""L4PortSetServiceEntry"",
                ""display_name"": ""$($Service_entry.display_name)"",
                ""destination_ports"": [
                    ""$($Service_entry | Select-Object -ExpandProperty destination_ports)""
                ],
                ""source_ports"": [
                    ""$($Service_entry | Select-Object -ExpandProperty source_ports)""
                ],
                ""l4_protocol"": ""$($Service_entry.l4_protocol)""
            },"
        }

        <#If the service entry type is 'L4PortSetServiceEntry' check if only source ports are specified and,
          if true, add the lines in $Service_entry_text to the request body ($Body variable). These are service
          entry properties#>
        If ($Service_entry.source_ports) {
            $Service_entry_text = "
            {
                ""resource_type"": ""L4PortSetServiceEntry"",
                ""display_name"": ""$($Service_entry.display_name)"",
                ""source_ports"": [
                    ""$($Service_entry | Select-Object -ExpandProperty source_ports)""
                ],
                ""l4_protocol"": ""$($Service_entry.l4_protocol)""
            },"
        }

        <#If the service entry type is 'L4PortSetServiceEntry' check if only destination ports are specified
          and, if true, add the lines in $Service_entry_text to the request body ($Body variable). These are
          service entry properties#>
        If ($Service_entry.destination_ports) {
            $Service_entry_text = "
            {
                ""resource_type"": ""L4PortSetServiceEntry"",
                ""display_name"": ""$($Service_entry.display_name)"",
                ""destination_ports"": [
                    ""$($Service_entry | Select-Object -ExpandProperty destination_ports)""
                ],
                ""l4_protocol"": ""$($Service_entry.l4_protocol)""
            },"
        }

        #Insert the text stored in variable $Service_entry_text at the end of the $Body variable
        $Body = $Body.Insert($Body.Length,"`n$Service_entry_text")
    }
}

#Add closing characters to body and remove the comma from the last service entry in the array
$ClosingText = '
    ]
}'

$Body = $Body.Insert($Body.Length,"`n$ClosingText")
$Body = $Body.Remove($Body.LastIndexOf(','),1)

NSX-T REST API Body: Request

I have already described the requests that use the Get method to read the Service objects. Now, we are going to use the Patch method to create the new Service in the destination data center. The Patch method updates resources but it works well for creating new ones as well. You can check the NSX-T Data Center API Guide for more details. The Put method should also work, however, when available, I prefer Patch over Put, because the latter completely overwrites the resource if it already exists, while Patch only makes updates to it.

The Authentication, SkipCertificateCheck, Method, Uri and Credential parameters have already been explained. That leaves us with Body and ContentType. Body is self-explanatory, it refers to the Body text we have put together, i.e. the $Body variable. ContentType tells PowerShell and most importanly, the API endpoint, that the body we are sending has a JSON format.

The comments explain in detail the message displayed by Write-Host and the increased $Counter variable. We use both to display progress.

#Send the REST request to NSX-T Manager
Invoke-RestMethod -Authentication Basic -SkipCertificateCheck -Method Patch -Uri "$DstNSXTServer/policy/api/v1/infra/services/$($Service.id)" -Credential $DstCred -Body $Body -ContentType 'application/JSON'
    
#Add 1 to counter
$Counter ++

<#Message to show progress, Write-Host used for this example. Write-Progress or Write-Verbose may be more 
appropriate but I personally like the ability to add color to the text.#>
Write-Host "Processed $Counter services of $($ServicesToAdd.Count)" -ForegroundColor Yellow

NSX-T REST API Body: Example

After running all the code, this is an example of how the value of $Body should look like.

{
    "description": "This is a Service body example",
    "display_name": "Service Example",
    "_revision": 0,
    "service_entries": [

        {
            "resource_type": "NestedServiceServiceEntry",
            "display_name": "MS-SQL-S",
            "nested_service_path": "/infra/services/MS-SQL-S"
        },

        {
            "resource_type": "NestedServiceServiceEntry",
            "display_name": "FTP",
            "nested_service_path": "/infra/services/FTP"
        },

        {
            "resource_type": "NestedServiceServiceEntry",
            "display_name": "SMTP",
            "nested_service_path": "/infra/services/SMTP"
        },

        {
            "resource_type": "L4PortSetServiceEntry",
            "display_name": "HTTPS",
            "destination_ports": [
                "443"
            ],
            "l4_protocol": "TCP"
        }

    ]
}

Full Script Code

And finally here is the full script code. As you can see, it has plenty of comments which should make it easier to understand.

#region credentials / authentication
$DstNSXTServer = 'https://mynsxtsrc.org'
$SrcNSXTServer = 'https://mynsxtdst.org'

$Username = 'admin'
$SrcPass = 'MySrcPass' | ConvertTo-SecureString -AsPlainText -Force
$DstPass = 'MyDstPass' | ConvertTo-SecureString -AsPlainText -Force

$DstCred = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $Username,$DstPass
$SrcCred = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $Username,$SrcPass
#endregion credentials / authentication


#region get new services information
#Get services in src site
$SrcResults = Invoke-RestMethod -Authentication Basic -SkipCertificateCheck -Method Get -Uri "$SrcNSXTServer/policy/api/v1/infra/services" -Credential $SrcCred |
Select-Object -ExpandProperty results

#Get services in dst site
$DstResults = Invoke-RestMethod -Authentication Basic -SkipCertificateCheck -Method Get -Uri "$DstNSXTServer/policy/api/v1/infra/services" -Credential $DstCred |
Select-Object -ExpandProperty results

#Compare dst with src
$ServiceDiff = Compare-Object -ReferenceObject $SrcResults -DifferenceObject $DstResults -Property id | Select-Object -ExpandProperty id

#Extract from src services ($SrcResults) the ones not present in dst
$ServicesToAdd = $SrcResults | Where-Object id -in $ServiceDiff

#Set counter to 0. It will be used to show progress
$Counter = 0
#endregion get new services information


#Main foreach structure to iterate through services from previous step
foreach ($Service in $ServicesToAdd) {
    
    #Add the first 4 lines of the request body to the $Body variable , these are service properties
    $Body = "
    {
        ""description"": ""$($Service.description)"",
        ""display_name"": ""$($Service.display_name)"",
        ""_revision"": 0,
        ""service_entries"": ["
    
    #Nested loop to iterate through each service entry
    foreach ($Service_entry in $Service.service_entries) {
        <#If the service entry type is 'NestedServiceServiceEntry' add the lines in $Service_entry_text
          to the request body ($Body variable). These are service entry properties#>
        If ($Service_entry.resource_type -eq 'NestedServiceServiceEntry') {
            $Service_entry_text = "
            {
                ""resource_type"": ""NestedServiceServiceEntry"",
                ""display_name"": ""$($Service_entry.display_name)"",
                ""nested_service_path"": ""$($Service_entry.nested_service_path)""
            },"
            $Body = $Body.Insert($Body.Length,"`n$Service_entry_text")
        }

        <#If the service entry type is 'L4PortSetServiceEntry' check if both source and destination ports
          are specified and, if true, add the lines in $Service_entry_text to the request body ($Body variable).
          These are service entry properties#>
        ElseIf ($Service_entry.resource_type -eq 'L4PortSetServiceEntry') {
            If ($null -ne $Service_entry.source_ports -and $null -ne $Service_entry.destination_ports) {
                $Service_entry_text = "
                {
                    ""resource_type"": ""L4PortSetServiceEntry"",
                    ""display_name"": ""$($Service_entry.display_name)"",
                    ""destination_ports"": [
                        ""$($Service_entry | Select-Object -ExpandProperty destination_ports)""
                    ],
                    ""source_ports"": [
                        ""$($Service_entry | Select-Object -ExpandProperty source_ports)""
                    ],
                    ""l4_protocol"": ""$($Service_entry.l4_protocol)""
                },"
            }

            <#If the service entry type is 'L4PortSetServiceEntry' check if only source ports are specified and,
              if true, add the lines in $Service_entry_text to the request body ($Body variable). These are service
              entry properties#>
            If ($Service_entry.source_ports) {
                $Service_entry_text = "
                {
                    ""resource_type"": ""L4PortSetServiceEntry"",
                    ""display_name"": ""$($Service_entry.display_name)"",
                    ""source_ports"": [
                        ""$($Service_entry | Select-Object -ExpandProperty source_ports)""
                    ],
                    ""l4_protocol"": ""$($Service_entry.l4_protocol)""
                },"
            }

            <#If the service entry type is 'L4PortSetServiceEntry' check if only destination ports are specified
              and, if true, add the lines in $Service_entry_text to the request body ($Body variable). These are
              service entry properties#>
            If ($Service_entry.destination_ports) {
                $Service_entry_text = "
                {
                    ""resource_type"": ""L4PortSetServiceEntry"",
                    ""display_name"": ""$($Service_entry.display_name)"",
                    ""destination_ports"": [
                        ""$($Service_entry | Select-Object -ExpandProperty destination_ports)""
                    ],
                    ""l4_protocol"": ""$($Service_entry.l4_protocol)""
                },"
            }

            #Insert the text stored in variable $Service_entry_text at the end of the $Body variable
            $Body = $Body.Insert($Body.Length,"`n$Service_entry_text")
        }
    }


    #region Add closing characters to body and remove the comma from the last service entry in the array
    $ClosingText = '
        ]
    }'

    $Body = $Body.Insert($Body.Length,"`n$ClosingText")
    $Body = $Body.Remove($Body.LastIndexOf(','),1)
    #endregion Add closing characters to body and remove the comma from the last service entry in the array


    #Send the REST request to NSX-T Manager
    Invoke-RestMethod -Authentication Basic -SkipCertificateCheck -Method Patch -Uri "$DstNSXTServer/policy/api/v1/infra/services/$($Service.id)" -Credential $DstCred -Body $Body -ContentType 'application/JSON'
    
    #Add 1 to counter
    $Counter ++

    <#Message to show progress, Write-Host used for this example. Write-Progress or Write-Verbose may be more 
      appropriate but I personally like the ability to add color to the text.#>
    Write-Host "Processed $Counter services of $($ServicesToAdd.Count)" -ForegroundColor Yellow
}

Feel free to leave comments and questions below.

The source code for this post is available in my GitHub repository.