Azure DevOps: Creating Variables During Release

More automation related information today, as we now look to cover Azure DevOps. Azure DevOps is a CI/CD tool that allows for multiple integrations. Natively with Azure, PowerShell, and many other tools, as well as allowing for expansion with additional tools, features, and self hosted agents.

I recently came across a situation where I created a pipeline, and wanted to use values generated in one task within another later in the pipeline. Research led me to some documentation, and later blog entries from Ngan, and Chaminda. Both entries were extremely helpful, and I thank them both for setting me out on the right foot in finding a solution.

Both of their solutions were almost exactly what I was looking for, but there were some changes that I was seeking specifically. When designing the pipeline, I knew that I would not be the one running it, and instead would configure and troubleshoot errors if necessary. With this in mind, I wanted a solution that kept the input clear of anything except values that needed to be changed for running the deployment.

This might not make much sense, but I can lay out the full example. The initial request was to reserve multiple IPs within an IPAM solution. From there we would use portions of the IPs requested to configure a Network Appliance. What I wanted was the deployment location and Appliance name, from there, I would handle the rest within my pipeline, without adding confusion for those running the initial tasks, nor having to continually reach back into IPAM to query for the information.

Authenticate DevOps API

As with most of my tasks that I have covered, I was looking to accomplish these tasks via PowerShell. As seen in the other posts, we need to configure our pipeline to be able to query the OAuth token, and configure a basic authentication header from it within our script.

Navigate to your DevOps instance, the stage, and then select the Agent Job task. From there, navigate below, and select the OAuth checkbox. This allows you to use the $env:SYSTEM_ACCESSTOKEN variable

Within the script, we can use the following commands to configure a basic authentication header from the token. By reading the value, and then performing a base64 conversion, we can get the necessary information. You can re-use this outside the pipeline using a personal access token as well by using the name and token value, and parsing them similarly.

$systemToken = $env:SYSTEM_ACCESSTOKEN
  
#Create Basic Auth Header
#$auth = ('{0}:{1}' -f $tokenname, $tokenvalue) Use for Personal Access Token
$auth = (':{0}' -f $systemToken)
$auth = [Text.Encoding]::UTF8.GetBytes($auth)
$auth = [Convert]::ToBase64String($auth)
$header = @{Authorization=('Basic {0}' -f $auth)}

Get Release Information

To update the release variables, we need to get information about the current release. We can craft the URL using the environment variables, SYSTEM_COLLECTIONURI, SYSTEM_TEAMPROJECT, and RELEASE_RELEASEID. Using these variables, and the DevOps API version 5.0, we concatenate the values and send a request to it.

#Get Environment Details from Release
$organization = $env:SYSTEM_COLLECTIONURI
$project = $env:SYSTEM_TEAMPROJECT
$releaseId = $env:RELEASE_RELEASEID

#Generate Release URL
$devOpsurl = ('{0}{1}/_apis/release/releases/{2}?api-version={3}' -f $organization, $project, $releaseId, $devOpsApiVersion)
Write-Host "URL: $devOpsurl"

#Get Release
$release = Invoke-RestMethod -Method Get -Uri $devOpsurl -Headers $header

The release variable now contains the full release JSON. We can parse through it to get the current variable settings, and then update them

Update Release Variables

Now that we have the current release, we will create a new variable from the item $release.variables, this will allow us to modify the setting, and then add back into the release later.

We have a new variable that we are able to add to, and from initial inspection, it should be simple to just add the new variables we need as entries, re-upload, and call it a day. However, because of how PowerShell parses information back into JSON, and how DevOps expects the release to be formatted we have some extra work to perform to successfully accomplish this.

When PowerShell performs a ConvertTo-Json command, hashtables get converted to braces ({}), and arrays get converted to brackets ([]). So we can expect conversions to look something like below (real code I’ll use in a post later).

@{
  refUpdates = @(
    @{
      "name" ="$refName"
      "oldObjectId" = "$objectId"
    }
  )
  commits = @(
    @{
      "comment" = "$comment"
      "changes" = @(
        @{
          "changeType" = "edit"
          "item" = @{
            "path" = "$filePath"
          }
          "newContent" = @{
            "content" = "$nsgConcat"
            "contentType" = "rawtext"
          }
        }
      )
    }
  )
}

Would get converted to the following within JSON.

{
  "refUpdates":  [
    {
      "name":  "refs/heads/master",
      "oldObjectId":  "0987654321234567890asdf"
    }
  ],
  "commits":  [
    {
      "changes":  [
        {
          "newContent":  {
            "content": "ContentData",
            "contentType":  "rawtext"
          },
          "changeType":  "edit",
          "item":  {
            "path":  "/path/to/data.txt"
          }
        }
      ],
      "comment":  "Added rule asdf to set testNSG.csv"
    }
  ]
}

This becomes important, because variables within the release are expected to be within braces when uploaded back into DevOps. Knowing this, we can properly add items back to the variable for conversion and re-upload. We can view this information by comparing the existing release variables within the portal, and after running a convertto-json on the release variable from earlier.

"variables":  {
  "testRead":  {
    "value":  "VariableAvailable"
  }
}

To create the new value within the variable we created from the release variables ($variableList), we have to jump through a few hoops. I take the variable info and convert that to a hashtable, that I can then add as a member of the parent variable. Running the operation with the -Force option allows us to also updated existing values. We can then re-convert the $release.variables list to be the updated value we have been modifying.

#Set Release Variables to object capable of edit
$variableList = $release.variables

#Add Items to Release Variable List
$variablevalueSet = @{value="$variablevalue"}
$variableList | Add-Member -MemberType NoteProperty -name $variableName -value $VariablevalueSet -Force

#Set Release Variables to updated list
$release.variables = $variableList

Once complete we can now look at re-uploading the release with the new variables.

Re-Upload Release

The good news is that because we have the release already set to a variable, and have only been updating it, sending the information back is not too difficult. We convert the release to JSON using depth 100. This depth is the maximum allowed through the CMDLET, but will thankfully parse all of the information that we need.

After sending to JSON, we read in the data in UTF format, and perform a PUT request to the initial URL we requested from with the same Basic Authentication header, and the UTF Data as the body.

$releaseBody = $release | ConvertTo-Json -Depth 100
$releaseBody = [Text.Encoding]::UTF8.GetBytes($releaseBody)
$release = Invoke-RestMethod -Method Put -Uri $devOpsurl -Headers $header -Body $releaseBody -ContentType 'application/json'

After performing the PUT request, you should be able to navigate back into the portal, and see your new variables on the release.

Adding Multiple Variables

While reading the last few lines, you have likely noticed a weakness with the method listed above, it is only adding a single variable to the release. This is a situation that limits the efficiency of the above, as multiple calls would be needed for multiple variables.

Thankfully, that has already been resolved. By creating a new function, and passing the name and value of new variables to the function, I can add multiple variables to a release using an array as a temporary holding space while parsing them into the JSON.

function New-DevOpsVariable{
 param(
  [string[]] $variableNames,
  [string[]] $variableValues,
  [ValidateSet('2.0', '2.1', '2.2', '2.3', '3.0', '3.1', '3.2', '4.0', '4.1', '5.0', '5.1')]
  [string] $devOpsApiVersion
 )
	
 #Get Environment Details from Release
 $organization = $env:SYSTEM_COLLECTIONURI
 $project = $env:SYSTEM_TEAMPROJECT
 $releaseId = $env:RELEASE_RELEASEID
 $systemToken = $env:SYSTEM_ACCESSTOKEN
    
 #Create Basic Auth Header
 $auth = (':{0}' -f $systemToken)
 $auth = [Text.Encoding]::UTF8.GetBytes($auth)
 $auth = [Convert]::ToBase64String($auth)
 $header = @{Authorization=('Basic {0}' -f $auth)}

 #Generate Release URL
 $devOpsurl = ('{0}{1}/_apis/release/releases/{2}?api-version={3}' -f $organization, $project, $releaseId, $devOpsApiVersion)
 Write-Host "URL: $devOpsurl"

 #Get Release
 $release = Invoke-RestMethod -Method Get -Uri $devOpsurl -Headers $header

 #Parse Variable Information
 $variableNameArray = @()
 $variableValueArray = @()
 $count = 0

 #Populate Variable Array
 foreach ($variableName in $variableNames){
  $variableValue = $variableValues[$count]
  write-host "Name: $variableName"
  write-host "Value: $variableValue"
  $variableNameArray += $variableName
  $variableValueArray += $variableValue
  $count++
 }

 #Set Release Variables to object capable of edit
 $variableList = $release.variables

 #Create Loop to add Items to Release Variable List
 $count = 0
 foreach ($variableName in $variableNameArray){
  $variablevalue = $variableValueArray[$count]
  $variablevalueSet = @{value="$variablevalue"}
  $variableList | Add-Member -MemberType NoteProperty -name $variableName -value $VariablevalueSet -Force
  $count++
 }

 #Set Release Variables to updated list
 $release.variables = $variableList

 #Re-Upload Release with new variables
 $releaseBody = $release | ConvertTo-Json -Depth 100
 $releaseBody = [Text.Encoding]::UTF8.GetBytes($releaseBody)
 $release = Invoke-RestMethod -Method Put -Uri $devOpsurl -Headers $header -Body $releaseBody -ContentType 'application/json'
}

I can re-use this function where I need, simply adding it to any additional scripts I am working on that are running within DevOps. Modifying it to work outside DevOps should not be too tricky, if you are so inclined, only needing to update the authentication header as mentioned before, and specifying the URL manually. To call it, just enter as many names and values as you like, and they should get added successfully

New-DevOpsVariable -variableNames 'variable1', 'variable2', 'variable3' -variableValues 'value1', 'value2', 'value3' -devOpsApiVersion 5.0

Azure DevOps: Creating Variables During Release

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.