Terraform: Merge a List ofObjects into One

The goal is to Merge a variable List of Objects from a Terraform Data Source into One Object using Terraform's Merge function.

 mergedExpanded = merge(local.policy_parameters[*].parameters...)  
 mergedHardcoded = merge(local.mergedExpanded[0].parameters,local.mergedExpanded[1].parameters,local.mergedExpanded[2].parameters)

Note, Terraform can Expand Function Arguments

If the arguments to pass to a function are available in a list or tuple value, that value can be expanded into separate arguments. Provide the list value as an argument and follow it with the ... symbol:

 myList = [{key=value,key2=value}, {key3=value,key4=value4},{key=value,key4=value4}]   
 merge(local.myList...)

The expansion symbol is three periods (...) and not a Unicode ellipsis character. Expansion is a special syntax that is only available in function calls.

Given Problem

We have the existing Policy Definitions:

 require-terraTest1-tag  
 require-terraTest2-tag  
 require-terraTest3-tag

Which we can extract using a data source:

 data "azurerm_policy_definition" "d_policy_definitions" {  
   count = 3  
   display_name = "require-terraTest${count.index}-tag"  
 }

We must then extract the parameters from the result, then decode it as JSON using the jsondecode() function:

 policy_parameters = [  
  for key,value in data.azurerm_policy_definition.d_policy_definitions:  
  {  
      parameters = jsondecode(value.parameters)     
  }  
 ]

We should now be able to Merge all of the Parameters into one variable.

 mergedExpanded = merge(local.policy_parameters[*].parameters...)  
 mergedHardcoded = merge(local.mergedExpanded[0].parameters,local.mergedExpanded[1].parameters,local.mergedExpanded[2].parameters)

Expected Behaviour:

Test1 and Test2 should have the same output.

Actual Behaviour:

Test1 returns an error:

"The expanding argument (indicated by ...) must be of a tuple, list, or set  type."

Test2 Works as expected.

Acceptance of the Limitation

As of today April 17, 2020. Terraform version 0.12 cannot merge a Tuple Variable by Expanding Function Arguments.

I have opened a Github Issue regarding this challenge.

The Workaround

I managed to find a workaround, which I do not claim to be optimized.

 policy_parameters = [  
   for key,value in data.azurerm_policy_definition.d_policy_definitions:  
    {  
     parameters = jsondecode(value.parameters)  
    }  
 ]  
 ph_parameters = local.policy_parameters[*].parameters  
 input_parameter = [for item in local.ph_parameters: merge(item,local.ph_parameters...)]

Break down:

Extract the parameter values into a list of JSON values

 policy_parameters = [  
   for key,value in data.azurerm_policy_definition.d_policy_definitions:  
    {  
     parameters = jsondecode(value.parameters)  
    }  
  ]

Reference the parameters as a variable

ph_parameters = local.policy_parameters[*].parameters

Merge all item content into each item.

 input_parameter = [for item in local.ph_parameters: merge(item,local.ph_parameters...)]

The 3rd step gives all items in the list the same value, so we can use any index. The safest index to use is 0.

Usage:

 parameters = "${jsonencode(local.input_parameter[n])}"

Conclusion

Even though this method works, I think this workaround is wasteful. Imagine having 200 indices with the same value. We'll just have to wait until the next Terraform Update.

JC Bagtas Weblog