my goal is to have some job steps (docker build and push) to run on every Dockerfile I add in a specific folder.

This is the main structure I'm trying:

...

stages:
- stage: Get_DF_Stage
  displayName: Get_Dockerfiles
  jobs:
    - job: Get_DF_Job
      steps:
        - task: bash@3
          inputs:
            filePath: 'helpers/from_files_to_yaml_object.sh'
          displayName: Get_Dockerfiles
          name: Get_DF_Task


# --- Take the latest image on the private repo and retags it with the current date
- stage: Docker_Image_Patch
  displayName: Docker_Image_Patch
  dependsOn:
    - Get_DF_Stage
  jobs:
  - job: docker_login
    displayName: Login to COMPANY CR

    steps:
    - task: Docker@2
      displayName: Login to COMPANY Container Registry
      inputs:
        command: login
        containerRegistry: COMPANY-cr
  
  - job: docker_patch
    displayName: Build and push Docker images

    strategy:
      # $[ stageDependencies.Get_DF_Stage.Get_DF_Job.outputs['Get_DF_Task.dockerfile_list'] ]
      matrix: 
        nginx:
          dockerfile: 'nginx'
        nginx-alpine:
          dockerfile: 'nginx-alpine'  

    steps:
    - task: Docker@2
      displayName: Build Docker image
      inputs:
        command: build
        containerRegistry: COMPANY-cr
        Dockerfile: dockerfiles/$(dockerfile)
        repository: $(dockerfile)
        buildContext: .
        arguments: --no-cache --pull
        tags: latest

    - task: Docker@2
      displayName: Push Docker image
      inputs:
        command: push
        containerRegistry: COMPANY-cr
        repository: $(dockerfile)
        tags: latest

My idea is to generate the YAML executing a bash task, that will have this output:

nginx:
  dockerfile: 'nginx'
nginx-alpine:
  dockerfile: 'nginx-alpine' 

For two files in the folder, named: 'nginx' and 'nginx-alpine'.

Then I would load that variable in the matrix through the stageDependencies command.

Is there a way to insert a YAML object from a bash script? Or is there a simpler way to iterate jobs on files?

CodePudding user response：

Yes this is possible in following way:

jobs:
- job: JobA
  steps:
  - pwsh: |
      $json="{'nginx': {'dockerfile': 'nginx'}, 'nginx-alpine': {'dockerfile': 'nginx-alpine'}}"
      Write-Host "##vso[task.setvariable variable=targets;isOutput=true]$json"
    name: setTargets
  - script: echo $(setTargets.targets)
    name: echovar

- job: buildSrc
  dependsOn: JobA
  displayName: Build source
  strategy:
    matrix: $[ dependencies.JobA.outputs['setTargets.targets'] ]
  variables:
    targets: $[ dependencies.JobA.outputs['setTargets.targets'] ]
  steps:
  - pwsh: Write-Host "${{ convertToJson(variables) }}"
    displayName: 'Print all variables via expression'

You can also check my answer to similar question here

CodePudding user response：

In the end I've found out my solution wasn't working because I was trying to generate a YAML or a JSON Array, instead of a JSON MAP.

So I've modified my bash task to generate a JSON map based on the files in a specific folder:

#!/bin/bash
filenames_list="$(ls -1 dockerfiles/ |tr '\n' ' ' )"
filenames_list=($filenames_list) 

filenames_tmp=""
for (( i=0; i<${#filenames_list[@]}; i   ))
do
    echo "> Found '${filenames_list[$i]}'"
    filenames_tmp="${filenames_tmp}, \"${filenames_list[$i]}\":{\"dockerfile\":\"${filenames_list[$i]}\"}"
done

# Remove first void comma
filenames_tmp="${filenames_tmp[0]:2}"
# Create JSON
filenames_list="{ ${filenames_tmp[*]} }"

## --- Output: { "nginx": { "dockerfile": "nginx" }, "nginx-alpine": { "dockerfile": "nginx-alpine" }}
echo "> Result <${filenames_list}>"
echo "##vso[task.setvariable variable=dockerfile_list;isOutput=true]`echo "${filenames_list}"`"

Then I call the dockerfile variable from the second stage like this

- stage: Docker_Image_Patch
  displayName: Docker_Image_Patch
  dependsOn:
    - Get_DF_Stage
  jobs:
  - job: docker_patch
  displayName: Build and push Docker images
  strategy:
    matrix: $[ stageDependencies.Get_DF_Stage.Get_DF_Job.outputs['Get_DF_Task.dockerfile_list'] ]

  steps:
    - pwsh: Write-Output "$(dockerfile)"
      displayName: Show current Dockerfile
    
    - script: echo "##vso[task.setvariable variable=CURRENT_DATE]`date  "%Y%m%d"`"

    - task: Docker@2
      displayName: Build Docker image
      inputs:
        command: build
        containerRegistry: company-cr
        Dockerfile: dockerfiles/$(dockerfile)
        repository: devops/$(dockerfile)
        buildContext: .
        arguments: --no-cache --pull
        tags: |
          $(CURRENT_DATE)
          latest

    - task: Docker@2
      displayName: Push Docker image
      inputs:
        command: push
        containerRegistry: company-cr
        repository: devops/$(dockerfile)
        tags: |
          $(CURRENT_DATE)
          latest