In this guide I will describe how to write an azure pipeline, The different parts of an azure pipeline, Gotchas to be aware of, and so on.

It is my hope that by writing this, I can spare you much frustration copy-pasting pre-existing azure pipelines and stack-overflow answers that you don't understand

What is an azure pipeline

An azure pipeline is basically a way of telling Azure DevOps that you would - like to run some process - using azure infrastructure

Here are some common use-cases for azure pipelines: - Build automation – testing, building, and pushing a docker image to a container registry - Managing deployments on a kubernetes cluster, or other cloud computing platform - Performing scheduled database maintenance - Miscellaneous automation of Azure infrastructure – using Azure service endpoints

Anatomy of an azure pipeline

I generally like to think of an azure pipeline in two parts:

• Frontmatter
• The Work

The frontmatter contains declarations for your trigger, pool, variables, parameters, and so on

The steps are where the actual work happens – these are your your tasks, scripts, etc.

Syntax of Frontmatter

99% of my pipelines have frontmatter like this:

  pool:
    vmImage: ubuntu-latest
  variables:
    - name: foo
      value: bar
    - name: duck
      value: soup
  # use parameters for more complex objects, for example
  # I often use these instead of variables if I need to create a pipeline using iteration.
  parameters:
    - name: 'environments'
      type: object
      default:
        - name: development
          namespace: dev
          filename: DevFileNameOne.txt
        - name: production
          namespace: prod
          filename: ProductionFilenameTwo.txt

Syntax of The Work

The Work contains the work that is “run” during an azure pipeline run. Depending on how complex your pipeline is, it may be organized like so:

• A pipeline consisting of one or more stages
• Each stage containing one or more jobs
• Each job containing one or more step

Each subdivision here can be thought of as an enclosing scope, from the most gross (pipeline), to the most granular (step).

This is an important point to get. Here’s the heirarchy again:

Pipeline > Stage > Job > Step

Here is a very bare example of the syntax used:

  stages:
    - stage: Build
    jobs:
      - job:
      displayName: "Build Job A"
      steps:
        - task: ...
        - script: ...
        - stage: Deploy
        ...

If your pipeline is simple (one stage, one job), then you may drop the enclosing stages/jobs declaration and just define an array named steps:

  steps:
    - task: ...
    - task: ...
    - script: ...

Variable and Parameter reference

I’m lifting this straight from the docs:

There are different ways of referencing/expanding variables, according to when you need the variable expanded

Macro syntax - $() - is used to expand variables at run-time, just before a task executes. When a variable is not found, the expression is rendered as-is - i.e., $(foo) remains $(foo) if the pipeline has no such variable.

Template Expression syntax - ${{ }} - is used to expand variables at compile-time. It can also be used to expand pipeline parameters, which are not modifiable at run-time. Because of this, variables expanded with ${{ }} may be used as structural elements of a pipeline – i.e. as keys as well as values.

If a template expression has no value (e.g. in the case a non-existent variable is referenced), then the compiler silently renders it with an empty string at compile-time

Runtime Expression syntax - $[ ] - is used to evaluate expressions at run-time. This is commonly used for conditional execution of jobs or stages

I usually use $(var), but it might be better to use ${{variables.var}} unless I’m explicitly referencing a variable I’m expecting to set during runtime.

Here's a handy table, which I stole:

Setting variables between steps

Sometimes you need to "export" a variable from one step to another. You can use logging commands for this.

steps:
  - script: |
      echo "##vso[task.setvariable variable=hey_there]hot_stuff"
  - script: |
      echo $(hey_there) # outputs "hot_stuff"

However, some variables are read-only. These are:

• Output variables, which are set with the flag `isOutput=true`: ##vso[task.setVariable variable=third;isOutput=true]three
• ReadOnly variables, which are set with the flag `isReadonly=true`: ##vso[task.setVariable variable=second;isReadonly=true]two
• System variables (like System.AccessToken), which are documented here

Such variables must be referenced using macro syntax or runtime expression syntax.

Remember that if a variable is not defined at compile time, template expressions will be empty

Template string operations

Maybe you want to do forbidden magic. I understand.

Interpolating parameters into a script

Use convertToJson to: convert to json

Handy if you know your way around jq

parameters:
- name: 'environments'
  type: object
  default:
  - environment: development
    namespace: dev
  - environment: production
    namespace: prod
steps:
  script: |
    printf '${{ convertToJson(parameters.environments) }}'