Eternities of Gitlab CI pipeline trial + error...only run with changes

I've spent many mind-numbing hours trying to get this Gitlab CI pipeline to work the way I want it to work. I'm not sure if I'm doing something wrong, or if GitlabCI is doing something wrong, but I'm becoming more and more convinced that it's GitlabCI.

Essentially, we have many different projects in our repo, separated by dirs. If nothing in a project changes, I don't want it to go through the CI process because it's a waste of time.

Luckily, GitlabCI solves for this...right...? I mean, yeah, you can define variables so they should work everywhere like you'd expect, right?

I started with something like this. Imagine like 10 of the one-project--plan all for different projects. I wanted to break out the only:changes part so that I don't have to repeat it everywhere.

one-project--plan:  
  stage: terraform-plan
  extends:
    - .install-cli-and-assume-role
    - .terraform-plan-only
  variables:
    TF_DIR: path/to/my/project
    WORKSPACE_NAME: my-tf-project-workspace-one

.terraform-plan-only:
  script:
    - cd "${TF_DIR}"
    - terraform init
    - terraform plan
  except:
    - master
  only:
    changes:
      - '${TF_DIR}/**/*'

Great - that makes sense, right? Sure, but it doesn't work. Hardcode the path, though, and it works. Ok, that's weird, but at least we have a baseline to work from.

A little digging shows that only is compiled at a different time than the rest of it, so you can't use variables in only/except. The recommended approach is to switch to using rules. Rules are fine. Not as nice-looking as only/except, and the logic is OR for some reason, but whatever.

Next iteration:

.terraform-plan-only:
  script:
    - cd "${TF_DIR}"
    - terraform init
    - terraform plan
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
      changes:
        - $TF_DIR/**/*

Imagine ~100 commits where I mess with various ways to quote that variable.

This link in the docs shows something disconcertingly similar:

docker build:  
  variables:
    DOCKERFILES_DIR: 'path/to/files'
  script: docker build -t my-image:$CI_COMMIT_REF_SLUG .
  rules:
    - changes:
        - $DOCKERFILES_DIR/*

There are also threads like this one in the Gitlab repo showing that something like this is supposed to work.

But it doesn't work.

I'm guessing it doesn't work because it's an inherited configuration entry and because rules:changes seems to be a sort of special case for Gitlab CI. Part of which means that it can't use variables that are defined in a stage, but only more global variables. But who knows. I'm just going to repeat that segment for every different project, which is...fine. Just fine.

TLDR;

This works:

variables:  
  MY_PROJECT: path/to/project

.terraform-plan-only:
  script:
    - cd "${TF_DIR}"
    - terraform init
    - terraform plan
  variables:
    TF_DIR: $MY_PROJECT
    WORKSPACE_NAME: my-project
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
      changes:
        - $MY_PROJECT/**/*

but this doesn't work:

variables:  
  MY_PROJECT: path/to/project

.terraform-plan-only:
  script:
    - cd "${TF_DIR}"
    - terraform init
    - terraform plan
  variables:
    TF_DIR: $MY_PROJECT
    WORKSPACE_NAME: my-project
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
      changes:
        - $TF_DIR/**/*

I hope you ran across this when there's still some of your day to get back.

Menu

TLDR;