Hitting (undocumented?) reusable workflow limit of 20

[JackCoplandGranta]

I've just started hitting a reusable workflow limit of 20:

I can't find any official docs of this limit, I'd expect it to be mentioned here. Is this mentioned somewhere I've missed?

I know the depth limit of reusable workflows was raised from 2 to 4 a bit ago, is there any hope of raising this limitation of them too?

[int128]

I tested 26 workflows in my public repository and it worked.
int128/sandbox#178

However, I tested it in the organization private repository, the same error occurred.

Also I tested calling 26 different workflows, and got the error.
int128/sandbox#179

too many workflows are referenced, limit is 20

[JackCoplandGranta]

This is still an ongoing issue for us, though the limitation is now at least documented. We're having to consider very tortured workarounds to deal with this.

[cruizba]

How it is possible that you try to apply "separation of concerns" and end up with more technical debt because of another undocumented limit? :(

[cruizba]

Did you find a workaround? @JackCoplandGranta

[JackCoplandGranta]

@cruizba There are some methods to achieve a similar effect, though none are as good as just a limit raise:

• Use composite actions (less capable feature, depth limit 10, not sure if there is a total use limit)
• Have a single reusable workflow with multiple jobs, and use job conditions (if:${{ inputs.job == 'job3' }}) to select which job to run using an input to the workflow. This 'counts' as a single reusable workflow, but can 'store' several jobs. If you have a complex job dependency structure, or lots of inputs, this may be hard to construct and maintain.
• Use the workflow_run trigger, to chain together workflows. Workflow runs have a call depth limit of 3, but you can skirt this by manually triggering workflows via the rest API.
• Cut the Gordian knot and tactically duplicate code rather than re-use.

A combination of the above as most appropriate for your workflow structure will work, but obviously introduces various issues.

This limit is entirely artificial, you can construct workflows of arbitrary job and step complexity by abusing the features listed above and they won't have any issue starting or running.

[rjmcnamara]

Have a single reusable workflow with multiple jobs, and use job conditions (if:${{ inputs.job == 'job3' }}) to select which job to run using an input to the workflow. This 'counts' as a single reusable workflow, but can 'store' several jobs. If you have a complex job dependency structure, or lots of inputs, this may be hard to construct and maintain.

@JackCoplandGranta have you tried that? That was my same thought while trying to work around this arbitrary 20 workflow limit, but when I implemented the workaround, I found that workflows are included towards the limit whether or not they are actually called.

[JackCoplandGranta]

@rjmcnamara It sounds like you're still listing the reusable workflows and conditioning them at the top level. That's not what i meant. I mean combining several of your re-usable workflows into one, something like:

parent.yml, where you call the re-usable workflows:

name: "Full Chain"
on:
  workflow_dispatch:

jobs:

  Combined_Workflow:
    uses: ./.github/workflows/combined.yml
    secrets: inherit

combined.yml:

name: Combined Workflows

on:
  workflow_call:

jobs:

  job1:
    name: Job1
    runs-on: [self-hosted, windows, x64]
    steps:
      - run: sleep 5
      
  job2:
    name: Job2
    runs-on: [self-hosted, windows, x64]
    steps:
      - run: sleep 5

where Job1 and Job2 used to be re-usable workflows that you invoked from parent.yml as separate items. And if you need control of the jobs to run, pass down inputs.

[FelixSchladt]

I just had the same problem and I am even using self hosted runners.
How is this still a limitation in 2024???

[potiuk]

We've also hit the limit of it in our "code reuse" quest in Apache Airflow - we managed it by combining some jobs together, but it would be nice to have bigger limit.

[mhaley37]

This is an incredible frustrating limit to run up against -- both because its' unexpected and that it feels completely arbitrary. Much like the arbitrary depth "call workflow" limit, there is no meaningful response from GitHub. Why include such rich features for re-use of workflows only to introduce limits like this? Open this work to community submissions and I'm sure you could find a contributor.