How one webhook URL stopped Bitbucket's 429s

Jenkins kept crashing at peak hours. Nobody could deploy. Bitbucket answered with 429, rate limit exceeded. Bitbucket Cloud gives you 1000 API calls per hour, and we were burning through them before lunch.

HTTP/1.1 429 Too Many Requests

The proposal on the table was to reduce the number of branches. Nobody had asked why one push turned into hundreds of API calls.

The technical answer was small. Getting there took two years.

Why it stayed broken

We were a medical company. Change control is real there, and it should be. But it had produced a strange asymmetry. Change one setting and you had to explain why. Install one more plugin and nobody asked anything.

Jenkins had accumulated hundreds of plugins that anyone could add, on a single controller that was the one point of failure for every deployment in the company, and that nobody wanted to touch because touching it was scary. I became the first person on the team to keep it updated, one version at a time, because leaving it alone was the bigger risk.

So instead of one big fix, I did it in pieces, each small enough to approve and reversible in minutes.

First: the webhook

Bitbucket was pointed at a generic git webhook plugin, built years ago for a bare git server that could not say what had changed. It received a ping, listed every branch, and diffed them all. Every push, every repo.

Bitbucket already sends the branch name in the payload, and the proper Bitbucket plugin was already installed. Nobody had switched the endpoint. One URL.

Then: the shared library

Every pipeline fetched our shared Groovy library from its own repo at the start of every build, more API calls before a single step ran. Jenkins has a cache for this. It was off. One checkbox.

Then: the scan triggers

When nothing seemed to happen, people pressed “Scan Multibranch Pipeline Now”, which fired another full scan against the same rate limit, so a discovery that should take seconds took hours.

I replaced the automatic rescans with a fixed schedule as a safety net, while the webhook still indexed the pushed branch immediately. Nobody waited for a build. A team with more branches or more activity no longer affected anyone else.

The result

API calls per push went from hundreds to a handful. The 429s stopped. Pipelines that had been taking hours, stuck in checkout waiting for Bitbucket, finished in minutes again.

And the controller’s CPU and memory came down with them: every stuck job had been an open thread holding its memory, and hundreds of those were what had been pushing the controller into OOM. That is how it should be: a controller schedules and stores, the work belongs on the agents. Nobody had to delete a branch, and nothing was rewritten.

What I took from it

In a regulated environment, the skill is not knowing the fix. It is breaking the fix into steps small enough that saying yes is easier than saying no, and being the person willing to touch the thing everyone is afraid of.