The Secret That Only Appeared in the Build, Never the Source

If your Azure Marketplace packaging pipeline ever patches a config file at build time — pulling values from CI secrets and writing them into a createUiDefinition.json or similar wizard file — a perfectly clean source repo can still ship a real credential to every customer who downloads your offer. This isn't hypothetical. It happened to us, and the way it happened is worth understanding because the source code review that would normally catch a hardcoded secret never had a chance to.

How does a clean source repo ship a real secret?

An internal API secret had been unset in CI for months — builds had always shipped with that config field empty, because there was nothing to populate it with. Then the secret got rotated and set as a CI variable for the first time. The very next tagged build picked it up and, for the first time ever, wrote the real value into the offer's wizard JSON as a defaultValue. Nobody touched the wizard file. Nobody reviewed a diff that looked suspicious. The build pipeline just did what it was designed to do, with a real secret available to it for the first time.

Microsoft's automated credential scanner caught it on submission — and the consequence was worse than a simple rejection. It didn't just reject the new version; it automatically pulled the entire existing live offer from distribution while the flagged version sat pending. A product that had been working fine for customers went offline over a build artifact nobody had looked at, because nobody had reason to.

What actually catches this before submission?

1. Don't review the source template — download the actual built package you're about to upload.
2. Extract the wizard/UI definition file and check every field with "key," "secret," "token," or "credential" in its name. defaultValue should be empty or an obvious placeholder.
3. Scan the whole file for long, high-entropy quoted strings even in fields that don't look secret-related — a real secret can show up in a field nobody thought to name carefully:

grep -oE '"[A-Za-z0-9+/=_.~-]{40,}"' createUiDefinition.json

You'll get occasional false positives (a public tracking ID, usually) — a few seconds of eyeballing beats a rejected, de-listed offer.

The durable fix, if the engineering time is available, is structural: never let a publisher-side secret travel inside a customer's deployment package at all. Route anything that needs a real credential through a small relay service authenticated by the customer's own deployment identity instead of a shared secret — that makes this entire class of bug impossible rather than something you have to remember to check every time.