Most integration projects look finished on the day the demo works. You click "Pay", Stripe says yes, the order shows up in the dashboard, and everybody goes home happy. Then real traffic arrives.
We've connected web apps to payment providers, CRMs, shipping carriers and a few APIs nobody has heard of. The first version is rarely the hard part. What costs money is the stuff that happens three weeks after launch, usually at 2 a.m., when a webhook arrives twice or doesn't arrive at all.
Here are the six problems we plan for now, because each one has bitten us or a client at least once.
1. The same event arrives twice
Webhook senders promise "at least once" delivery. That phrase means duplicates are normal. Stripe, for example, keeps retrying a failed webhook for up to 3 days, and a retry can land after your server already processed the first attempt but answered too slowly.
If your handler creates an order every time it's called, you'll ship two parcels for one payment.
The fix is boring. Store the event ID, check it before doing any work, and skip anything you've seen. A unique index in Postgres or MySQL does this in one line and it never forgets.
2. Your own request gets sent twice
It works in the other direction too. Your server calls the API, the network hiccups, you never see the response. Did the charge go through? You can't tell, so the code retries.
Good APIs give you an idempotency key for this. You send a random string with the request, and if the same string shows up again the provider returns the original result instead of charging the card a second time. Stripe keeps those keys for 24 hours. Use them on every POST that creates or moves something. It's one Idempotency-Key header, and it's free.
3. Rate limits you only meet in production
In development you make ten requests a day. On Black Friday you make ten a second, and the API starts answering 429 Too Many Requests.
Shopify's REST Admin API is a well known case: a standard store gets a bucket of 40 requests that refills at 2 per second. A nightly sync that loops over 5,000 products without pausing will hit that wall in the first minute.
So we put a queue between the app and the provider. Jobs go in, a worker takes them out at a pace the API accepts, and a 429 means "wait and try again" rather than "lose the update". Read the Retry-After header when there is one. Guessing is slower.
4. Webhooks that nobody verified
A webhook endpoint is a public URL that changes data in your system. Anyone who finds it can post to it.
Every serious provider signs its webhooks. Stripe puts an HMAC-SHA256 signature in the Stripe-Signature header, GitHub uses X-Hub-Signature-256. Checking it takes about ten lines, and we still inherit projects where it's commented out "for testing". Verify the signature, reject anything older than 5 minutes, and answer 200 quickly. Do the slow work afterwards in a background job, because most providers give up after a few seconds and count it as a failure.
5. The API changes and nobody tells you
They do tell you, to be fair. The notice goes to an inbox that belonged to a developer who left in 2023.
Two habits help. Pin the API version in your code instead of floating on "latest", so an upgrade is something you choose on a Tuesday morning. And put the deprecation emails on a shared address that a human reads.
6. No way to see what happened
A customer writes: "I paid but my account is still locked." Can you find that payment's webhook in under five minutes?
If the answer is no, you'll spend an afternoon on it. We log every inbound event and every outbound call with its ID, status and timing, and keep them for at least 30 days. One table is enough. It doesn't need a fancy tool, it needs a search box.
What this means for your budget
When we estimate an integration, the "happy path" is the small line. Retries, queues, signature checks and logging take more hours than the call itself. A quote that only covers the demo isn't cheaper, it just moves the bill to later.
If you're planning an integration, or you already have one that misbehaves, send us a note. We'll look at the flow and tell you which of these six is most likely to hurt first.