How do banks stay up when a dependency fails?
Key takeaways
- Treat dependency failure as routine. Contain it at runtime, because a rebuild and redeploy is too slow while customers are affected.
- Keep feature flag evaluation local and serve cached state, so a control plane or third-party outage does not take your own controls offline.
- Scope each kill switch to the smallest unit of risk, and define the fallback (previous path, cached response, hidden feature, or queued write) before you need it.
- A tested containment path is the evidence you can show.
- Test each kill switch with the feature flag service unreachable, and govern who can flip it with four-eyes approval and an audit trail.
A payment provider starts responding slowly. A third-party API begins rate-limiting your requests. An internal service times out under load. In distributed systems, failures like these are routine, and a bank runs on a long list of dependencies.
Banks stay up by containing failures at runtime. The controls that do the containing run inside the bank’s own infrastructure, so they keep working when the dependency does not. Those controls are local evaluation, cached feature flag state, and kill switches with defined fallbacks. You also prove they work before an incident.
What a dependency failure looks like inside a bank
Banking teams ship across account access, payments, lending, and digital channels. Each of those areas depends on internal services and third parties. A failure in any one of them can spread to the others unless something limits it.
Recovery restores the failed component, while containment limits the damage during the failure. A failing dependency usually needs a runtime change within seconds, while customers are still affected.
Without granular controls, the default response is a full outage. Your operators cannot switch off one capability, so they hesitate and debate whether the issue justifies shutting down everything.
With fault tolerance, you try to prevent failure. With graceful degradation, you assume failure will occur and limit its impact by keeping at least some functionality running when other parts are not working.
What regulators expect: containment, not a paper plan
EU and DORA
DORA became applicable in January 2025 and requires financial entities to activate containment without delay when something goes wrong. Kill switches for financial services are framed as a legal requirement for operational resilience under the regulation. If your containment path depends on a ticket queue or a release train, it is hard to argue you can act without delay.
From recovery objectives to containment
Recovery objectives measure how fast you restore service. Containment is what you do while the failure is happening, and it is the part you can exercise and evidence.
Keep your controls independent of the failing dependency
A kill switch that fails when the network fails gives you no control at the moment you need it. Network latency and partitions can make remote-controlled kill switches useless. If the feature flag service fails, your application should keep running.
Evaluate locally and serve cached state
Unleash backend SDKs fetch configuration into an in-memory repository and refresh it on a schedule. Evaluation is then a local lookup, which happens in nanoseconds. When you evaluate flags locally and the central feature flag service becomes unreachable, the SDK keeps serving the last known configuration. An already engaged or cached state keeps being honored even when the uplink is down.
A new flip made centrally cannot reach instances until connectivity returns. Pre-position fallbacks for the dependencies you worry about, and test that case explicitly.
Set safe defaults and bootstrap
Choose SDK defaults so a cold start with no network still behaves safely. Bootstrap the SDK from a local file or object store so a restart during an outage does not leave the application running on empty defaults.
Also check your refresh interval. A 15-second poll means a change can take up to 15 seconds to reach every instance. If that is too slow for a given capability, shorten the interval or move closer to real time with Unleash Enterprise Edge’s streaming.
Run an edge layer inside your infrastructure
For frontend use cases, an edge layer inside your own infrastructure can handle evaluation and metrics collection locally. It continues serving evaluations if the central feature flag service goes down. For regulated workloads, self-hosted or single-tenant deployment combined with local evaluation keeps customer and account targeting context inside your perimeter. Teams that run feature flags in banking often start from this deployment model.
Build kill switches and fallbacks for each dependency
Scope each kill switch to the smallest logical unit of risk: one code path, one integration, one counterparty or client, or one UI feature. A single monolithic big red button causes hesitation and cascading failures. Granular targeting isolates the specific feature or algorithm that is misbehaving and leaves the rest of the bank running.
Make “off” the safe state. Use an inverted flag so the default value, off, leaves the kill switch disengaged and normal behavior in place. Name it so anyone reading an alert at 2 a.m. knows what flipping it does. `disable-checkout-v2` tells the reader what happens; `checkout-flag-3` does not.
Decide the fallback when you write the kill switch. Common patterns include:
- Previous implementation: route back to the code path that worked before.
- Static or cached response: serve the last known good data for a slow analytics call or reference lookup.
- Hide the feature: remove the affected option from the interface when a payment provider is failing.
- Queue the write: hold the operation for later, but only if it tolerates delay.
Graceful degradation applies the same thinking to the user experience. Instead of a 500 error, you can show cached or partial data, disable a problematic UI element, route to safer fallback logic, or skip a slow backend dependency.
A canary release limits exposure on the way in, while a kill switch gives you rapid damage control once an issue has already started.
At Lloyds Banking Group, which serves 23 million digitally active customers, a defect in one feature used to roll back all changes together and affect unrelated teams. That is a case for isolating change, and the same principle lets you reduce blast radius when a dependency fails.
Test containment before you need it
A plan that has never run is not containment. Treat every kill switch as unproven until you have exercised it.
Run these checks on a schedule for each critical kill switch:
- Confirm cached state is still served while the feature flag service is unreachable, and that the fallback path runs.
- Confirm cold-start defaults behave safely with no network.
- Confirm the fallback path renders something usable, not a blank screen or an error.
- Confirm who can flip the kill switch and how long the change takes to reach every instance.
These checks are practical good practice. Each run gives a risk or compliance lead a dated result to review. The checks also start after detection, so decide in advance what alert or threshold tells someone to flip a kill switch.
The kill switch itself also needs governance. A kill switch is itself an attack vector, so protect it with dual control (four-eyes approval) and immutable audit logs. Flipping a feature flag in production is a change, and at Prudential it is audited as a production change, with approvals that sync automatically to ServiceNow. Your audit trail should show who changed what, who approved it, and what was affected.
Conclusion
Banks stay up when a dependency fails because containment runs inside their own infrastructure. Unleash supports this FeatureOps approach to runtime control: local evaluation keeps serving the last known configuration when the central server is unreachable, kill switches let you turn off one capability, and four-eyes approvals govern who can flip them. Contain failures at the smallest unit, keep your controls local, and test them before you need them, so one failing vendor, API, or control plane degrades a single capability instead of the whole bank.
Try a containment test on one dependency
Pick one dependency your teams already worry about, such as a payment provider or a third-party API. Start a free trial of Unleash Enterprise and build a single kill switch with a defined fallback. Then check that cached state is still served and the fallback path runs when the feature flag service is unreachable.
FAQs
Why do banks lose whole services when one dependency fails?
Without granular controls, operators cannot switch off one capability. They hesitate and debate whether the issue justifies shutting everything down, and a full outage becomes the default response.
What is the difference between containment and recovery?
Recovery restores the failed component, while containment limits the damage during the failure. A failing dependency usually needs a runtime change within seconds, while customers are still affected.
How can a kill switch work if the feature flag service is down?
Backend SDKs evaluate from an in-memory repository, so evaluation is a local lookup. If the central feature flag service becomes unreachable, the SDK keeps serving the last known configuration. A new change made centrally cannot reach instances until connectivity returns, so pre-position fallbacks. Safe defaults and bootstrapping from a local file or object store cover cold starts.
What should a fallback look like?
Decide it when you write the kill switch. Common patterns are routing to the previous implementation, serving a static or cached response, hiding the feature, or queuing the write if the operation tolerates delay.
Which regulations apply to bank operational resilience?
In the EU, DORA has applied since January 2025 and requires financial entities to activate containment without delay. Check with your compliance team which other regimes apply to your institution.
How do you govern a kill switch?
Treat it as a production change. Use four-eyes approval, immutable audit logs, and an audit trail that shows who changed what, who approved it, and what was affected.
