Imagine an internal approval application moving from one server to two. The home page becomes faster. An employee fills in a form, clicks Save and gets an empty screen: the next request reached a different server, which knows nothing about the draft. The infrastructure has scaled, but the application still behaves as though it has only one instance.
A load balancer distributes incoming work across application instances. This makes it possible to add capacity and take individual instances out of service. First, though, the team must establish which work can move freely between them and which state still belongs to one particular machine.
Two instances must be able to serve the same user
If a session, uploaded file or unfinished operation exists only in one process's memory, another instance cannot reconstruct it. The application needs an appropriate way to manage state: shared storage, suitable session handling and durable results before acknowledging important changes. None of that automatically requires microservices.
Sticky sessions can bind a user to a particular instance and may help an older application. But affinity does not replicate state or preserve it when that instance disappears. Busy users may also end up distributed unevenly.
Check background tasks separately. If every instance launches the same nightly mailing, a second server may double the emails rather than just the capacity. The balancer controls incoming requests; coordination of background work remains another responsibility.
Equal request counts do not mean equal workloads
Opening a request record and exporting a year's history are different amounts of work. Round-robin routing can suit requests of similar cost. When durations vary, choosing the instance with fewer outstanding requests may help. Even that measurement cannot predict how expensive the next operation will be.
The routing boundary also matters. Some balancing decisions select a destination for an entire network connection; others operate at the HTTP-request level. A long-lived connection such as a WebSocket generally stays with its selected server. Adding a third instance does not automatically redistribute existing connections.
Look at response times per instance, active connections and the mix of operations, not just aggregate request counts. The Application Load Balancer documentation describes these considerations alongside routing algorithms and session affinity. Another implementation may have different behaviour.
The surviving capacity has to be sufficient
Take an illustrative example. Each of two identical instances can sustainably handle one hundred operations a minute, while the product needs one hundred and twenty. Both together have enough capacity. After one fails, the balancer routes work to the other. Routing cannot make that remaining instance perform one hundred and twenty operations at the same service level.
Assess spare capacity against the failure the company intends to tolerate: one instance, an entire location, or capacity removed for maintenance. Include the time it takes to bring a replacement into service. Autoscaling can help, but customers still need somewhere to send requests while it reacts.
This does not mean keeping half the infrastructure idle in every product. The company may accept less spare capacity and temporarily restrict optional work instead. That should be a deliberate choice, not something discovered during the first outage.
Taking an instance out of rotation does not finish its work
For a planned update, stop sending new work to the instance and allow accepted requests to finish within an agreed period. This is commonly called draining. It must be coordinated with application shutdown: a balancer's grace period cannot save a request if the process is killed first.
A sudden failure leaves no opportunity for an orderly finish. Some requests will break. Some writes may already have committed before their responses were lost, so automatically replaying every operation on another instance is unsafe. Important changes need status checks and protection against repeated execution.
Test deployment using normal user activity. An employee uploads an attachment during an update: was the file saved, can the action continue, and is any failure explained rather than silently discarding work?
What is still shared behind the balancer?
Several instances offer little help when all depend on the same failed database, file store or broken configuration. The load balancer can itself become a single point of failure unless its deployment provides redundancy. Counting boxes in a diagram does not establish resilience.
Health checks determine which instances should receive traffic. A shallow check leaves broken instances in service. An overbroad check may reject every instance because of one shared dependency. Verify what the chosen product does when no targets are healthy: some configurations continue routing to targets whose health checks have failed.
Shared settings deserve attention too. Certificates, request-size limits and connection timeouts can affect every instance equally. A managed balancer removes some infrastructure maintenance, but someone still owns those choices.
What the acceptance test should demonstrate
Useful exercises include removing one instance under normal load, deploying while users are active, and bringing a new instance into the pool. Observe completed user actions, latency and errors, not merely whether all the indicators eventually return to green.
Look at particular groups as well as averages. Users with long-lived connections or large uploads may have a very different experience from people loading a small page. A capacity test that only fetches the home page will miss those differences.
If more servers produce intermittent logouts, duplicate jobs and greater database pressure, the work is not finished. A small product with stable demand may reasonably remain on one instance, provided the recovery time is understood and acceptable. The relevant comparison is the cost of that interruption against the cost of operating the more complex arrangement.