A supplier-catalogue import service may have a distinctly uneven workload. Small updates arrive throughout the day, then several partners submit large files near a shared deadline. Keeping enough workers for the peak at all times is expensive. Adding them manually after customers complain is hardly an attractive alternative.
Autoscaling adjusts available capacity in response to observed demand. Here the focus is horizontal scaling: adding application instances or background workers. That reaction takes time and has limits. Sometimes the right policy includes preparing capacity in advance, even though reducing permanently idle capacity was the original motivation.
Measure the work the customer is waiting for
For an import, accepting the upload is not the finished result. If the HTTP request completes quickly but processing sits in a queue, the product still feels slow. CPU utilisation may not reveal the bottleneck at all: workers could be waiting for an external service.
For independent background jobs, useful signals include outstanding work per available worker and the age of waiting jobs. Queue length alone lacks context: one worker and twenty workers drain the same backlog differently. The SQS-based scaling guidance describes the backlog-per-instance approach.
Even job count can mislead. A small amendment and an annual catalogue export may each count as one task. Account for variation in job sizes and duration, and watch the oldest waiting work. An acceptable average can conceal an unacceptable delay for particular customers.
Requested capacity is not ready capacity
Between detecting load and getting useful capacity come metric collection, a controller decision, resource allocation, application startup and warm-up. A new process may need configuration, connections and reference data before it can contribute. Counting it as useful capacity immediately creates an optimistic picture of recovery.
A brief spike may be over before the extra workers arrive. A small permanent reserve can be the cheaper answer. For a known opening time or scheduled bulk import, raising minimum capacity beforehand and returning to the normal policy afterwards is perfectly reasonable.
The controller may also request instances that the platform cannot place. Desired capacity rises while actual capacity stays unchanged. Monitor ready instances and launch failures, check quotas and resource availability, and do not confuse an accepted scaling instruction with a completed capacity increase.
More application instances can make the database slower
Test whether additional workers increase completed throughput. If they barely help, the limit may be a shared database, lock contention, storage or a partner's API allowance. Continuing to add workers then produces a more expensive version of the same queue.
Each instance also adds connections and background activity. The maximum scale must respect the combined connection budget, shared infrastructure and external quotas. A maximum chosen only from the spending budget can still be technically unsafe.
Check how overload protection interacts with the scaling signal. If early rejection keeps CPU low, a CPU-only controller may fail to observe unmet demand. The Amazon Builders’ Library discussion of load shedding describes this interaction. Rejected traffic matters alongside the work that was admitted.
Removing capacity needs its own design
The visible queue empties and the controller removes half the workers. Yet some tasks have already left the queue and are still running. Terminating processes immediately may cause repeat processing or even lost work if acknowledgements were designed incorrectly.
A retiring worker should stop accepting new jobs and either finish existing work or hand it back through a defined recovery mechanism. Redelivery needs safe duplicate handling. Long jobs may benefit from checkpoints or supported protection against scale-in while they are active.
Growth and reduction need not happen at the same speed. Reacting instantly to every small dip can cause repeated startup and shutdown. Stabilisation windows and limits on the rate of scale-in help. The Horizontal Pod Autoscaler documentation gives one concrete implementation of those controls.
Reaching the maximum requires a product decision
Automation needs limits and an explicit response when it reaches them. Once maximum capacity is busy, new work does not disappear. The product must restrict admission, accept it with an honest waiting time, or reject it before promising a result it cannot deliver.
A budget notification is not necessarily an immediate spending cutoff. Storage, network traffic and dependent services also cost money. Someone needs visibility into the total, authority to stop abnormal growth and a response for abusive traffic or a looping integration.
Evaluate savings per completed useful task. A lower compute bill may come with longer waits, more repeated work and more support intervention. For a modest, steady workload, fixed capacity can remain cheaper when the full operating cost is considered.
The test ends only after demand falls again
A graph of instance count following traffic is not sufficient evidence. Exercise a sudden increase, the configured ceiling, delayed launches, a decline and another burst. Real jobs should be in progress while capacity is reduced; otherwise the most fragile part of the cycle remains untested.
Measure customer waiting time, missing or repeated work, pressure on the database and cost across that whole cycle. Test what happens when metrics disappear, too: the policy should have an understood response instead of depending on an operator noticing an empty dashboard.
The result may justify a larger minimum reserve or a lower concurrency limit. Neither contradicts autoscaling. The team chooses the acceptable balance of waiting, risk and expense; the controller implements it. For a predictable workload, the test may also show that a schedule and a small fixed reserve are sufficient, without a more elaborate reactive policy.