Skip to main content

Azure Load Balancer Best Practices for Production Workloads

A correctly configured Azure Load Balancer reliably distributes TCP and UDP flows across healthy backends. But production‑grade load balancing is more than a frontend IP and a rule. It requires deliberate decisions about health signalling, failure domains, network security, outbound connectivity, deployment safety, and observability. When these decisions are made in isolation, the load balancer can hide rather than prevent problems.

This article describes practices that reduce the probability and impact of production incidents for services that depend on Azure Load Balancer. Every recommendation is tied to a specific failure mode, explains the reasoning, and—where practical—tells you how to validate that the practice is working. The guidance applies primarily to the Standard SKU and assumes that you are operating a workload where availability matters.

1. Choose the correct service and scope​

Before you create a load balancer, confirm that a Layer 4 (transport‑layer) distributor is the right tool for the job. Azure Load Balancer is designed for regional, high‑throughput TCP/UDP traffic. It does not provide HTTP path routing, host‑based routing, TLS termination, or a web application firewall.

Evaluate these alternatives when your requirements extend beyond Layer 4:

  • Azure Application Gateway – regional Layer 7 reverse proxy with URL routing, TLS offload, and WAF.
  • Azure Front Door – global HTTP/S entry point with edge caching and WAF.
  • Azure Traffic Manager – DNS‑based global endpoint selection and failover.
  • Azure NAT Gateway – purpose‑built for outbound internet connectivity, not inbound load balancing.

These services can be layered together. For example, Front Door → Application Gateway → internal Load Balancer is a common design that separates global entry, application‑layer processing, and internal traffic distribution. If you need any of the Layer 7 capabilities, do not try to force them into a Layer 4 load balancer; you will create an architecture that is fragile and difficult to operate.

Also decide early whether the frontend must be public (internet‑facing) or internal (private VNet address). This choice defines the trust boundary, the NSG rules you will need, and the operational exposure of your service. Prefer an internal load balancer for middle‑tier and data‑tier services; only expose a public frontend when internet clients must reach the service directly.

2. Select and govern the SKU​

Today Standard Load Balancer is the appropriate SKU for new production deployments. Basic Load Balancer has been retired and is no longer available for new resources. Standard provides zone‑redundant frontends, a secure‑by‑default network posture, support for IP‑based backends, and a 99.99 % SLA when you deploy at least two healthy backend instances across availability zones.

Before you commit to a design, verify the current SKU capabilities and limitations in the official Azure documentation, particularly:

  • Supported regions and availability zone status.
  • Backend pool size and membership types.
  • Feature differences (e.g., HA ports, outbound rules).
  • Pricing dimensions (data processed, public IPs, NAT Gateway if used).
  • Migration implications if you are moving from Basic.

Do not assume that a feature available in one SKU or region is universally available. Always check the current documentation and incorporate that check into your architecture review process.

3. Design backend pools for availability​

A load balancer that points to a single virtual machine or a single scale‑set instance does not improve availability—it adds a network hop. For production workloads, place at least two backend instances in the pool and ensure they are functionally interchangeable for the traffic that the rule distributes.

Prefer Virtual Machine Scale Sets when you need managed instance membership, horizontal autoscaling, and automated replacement of unhealthy VMs. Scale sets integrate natively with backend pools and simplify capacity management.

Keep backend instances stateless where possible. State that lives only in instance memory is lost when a backend fails or is removed. Store sessions, caches, and shared data in external services (Azure Cache for Redis, a database, or a shared storage layer). Stateless design allows any healthy backend to handle any request, which is fundamental to transparent failover.

Plan for reduced capacity. When one instance or an entire zone fails, the remaining backends must absorb the traffic. Size your pool so that you can lose the largest expected failure domain and still have enough headroom to handle peak load. This is not a load‑balancer configuration knob; it is a capacity‑planning decision that must be validated with load tests.

4. Design health probes around readiness​

A health probe answers a single operational question: should this backend receive new connections right now? The probe is the gate between the load balancer and your application; getting it wrong causes two common failures—sending traffic to a deadlocked instance, or removing all instances because a non‑critical dependency was temporarily slow.

Probe depth trade‑offs​

Probe depthWhat it catchesRisk
TCP port open (shallow)Process has not crashed, port is listeningDeadlocked process may still pass; no application context
HTTP /health returning 200 (application‑level)Application framework is respondingApplication logic may be broken; dependencies not checked
HTTP /ready checking local resources only (moderate)Core service can process a request; local config/secrets loadedRemote dependency failure may not be detected
HTTP /ready checking all critical downstream services (deep)Full readiness verifiedCascading probe failures if one shared dependency slows

There is no single correct depth. The principle is to probe enough to avoid serving broken instances, but not so deeply that every transient dependency problem evicts your entire service. A dedicated readiness endpoint that verifies the minimal local prerequisites—configuration loaded, worker threads started, essential local resources available—is usually the best compromise.

Operational requirements for probes​

  • Probe traffic must be explicitly allowed. With Standard Load Balancer, you must add NSG rules on the backend subnet (or NICs) that permit inbound traffic from the AzureLoadBalancer service tag on the probe port. A missing rule causes all backends to appear unhealthy.
  • Bind the service to 0.0.0.0 or the specific interface the probe expects. An application listening only on localhost will fail health checks.
  • The probe path should be fast, deterministic, and not trigger side effects. Avoid calling external APIs, writing to databases, or performing expensive computations inside a health check.
  • Monitor probe failures as an operational signal. A sudden drop in healthy backend count is an incident that demands immediate attention, not a background metric.

Representative readiness endpoint​

GET /health/ready HTTP/1.1
200 OK

The implementation should return 200 only when the instance can accept production traffic for the specific service port. When the instance is shutting down or initialising, it should return a non‑200 status to allow the load balancer to drain connections gracefully.

5. Configure rules deliberately​

Each load‑balancing rule ties a frontend IP:port, a protocol, a backend pool, and a health probe together. Keep these mappings simple and well‑documented:

  • One rule for one traffic intention. Avoid reusing a probe for multiple rules unless the services are identical and the probe semantics genuinely apply.
  • Explicitly choose the backend port. The frontend port and backend port can differ; document why and make the choice visible in your Infrastructure as Code.
  • Session persistence (source IP affinity) is not a replacement for external state. Use it only when a legacy application genuinely requires it, and understand that failover becomes coarser.
  • Review idle time‑outs. The default TCP idle time‑out is 4 minutes. Long‑lived idle connections (e.g., database replicas) may need a longer time‑out; applications can also use TCP keep‑alives to keep the connection alive.

6. Secure public and internal exposure​

Security is not a checkbox on the load balancer; it is a set of layered controls that must be deliberately designed.

Public load balancers​

  • Restrict the ports and protocols exposed on the public frontend. If clients only need TCP 443, do not open TCP 80 or any other port.
  • Never assign public IPs directly to backend VMs unless there is an explicit, documented requirement. Backends should remain in private subnets.
  • Apply Network Security Groups to the backend subnet with rules that allow only the expected client source ranges and the necessary health probe traffic.
  • Consider Azure DDoS Protection Standard for workloads that are likely targets of volumetric attacks.
  • TLS must be terminated somewhere; Azure Load Balancer does not do it. Terminate TLS on the backend instances or place Application Gateway/Front Door in front.

Internal load balancers​

  • A private frontend IP is not automatic trust. Apply NSGs that restrict traffic between tiers to only the required ports and protocols.
  • Document the exact VNets, peerings, VPN, or ExpressRoute connections that are expected to reach the internal frontend. Unintended reachability is a security incident.
  • Use separate internal load balancers for different trust zones (e.g., one for the application tier, another for the data tier).

Management access​

Do not expose SSH (port 22) or RDP (port 3389) to the internet through broad inbound NAT rules. Prefer:

  • Azure Bastion – browser‑based, managed access without public IPs.
  • Just‑In‑Time (JIT) VM access – time‑bound, role‑based access through Microsoft Defender for Cloud.
  • Private jump hosts – accessed via VPN or ExpressRoute.

If inbound NAT rules are unavoidable, restrict the source IP prefixes to specific management networks and monitor access logs.

7. Separate inbound and outbound design​

Inbound load balancing and outbound internet connectivity are different architectural paths. A public load balancer can provide Source Network Address Translation (SNAT) for outbound connections from backend instances, but this is not its primary purpose and can lead to port exhaustion under high connection rates.

Use Azure NAT Gateway for production outbound connectivity when you need:

  • Predictable, static public IP addresses for egress.
  • Large SNAT port allocations per VM.
  • Independent scaling of outbound capacity.
  • Egress that does not depend on the presence of a public load balancer.

Even if you use a public load balancer with outbound rules, be aware of SNAT port limits and monitor SNAT connection metrics. When outbound connections start failing intermittently while inbound traffic flows normally, SNAT exhaustion is a likely cause.

For internal load balancers, outbound internet access must be provided by a NAT Gateway or another egress solution; the internal load balancer alone offers none.

8. Design for zones and failure domains​

Standard Load Balancer supports both zonal and zone‑redundant frontend configurations. A zone‑redundant frontend remains reachable even if an entire availability zone fails, provided you have healthy backends in the surviving zones.

  • Distribute backend instances across at least two availability zones. Three zones are preferable for workloads that require high availability.
  • Do not place all capacity in a single zone, even if you use a zone‑redundant frontend. If that zone fails, every backend fails, and the frontend has nothing to forward to.
  • Verify that your backend capacity in the remaining zones can handle the full production load after a zone outage. This is a capacity sizing exercise, not a load‑balancer setting.
  • Test zonal failure assumptions. Simulate the loss of an entire zone (where your testing permissions and environment allow) and measure recovery time and traffic impact.

For cross‑region resilience, Azure Load Balancer alone is insufficient. You must combine it with a global traffic service such as Azure Front Door (for HTTP/S) or Traffic Manager (for DNS‑based failover).

9. Plan capacity and connection behaviour​

Capacity planning for a Layer 4 load balancer must account for behaviour that is different from an HTTP reverse proxy.

  • Connections, not requests. The load balancer distributes transport‑layer flows. A single HTTP/2 connection can carry many requests to the same backend. Do not assume that request counts will be evenly spread.
  • Long‑lived connections. gRPC streams, database replication links, and WebSocket connections hold open TCP sessions that can skew distribution. Monitor per‑instance connection counts and application‑level load.
  • Scaling latency. When you add a new backend, it only receives new flows. Existing flows remain pinned to their original backends. This means traffic redistribution after a scale‑out event is gradual, not instant.
  • UDP flows. For UDP, a flow is defined by the same 5‑tuple over time. Ensure your application and client libraries handle the stateless nature of UDP and any idle time‑out behaviour.

Validate your assumptions with load tests that create realistic connection patterns, not just high request rates over a few persistent connections.

10. Deployment, scaling, and rollback​

Changing backend instances without disrupting production traffic requires coordination between the load balancer’s health model and your deployment process.

  • Adopt immutable deployments. Replace instances rather than patching them in place. Build a new VM image or container, add instances to the pool, and remove old ones after the new instances are healthy.
  • Wait for readiness. Automation must not add an instance to the backend pool until it is fully initialised. That means configuration applied, secrets mounted, dependencies reachable, and the health probe passing.
  • Drain connections before shutdown. Signal the instance to fail its readiness endpoint, wait for existing connections to drain (or a graceful timeout), then terminate the instance. The load balancer will stop sending new flows once the probe fails.
  • Never remove all old instances simultaneously. Keep enough capacity active to handle traffic during the rollout. Use rolling strategies with health checks between each batch.
  • Test rollback. Verify that you can revert to a previous deployment by simply pointing the backend pool to a set of known‑good instances or a previous scale‑set image.

Define the load balancer and its dependencies in Infrastructure as Code (Bicep, Terraform, ARM). Review changes to rules, probes, and frontends with the same scrutiny as application code changes.

11. Build observability around user impact​

Operational visibility requires correlating load balancer signals with backend and application telemetry.

Key metrics to monitor (verify exact names and availability in current Azure documentation):

  • Data path availability – indicates whether the load‑balancing data plane is operational.
  • Health probe status – number of healthy versus unhealthy backends. An alert should fire when the count of healthy instances drops below a safe threshold.
  • SNAT port usage (when using outbound rules) – approaching exhaustion.
  • Backend instance metrics – CPU, memory, application‑level error rates, request latency, and connection counts.

Supplement these with NSG flow logs (to debug blocked traffic) and Network Watcher connection troubleshoot (to isolate reachability issues).

Design alerts around symptoms, not just thresholds:

  • “No healthy backends” → service unavailable.
  • “Healthy backend count below 2” → degraded resilience.
  • “Elevated connection reset count” → possible application failure or idle time‑out mismatch.
  • “Outbound connection failures rising” → possible SNAT exhaustion.

When an alert fires, the runbook should direct an engineer to check health probe logs, NSG effective rules, backend listener status, and any recent deployment or configuration changes.

12. Test failure and recovery​

A configuration that passes a simple connection test may still fail under realistic failure conditions. Regularly validate:

  • Single backend failure: Stop the application process on one instance. Confirm the health probe removes it from rotation within the expected time, and that the remaining instances handle the traffic.
  • Full backend pool failure: What happens when every instance fails its probe? Does the load balancer stop forwarding, and does your monitoring detect it?
  • Probe misconfiguration: Temporarily block the probe port with an NSG rule. Verify that alarms fire and that the runbook correctly diagnoses the cause.
  • Zone failure (simulated): If your architecture permits, test the loss of all instances in a single zone. Confirm traffic continues through the other zones.
  • Outbound dependency failure: If the backend needs an external endpoint, simulate that endpoint’s unavailability. Observe how the health probe (and the application) behaves.
  • Deployment drain and rollback: Execute a full deployment cycle and verify zero‑downtime and successful rollback.

Document the expected behaviour for each test and incorporate the results into operational runbooks.

13. Cost and governance​

While technical correctness is the primary concern, cost efficiency and governance prevent technical debt.

Cost drivers to watch:

  • Standard Load Balancer has an hourly charge plus data processed. Idle frontends, unused rules, and orphaned public IPs add cost without value.
  • NAT Gateway is billed separately; right‑size the number of public IPs and monitor data transfer.
  • Cross‑zone traffic within a region typically incurs a small data transfer cost; be aware of it if you tightly couple highly chatty services across zones.

Governance practices:

  • Use consistent naming, tagging, and resource group structures. Tags like env, service, owner, and data-classification simplify cost allocation and incident response.
  • Apply Azure Policy to prevent the creation of public IPs on backend NICs or to enforce NSG rules on subnets hosting load‑balanced workloads.
  • Regularly review backend pool membership, inbound NAT rules, and outbound rules to remove stale or overly permissive configurations.

14. Common anti‑patterns​

Anti‑patternAssumptionConsequenceDetectionCorrection
Using a load balancer for HTTP path routingLoad balancer works like a reverse proxyRouting fails silently; clients get wrong contentApplication logs show unexpected routingReplace with Application Gateway or Front Door
TCP‑open probe as sole health checkListening port means healthy applicationDeadlocked app continues receiving trafficApplication error rate rises despite healthy probeSwitch to an HTTP readiness endpoint
Deep health probe checking all dependenciesEarly detection of any dependency failureCascading eviction of all backends on transient issueAll instances go unhealthy simultaneouslyProbe only local readiness; monitor dependencies separately
Adding instance to pool before application readyApplication starts quickly enoughRequests fail until app initialisesHigh error rate immediately after deploymentWait for probe success before placing instance in pool
Relying on source IP affinity for session stateAffinity will always send user to same backendUser sessions break after client IP change or failoverLogin failures, lost shopping cartsExternalise session state (Redis, database)
Single backend behind a load balancerLoad balancer alone provides high availabilityInstance failure → outageEntire service downMinimum two backends in different zones
Exposing RDP/SSH via broad NAT ruleNo one will scan the portBrute‑force attacks, compromiseAuth failures, unusual activityUse Azure Bastion or JIT; restrict source IPs
Assuming inbound load balancer provides scalable outboundOutbound rules sufficient for all workloadsSNAT exhaustion, intermittent outbound failuresOutbound connection errors, SNAT metrics maxedUse NAT Gateway for production egress
All backends in one availability zoneZone‑redundant frontend protects usZone outage → all backends lostService downDistribute backends across zones

15. Representative production scenario: internal API tier​

A team operates a stateless REST API that is called exclusively by internal application workloads. The API runs on a Virtual Machine Scale Set and must be highly available within a single Azure region.

Design decisions:

  • Internal Standard Load Balancer with a private frontend IP (10.30.2.100).
  • Backend pool: Virtual Machine Scale Set distributed across three availability zones, minimum three instances.
  • Health probe: HTTP GET on /ready port 8080. The endpoint verifies that the local configuration is loaded and the main request loop is running. It does not check downstream databases or partner APIs.
  • Load‑balancing rule: TCP 8443 → TCP 8443 (TLS terminated on the backend VMs).
  • Outbound connectivity: NAT Gateway attached to the backend subnet, providing a static outbound IP for calling external services.
  • NSGs: backend subnet allows inbound TCP 8443 from the application VNets, and TCP 8080 from AzureLoadBalancer. All other inbound traffic denied.
  • Deployment: rolling upgrade using new scale‑set model. Each new instance passes the /ready probe before the old instance is removed. Graceful shutdown: first signal failure on /ready, wait 30 seconds, then stop.

Why this design works:

  • The internal load balancer isolates the API from the internet; all client traffic remains private.
  • Zone distribution with a zone‑redundant frontend ensures survivability of a single‑zone failure.
  • The health probe is deep enough to catch a hung process but avoids cascading failures if a downstream API becomes slow—that would be detected by application‑level circuit breakers and monitoring, not by removing the instance from the pool.
  • Outbound connectivity is decoupled from inbound load balancing via NAT Gateway; the API can call external services predictably and without SNAT exhaustion.
  • Deployment automation respects probe readiness and drain time, preventing dropped connections during rollout.

Rejected alternative: A public load balancer with NSG restrictions was considered but rejected because the API must never be reachable from the internet, even through a restricted source IP list. The internal design aligns with a zero‑trust network model.

When this design would need to change: If the API later required URL‑based routing (e.g., /v1 vs /v2 on the same frontend), TLS offloading at the edge, or a web application firewall, an Application Gateway would be placed in front of the internal load balancer.

Explore further​