Rebalance shards across servers in production: 265k objects in 12s

TL;DR You can now rebalance shards across servers with the solver Meta runs 40 million times a day: Rebalancer, open-sourced in September 2026 under Apache 2.0. Its local-search engine explores a neighborhood of roughly 268,200 candidate moves instead of the 848 million binary variables the equivalent Mixed Integer Program would need, which is how a 265,000-object, 3,200-container placement lands in 12 seconds at p99. Model your fleet as objects, containers, dimensions and specs, then pick the solver that matches your scale.

What is Rebalancer, and which problem does it actually solve?

Rebalancer is a domain-specific language for assignment problems: putting N objects into M containers while satisfying hard constraints and optimizing something you care about. Meta open-sourced it on September 21, 2026 under Apache 2.0, with a Python package on PyPI.

That description sounds abstract until you notice how many infrastructure problems have exactly that shape. Shards to database servers. Servers to services. Edge traffic to backend datacenters. Serverless functions grouped for locality. Online ML training workloads spread across a pool. Meta reports running 40 million of these problems a day across more than 30 distinct formulations — an average of about 463 solves per second, every second.

The reason this matters more than another optimization library: most teams write this logic by hand. A greedy loop that picks the emptiest host, plus a tie-breaker, plus a special case for the rack-diversity rule someone added in 2023. It works until the constraint list grows, and then nobody can say whether the placement is good or just the first thing the loop found.

Why hand-rolled placement stops working

The search space is the part people underestimate. Take Rebalancer’s own published benchmark size — 265,000 objects across 3,200 containers. The number of distinct assignments is 3,200 raised to the power of 265,000. Written out, that number is nearly 929,000 digits long.

No amount of cleverness enumerates that. So every real system uses a heuristic, and the question is only whether your heuristic is written down as a model or buried in a loop. A model you can audit, constrain, and re-solve. A loop you can only rewrite.

How to rebalance shards across servers with Rebalancer

The library separates specification from storage from solving from debugging, and the modeling flow follows that same order. Here is the shape of a minimal run:

1. Install it. pip install rebalancer pulls the Python package; Debian packages and a Homebrew formula are published on the repo as well.
2. Name your two sides. set_object_name("shard") and set_container_name("host") — objects get placed, containers receive them.
3. Seed the current assignment. set_assignment({"host0": ["shard0", "shard1"], "host1": ["shard2"]}). This is what makes it a rebalance rather than a cold placement, and it is the hook you need for churn control later.
4. Declare dimensions. add_object_dimension("memory", {...}) says what each shard consumes; add_container_dimension("memory", {}, default_value=20.0) says what each host offers.
5. Add constraints as specs. A CapacitySpec scoped to host on the memory dimension is the hard rule that no host is overcommitted.
6. Add objectives. A balance spec spreads a dimension evenly across containers once the hard rules are satisfied.
7. Choose a solver. A LocalSearchSolverSpec takes a moveTypeList — SingleMoveTypeSpec moves one object, SwapMoveTypeSpec exchanges two — and those move types are literally the neighborhood the search explores.

from rebalancer import ProblemSolver
from rebalancer.specs import (
    CapacitySpec, ConstraintSpec, LocalSearchSolverSpec,
    MoveTypeSpec, SingleMoveTypeSpec, SwapMoveTypeSpec, SolverSpec,
)

solver = ProblemSolver(service_name="rebalancer", service_scope="example")
(solver
    .set_object_name("task")
    .set_container_name("host")
    .set_assignment({"host0": ["task0", "task1", "task2"], "host1": ["task3"]})
    .add_object_dimension("memory", {"task0": 10, "task1": 10, "task2": 10, "task3": 10})
    .add_container_dimension("memory", {}, default_value=20.0)
    .add_constraint(ConstraintSpec(capacitySpec=CapacitySpec(
        name="memory_capacity", scope="host", dimension="memory")))
    .add_solver(SolverSpec(localSearchSolverSpec=LocalSearchSolverSpec(
        moveTypeList=[MoveTypeSpec(singleMoveTypeSpec=SingleMoveTypeSpec()),
                      MoveTypeSpec(swapMoveTypeSpec=SwapMoveTypeSpec())])))
)
solution = solver.solve()

Underneath, constraints and objectives compile into an expression graph with recursive operations — sum, max, square, absolute value — over utilization leaves. That graph is what gets re-evaluated on every candidate move, which is why the choice of solver is a scaling decision rather than a style preference.

Local search vs a MIP solver: which one fits your problem size?

Rebalancer ships two engines, and they fail in opposite directions. The optimal solver translates your expression graph into a Mixed Integer Program for HiGHS, Gurobi, or FICO Xpress, and gives you a provably optimal answer — if it finishes. Model size is worst-case proportional to objects times containers.

Run that against the 265,000-object benchmark and the MIP needs on the order of 848 million binary variables. Local search, whose neighborhood is proportional to objects plus containers, considers about 268,200 moves per step — roughly 3,162 times smaller. At a million objects and 5,000 containers, the MIP formulation crosses 5 billion variables while local search still returns in an average of 171 seconds.

The honest read: use the optimal solver as a correctness oracle on a scaled-down instance, then run local search in production. Meta’s own largest problems are explicitly described as too big for any MIP solver, which is a useful admission from the team that built both paths.

Those 3,400 large runs a day are worth converting into a capacity number before you adopt this. At 171 seconds each, that is 581,400 solver-seconds per day — about 6.7 machine-days of compute consumed every single day. Rebalancing at fleet scale is a background service with its own fleet, not a cron job on the control plane.

What breaks if you rebalance greedily instead?

Three failure modes, in the order teams hit them.

Constraint creep. The first version has one rule. The fifth has capacity, rack diversity, a version-affinity rule, and a customer-isolation rule, and they interact. A greedy loop cannot tell you a combination is infeasible; it just returns something and lets you find out in production.

Churn. A rebalancer that does not know the current assignment returns the layout it likes best, which can move nearly everything. Seeding set_assignment with today’s placement and letting the objective pay for movement is what separates a rebalance from a reshuffle. Getting this wrong turns an optimizer into a self-inflicted traffic event that looks a lot like a yo-yo DDoS — repeated, self-triggered load.

No way to debug the answer. When the solver returns a placement you did not expect, you need to see which constraint bound and which objective dominated. Rebalancer ships an Explorer web UI in a container for exactly that, and the team notes that once modeling gets easy, most of the engineering effort shifts to debugging.

When should you not reach for a solver?

If your placement problem has one constraint and fewer than a few hundred objects, a sorted loop is the right answer and adding a solver is resume-driven architecture. The threshold is not object count, it is constraint count: the moment two rules can conflict, you want a model that can report infeasibility instead of a loop that silently picks a winner.

The second case to skip it is when placement is not actually your bottleneck. If you are moving data between nodes, the PCIe and NIC path usually dominates any gains from a smarter layout. And if you are running a small cluster, the $166-a-year answer frequently beats the clever one.

For related reading, the same infrastructure discipline shows up in the cloud integrations you have to build yourself on bare-metal Kubernetes and in planning a migration with no downtime window. If the workloads you are placing are model training or inference jobs, the LLM engineering topic hub collects the adjacent capacity and latency work, including why multi-agent text handoffs cost 2.5x in latency.

FAQ

Do I need a commercial solver license to use Rebalancer? No. The optimal solver path compiles your model into a Mixed Integer Program that can run on HiGHS, which is open source, alongside the commercial Gurobi and FICO Xpress backends. The local-search solver needs no external solver at all, and that is the path Meta uses for its largest problems anyway.

How large a fleet can the local-search solver actually handle? Meta reports averaging 171 seconds on problems with more than a million objects across 5,000 containers, and runs over 3,400 of those per day. The reason it scales is that each search step evaluates a neighborhood of about objects-plus-containers candidate moves, not the objects-times-containers decision matrix a MIP has to build. Practically, the limit you hit first is usually how long you are willing to wait, not the model size.

Is this only useful for sharded databases? No — sharding is just the most recognizable instance. Meta runs the same library for assigning servers to services, routing edge traffic to datacenters, grouping serverless functions for locality, and balancing online ML training workloads. Anything shaped as put these N things into these M places without violating constraints is the same problem.

How do I stop a rebalance from moving everything at once? Model churn as a cost rather than trying to bolt on a limit afterwards. You seed the solver with the current assignment through set_assignment, then express movement as something the objective penalizes, so a move has to buy more balance than it costs in disruption. A solver that does not know the current placement will happily return a mathematically better layout that is operationally a full reshuffle.

What is the difference between a constraint and an objective here? A constraint is a hard rule the solution must satisfy, like a container never exceeding its memory dimension. An objective is what gets optimized once the hard rules hold, such as spreading a dimension evenly across containers. Getting this split wrong is the most common modeling mistake: encoding a preference as a constraint is how you end up with an infeasible problem and no solution at all.

Sources

• Meta Engineering: Open-sourcing Rebalancer — the announcement, production scale figures, solve-time benchmarks, and the list of internal systems built on it.
• facebook/rebalancer on GitHub — Apache 2.0 source, install instructions, and the Python example this post’s code is based on.
• Rebalancer documentation — the spec catalog, expression API reference, and the Explorer debugging tool.