> often an order of magnitude out of whack for companies that prefer a horizontal-first scaling approach
This is practically worth its own thread, so I will.
I think it’s the principle of the excluded middle biting us on the ass again. Vertical first risks having people assume shared state across events. Everything collects in memory, because look at all of the “unused” memory that nobody will complain about me using for my dumb feature because nobody is looking yet (when the moments are still teachable).
To your hardware point, many people don’t understand how that “free” memory is used by the OS to reduce mean response time. So they overload the machine and don’t bother trying to do anything until they hit 80%, when the penalties have started racking up at 60%.
So we shall force the issue by giving them tons of tiny machines, as if going to the other extreme ever fixed a goddamned thing.
You don’t want tall. You don’t want wide. You want square.
I saved us almost 10% on hardware by swapping in larger VMs and fewer of them. We don’t have a homogenous workload, but a good fraction is, so larger machines can share a bit of OS cache and statistical clusters of slow events can be load balanced better, improving p95. But at some point the performance improvements we’ve made will take us in the other direction, because then we’ll have too high a fraction of traffic on a single VM. That’s not a problem by itself, but could be half of a production incident.
This is practically worth its own thread, so I will.
I think it’s the principle of the excluded middle biting us on the ass again. Vertical first risks having people assume shared state across events. Everything collects in memory, because look at all of the “unused” memory that nobody will complain about me using for my dumb feature because nobody is looking yet (when the moments are still teachable).
To your hardware point, many people don’t understand how that “free” memory is used by the OS to reduce mean response time. So they overload the machine and don’t bother trying to do anything until they hit 80%, when the penalties have started racking up at 60%.
So we shall force the issue by giving them tons of tiny machines, as if going to the other extreme ever fixed a goddamned thing.
You don’t want tall. You don’t want wide. You want square.
I saved us almost 10% on hardware by swapping in larger VMs and fewer of them. We don’t have a homogenous workload, but a good fraction is, so larger machines can share a bit of OS cache and statistical clusters of slow events can be load balanced better, improving p95. But at some point the performance improvements we’ve made will take us in the other direction, because then we’ll have too high a fraction of traffic on a single VM. That’s not a problem by itself, but could be half of a production incident.