Most "modern" GUI terminals make this artificially fast by not rendering every flushed buffer; instead, they simulate what ought to be on screen, and then render at 60Hz or whatever your refresh rate is.
The point being that it's not measuring latency, nor throughput of the rendering, but rather throughput of the emulation.
You're describing exactly what makes them feel slow. All of that buffering is adding latency. Everything is buffering: the screen, the GPU, the OS, the window manager, the shell.
People notice the extra latency though. I sure do. I remember what it's like to have a CRT getting photons displayed nearly immediately after a keystroke. That's exactly what makes an old 286 feel snappier than a 2017 macbook pro while typing.
I remember how slow it was accessing text mode video RAM on the ISA bus compared to constructing a display in main RAM, and indeed buffering changes there and rendering them across en bloc to video RAM. Buffering is not a recent invention.
The point being that it's not measuring latency, nor throughput of the rendering, but rather throughput of the emulation.