You're sorta describing TPUs, especially early ones: https://cloud.google.com/blog/products/ai-machine-learning/a...

That first generation also included the ability to apply a handful of fixed activation functions, but really that's about it. The array is bigger than 32x32, also.