Because having a button there, even if it isn't active, reduces the number of people who will walk when they shouldn't, because they think that the light will change sooner because they pushed the button.
No, it increases the number of jaywalkers. If you know that the light is on a timer and you will have your turn in 15 seconds, you have little reason not to wait the 15 seconds. But if you have to press a button for something, you deserve quick response to the button push, and if it doesn't come quickly, you give up in disgust and cross against the light.
Even when those buttons DO work they aren't instantaneous. You presume a world where pedestrians have almost no patience. A more likely scenario is that people start pushing the button more and more trying to get it to work before they "give up in disgust".
