It works with select because it takes a pointer to the timespec object, which is read kernel side [1]. I misremembered poll doing the same thing but it takes a plain integer parameter.
[1] not guaranteed of course, but it is historical Unix behaviour
[1] not guaranteed of course, but it is historical Unix behaviour
Edit: btw, i do agree with your final sentence!