People should stop looking for a silver bullet. If you are using react + relay, you must be building a very complex solution, where the overhead for such abstractions is justified.
If you are at a point where your task doesn't justify these abstractions - don't use them.
I've found that the benefits in terms of developer experience, and being able to build good UI quickly, start accumulating almost immediately, even on tiny projects.
My initial reservations about the amount of boilerplate in setting up Relay, were in large part due to the server-side half of the equation. Since switching to building my backend in a different language (now using Elixir with absinthe, rather than node and graphql-js) I've found my productivity has skyrocketed.
At this point the overhead in terms of file-size (and the still pending support for server-rendering) are the only things giving me pause for thought.
If you are at a point where your task doesn't justify these abstractions - don't use them.
Find the right tool for your job.