In some fields, such as HCI (and research through design) I think there could be value in moving to a system where instead you version papers and also publish the criticisms that led to its refinement.
If you've explored a problem, designed a piece of technology, and taken it back to that audience to gain further insights into the problem, what I can do in reviewing it isn't the same kind of analysis as a maths proof. Even if I you were to give me the tech or the code, I wouldn't have the same people to test it with, and would not expect to get exactly the same results. Instead, peer review tends to criticise the writing, missing elements in the argument, pieces of literature that also need considering, aspects of the methodology, and suggest more things you could do.
All of which is very valuable, and sometimes peer review is the best (only) mentoring you'll get if you're in a small university. But it's not quite the same as "we have to review this or a wrong claim might get the imprimatur of being science". Design questions don't make the same claims of completeness or context-independence. It's more of an argument based in the kind of muddy evidence you have to put up with when tackling muddy problems.
Which makes me think that instead of peer review being a hurdle to publication, perhaps we should just make the whole argument and refinement public. Give the reviewers more kudos as instead of being invisible entities whose thoughts are only received by the author and editor, their role in the development of that idea becomes a published item too.
If you've explored a problem, designed a piece of technology, and taken it back to that audience to gain further insights into the problem, what I can do in reviewing it isn't the same kind of analysis as a maths proof. Even if I you were to give me the tech or the code, I wouldn't have the same people to test it with, and would not expect to get exactly the same results. Instead, peer review tends to criticise the writing, missing elements in the argument, pieces of literature that also need considering, aspects of the methodology, and suggest more things you could do.
All of which is very valuable, and sometimes peer review is the best (only) mentoring you'll get if you're in a small university. But it's not quite the same as "we have to review this or a wrong claim might get the imprimatur of being science". Design questions don't make the same claims of completeness or context-independence. It's more of an argument based in the kind of muddy evidence you have to put up with when tackling muddy problems.
Which makes me think that instead of peer review being a hurdle to publication, perhaps we should just make the whole argument and refinement public. Give the reviewers more kudos as instead of being invisible entities whose thoughts are only received by the author and editor, their role in the development of that idea becomes a published item too.