Yeah, I was thinking the same thing. If you can tell which group a particular user belongs to then you can train two models and optimize them independently. Then you just select the most appropriate model for the user.