No, rabbits definitely shouldn't be housed with cavies - there are MANY reasons.
Food - Cavies need large amounts of Vit. C in their diets, which their pellet food is full of, rabbit pellet food won't contain this, and gp food won't contain all the nutrients a rabbit needs. It would be impossible to ensure they got the correct amount of their own food.
Have you ever saw two bonded rabbits together? Constantly grooming each other and snuggling, they are very close, unlike piggies. Yes, they are a social animal but they are not affectionate to each other like rabbits. You said it yourself, your bun grooms your piggies, it is very frustrating for him to not be groomed back, it's what he expects as that's what a bunny would do. It's not fair.
A rabbits back legs are so so powerful, if a rabbit is spooked they could react by jumping, if they are next to a cavy that is one nasty smack, and could result in death. The rabbits don't mean to hurt them, they just do, even an ND bun has a very powerful kick.
There is bacteria in a rabbits gut that can cause infection in a piggie - that happened to someone I know, and the little piggie died.
I'd have liked to give you more detail but I haven't time, I'm sure others will chime in here too.
