Sounds like lactose issue. A lot of the lactose is processed out by the time it becomes yogurt or cheese (depending on the cheese). I am similar - milk doesn’t sit right but many cheeses are fine. I am not fully lactose intolerant, but there is a spectrum