Married sages of India?

Celibacy and living in a monastery was always strange to Hindu religion. In fact almost all the great sages of Puranas and Vedas are householders. The few exceptions that come to my mind are Narada and Shuka. Among the gods, Ganesa (in south) observes celibacy and Subrahmanya (in north ) observes celibacy and God Ayyappa observes celibacy in Kerala. Ganesa has two wives in North, Subrahmanya has two wives in south and ayyappa has two wives out side Kerala.
Most probably observing celibacy and the monastery system came in to being in India though Buddhism and Jainism . Adhi Shankara who wanted to drive away these religions was the first great sage who observed celibacy and also established monasteries in different parts of India. Later saints followed this trend. So present day people think that to be a Hindu sage , you should never marry and observe celibacy. As explained above this concept was not there in Hinduism.
But in the present day concept clearly believes that a Swamiji (sage) should never marry and observe celibacy. This goes against human nature. Because of this many well known sages who come to the lime light are forced to observe celibacy. A common man believes that if a swamiji , indulges in sex(a very important human activity) he is doing a sin. Swami Poornanda Theertha of Parlikkad Ashram was a Swamiji and decided that he would get married and married his secretary. Similarly a musician Swamiji of the Madhwa mutt decided to get n married and did. At that time this was talked about by public but these people were soon forgotten. Besides this are happenings in Ashrams where the cornered Swamijis indulge in sex with available females secretly. Once this is brought to the knowledge of people, there is a big trouble,. All people talk about these swamijis for some time. The government talks of taking action, their photos are burnt etc.
Is it correct? Should we not go back to Vedic times and have great saints who had a family of their own?