There is a difference between sexual act done only for pleasure, under the stress of physical attraction, and the same act done as a natural expression of love awakening between two human beings. The former ties us to groves of animal nature and if persisted slowly takes away the possibility of deep and stable emotional relationships. It is like getting hooked and addicted to anything. One slowly loses all sense of reason and discernment and is driven to seek physical pleasure in the easiest way possible.
The other problem with the sexual act is that there is a deep interchange of energies in the intimate act. There is a mutual exchange and transfer of material from one to the other which often is not of a very desirable nature. It confuses the inner being turning the inner rooms of our mind and heart and life into a platform. In ordinary life where anyways there is enough confusion it may hardly matter as it is then like mixing two dirty streams of energy. But if one is slightly developed inwardly then one may bring in much dirt inside which would take days and weeks to clean up.
And if one is an aspirant for a higher life, the spiritual life, then these kind of sexual interchanges may become dangerous by opening the doors to hostile energies, sickness, weaknesses through the intimate contact with random people, especially human beings of a low inner consciousness who often are endowed with an outer charm that attracts. Besides, the sexual act as such, whether casual or sanctioned by marriage does lead to the fall of consciousness if it was on the path of ascension. That is why steady partner who is walking the same path and carries the same intensity of aspiration is always better if the urge does need an outer expression. Though the fall is still there, an equilibrium is easily reached and there are useful complimentary elements that pass into each other.
Of course, finally one has to get rid of the sexual stamp of our animality altogether as otherwise it holds back the inner spiritual ascension for long and dissipates whatever is received and is developing within through the yoga.
Affectionately,
Alok Da