Vasubandhu in his Abhidharmakośa-bhāṣya states, "Defilement arises from defilement, as attachment arises from desire. Action arises from defilement, as consciousness from attachment, or saṃskāras from ignorance." Elsewhere he also states, "Because of attachment, accumulated action produces a new existence: this is bhava."
Your mind is constantly filled with defilements (kleśa). If it is not anger that motivates your strong speech, it is the lure of attractive forms that draws your gaze. Despite vowing to overcome your desires, your mental autonomy constantly wavers. Your desires lead to attachment, which in turn adds fuel to your saṃsāra. It is you and you alone who will suffer for this. The result (vipāka) of unwholesome karma (akuśala-karma) is none other than suffering (duḥkha). To engage in any lustful thought or act is to create the causes for future suffering. You are in the flames yet constantly fail to leap out.
Moreover, there is no lasting satisfaction to be found in mundane pleasures as they all entail suffering of change (vipariṇāma-duḥkhatā). What is this? It is the failure of happy moments to last. It is the failure of physical satisfaction and ease to last. No matter how much you enjoy yourself, you will always later find yourself unsatisfied and seeking the same experience again. Hence what you perceive of as pleasure is just a type of suffering. It is the immediate absence of more evident forms of suffering. By continually engaging such mental states you only amplify the addiction to them.
Even in this life if you do not feel these effects, so long as sexual desire remains, the doors to the lower realms remain open. As it is said, tomorrow morning or your next life. Death is certain to occur. It is uncertain when it will occur. Therefore be mindful of this and avoid creating causes that drag you deeper into saṃsāra.
Yet on a more immediate level the consequences of your desires should be evident. It keeps you bound to the desire realm (kāma-dhātu), from whence you cannot free yourself so long as that desire remains present and active. The form (rūpa-dhātu) and formless (ārūpya-dhātu) realms remain distant goals so long as you allow yourself to experience craving. Your aspirations as a yogi are shallow indeed if this is so.
Moreover, without experiencing the higher realms you will lack the empirical knowledge of the subtle sufferings they entail and without that point of reference your supposed compassion for all beings, including those in said states, will remain nothing more than intellectual speculation. Your talk of liberation will be mere wordplay. Your perspective will be confined to what you know in the desire realm. Your compassion amounts to crow's teeth and second heads -- such things can be conceived, but are not to be seen anywhere. You are like a frog in a well thinking how vast the sky is from such a limited perspective.
This is why the bodhisattva abandons desire. You cannot liberate others unless you yourself are liberated. You cannot have true compassion unless you truly understand the suffering all beings experience including the lowly pretas and Brahma himself. Brahma may not crave sex and food as you do, but he does crave nourishment gained through cognition (vijñāna-āhāra). This too is desire, albeit subtle and elusive.
So long as you thirst and grasp unto all aspects of bhava, you will be bound to saṃsāra. The migrations of beings through the six realms is just as applicable to you, hence the hell realms are always a possibility. Desire will ensure you have another mundane rebirth. Do not make excuses. Yes, desire for food and sensual pleasures is natural, but the natural order of things is saṃsāra. To free yourself you must go against the flow lest you will be swept away by it. You must defy the common way of existing. If you have any ordinary thoughts you must think, "If I have ordinary aspirations, may they never succeed!"
There ultimately is no happiness in saṃsāra. There is only suffering which is either apparent or unapparent. Therefore the holy and noble all teach abandoning desire; the cause for continued existence in this nightmare.