I feel that sometimes as well. But on the other hand, if it's a sports car I also tell myself it's what they are meant to do. Not all the time, but a spirited drive once in a while makes me think they feel alive and fulfilled. Although I suppose there will be more wear on the tyres, brakes, engine, etc.
I felt a bit guilty by taking my 7R to a track after a week.