Does he love me or is he just scared?

[IonnaStar]

I've been having issues with my pony about leaving him in the paddock(he tries to jump the fence), in his stable(I'm scared he's going to injure himself) and tethered(He always breaks free). It doesn't matter how small or large - once I went to say hi to my friend in the stable opposite and he panicked. I don't know why he doesn't want me to leave. Does he just love me and not want me to go, or his he scared of something?