Well I think it can get annoying for the person you're stalking but it should be settled in RP, just as a hostage situation, police chase or any other kind of RP can get ''annoying'' to some people.
Very thin line between annoying and being annoyed, let's not forget that.