That can happen, and I've yet to see a fix for it. Best thing you could do is switch to windowed mode (Alt+Enter), then go do whatever you want, and when you switch back to GTA, it should load everything instantly, then just switch back to fullscreen (also Alt+Enter).