Nah, I don't see a problem with this at all, alot better than hitting someone else...
I just recently did the same with my wall right next to me... Instead of going back of stairs to start swinging on people, I just took my stress out on the wall, The worst that happened was a couple of little cuts on my hand and a small hole in the wall.