To give you some insight, five is an average rating, and is handed out to unedited, plain avatars. They need to be crisp, clean images without any noticeable defects, otherwise they get a four. Your avatar missed that rating for minor issues. As I mentioned before, my rating policies have little to no barring on how much I actually like the avatar.
I'm not sure if a border would be good. I don't think white would look very nice, and I don't think black shows up very well on these forums, due to the darker color scheme. Most other colors tend to be distracting and gaudy looking, in my opinion.