First of all, most frets below the 5th have much more wear on them than frets above the 5th. If the frets dented due to impact, the dent should be rather crisp in the sense that if you bend a string to the dent, the string wants to fall and stay in the dent. If the fret is just worn it won't fall and stay; it'll just
slide.
You also say you removed a string. Did you replace it? If not, the relieft adjustment in your trussrod is overwhelming the remaining 5 strings and that will certainly cause them to buzz.
btw, what kind of guitar is this?