Hi, Mat.
The simple answer is: songs don't have to
stay in key to sound good, in fact moving out of key is the best way to keep the listener interested. So, yes, it moves out of key once, then back in, them out again, then back in again. The fact that it does move away from E a couple of times doesn't mean it's not in E ... it returns home after a couple of detours and we're all the more satisfied for it.
You could flesh those tenths out and make a chord of each:
E > B7/F# > Gdim > E7/G#
