I'm sure you are right; the Germanic peoples of this main island was just about sorting themselves into one nation, which I assume would have been called Saxonia or something vaguely similar, when the Normans invaded and decided they had conquered the Land of the Angles (Angleterre). Consequently there has been little development of any nationalistic fervour (in its most attractive sense).
After taking early retirement in the mid 90s (the years, not my age) I went into retail as life was boring. It took me three years to convince a manager that he should have some tastefully displayed English flags around on St George's day. He was afraid to put on display that flag for the very reason you mention. My argument was that it was long past time when the English should take back that flag and not leave it to the neo fascists to claim as theirs. I'd ban only one flag - and that's the version of the Cross of St George that has the word "England" across its centre. People who use that probably have the words "left" and "right" stencilled on their shoes!
That the bird should be protected I have no doubt...but waving a flag at it will make no difference.