Zoo - puzzling puzzle

I'm in a zoo
But not in a jungle
I rarely come in pairs
But I'm in every puzzle


Some think I'm in a xylophone
But, I most certainly am not
You don't see me with a loan
But very strangely in a zealot


I'm in the magical prison of Azkaban
Well, can't you see?
I'm in the buzz of a fan
But, my oh my what could I be?