I would guess
he continually upbraided them for what he considered their "foolishness" and what he's describing is their reaction to it. Bait and then complain when you get someone to bite. A true Atheist wouldn't care, only God Haters make such a stink. A lot of those hide behind the label of "Atheist".