Take anything you like. A cause, religious group, political party, sports fanbase, etc.

Some subset of each group are generally decent people, who do their best to be good to the people around them, and to live their lives in a generally moral and ethical manner.

Some subset of each group have no interest in morals or ethics, and attach themselves to the group for purely selfish reasons.

Judging the entire group based on the selfish or amoral subset is a logical fallacy of the most basic sort, and is bordering religiosity. This is even more problematic when you look at the kinds of negative EA situations that have (rightly) caused controversy. The high profile big money cases get attention.

If confronted with someone who ascribes to the EA philosophy and by all measurable indicators has done incredibly good things for the world, would you be willing to change your mind?