Nothing is just a fact. If it's fair to point out that men are charged/arrested/convicted at a higher rate, it's also fair to point out that black people are charged/arrested/convicted at a higher rate. Neither of those statistics are "just a fact". They are complex phenomena that we don't fully understand the cause of and should avoid drawing unsupported conclusions from.