Why? I'm pretty sure it could do this kind of task - attention is computed between all pairs of tokens. Yes, it's a lot of compute.