I'm far from an evolutionary biologist but AFAIK the majority of our brain is a late addition in the scheme of our evolution from sludge to human and vision is connected to one of the older parts and the new parts are up towards the front. Another case of the iterative nature of evolution, the frontal cortex where most emotion processing happens came about and enlarged a lot through the evolution of chimps that eventually branched off into humans.