The Brexit referendum has divided the country and created highly salient identities that cut across traditional partisan and other identities. The Brexit identities have led to strong affective polarization over issues that go far beyond questions of EU integration. However, we do not know much about the individual-level factors that combat or contribute to mass polarization. Political theorists and the media regularly float the idea that more empathy could foster understanding between the two sides in the Brexit debate. Hence, in this article, I use a large survey of UK citizens (N=7691) and two original survey experiments (N=7000 and N=1600) to test the proposition that citizens with higher empathic ability are less polarized over Brexit. Empathic ability is a personality trait that is central to group-based behavior, which is why I argue that empathy is a key personality trait that contributes to intra-group favoritism, instead of reducing inter-group conflict. In other words, high empathic ability leads to stronger intra-group sympathy, which in turn intensifies inter-group hostility between supporters of the Leave and Remain camps.