The better we know someone—the more we know about them or know them as a person—the better we are at telling what they are thinking and feeling. Yet in responding to the epistemological problem of other minds—how we know others’ mental states—perceptualists and inferentialists fail to develop how such an everyday intuition may fit in to their accounts. This paper offers a potential explanation of how knowledge about and of another may privilege our knowledge of their mental states. It argues that while both perceptualist and inferentialist accounts can accommodate certain accumulated propositional knowledge about another, the inferentialist account does so with a broader range of cases. It then shows that interpersonal knowledge—knowing another as a subject—can be taken as a non-distinctive kind of knowledge, a distinctive way of acquiring non-distinctive knowledge, or a distinctive kind of knowledge. While perceptual and inferential accounts may be aided by the first and second, they have severe limitations for the third way of understanding interpersonal knowledge. If this is right, perceptual and inferential accounts fail to exhaust the epistemic relations towards other minds, since interpersonal knowledge involves a distinctive kind of practical relation to another.