The knowledge that wardens have files on their assigned inmates is a commonly known fact. The fact that he seems to think you don't know about them isn't exactly reassuring in terms of his intelligence.