Psychiatric patients are often victims of crime and discrimination and are often socially withdrawn. This has negative consequences for their health and recovery. We examined whether such discrimination mediates the association between victimization and social withdrawal, and whether these associations differ between men and women. We also determined the prevalence of social withdrawal and the discrimination experienced by patients suffering from a severe mental illness. This study is embedded in the Victimization in Psychiatric Patients study. Information on discrimination, social withdrawal and victimization was obtained using structured self-report questionnaires (N = 949). We reported the 12-month prevalence of these phenomena and used path analysis to estimate the direct path between personal and property victimization and social withdrawal, and the indirect path through the discrimination experienced. The impact of gender was assessed by testing interaction terms. Social withdrawal was reported by 20.6% (95%CI 18.1–23.2) of participants, and being discriminated against in the past 12 months by 75.3% (95%CI: 72.6–78.0%). While crime victimization had no direct effects on social withdrawal, personal crime victimization (B = 0.47; 95%CI 0.25-0.72; p < 0.001) and property crime victimization (B = 0.65; 95%CI 0.42-0.93; p < 0.001) had significant indirect effects on social withdrawal, which were mediated by the discrimination experienced. In men we found a direct negative effect of property crime on social withdrawal (B = -0.68; 95%CI: −1.21to −0.11, p = 0.014). We conclude that personal and property victimization, for both men and women, was associated with higher levels of social withdrawal, and this was fully mediated by the discrimination experienced.