Social anxiety disorder is an invalidating psychiatric disorder characterized by extreme fear and avoidance of one or more social situations in which patients might experience scrutiny by others. The goal of this two-generation family study was to delineate behavioral and electrocortical endophenotypes of social anxiety disorder related to social evaluation. Nine families of patients with social anxiety disorder (their spouse and children, and siblings of these patients with spouse and children) performed a social judgment paradigm in which they believed to be evaluated by peers. For each peer, participants indicated their expectation about the evaluative outcome, after which they received social acceptance or rejection feedback. Task behavior, as well as the feedback-related EEG brain potentials (N1, FRN, P3) and theta power were tested as candidate endophenotypes based on two criteria: co-segregation with social anxiety disorder within families and heritability. Results indicated that reaction time for indicating acceptance-expectations might be a candidate behavioral endophenotype of social anxiety disorder, possibly reflecting increased uncertainty or self-focused attention and vigilance during the social judgment paradigm. N1 in response to expected rejection feedback and P3 in response to acceptance feedback might be candidate electrocortical endophenotypes of social anxiety disorder, although the heritability analyses did not remain significant after correcting for multiple tests. Increased N1 possibly reflects hypervigilance to socially threatening stimuli, and increased P3 might reflect that positive feedback is more important for, and/or less expected by, participants with social anxiety disorder. Finally, increased feedback-related negativity and theta power in response to unexpected rejection feedback compared to the other conditions co-segregated with social anxiety disorder, but these EEG measures were not heritable. The candidate endophenotypes might play a new and promising role in future research on genetic mechanisms, early detection and/or prevention of social anxiety disorder.