In this survey study of 7,208 Dutch healthcare workers, we investigate whether healthcare workers dealing with COVID-19 patients experience lower general health, more physical and mental exhaustion and more sleep problems than other healthcare workers. Additionally, we study whether there are differences in well-being within the group of healthcare workers working with COVID-19 patients, based on personal and work characteristics. We find healthcare workers who are in direct contact with COVID-19 patients report more sleep problems and are more physically exhausted than those who are not in direct contact with COVID-19 patients. Mental exhaustion and general health do not significantly differ between healthcare workers who are in direct contact with COVID-19 patients and those who are not. Among healthcare workers in direct contact with COVID-19 patients, lower well-being on one or more indicators is reported by those who are female, living alone, without leadership role, or without sufficient protective equipment. Regarding age, physical exhaustion is more prevalent under healthcare workers older than 55 years, whereas mental exhaustion is more prevalent under healthcare workers younger than 36 years. These results stress the need of mental and physical support of healthcare workers during a pandemic, catered to the needs of healthcare workers themselves.