A high dosimetric accuracy is required for radiotherapy treatments where IMRT in combination with narrow treatment margins is applied to achieve optimally conformal dose distributions. In order to routinely verify the in vivo fluence delivery (i.e., during the actual patient treatment), our method for predicting portal dose images with a patient in the beam was validated. A unique feature of this method is that it is fully based on calibration measurements with an EPID. The portal dose image (PDI) behind a patient is dependent on the transmission of primary radiation through the patient and a contribution of scattered radiation from the patient. To derive both components, the patient geometry as observed in the planning CT scan is converted into an equivalent homogeneous phantom. A limited set of EPID measurements is required to derive the input parameters of this model. The accuracy of the in vivo PDI prediction was verified using measurements behind phantoms and four prostate cancer patients treated with IMRT. Behind homogeneous slab phantoms, the local differences between measured and predicted PDIs were within 2% inside the field, while behind a lung and a pelvic phantom, the agreement was within 3% or within 3 mm in regions with steep gradients. Outside the fields, the PDIs agreed within 2% of the global dose maximum. Evaluation of the in vivo PDI measurements behind patients showed that, on average, 87% of the pixels inside the field fulfilled the 3% local dose and 3 mm distance-to-agreement criteria. For half of the failing pixels the differences occurred due to changes in patient geometry with respect to the planning CT or due to beam attenuation by the treatment couch that was not accounted for. A fully EPID-based method for predicting portal dose images using the planning CT scan has been implemented and validated for phantoms and clinical patients.