Patient-derived tumor organoids have been established as promising tools for in vitro modelling of multiple tumors, including cholangiocarcinoma (CCA). However, organoids are commonly cultured in basement membrane extract (BME) which does not recapitulate the intricacies of the extracellular matrix (ECM). We combined CCA organoids (CCAOs) with native tumor and liver scaffolds, obtained by decellularization, to effectuate a model to study the interaction between epithelial tumor cells and their surrounding ECM. Decellularization resulted in removal of cells while preserving ECM structure and retaining important characteristics of the tissue origin, including stiffness and presence of desmoplasia. The transcriptome of CCAOs in a tumor scaffold much more resembled that of patient-paired CCA tissue in vivo compared to CCAOs cultured in BME or liver scaffolds. This was accompanied by an increase in chemoresistance to clinically-relevant chemotherapeutics. CCAOs in decellularized scaffolds revealed environment-dependent proliferation dynamics, driven by the occurrence of epithelial-mesenchymal transition. Furthermore, CCAOs initiated an environment-specific desmoplastic reaction by increasing production of multiple collagen types. In conclusion, convergence of organoid-based models with native ECM scaffolds will lead to better understanding of the in vivo tumor environment. STATEMENT OF SIGNIFICANCE: The extracellular matrix (ECM) influences various facets of tumor behavior. Understanding the exact role of the ECM in controlling tumor cell fate is pertinent to understand tumor progression and develop novel therapeutics. This is particularly the case for cholangiocarcinoma (CCA), whereby the ECM displays a distinct tumor environment, characterized by desmoplasia. However, current models to study the interaction between epithelial tumor cells and the environment are lacking. We have developed a fully patient-derived model encompassing CCA organoids (CCAOs) and human decellularized tumor and tumor-free liver ECM. The tumor ECM induced recapitulation of various aspects of CCA, including migration dynamics, transcriptome and proteome profiles, and chemoresistance. Lastly, we uncover that epithelial tumor cells contribute to matrix deposition, and that this phenomenon is dependent on the level of desmoplasia already present.