artificial intelligence (ai) research has emerged as a powerful tool for health-related applications. with the increasing shortage of radiologists and oncologists around the world, developing an end-to-end ai-based clinical decision support (cds) system for fatal disease diagnosis and survivability prediction can have a significant impact on healthcare professionals as well as patients. such a system uses machine learning algorithms to analyze medical images and clinical data to detect cancer, estimate its survivability and aid in treatment planning. we can break the cds system down into three main components: the computer-aided diagnosis (cad), the computer-aided prognosis subsystem (cap) and the computer-aided treatment planning (catp). the lack of trustworthiness of these subsystems is still considered a challenge that needs to be addressed in order to increase their adoption and usefulness in real-world applications. in this thesis, using the breast cancer use case, we propose new methods and frameworks to address existing challenges and research gaps in different components of the system to pave the way toward its usage in clinical practice. in cancer cad systems, the first and most important step is to analyze medical images to identify potential tumors in a specific organ. in dense prediction problems like mass segmentation, preserving the input image resolution plays a crucial role in achieving good performance. however, this resolution is often reduced in current convolution neural networks (cnn) that are commonly repurposed for this task. in chapter 3, we propose a double-dilated convolution module in order to preserve spatial resolution while having a large receptive field. the proposed module is applied to the tumor segmentation task in breast cancer mammograms as a proof-of-concept. to address the pixel-level class imbalance problem in mammogram screenings, different loss functions (i.e., binary crossentropy, weighted cross-entropy, dice loss, and tversky loss) are evaluated. we address the lack of transparency in current medical image segmentation models by employing and quantitatively evaluating different explainability methods (i.e., grad-cam, occlusion sensitivity, and activation visualization) for the image segmentation task. our experimental analysis shows the effectiveness of the proposed model in increasing the similarity score and decreasing the miss-detection rate. [...]