Inefficient resource allocation and planning, expensive treatment costs, and low patient adherence to medication plans lead to undesired health outcomes in patients with chronic diseases. Operation research and stochastic decision process models have provided significant opportunities to assist physicians and healthcare providers with these complexities. Using advanced stochastic decision-making processes, this dissertation contributes to the field of medical decision making in patients with chronic diseases. We use Markov decision process and partially Markov decision process models to address our research questions. The first contribution investigates the impact of patient adherence on health outcomes and medication plans in patients with Type 2 diabetes. The second contribution, that is an extension of the first contribution, investigates the financial effects of nonadherence to medication plans in patients with Type 2 diabetes. The experimental results of these two studies reveal the importance of higher adherence to the medication in achieving desired health outcomes and expenses. Finally, the third contribution focuses on patients with a risk of Alzheimer’s disease and aims to provide observation-based screening plans that consider patient risk factors.