Generally the processing time of a program can be improved by splitting the program into several portions and executing each portion on its own computing core. This process allows us to take advantage of as much of the computer system hardware as we can. However, this also can cause some complexities to arise such as dependency issues. Sparse matrices are, in the most general form, matrices with a large number of zero entries relative to their number of nonzero entries. These matrices are commonly occurring in applications like in a Partial Differential Equation (PDE). Naturally, systems of linear equations involving these sparse matrices are common and there are several methods for solving these systems. As most of these matrices are relatively large, it can be inefficient to solve them directly. Instead many ways to solve these systems involve the use of iterative methods. Combining parallel processing with these iterative methods, we can quickly and efficiently solve large sparse systems. The following thesis contains the use of two types of parallel processing, provides applications of both, shows the value of using iterative methods with sparse matrices, and examines solving partial differential equations using several iterative methods with parallel processing.