Variable selection is an important topic in linear regression analysis and attracts considerable research in this era of big data. It is fundamental to high-dimensional statistical modeling, including nonparametric regression. Some classic techniques include stepwise deletion and subset selection. These procedures, however, ignore stochastic errors inherited in the stages of variable selections, and the resulting subset suffers from lack of stability and low prediction accuracy. Penalized least squares provide new approaches to the variable selection problems with high-dimensional data. The least absolute shrinkage and selection operator (LASSO), which imposes an L1-penalty on the regression coefficients, and the Elastic Net which combines an L1 and an L2 penalties are popular members of the penalized regressions. In this dissertation, we develop penalized linear regression with a universal penalty function, which includes the widely used ridge and Lasso penalty functions as special cases. A Monte Carlo simulation approach is developed to illustrate that the Elastic Net is also a special case of our model. The structure and properties of the universal penalty are studied, and the corresponding algorithm to solve the regression coefficients is developed. Furthermore, we apply our model to a real U.S. economic and financial data example. Simulation studies and real-data support the advantageous performance of the proposed method.