Usually, model selection is based on the merits of the model as such. However, in regression settings where the measurement we wish to estimate or predict is dependent on gathering relevant covariates, there are types of applications where there are certain costs associated with getting hold of some or all of these covariate values. This may apply, for instance, to medical diagnostic, weather forecasts and financial forecasts. In these cases there can be a trade-off between using models providing highest information-value and the cost of gathering covariates to use in the estimation or prediction. This study aims at exploring principles for optimally deciding on the trade-off between information-value and costs in such settings. The study, hence, seeks to explore methods for a cost information-value trade-off in regression covariate selection. We delineate this study to generalized linear regression models (GLM), although the methods are also applicable in more general settings.