The One Versus Rest (OVR) method of building classifiers has fallen out of favor with machine learning researchers in recent years. When used, it is mostly only with older binary linear classifiers such as Support Vector Machines. We suspected that one could also improve the classification performance of more complex neural network architectures by setting them up as individual binary classifiers for each class and combining the output. In our research, we have tested the OVR style of building a classifier on several modern neural network architectures, including DenseNet, Inception v3, Inception ResNet v2, Xception, NASNet, and MobileNet. We have compared several aspects of their performance during training and testing, in both an OVR style and conventional multiclass single-network style. We have compared hardware resource use, classification speed, and several classification accuracy metrics. Also, we trained and tested a total of 186 networks; 50 of these were multiclass, and 136 were individual binary networks. Using our final selection of 99 networks (11 multiclass and 88 binary), we compared the results of the 11 multiclass networks with the 11 OVR style networks built from the 88 individual binary classifiers using the Kvasir v2 dataset, which contains thousands of classified frames from colonoscopy videos. We chose The Kvasir dataset as it could provide a good indication of how applicable our method is to clinical research. We found that overall, there was a substantial increase in the average and median classification metrics when using our methods and applications. Using the OVR style resulted in a 7% increase in average F1 score, a ~1% increase in average accuracy, a 6% increase in Matthews Correlation Coefficient, a 1% increase in precision and finally a 4% increase in average recall. Specificity remained mostly unchanged. Also, the median values for all of these metrics increased significantly in the OVR style, with the median F1, MCC and recall scores increasing by over ~15%. The most improved network in the OVR configuration saw an increase in F1 of 45% and an MCC increase of 40%. However, on average our OVR multi-network style was 7.6 times slower to classify than a single network multiclass implementation. These collective findings lead us to conclude that the OVR method can be applied to modern neural networks structures, and will often result in increased classification accuracy, but at the cost of classification speed.