Abstract
The Unified Modeling Language (UML) is becoming the standard notation for expressing object-oriented analysis and design models. However, to date, there are only very few empirical studies that have attempted to evaluate UML, that is, to evaluate the potential costs, benefits and appropriate use of UML. For example, a common claim is that the UML documentation will improve the ease of understanding how to perform changes to the software it represents. This might be very important as the cost of maintaining software is very high. However, there are only a few studies attempting to evaluate the actual benefits of UML during program comprehension. Most of these existing studies have been based on subjective assessments of the value of UML as opposed to more objective indicators of the benefits such as development time savings or improved quality during actual software development in a controlled setting. Furthermore, using UML may incur additional costs, because the UML documents need to be developed and maintained. Thus, the tradeoffs between the costs and benefits of using UML need to be investigated.
This thesis describes a controlled experiment to evaluate the effect of UML documentation on the maintainability of object-oriented software. In the experiment, 20 students designed and coded several Java maintenance tasks. About half of them received UML documentation, whereas the other half did not. The subjects who received UML documentation also had to update the documentation using a tool, Tau UML.
The primary goal of the experiment was to evaluate to which extent the access to UML class and sequence diagrams improves the ease of understanding and changing object-oriented software. To assess the costs and benefits of using UML in object-oriented development projects, the experiment also attempted to assess the additional costs associated with updating the UML documentation. The dependent variables of the study were effort (in minutes spent to solve the tasks) and correctness. The effort data was reported by each subject using a task questionnaire. The correctness of the task solutions was assessed by the researchers. The subjects spent between five and eight hours solving tasks. Five observers were present during the experiment.
The results of the experiment reported in this thesis indicate that having UML documentation may increase the chance of producing correct solutions by as much as 50 percent for the most complex tasks, that is, for the tasks that require an in-depth understanding of the system to perform the changes to the code. For simpler maintenance tasks, the effect of having access to UML on program correctness is much smaller but still positive. The subjects who received UML documentation also spent less time on coding. However, because they also had to update the UML documents, the total effort to perform the changes was about 20 percent higher than for the subjects who did not receive UML documents.
In summary, the results of this thesis suggest that UML may have a positive effect on the maintainability of object-oriented software. The suggested 20 percent increase in change effort seem to be an acceptable amount of overhead as the benefits (savings) associated with less faulty software probably more than outweigh the additional costs incurred by updating the UML documentation. However, further research through replications is required to assess the extent, to which the results can be generalized outside the experimental conditions, e.g., to professional developers solving real maintenance tasks: In this experiment, the subjects were students. Furthermore, the tasks and systems were small and perhaps not representative of “typical” maintenance tasks.