Evaluating Software Documentation Quality

• Author / Creator

  Tang, Henry K

• Software libraries provide reusable code that allow developers to include needed functionality without committing time and effort to develop the functionality themselves. To benefit from the code reuse, developers first compare multiple libraries that offer the needed functionality and spend time learning how to use the library. This is typically done through a combination of trying the library, as well as reading the library's documentation. However, ``bad quality'' documentation is not only ineffective in aiding developers learn a library, but may even be detrimental if there is incorrect information. Thus, it would be useful if developers already had an idea about the documentation quality of a library before they invest time in exploring it.

  In this thesis, we investigate a metric-based evaluation of software documentation quality. We start by exploring the literature to confirm the importance of documentation when learning a software library, before finding descriptions of documentation aspects, as well as previous attempts to evaluate documentation quality. We summarize the different quality aspects of documentation, then narrow our focus to six different documentation aspects: Code Examples, Completeness, Ease of Use, Readability, and Up-to-date. From these documentation quality aspects, we utilize the Goal-Question-Metric paradigm to determine the metrics that can be used to measure them. We validate these initial metrics through an interview study of three industry professionals, where we integrate the feedback back into our metrics. After incorporating the interview feedback, we create a summary tool presenting each of our metrics for a given library and conduct a survey with 25 developers on the usefulness of our tool and metrics. Participants found our documentation quality summary useful with a median score of four on a five-point Likert scale rating system. Participants especially found the metrics relating to text readability and code example readability as useful documentation quality metrics. We also discuss opportunities and future applications of our work.

• Subjects / Keywords

  • Software documentation--Evaluation

• Graduation date

  Spring 2023

• Type of Item

  Thesis

• Degree

  Master of Science

• DOI

  https://doi.org/10.7939/r3-3x36-zp10

• License

  This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.