Impact of usability mechanismsAn experiment on efficiency, effectiveness and user satisfaction

  1. Juan M. Ferreira 1
  2. Silvia T. Acuña 2
  3. Óscar Dieste 3
  4. Sira Vegas 3
  5. Adrián Santos
  6. Francy Rodríguez 2
  7. Natalia Juristo Juzgado 3
  1. 1 Universidad Nacional de Asunción
    info

    Universidad Nacional de Asunción

    San Lorenzo, Paraguay

    ROR https://ror.org/03f27y887

  2. 2 Universidad Autónoma de Madrid
    info

    Universidad Autónoma de Madrid

    Madrid, España

    ROR https://ror.org/01cby8j38

  3. 3 Universidad Politécnica de Madrid
    info

    Universidad Politécnica de Madrid

    Madrid, España

    ROR https://ror.org/03n6nwv02

Book:
Actas de las XXV Jornadas de Ingeniería del Software y Bases de Datos (JISBD 2021): [Málaga, 22 al 24 de septiembre de 2021]
  1. Rafael Capilla (coord.)
  2. Maider Azanza (coord.)
  3. Miguel Rodríguez Luaces (coord.)
  4. María del Mar Roldán García (coord.)
  5. Loli Burgueño (coord.)
  6. José Raúl Romero (coord.)
  7. José Antonio Parejo Maestre (coord.)
  8. José Francisco Chicano García (coord.)
  9. Marcela Genero (coord.)
  10. Oscar Díaz (coord.)
  11. José González Enríquez (coord.)
  12. Mª Carmen Penadés Gramaje (coord.)
  13. Silvia Abrahão (col.)

Publisher: Sociedad de Ingeniería de Software y Tecnologías de Desarrollo de Software (SISTEDES)

Year of publication: 2021

Congress: Jornadas de Ingeniería del Software y Bases de Datos (JISBD) (25. 2021. Malaga)

Type: Conference paper

Abstract

Context: As a software quality characteristic, usability includes the attributes of efficiency, effectiveness and user satisfaction. There are several recommendations in the literature on how to build usable software systems, but there are not very many empirical studies that provide evidence about their impact. Objective: We report an experiment carried out with users to understand the effect of three usability mechanisms +IBQ-Abort Operation, Progress Feedback and Preferences+IBQ on efficiency, effectiveness and user satisfaction. Usability mechanisms are functionalities that should, according to the HCI community, be implemented within a software system to increase its usability. Method: The experiment was conducted with 168 users divided into 24 experimental groups. Each group performs three online shopping tasks. We measure efficiency variables (number of clicks and time taken), effectiveness (percentage of task completion) and user satisfaction gathered from a questionnaire. Results: The adoption of Abort Operation has a signi++wE-cantly positive effect on efficiency (time taken), effectiveness and user satisfaction. The adoption of Progress Feedback does not appear to have any impact on any of the variables. The adoption of Preferences has a signi++wE-cantly positive effect on effectiveness and user satisfaction but no in++wI-uence on efficiency. Conclusions: We provide relevant evidence of the impact of the three usability mechanisms on efficiency, effectiveness and user satisfaction. In no case do the usability mechanisms degrade user performance. The effort to adopt Abort Operation and Preferences appears to be justi++wE-ed by the bene++wE-ts in terms of effectiveness and user satisfaction. Also Abort Operation enables the user to be more productive. We believe that the effects on efficiency, effectiveness and satisfaction depend not only on mechanism functionality but also on the problem domain. The impact of a mechanism in other contexts could differ. Therefore, we need to conduct further experiments to gather more evidence and con++wE-rm these results.