Creating Trustworthy Health Apps Using Speculative Design
My Role: UI/UX designer & Researcher
Timeline: December 2022 - September 2023
Tools: Figma, FigJam, Adobe, Canva, User Interviews, Surveys, Overleaf
Project Overview
This portfolio entry provides an in-depth account of an individual user experience design project focused on creating trustworthy smart health applications. My objective was to understand and incorporate factors that foster user trust in these applications through speculative design, society descriptions, prototype development, and user evaluations. This project was a result of the design I conducted for the second stage of my master's thesis, which aimed to help users and designers reach more usable and trustworthy health apps.
Background and motivation
I discovered the lack of trust as a serious problem for smart health application users in the previous stage of my thesis. To address this issue, I aimed to study user perceptions toward the trustworthiness of smart health applications. Trust is a complex and multilayered phenomenon with many factors leading to its buildup. For the purposes of this study, I used a well-established definition of trust by Simpson that states, "I trust based on an assessment of the other party's trustworthiness. There is also a willingness to render myself vulnerable. An object or person is, in turn, considered to be trustworthy if it has the ability, benevolence, and integrity."
Building on this foundation and considering that interpersonal trust and trust in technology have some fundamental differences in the way that they are formed, I turned to previous work studying trust in smart technologies specifically. Toreini et al. explored the trust foundations of artificial intelligence and human interaction. They established four criteria that are essential for an AI-based system to be considered trustworthy: Fairness, Explainability, Audibility, and Safety. I leveraged this framework for my study regarding smart health applications to investigate whether these criteria are effective indicators when it comes to user trust.
Especulativ Design
Start of Speculative Design
As discussed, I chose the exploratory method of speculative design to navigate the rapidly changing technological capabilities and limitations of smart systems and to investigate the broader development of trustworthy smart health applications. The trajectory of smart health applications, as well as smart systems in general, is undeniably influenced by the overall development and societal trends. Thus, envisioning the future equivalent of a smart health application necessitates consideration of the broader societal context in which it operates. To initiate my design process, I developed several fictional societies, each evolving in a way that allowed for varying degrees of fairness, explainability, audibility, and safety. Each society description was carefully crafted to provide sufficient background information about the fictional state of the society and detailed insights into its health systems and smart health systems while still encouraging imagination and creative interpretation, crucial aspects of speculative design.
Toreini et al.’s framework which was used to extract the four criteria of trustworthiness, only provided very high-level definitions of the four trustworthy criteria. In order to apply these definitions to my speculative design process, I refined these definitions and considered the meaning of both extremes of each criterion in my design context:
Fairness: technologies focused on the detection or prevention of discrimination and bias in different demographics and abstained from generalizing toward a dominant group. Low fairness: Not addressing individual differences. Unfair to an under-resourced population. High fairness: Sensitive to discrimination and bias. Address individual differences.
Explainability: technologies focused on explaining and interpreting the outcome to the stakeholders (including end-users) in a humane manner. Low explainability: No one knows how the technologies work; people just blindly follow the recommendations. High explainability: People understand the suggestions/recommendations that the technologies provide and follow them with discretion.
Audibility: technologies focused on enabling third parties and regulators to supervise, challenge, or monitor the operation of the model(s). Low audibility: No one knows what the companies are doing. High audibility: The company needs to give sensitive information when requested.
Safety: technologies focused on ensuring that the system is medically safe (data is correct and the app is safe to use, not harmful to users) and the operation of the model works as intended in the presence of active or passive malicious attackers. Low safety: The technology is not necessarily medically safe or accurate. Attackers can easily gain access to technologies, and thus personal data, etc. High safety: There is a reasonable level of medical safety and using them carries little risk to users both in terms of potential medical harms and in terms of possible data breaches and privacy transgression.
I started by creating 16 fictional societies that encompassed all possible combinations of the extremes of the four criteria (e.g., low-low-low-low, low-low-low-high, etc.). To emphasize the significance of each criterion and maximize differences between the societies, I intentionally focused on the extreme ends of the spectrum. This meant that each hypothetical society exhibited a particular criterion in either very high or very low degrees. After inspecting those fictional societies, I found that the 16 society descriptions shared numerous similarities, with some scenarios appearing highly improbable and far-fetched. For instance, envisioning a world where smart systems were incredibly fair, transparent, and easily understood, yet remarkably unsafe, posed significant challenges. To address these issues and streamline my user study, I undertook a trimming-down process.
During the trimming-down process, it became evident that Fairness and Explainability were closely related – a fair society necessitates providing information to reassure individuals about the system’s fairness and suitability; and similarly, a transparent society would naturally produce diverse and unbiased products, as its citizens can access relevant information and make informed decisions. Likewise, it was apparent that a robust system of checks and balances by regulatory bodies and trusted organizations (i.e., Audibility) leads to safe technologies (i.e., Safety). Developing and maintaining genuinely safe products without such checks is highly challenging. These pairings guided my selection process, resulting in the development of four final fictional societies.
Society Design
The four selected societies were specifically designed to explore the effects of the four trustworthiness criteria and compare the importance of each pairing. Here are the descriptions of the final four selected societies, labeled with the considerations of the four trustworthy criteria:
Society A - Low Fairness - Low Explainability - Low Audibility - Low Safety:
In society A, most people are treated like homogeneous bricks without much individual worth. The continuous growth of the world population, the decline of natural resources, and the effects of global warming have rendered most people expendable cogs in the machine. The governments in most countries prioritize low cost and minimize the burden on the health sector through solving most common problems with general solutions. The health services are working based on the data received from the most populated groups and cultures. As a result, members of the more underprivileged groups are mostly ignored. Most people receive general advice through public websites and applications accessible through mass-produced health check devices with medium to low sensitivity sensors worn on their wrists, waist, or around their necks. The general effectiveness of these methods is never checked. And as long as there are no catastrophic results, these technologies continue to be used. People can access their files and info on any personal computing device, but they cannot check the reasoning behind a suggestion or health tip. As a result of the emphasis on keeping the cost down, the vulnerability toward nefarious online attacks is high, and it is not rare for public data to be leaked. Additionally, the government can access the health data of citizens without their consent. The mass solution to health problems, however, keeps the overall process financially cheap.
Society B - Low Fairness - Low Explainability - High Audibility - High Safety: After the globalist centralization of the world government and decades of uniform education provided to most people, many of the individual and cultural differences have started to fade. Groups that do not conform with what the society believes are marginalized and their interests are discarded when it comes to making public policy decisions. To reflect on this worldview, each citizen receives tracking devices that record their exact physiological data since birth. They are required to wear their wearable devices at all times. All of the services and devices used to monitor and improve public health have to pass a rigorous examination process by government institutions so that their functionality is scientifically proven. On the other hand, the diversity of the available products is limited – new and exciting products cannot enter the market before going through the lengthy process for approval. Additionally, people receive tips and recommendations, and notifications constantly. However, they cannot know the exact reason behind these suggestions or check the track record of the methods offered to them. This forced ignorance can cause people to miss better options for health services and solutions. The public hospitals and medical centers have a mixture of diagnostic supercomputers which are directly connected to the database storing the health information of people in their district and some human healthcare workers, who are also important for solving more severe health issues. As a result, each individual’s recommended health program is made to minimize any danger or harm. Moreover, the gathered health data is highly guarded, and there are multiple protections in place to prevent a data attack. However, as a product of the overall philosophical approach of society, an individual’s physical and mental characteristics are not part of the health information being recorded. Their personal preferences are never taken into consideration. As a result, each individual is confined to the highly detailed but general list of health services provided to them.
Society C - High Fairness - High Explainability - Low Audibility - Low Safety: In Society C, people are used to personalized AIs that feel like a friend. It’s very important for them that they have their “friends” built in the specific way that they prefer. With the emphasis on multiculturalism and representation of each group, most service providers try to have a wide variety of services for different groups and demographics, genders, ethnic groups, as well as sexual orientations and identities. Companies also prioritize having a diverse development and design team. This emphasis, however, sometimes can turn into catering to a marketing strategy – some good products can be rebuffed just because they do not look diverse enough. Basically, each user in Society C is the master of their own health journey. Thus, health systems emphasize providing users with abundant customization options. On the downside, giving so much control and responsibility to the users causes some citizens of Society C to fall into the trap of believing themselves to be professionals capable of making every health decision on their own. In practice, this can prove to be quite dangerous. Similarly, the lack of oversight means that sensitive health data can move through very unsafe channels which are extremely vulnerable to attacks. Moreover, as so many health-related decisions are made by the users and the process is distributed through a very large and open network, any form of regulated testing and universally reputable certificates are impossible to employ properly.
Society D - High Fairness - High Explainability - High Audibility - High Safety: In Society D, automation has grown to take over a substantial part of human responsibilities. Here, different and competing automated services are offered to people in any domain, although some human oversight in key and sensitive domains such as health care is still deemed necessary. This society offers each individual a personalized and detailed set of services curated just for their habits, personal traits, and preferences, emphasizing the diversity of users in the design process of apps and services. It is important for each health service provider to prove that they used a diverse and reflective set of users in the testing process and employed the help of relevant experts to make sure that the service provides valid information to each demographic. Although beneficial to avoid injustice and bias, this approach can make the testing and development process harder and can push functional products that are not specially designed to emphasize diversity out of the market. The users are free to check the reasoning behind a decision or suggestion; they can even look at the inner workings of an algorithm. They also can see the history and past data for the available health methods, which can help them make a more informed decision. The access to personal data, however, is limited to what has been explicitly granted to them by the users. The vigorous safety testing ensures that the technologies are capable of protecting users’ data. Only AI systems passing the safety tastes and medical trials are allowed to offer health services. Users still can apply some personal preference to the systems and choose between the variety of available options. However as the general census is inclined to opt for safer, more well-known options, and due to the expensive trials and medical technologies required to perform specialized practices, most health services are offered by a number of big companies.
Prototype Process
Preliminary Low-Fidelity Prototype Design
Upon creating the descriptions of the different societies, I proceeded to develop four high-fidelity prototypes that closely resembled smart health applications in terms of visual and aesthetic properties. This approach aimed to provide users with a familiar interface, allowing them to concentrate on the four criteria represented in the prototypes and their overall user experience. By utilizing a design that is familiar to the user, I create a possible-plausible effect that is essential to speculative design, allowing users to reflect on alternative and futuristic values based on their understanding of the current reality. However, it’s worth noting that the prototype corresponding to Society C deviated slightly from this approach to represent the individuality and occasional quirkiness of the products within that society; this decision is made because being unique and sometimes unconventional is an integral characteristic of the systems in Society C. This diversion, however, was still minor and mostly reflected on the aesthetics and did not affect the way that the users interacted with this prototype.
To avoid introducing bias, the prototypes did not feature any explicit visual indicators of their corresponding societies or the levels of fairness, explainability, audibility, and safety they intended to represent. Instead, these qualities were subtly conveyed through the functionalities of the prototypes, the information provided, and the user experience they offered. Fair systems presented personalized and flexible options tailored to users’ needs and characteristics while allowing for choices. Explainable systems made the reasoning behind their suggestions and the accuracy of their methods clear and understandable. Audible systems highlighted the tests and checks conducted to ensure product effectiveness. Lastly, safe products refrained from recommending risky or unhealthy suggestions or services while safeguarding user privacy.
Each prototype depicted a different health-related function, although participants were informed that most prototypes were capable of performing all regular health-related functions, such as diet, fitness, activity tracking, mental health, and diagnostic tools. This approach served two purposes. Firstly, it diversified the range of activities and scenarios encountered by the users, expanding their imagination and reflection regarding the applications, in line with the goals of speculative design. Secondly, it prevented participants from fixating on the minute technical details of how each prototype performed a similar function and encouraged them to focus on the broader and inherent differences in the design ideology of each prototype.
After the first set of prototypes was created, I conducted two rounds of preliminary user studies to iteratively refine the prototype designs. Three participants aged from 23 to 25 participated in those user studies, including one male and two females. During the study sessions, the participants were first asked to explain their background experiences with smart health applications and answer a number of demographic questions. The next step of the study asked participants to work with the four designed prototypes, finish a series of tasks, and share their opinions regarding the perceived trustworthiness of these prototypes. Lastly, participants were presented with the four fictional society descriptions. They were then asked to read these descriptions and share their thoughts about the societies. Specifically, they were to consider how suitable these societies appear for the development of trustworthy smart applications. The sessions were recorded, and their responses were then analyzed to extract the major problems of the study and refine the prototypes. Based on the participants’ feedback, I made various changes to the prototypes. I describe two major changes below.
Initial Low-Fidelity Prototype Design Changes
The first major change is related to the color schemes used. Initially, the low-fidelity prototypes featured distinct color themes intended to emphasize the individuality of each prototype. For instance, the first prototype had a grey and black color theme to reflect the austere nature of the society it represented. However, during the first round of pilot studies, the first participant pointed out that these color choices could introduce bias and unduly influence their perceptions of the prototypes. Taking this feedback into consideration, I decided to transform all the designs to incorporate green, blue, and white colors. These colors are commonly associated with health and healthcare, ensuring a more neutral and unbiased visual presentation across all the prototypes.
As an example here is the overview of the original concept for the first page of Prototype 1:
The second major change was related to the design limitations that inadvertently appeared as intentional design choices. For instance, the concept version of prototype 3, representing Society C, included a recipe recommendation system, where users were able to get recipes based on either a health-oriented plan or a plan based on the foods they liked. This prototype was originally perceived by two participants as restrictive due to its display of only one recipe for each plan. This experience originated from the prototype’s incomplete and non-functional nature, rather than a deliberate design decision. To address this issue and provide a more comprehensive user experience, I added more pages and functions to the prototypes to offer a broader range of options and enhance overall usability, which are shown below. These enhancements aimed to ensure that users could experience a more realistic representation of the potential application’s capabilities.
Final Prototype Design
Below is a short description of the final versions of each of the prototypes:
Prototype 1 corresponds to society A with low fairness, low explainability, low audibility, and low safety:
Prototype 1 served as an activity recommendation and tracking system. On the main page, users could view their citizen health points, representing rewards earned upon successful completion of recommended activities. Users lost points if they were unable or unwilling to finish an activity. It was explained to the participants that these points contributed to their social standing within Society A. The main page provided users with daily activity instructions. The daily recommended activity section allowed users to mark completed activities with a ticked box, while missed activities were identified with a cross mark. Activities that remained unfinished had an accompanying empty tick box. Clicking on an activity led users to an activity page with instructions on how to perform the task. Although the prototype focused on the daily exercise section for the user study, participants were informed that it aimed to represent a comprehensive daily health-related activity recommender and tracker. Other sections were not developed due to time constraints. For individual exercise sessions, a video demonstrating the recommended exercise was provided to users. The audio was intentionally removed to add difficulty. Two options, “Successfully finished the exercise” and “Could not finish the exercise,” were presented. Selecting the first option resulted in users being informed that they did not properly complete the exercise according to the tracking data, without offering a solution or further explanation. This led to no points gained or lost. Choosing the second option notified users of lost health points, again without providing a solution or justification. The lack of fairness was portrayed through the impersonal and cold interface treating citizens as mere numbers, as well as the inflexible nature of recommended activities that disregarded personal preferences. To underscore this unsuitable nature, all users were recommended a relatively difficult daily exercise with inadequate instructions. The absence of explainability was demonstrated by not providing information about the basis for suggestions or recommendations and informing users of exercise failure even when they selected the successful completion option, without any further explanation. The lack of audibility was evident in the absence of information, reassurance, or certification regarding any tests or checks conducted to ensure the functionality of the product. Similarly, the lack of safety was depicted through the absence of privacy information and ambiguous tracking without user consent. Additionally, the difficult recommended exercise, regardless of users’ medical and physical conditions, highlighted the disregard for safety.
Prototype 2 corresponding to society B with low fairness, low explainability, high audibility, and high safety:
Prototype 2, representing Society B, was a comprehensive health tool. The first page featured sections for daily activities, checkups, diet, and general health tracking. While the daily activity section was explored in the prototype and user study, participants were informed that the hypothetical application it represented included all sections. Users had the flexibility to access any recommended activity by clicking on it in the daily exercise section. They could then view instructions for the activity and begin it. For example, in the daily walk activity, the prototype informed users about the optimal time for their walk and allowed them to schedule it within the specified time slot, with a suggested distance and path. Upon completing an activity, such as a daily walk, users were congratulated, and the activity was marked as finished. The high level of audibility in Society B was demonstrated by informing users that the application had been tested, approved, and endorsed by medical professionals. The emphasis on safety was evident through assurances regarding the privacy of users’ data and the mention of vital signs being monitored to prevent any potential harm. Furthermore, the app informed users that their walk had been scheduled during the most medically beneficial time, adding to the medical safety aspect. However, the lack of fairness was depicted by the provision of a strict and inflexible schedule that did not consider individual needs. Despite claiming to be medically optimal, it provided no flexibility or choice for users with time or other limitations. The absence of explainability was conveyed by not providing users with explanations as to why certain activities were suggested or why they were unable to choose an alternative time for their daily walk. While it was mentioned that their vital signs were being recorded, no further information or conclusion was provided, highlighting the lack of explainability once again.
Prototype 3 corresponding to society C with high fairness, high explainability, low audibility, and low safety:
Prototype 3, representing a fictional application in Society C, featured the most visual differences compared to the other prototypes to emphasize the unique and highly individualistic nature of this society. Users were able to select from different meal options depending on their personal preferences, dietary restrictions, and health goals. The application ensured that users had a wide variety of choices catering to diverse tastes and cultural backgrounds. To explore the prototype during the user study, participants were instructed to select a recipe for dinner. On the first page, users were able to view various meal options, each featuring a picture of the dish and a brief description of its ingredients and nutritional content. Users could click on a meal to view its detailed recipe, preparation instructions, and nutritional information. Users were also provided with options to filter meals based on dietary preferences and health goals. Upon selecting a meal, users were informed about the ingredients required and the preparation time. However, the lack of fairness was demonstrated by a somewhat overwhelming number of choices and the absence of any active recommendations or personalization. Despite offering extensive customization options, the application provided little guidance or assistance in making a decision, placing the onus entirely on the user. The absence of audibility was apparent in the lack of any information regarding the app’s development, testing, or endorsement by medical professionals. Similarly, the lack of safety was conveyed through the application’s inability to verify the safety of recipes or nutritional advice, leaving users to rely on their own judgment and potentially exposing them to risks. Despite being user-centric and valuing individuality, the application failed to address potential safety hazards or provide users with any safeguards against making unhealthy choices. Furthermore, the application neither informed users about the sources of their recipes nor provided any validation or certifications regarding their nutritional accuracy. The app also lacked privacy information, data security assurances, and safeguards against potential data breaches or unauthorized access.
Prototype 4 corresponding to society D with high fairness, high explainability, high audibility, and high safety:
Prototype 4 was designed to represent the fictional Society D, embodying high levels of fairness, explainability, audibility, and safety. This prototype focused on diet and meal planning, allowing users to select recipes based on their dietary preferences, health goals, and taste preferences. Users could also find detailed information about the nutritional content of each meal, its health benefits, and cooking instructions. To experience the prototype during the user study, participants were instructed to select a recipe for dinner. On the first page, users were presented with a range of meal options, each featuring a picture of the dish, a brief description, and its suitability for different dietary preferences and health goals. Users could click on a meal to access detailed information about its ingredients, nutritional content, preparation instructions, and potential health benefits. Users were also able to filter meals based on their preferences. However, unlike prototype 3, the application in this society offered active recommendations and personalized suggestions based on user profiles and health data. It considered individual characteristics, health goals, and dietary restrictions to provide tailored meal options. The application highlighted the use of diverse user profiles and expert evaluations during its development and testing phases. It emphasized its transparency by offering users the ability to view the reasoning behind each recommendation, access the inner workings of the recommendation algorithms, and explore the testing procedures it underwent. The application also assured users of its commitment to privacy and data security. It provided information about data encryption, access control, and regular security audits to safeguard user data. Moreover, it facilitated user consent and control over data sharing and access permissions. The high level of fairness was reflected in the application’s active efforts to ensure diversity and inclusivity in its design, development, and user evaluation processes. It emphasized the representation of various demographic groups in its user base and expert panels to minimize bias and discrimination. The absence of unfairness was underlined by the application’s personalized and flexible recommendations, ensuring that each user had access to suitable meal options that aligned with their preferences and dietary needs. The high explainability was manifested through the detailed information provided about the nutritional content, health benefits, and preparation instructions for each meal. Users could explore the inner workings of recommendation algorithms and the criteria used to suggest specific meals. The application aimed to empower users with the knowledge and understanding needed to make informed dietary choices. The high audibility was evident in the application’s openness about its development, testing, and evaluation processes. It disclosed the involvement of diverse stakeholders, including medical professionals, nutritionists, and user communities, in its development and evaluation. Users were assured of the transparency and accountability of the application’s operations.
User Study
Study Design
Participants and Ethics
Six technologically literate participants were recruited for the study, which was ethically approved by Polytechnique Montreal.
Study Structure
Each online study session, lasting about 90 minutes, consisted of three parts:
Part 1: Prior Experiences and Expectations:
Participants shared past experiences with smart health apps and discussed what made them trustworthy or not. They also discussed broader issues influencing trust in smart tech.
Part 2: Evaluation of Fictional Societies:
Participants read descriptions of four fictional societies and shared their opinions on trustworthiness. They ranked these societies based on perceived trust.
Part 3: Evaluation of Prototypes :
Participants interacted with prototypes presented in random order, providing feedback, trust assessments, and associations with fictional societies to assess how well prototypes represented them.
User Study Script
The user study script is provided below:
Data Analysis
The data collected from the user study, including participants’ questionnaire responses, open-ended comments, and reflections on the fictional societies, were analyzed to gain insights into the perceived trustworthiness of the prototypes and the influence of the trustworthiness criteria (fairness, explainability, audibility, and safety). The analysis involved both quantitative and qualitative approaches to provide a comprehensive understanding of participants’ perspectives.
Key Findings
This study revealed crucial insights:
1- Four Essential Criteria: Fairness, explainability, audibility, and safety are indispensable for building trust in smart health applications.
4- User Agency and Freedom: Empowering users to make informed decisions and respecting their autonomy enhances trust.
2- Personalization Matters: Users highly value personalized health solutions, fostering a sense of fairness and control.
5- Trustworthy Intentions: Trusting the application and its developers' intentions is paramount and can be conveyed through reputation and design.
3- Transparency and Understanding: Clear explanations for recommendations are crucial; users must comprehend the application's reasoning to trust it.
Based on these findings, several design considerations for trustworthy smart health applications are proposed:
-
Comprehensive Integration: Seamlessly integrate fairness, explainability, audibility, and safety into the application's design and functionality.
-
Personalization Focus: Prioritize personalization and customization to cater to individual preferences and foster user trust.
-
Transparent Explanations: Offer clear explanations for recommendations and decisions to ensure user understanding.
-
User Autonomy: Empower users to make choices, respect their autonomy, and allow them to navigate their health journey.
-
Balancing Safety and Freedom: Strike a balance between ensuring user safety and preserving their freedom, particularly in health decisions.
Conclusion
This project underscores the paramount importance of trust in smart health applications. Through speculative design, society descriptions, and prototype development, I've explored user-centered approaches that transcend technological limitations, fostering trust and adapting to evolving needs.
This journey has not only deepened my understanding of trust dynamics but also enriched my knowledge of user-centered design. Iteratively refining the prototypes based on user feedback has underscored the pivotal role of user-centered design. In this enlightening experience, I've learned that building trust is an ongoing journey that demands adaptability, empathy, and an unwavering commitment to user well-being.