Role: Head UX/UI Designer
Timeline: 4 months
TERRAFirme
Developing an Accessible Risk Reporting App: AI-Powered Image Recognition for Civil Defense's Landslide Risk Identification in Dense Urban Areas
Developing an Accessible Risk Reporting App: AI-Powered Image Recognition for Civil Defense's Landslide Risk Identification in Dense Urban Areas
Developing an Accessible Risk Reporting App: AI-Powered Image Recognition for Civil Defense's Landslide Risk Identification in Dense Urban Areas
The Challenge.
In light of the significant loss of lives, with approximately 1,500 fatalities attributed to landslides in Brazil over the past 10 years, my objective was to develop a solution that tackles this grave issue. By prioritizing effective identification of landslide risks and improving reporting and communication processes, the aim was to enable timely intervention by civil defense authorities.
Short onboarding with the options to learn landslide warning signs
Includes images to help users see real life examples
Quick and easy sign in
Short onboarding with the options to learn landslide warning signs
Includes images to help users see real life examples
Short onboarding with the options to learn landslide warning signs
Includes images to help users see real life examples
Quick and easy sign in
Users with visual or cognitive impairments can utilize Assisted Reporting or 360° Reports feature to identify landslide risk signs in their homes. Allowing new segments of the population to actively report risks.
Assisted Reporting utilizes a pre-trained AI model to identify 45° cracks on walls in pictures taken by users.
360° Reports enables users with visual impairments to "hear" cracks using voiceover and a pre-trained AI model, allowing them to actively report landslide risks.
Allowing users with cognitive and physical disabilities to have assisted reporting
Users with visual or cognitive impairments can utilize Assisted Reporting or 360° Reports feature to identify landslide risk signs in their homes. Allowing new segments of the population to actively report risks.
Assisted Reporting utilizes a pre-trained AI model to identify 45° cracks on walls in pictures taken by users.
360° Reports enables users with visual impairments to "hear" cracks using voiceover and a pre-trained AI model, allowing them to actively report landslide risks.
Allowing users with cognitive and physical disabilities to have assisted reporting
Shortening the flow from the moment of risk analyzed to the final report being made.
Allows users to maintain awareness of their communities risk levels while incentivizing a culture of vigilance of current environment.
Notifications of “are you sure you don't want to report” Allow users to rethink choices eventually report a risk.
Notifications on identified risks in images and/or awareness of local reports
Shortening the flow from the moment of risk analyzed to the final report being made.
Allows users to maintain awareness of their communities risk levels while incentivizing a culture of vigilance of current environment.
Notifications of “are you sure you don't want to report” Allow users to rethink choices eventually report a risk.
Notifications on identified risks in images and/or awareness of local reports
The Solution
Quick and easy flow to report signs of risk
Allows users to send multiple pictures. Autofill of previously given information accelerates the fill out process.
Creating a fast and efficient way to report
The Solution
Quick and easy flow to report signs of risk
Allows users to send multiple pictures. Autofill of previously given information accelerates the fill out process.
Quick and easy flow to report signs of risk
Allows users to send multiple pictures. Autofill of previously given information accelerates the fill out process.
Creating a fast and efficient way to report
Creating a fast and efficient way to report
Citizens located in gang/ militia dominated areas are able to anonymously report risk without being blamed for the report.
Allows members of own family to report problems in a household worried about risk
Gives data on locations not very accessible before
Allowing for anonymous reporting
Citizens located in gang/ militia dominated areas are able to anonymously report risk without being blamed for the report.
Allows members of own family to report problems in a household worried about risk
Gives data on locations not very accessible before
Allowing for anonymous reporting
The Secondary Research
The Secondary Research
The importance of quality data for landslide early warning systems
My research included:
Getting wide angle understanding of the causes of landslides
Understanding the main issued that arise in the prevention of landslide
Common issues with prevention technologies today
The populations relationship with reporting risks
Ways to identify landslide risk
Initially having gone into the research with an extremely generalist approach i felt very much worried about the inconsistency in the subject matter id finding important information on. Fortunately many of the articles and research papers pointed me into the very interesting direction. I first choose to go extremely generalist because we were indeed going in into this with the goal to solve an actual problem that the civil defense had. To truly comprehend the scale of the problem and its complexities studied areas slightly out of topic made witch made perfect sense of researching about because later in my research was able to validade the final research and understand the true value i was bringing to this urging problem
Discoveries
Prevention
Prevention efforts are done defensively in most cases
Soil types vary and difficult prevention efforts
Depending on the risk level and time given, prevention efforts can be done using trees and not retaining walls
Schools in Brazil don't have natural disaster education as a mandatory activity
The list of signs of risk is given out via pan-flit form
Organized visits to schools initiated on the behalf of civil defense have shown increased reporting amongst children's parents
Education
Although most systems were effective they were too expensive to place
Low precision on timing of evacuation difficult efficacy of prevention efforts
Current systems to report signs of landslides are limited by phone calls
Current systems
Satellites monitoring land shifts offer a potential solution for detecting hazards, but their single-pixel size, typically around one square meter, limits their ability to visualize detailed features like cracks on rooftops or cracks on vertical structures like walls. Despite the existence of early warning sensor technologies, their high costs and challenges in widespread implementation on densely occupied hills make them less feasible.
Interviews
How have apps/services facilitated the transfer of information to organizations?
How have similar apps incentivized engagement to report?
What new market trends or technological advancements are my competitors leveraging?
Goal of the analysis
Competitive analysis
Some products automated irregularity detection in movement.
Interactive maps enhanced the sense of security in many products.
A diverse range of input options was available initially.
Strength
Lack of visual hierarchy or contrasting colors affected clarity.
Chat features in some products became ineffective due to message overload.
Attempting to address multiple issues resulted in a useless outcome.
Weaknesses
Some products incorporated features to aid communication with first responders.
Majority lacked engagement, while some gamified the experience.
Enhanced customizability for users with disabilities was desired in many cases.
Opportunities
Implementing AI or sensors, as demonstrated in Apple's fall detection
Gamification may attract users, but it also has the potential to be borderline offensive.
Some apps already possess substantial user data.
Threats
Initially, we faced uncertainty about the competitors in the space and the potential benefits of conducting a competitive analysis. However, we decided to break down the analysis into various services and prioritize their key features, particularly focusing on education, identification, and communication of risks. Fortunately, conducting the competitive analysis proved to be highly beneficial. Among the 45 products and services we examined, we were able to identify a few that sparked the leading idea for our final product.
My Shake app
Citizen app
Fixmystreet.brussels
Reportesmed app
FEMA app
Redes Socias
My Shake
Reportemed
SeeClickFix
mPERS Device
Onde Tem Tiroteio
Fall Detection
Apps
Produtos
Apps
Notifications
Instructions on risk assessment
Input options
Inclusivity
Some form of autonomy
Visual clarity and hierarchy
Interview questions that I came up with based on my secondary research:
What is your relationship with landslides? how have they affected you? or have you affected them?
How do you deal with safety measures inside your home?
Do you know the signs of risk of a landslide?
Is search and rescue more important than prevention on a large scale?
What would you say is the largest contributor to the prevention of landslides?
What can the population do to keep themselves safe?
Interview questions that I came up with based on my secondary research:
What is your relationship with landslides? how have they affected you? or have you affected them?
How do you deal with safety measures inside your home?
Do you know the signs of risk of a landslide?
Is search and rescue more important than prevention on a large scale?
What would you say is the largest contributor to the prevention of landslides?
What can the population do to keep themselves safe?
The amount of research invested in identifying stakeholders for this project was tremendous. We interviewed 17 individuals, including victims, geologists, the Secretary of Civil Defense, and the Director of the Red Cross. It was an eye-opening experience to meticulously interview each and every person involved in this significant problem.
end
flip
mute
effects
Camera Off
Speaker
Interviews
Affinity mapping
After carefully reviewing the extensive collection of notes we had accumulated, my partner and I diligently sorted and categorized them into major groups. Initially, we separated our findings from government officials and the general population into two distinct groups. Further classification was done based on themes related to prevention, communication, behavior, worries, and critiques. However, in the final analysis, we consolidated these groups into three overarching categories: prevention, communication, and behavior. T
Data on first person reports, Accessibility and lack of ways to report a risk were hinging efforts immensely.
My final research findings
The civil defense wasn't very efficient in the sharing of information, especially signs of risk
Some victims felt neglected and forgotten about in some areas and said they never were warned of anything.
Directed people to their phone number for any reports (199)
Communication
Older citizens and disabled of the population didn't have ways of reporting through phone call
People were either too lazy to report risks or were scared about the consequences of reporting
Not everyone is able to memorize/ visualize signs of risk
Accesability
A lack of knowledge and urgency dis motivated users
The population never thought it could happened to them even after happening next door
Autonomous systems do work. Why don't they exist in a field so crucial for survival?
Behavior
“The majority of citizens don't want to spend a second avoiding something thats never directly affected them” - USER #3
“ The people that spent more time inside their homes either dont know how to report or have difficulty reporting due to disabilitiese/age etc.” USER #8
“ People are truly worried of telling us and having to evacuate their only thing they consider theirs on this earth. ” USER #10
“ If citizens were actually aware of risks and reported them we would have prevented so much suffering from landslides ” USER #10
My final research findings
The civil defense wasn't very efficient in the sharing of information, especially signs of risk
Some victims felt neglected and forgotten about in some areas and said they never were warned of anything.
Directed people to their phone number for any reports (199)
Communication
Older citizens and disabled of the population didn't have ways of reporting through phone call
Not everyone is able to memorize/ visualize signs of risk
Accesability
Autonomous systems do work. Why don't they exist in a field so crucial for survival?
People were either too lazy to report risks or were scared about the consequences of reporting
A lack of knowledge and urgency dis motivated users
The population never thought it could happened to them even after happening next door
Behavior
My final research findings
The civil defense wasn't very efficient in the sharing of information, especially signs of risk
Some victims felt neglected and forgotten about in some areas and said they never were warned of anything.
Directed people to their phone number for any reports (199)
Communication
Older citizens and disabled of the population didn't have ways of reporting through phone call
Not everyone is able to memorize/ visualize signs of risk
Accesability
Autonomous systems do work. Why don't they exist in a field so crucial for survival?
People were either too lazy to report risks or were scared about the consequences of reporting
A lack of knowledge and urgency dis motivated users
The population never thought it could happened to them even after happening next door
Behavior
Data on first person reports, Accessibility and lack of ways to report a risk were hinging efforts immensely.
Field Research
Visits to the Civil Defense to understand what they currently do on the prevention side
Discoveries
Prevention
Prevention efforts are done defensively in most cases
Soil types vary and difficult prevention efforts
Depending on the risk level and time given, prevention efforts can be done using trees and not retaining walls
Schools in Brazil don't have natural disaster education as a mandatory activity
The list of signs of risk is given out via pan-flit form
Organized visits to schools initiated on the behalf of civil defense have shown increased reporting amongst children's parents
Education
Although most systems were effective they were too expensive to place
Low precision on timing of evacuation difficult efficacy of prevention efforts
Current systems to report signs of landslides are limited by phone calls
Current systems
Interviews
My Shake app
Citizen app
Fixmystreet.
brussels
Reportesmed app
Some products automated irregularity detection in movement.
Interactive maps enhanced the sense of security in many products.
A diverse range of input options was available initially.
Strength
Lack of visual hierarchy or contrasting colors affected clarity.
Chat feature in some products became ineffective due to message overload.
Attempting to address multiple issues resulted in a useless outcome.
Weeknesses
Some products incorporated features to aid communication with first responders.
Majority lacked engagement, while some gamified the experience.
Enhanced customizability for users with disabilities was desired in many cases.
Opportunities
Implementing AI or sensors, as demonstrated in Apple's fall detection
Gameification may attract users, but it also has the potential to be borderline offensive.
Some apps already possess substantial user data.
Threats
FEMA
app
Redes
Socias
My Shake
Reportemed
SeeClickFix
mPERS
Device
Onde Tem
Tiroteio
Fall
Detection
Apps
Produtos
Apps
Notifications
Instructions
on risk
assessment
Input
options
Inclusivity
Some form
of autonomy
Visual
clarity and
hierarchy
How have apps/services facilitated the transfer of information to organizations?
How have similar apps incentivized engagement to report?
What new market trends or technological advancements are my competitors leveraging?
Goal of the analysis
Competitive analysis
Initially, we faced uncertainty about the competitors in the space and the potential benefits of conducting a competitive analysis. However, we decided to break down the analysis into various services and prioritize their key features, particularly focusing on education, identification, and communication of risks. Fortunately, conducting the competitive analysis proved to be highly beneficial. Among the 45 products and services we examined, we were able to identify a few that sparked the leading idea for our final product.
Discoveries
Prevention
Prevention efforts are done defensively in most cases
Soil types vary and difficult prevention efforts
Depending on the risk level and time given, prevention efforts can be done using trees and not retaining walls
Schools in Brazil don't have natural disaster education as a mandatory activity
The list of signs of risk is given out via pan-flit form
Organized visits to schools initiated on the behalf of civil defense have shown increased reporting amongst children's parents
Education
Although most systems were effective they were too expensive to place
Low precision on timing of evacuation difficult efficacy of prevention efforts
Current systems to report signs of landslides are limited by phone calls
Current systems
Interview questions that i came up with based on my secondary research:
What is your relationship with landslides? how have they affected you? or you affected them?
How do you deal safety measure inside your home?
do you know the signs of risk of a landslide?
Is search and rescue more important than prevention on a large scale?
What would you say is the largest contributors to the prevention of landslides?
What can the population do to keep themselves safe?
The amount of research invested in identifying stakeholders for this project was tremendous. We interviewed 17 individuals, including victims, geologists, the Secretary of Civil Defense, and the Director of the Red Cross. It was an eye-opening experience to meticulously interview each and every person involved in this significant problem.
end
flip
mute
effects
Camera Off
Speaker
Interviews
Field Research
Visits to the Civil Defense to understand what they currently do on the prevention side
Affinity mapping
After carefully reviewing the extensive collection of notes we had accumulated, my partner and I diligently sorted and categorized them into major groups. Initially, we separated our findings from government officials and the general population into two distinct groups. Further classification was done based on themes related to prevention, communication, behavior, worries, and critiques. However, in the final analysis, we consolidated these groups into three overarching categories: prevention, communication, and behavior. T
The civil defense wasn't very efficient in the sharing of information, especially signs of risk
Some victims felt neglected and forgotten about in some areas and said they never were warned of anything.
Directed people to their phone number for any reports (199)
Communication
Older citizens and disabled of the population didn't have ways of reporting through phone call
Not everyone is able to memorize/ visualize signs of risk
Accesability
Autonomous systems do work. Why don't they exist in a field so crucial for survival?
People were either too lazy to report risks or were scared about the consequences of reporting
A lack of knowledge and urgency dis motivated users
The population never thought it could happened to them even after happening next door
Behavior
My final research findings
Data on first person reports, Accessibility and lack of ways to report a risk were hinging efforts immensely.
The civil defense wasn't very efficient in the sharing of information, especially signs of risk
Some victims felt neglected and forgotten about in some areas and said they never were warned of anything.
Directed people to their phone number for any reports (199)
Communication
Older citizens and disabled of the population didn't have ways of reporting through phone call
Not everyone is able to memorize/ visualize signs of risk
Accesability
Autonomous systems do work. Why don't they exist in a field so crucial for survival?
People were either too lazy to report risks or were scared about the consequences of reporting
A lack of knowledge and urgency dis motivated users
The population never thought it could happened to them even after happening next door
Behavior
My final research findings
Data on first person reports, Accessibility and lack of ways to report a risk were hinging efforts immensely.
Interviews
Implementing AI or sensors, as demonstrated in Apple's fall detection
Gameification may attract users, but it also has the potential to be borderline offensive.
Some apps already possess substantial user data.
Threats
Some products incorporated features to aid communication with first responders.
Majority lacked engagement, while some gamified the experience.
Enhanced customizability for users with disabilities was desired in many cases.
Opportunities
Lack of visual hierarchy or contrasting colors affected clarity.
Chat feature in some products became ineffective due to message overload.
Attempting to address multiple issues resulted in a useless outcome.
Weeknesses
Some products automated irregularity detection in movement.
Interactive maps enhanced the sense of security in many products.
A diverse range of input options was available initially.
Strength
My Shake app
Citizen app
Fixmystreet.
brussels
Reportesmed app
How have apps/services facilitated the transfer of information to organizations?
How have similar apps incentivized engagement to report?
What new market trends or technological advancements are my competitors leveraging?
Goal of the analysis
Competitive analysis
Initially, we faced uncertainty about the competitors in the space and the potential benefits of conducting a competitive analysis. However, we decided to break down the analysis into various services and prioritize their key features, particularly focusing on education, identification, and communication of risks. Fortunately, conducting the competitive analysis proved to be highly beneficial. Among the 45 products and services we examined, we were able to identify a few that sparked the leading idea for our final product.
Apps
Apps
Onde Tem
Tiroteio
Fall
Detection
Notifications
Instructions on risk assessment
Input options
Some form of autonomy
Visual clarity and hierarchy
Inclusivity
Apps
Apps
FEMA app
Redes Socias
My Shake
Notifications
Instructions on risk assessment
Input options
Some form of autonomy
Visual clarity and hierarchy
Inclusivity
Apps
Apps
Reportemed
SeeClickFix
mPERS
Device
Notifications
Instructions on risk assessment
Input options
Some form of autonomy
Visual clarity and hierarchy
Inclusivity
Biggest Setback
Biggest Setback
We put too much value on a the wrong group of people at the wrong time. During a significant part of our research process, my partner and I gained a profound understanding of the value and true essence of empathy. Initially, our plan was to engage with as many victims as possible, aiming to grasp their firsthand experiences and learn how the system could have better prevented such catastrophes. However, after facing several denials, we realized the delicate nature of the subject. It became evident to us that empathy could be cultivated not only through direct interaction with the afflicted but also through comprehending the experiences and efforts of other stakeholders. This included those working to combat the issue, individuals living with the constant fear of imminent danger, as well as activists and researchers dedicated to preventing such incidents from recurring.
We put too much value on a the wrong group of people at the wrong time. During a significant part of our research process, my partner and I gained a profound understanding of the value and true essence of empathy. Initially, our plan was to engage with as many victims as possible, aiming to grasp their firsthand experiences and learn how the system could have better prevented such catastrophes. However, after facing several denials, we realized the delicate nature of the subject. It became evident to us that empathy could be cultivated not only through direct interaction with the afflicted but also through comprehending the experiences and efforts of other stakeholders. This included those working to combat the issue, individuals living with the constant fear of imminent danger, as well as activists and researchers dedicated to preventing such incidents from recurring.
Biggest Setback
We put too much value on a the wrong group of people at the wrong time. During a significant part of our research process, my partner and I gained a profound understanding of the value and true essence of empathy. Initially, our plan was to engage with as many victims as possible, aiming to grasp their firsthand experiences and learn how the system could have better prevented such catastrophes. However, after facing several denials, we realized the delicate nature of the subject. It became evident to us that empathy could be cultivated not only through direct interaction with the afflicted but also through comprehending the experiences and efforts of other stakeholders. This included those working to combat the issue, individuals living with the constant fear of imminent danger, as well as activists and researchers dedicated to preventing such incidents from recurring.
The Design
The Design
Principals i set based on my research were:
Has to be extremely accessible and be WCAG AA/AAA certified
Has to have extremely short flows.
Has to overdo its expectations to actually be effective
Wireframes and User tests
Wireframes and User tests
1
3
4
Autoill is mandatory
2
IncludedStep by step
Most of our elderly users/ felt lost once first entering the app. They didn't know where to start nor what to do.
My colleague mentioned one of the new WCAG 2.2 requirements regarding autofill functionality. Implementing autofill in this app would greatly improve reporting speeds and reduce cognitive load for users.
The initial wireframes placed the ability to change between report types on the reporting screen itself. However, to avoid accidental privacy invasions, this feature was relocated to the settings screen.
Account setting was re organized
Users thought the 360° reporting voiceover would be jumbled due to the UI hierarchy
We though the more touching points the better. We were wrong
We minimized actions and focused on users only having their thumb on one button so itl easily be said to them
2
IncludedStep by step
Most of our elderly users/ felt lost once first entering the app. They didn't know where to start nor what to do.
3
Autoill is mandatory
My colleague mentioned one of the new WCAG 2.2 requirements regarding autofill functionality. Implementing autofill in this app would greatly improve reporting speeds and reduce cognitive load for users.
4
The initial wireframes placed the ability to change between report types on the reporting screen itself. However, to avoid accidental privacy invasions, this feature was relocated to the settings screen.
Account setting was re organized
1
We though the more touching points the better. We were wrong
We minimized actions and focused on users only having their thumb on one button so itl easily be said to them
Users thought the 360° reporting voiceover would be jumbled due to the UI hierarchy
The Final Products.
The Final Products.
Click here to test out the prototype
Click here to test out the prototype
Click here to test out the prototype
The Style Guide
Header 2 (Semibold) 16 pt
Header 1 (Medium) 22 pt
Main body text (Medium) 16 pt
Sec. body text (Regular) 14 pt
Button (Medium) 18 pt
Primary
Colors
Accent
Logo
Components
Application icon
Typography
8 px layout grid
60 px layout grid
4 column grid /
16 px margins
Grid + Spacing
Onboarding illustration
The Style Guide
Header 2 (Semibold) 16 pt
Header 1 (Medium) 22 pt
Main body text (Medium) 16 pt
Sec. body text (Regular) 14 pt
Button (Medium) 18 pt
Primary
Colors
Accent
Logo
Components
Application icon
Typography
8 px layout grid
60 px layout grid
4 column grid /
16 px margins
Grid + Spacing
Onboarding illustration
Primary
Colors
Accent
8 px layout grid
60 px layout grid
4 column grid /
16 px margins
Grid + Spacing
Application icon
Logo
Components
Header 2 (Semibold) 16 pt
Header 1 (Medium) 22 pt
Main body text (Medium) 16 pt
Sec. body text (Regular) 14 pt
Button (Medium) 18 pt
Typography
Onboarding illustration
The Style Guide
Things I’d do differently
Things I’d do differently
Find empathy in different ways - Through our project, TERRAFIRME, we gained valuable insights into the search for empathy. However, we underestimated the challenges victims face in sharing their stories. In our future projects, we aim to identify all true stakeholders and prioritize learning from them. When interviewing victims, we intend to engage them during the final stages of research, ensuring we approach their experiences with extensive knowledge. This approach will allow us to better understand and respect their perspectives.
Id spend more time perfecting - During the course of this project, we experienced significant gaps between our meetings due to initial time constraints that were later eased, we would go for weeks without communicating with our contractors. Nevertheless, the work continued and was refined. However, I believe that with the additional time, we could have taken it to a higher level of excellence and achieved even greater results.
More user testing with individuals with disabilities- While we had the opportunity to conduct user tests with individuals with disabilities, we firmly believe that there can never be enough of such tests. The first two rounds of testing provided valuable insights, and I strongly believe that conducting four or more rounds would have ensured the highest possible levels of accessibility.
