Responsible AI
Materials — open to everyone, no sign-in
Topic: Responsible AI
Presenter: Ying Ying Liu
Additional Resources:
System Design Presentation - Responsible AI
Join Us on Wechat
Subscribe to Our YouTube channel https://commitway.com/eventyoutube
2 key points in responsible AI
Privacy
Fairness
2 definitions of responsible AI
China’s law and regulation on AI
United States’ law and regulation: HIPPA etc
Canada: more detailed regulation than United States, at national, provincial level
China is most advanced in law development
US&Canada: more regulation for government use of AI
Cases
Royal free, deepmind collaboration
Deepmind received personally identifiable information
Clearview AI
Collection of images online
Used by law enforcement
2020 Canada investigated Clearview AI collection methods
Deep fake
Emma
Taylor Swift
Canadian regulation and advices
Legal authority and consent
Appropriate purposes
Necessity and proportionality
Openness
Accountability
Individual access - user can see the process
Limiting collection, use and disclosure
Accuracy
Safeguard
Microsoft AI privacy
Differential privacy
grouping/aggregation
Anonymity vs utility
Adding Laplace distribution noise to raw data
Can be controlled by epsilon
Epsilon increases, utility increases, privacy decreases
Used by Apple, Google and Microsoft
The model performs well.
Need intuition
Need to collaborate across teams and companies
Understand laws and regulations
Feature: name, age, address, occupation, etc
Understand the source of the data
Try to have minimum usage
Should document different points that touch the data
Need to adjust epsilon
Collaboration
Transparent note (microsoft), privacy assessment (canadian government)
Fairness
Unfair treatment
Transparency
Cases for fairness
2007 Amazon AI hiring tool discriminated
Gender classification of people of darker skin color
2019 Racial bias in health scores
Labeling, using healthcare cost as measurement of health
Enhancement: use a combination of health and health care cost
[why not remove the healthcare cost?]
[ Probably due to the fact it’s for health insurance ]
2023 AI index report
LLM shows gender bias
2023 Quebac sues Facebook for ads to discriminate against age and gender
FBPML (foundation for best practices in machine learning)
Fair AI
Currently AI fairness is only on the surface
Religion
Fair machine learning
De-bais data
Fairlearn
How to improve fairness
Genetic programming
Generic algorithm - fixed array
Genetic programming - evolve the code. Using tree
Find optimal algorithm
Symbolic regression
[huge search space?]
Fair ML through Genetic coding
Objective 1: accuracy
Objective 2: fairness
Summary
Fairness is not mature yet, still a huge challenge
Privacy is more mature
Countries may learn from China
Developers:
Should be sympathetic
Follow best practice and tools
Understand the transparency
Transparency note - product deliverable of
[what are typical deliverable]
Human in the loop - subject matter expert can review the process
Structural injustice - human society is unfair - cannot be completely fair, cannot be 100%
Architecture diagram
Transparency note: a master document - small documents (privacy impact assessment)
Audience
Q: Is unfairness due to structural injustice or error in data?
A: There are some standards to measure. Gender and ethnicity
Q: What other aspects are mature: Transparency, useability
A: reliability, safety, security are more mature. Technically solvable problem is more mature
Q: genetic programming
A: Tree node: operator. Leaf node: are input data
A tool for tuning. May accomplish simple tasks
Q: transparency note, anything else?
A: Algorithm design
A collection of documents.
Human in the loop. Who to collaboration
Why fairness? Add business rule Customer service
Privacy: privacy impact assessment. Long document at provincial level
Security
Accountability: racy - matrix who is responsible for what.
Transparency note - workflow for the algorithm
Master document
Operation guide
End-user guide
Budget / expenditure
Most important is security and privacy question
Lots of paperwork for data science projects
Q: do different departments have different opinion about fairness
A: yes. Need communication across departments and compromises