Part of my series of notes from NAACL-HLT 2019 in Minneapolis.
Six Observations on Deception
1. Deception precedes language
- present elsewhere in the animal world
- learning to speak is a kind of lying
- talking about things that are non-existent
2. Deception is very frequent
- frequency changes throughout life (peaks in adolescence)
- people lie about twice a day
3. Deception is very diverse
- many reasons people lie (good or bad…)
4. Humans are very poor deception detectors
- many studies confirming this – people perform close to chance
- relying on weak behavioural cues
- really impacts how we solve this problem, gather data, etc.
- corollary 4.1: standard data annotation strategies don’t work
- other strategies: specifically elicit lies, look for other sources / verification
- corollary 4.2: deception detection isn’t a big data problem
- 100s, maybe 1000s of data points
5. Deception (detection) is central to many domains
- case studies from a decade of research
- The Lie Detector, ACL 2009
- look at word categories present in lies vs. truths
- validates some psycholinguistic behaviours, uncovers some new ones
- Experiments in Open Domain Deception, EMNLP 2015
- instructions to participants: “lie about anything”
- also syntactic markers of deception
- Real-Life Trial Data, ICMI 2015 – multimodal studies using public trial videos
- using all features helps (textual / audio / visual)
- can look at gestures that correspond with lying
- people try harder to convince you when they’re lying
- Identity Deception Detection, IJCNLP 2017
- e.g. gender, age
- easier to detect female and older people lying
- Box of Lies, NAACL 2019
- corollary 5.1: Jimmy Fallon is a better liar than his guests
- more data from a single individual doesn’t seem to help detection
- corollary 5.1: Jimmy Fallon is a better liar than his guests
6. Deception (detection) has many social implications
- systems have 65-75% accuracy
- better than humans, but not ready for prime time
- ethics / bias of use
Q&A
- for AIs to become social, should they have the ability to deceive???
- comparison with polygraphs?
- hard find out how evaluated etc.
- intentional vs. unintentional lying
- misinformation about e.g. health – not true, but not deception
- cultural specificity in markers?