Music Clash - 1 Fought the Law and the Law Won Life Inside a Skinner Box Confronting our I Future of Automated Law Enforcement Greg Conti gregory. conti@usma. edu Lisa Shay lisa. shay@usma. edu Woody Hartzog whartzog@samford. edu Disclaimer The views in this talk are the authors' and don't reflect the official policy or position of the United States Military Academy, Samford University, the Department of the Army,the Department of Defense, or the United States Government. Why? We don't want to live in a police state http://www.nationalguard.mil/news/archives/2008/09/images/090808-secuirty_forces-full.jpg Meditation on the Law Driving the Speed Limit on the Atlanta Perimeter http://www.youtube.com/watch?v=1B-OxOZmVIU Three Points 1 . Networked technologies have the potential to automate enforcement of the law. 2. Reckless and pervasive automation can have disastrous and unintended consequences. 3. This audience is in a unique position to defend society and appropriately confront law enforcement systems. Automated Law Enforcement Any computer-based system that uses input from unattended sensors to algorithmically determine that a crime has been or is about to be committed then takes some responsive action, such as to warn the subject or inform the appropriate law enforcement agency. Additionally, these systems will be capable of automatically imposing some form of punishment. Our proposed conceptualization of automated law enforcement is based on a model that is subject to automation at various points. The Precursors are in Place Increased Proliferation of Sensors Increased Diversity and Sensitivity of Sensors State Police Pull Over CT Medical Patient for Tripping Radioactivity Detector http://wwwxtpostxom/default/article/Radioactive-man-Milford-resident-pulled-over-by-3549631.php#photo-2920568 Compulsory Sensing US Senate Passes Bill Requiring All New Cars in America to Come Equipped with a Black-box by 2015 http://venturebeat.com/2012/04/19/black-boxes-for-cars/ httn://en.\viki jxxlia.org/uiki/File: Head On_ CollKion.jp;; Mobile Sensors Honeywell T-Hawk "Intended to provide the dismounted soldier with Reconnaissance, Surveillance, and Target Acquisition (RSTA) and laser designation. Total system weight, which includes the air vehicle, a control device, and ground support equipment is less than 51 pounds (23 kg) and is back-packable in two custom MOLLE-type carriers." http://en.wikipedia.org/wiki/File:Class1 Soldiers2.jpg http://en.wikipedia.org/wiki/Honeywell_RQ-16_T-Hawk http://www.wsvn.com/news/articles/local/21 0031 981 89967/ 3 Billion+ Mobile Devices Connected Cars OnStar http://www.blogcdn.com/www.autoblog.com/media/2011/09/onstar-630.jpg http://app.ddot.dc.gov/services_dsf/traffic_cameras/map_10.asp We are Losing Control of our Technology So when I get into a car — a computer that I put my body into — with my hearing aid* — a computer I put inside my body — I want to know that these technologies are not designed to keep secrets from me, or to prevent me from terminating processes on them that work against my interests. - Cory Doctorow * Note that this quote is taken a bit out of context. Mr. Doctorow was speculating that he may require a hearing aid at some point in the future. http://boingboing.net/2012/01/10/lockdown.html http://www.youtube.com/watch?v=HUEvRyemKSg Facial Recognition http://wwwxngrAlu.edu/iiinKilKHii7nicturcs/facial_feature.JPG See also "Faces of Facebook: Privacy in the Age of Augmented Reality," Alessandro Acquisti, Ralph Gross, and Fred Stutzman. BlackHat USA, 201 1. Crowdsourced Identification Identify the rioters ■te! de ~ people behind the Vancouver riots on Wednesday, June 15th Can you identify the (alleged) rioter or looter in this photo? POLICE LINE DO NOT CROSS POLICE LINE DO NOT CROSS POLICE LINE DO NOT CROSS Vancouver Riots www.identifyrioters.com Improved Algorithms Cordon: 32 Vehicle Tracking Across 4 Lanes http://www.peakgainsystems.com/en/cordon.html http://www.engadget.com/201 1/1 0/31 /cordon-multi-target-photo-radar-system-leaves-no-car-untagged-v http://tech.slashdot.org/story/11/10/31/1846223/multi-target-photo-radar-system-to-make-speeding-riskier Crowdsourced Crime Detection http://interneteyes.co.uk/ Tech Transfer Miami-Dade Police Department's Drones Ready to Fly http://www.wsvn.com/news/articles/local/21 0031 981 89967/ http://www.nbcmianii.con]/nc\\'>/l(K\il/\liaiiii-[)adc-P()licc-DcpaifnicnlN-Dr(>ncs-Rc;!d\'-T() Fly- 1 37434223 .html Augmented Reality Google's Project Glass: One Day... http://www.youtube.com/watch?v=9c6W4CCU9M4 http://www.wsvn.eom/news/articles/local/2 1 003 1 98 1 89967/ http://www.nbcmianii.con]/nc\\'>/l()cal/\li;!iiii-[)adc-P(ilicc-Dcpai1nicnls-Dr(>ncs-Rc;!d\ To-Fly-137434223.html Conversion of Analog Systems to Digital http://www.bobspropshop.com/img/2015.jpg Increased Analytics Wide Area Motion Imagery (WAMI) See also Persistent Stare Exploitation and Analysis System (PerSEAS) http://defense-update.com/features/2010/july/persistent_wide_area_surveillance_04072010.html http://www.kmimediagroup.com/mgt-home/345-gif-2011-volume-9-issue-6-september/4656-wide-area-intelligence.html Reid Porter, Andrew Fraser, and Don Hush. "Wide-Area Motion Imagery." IEEE Signal Processing Magazine, September 2010 Increased Linkages National Identity Registries Location Tracking PI Get The Best of Both Worlds With Our Top Two Products Live GPS i Spark Nano GPS Tracker • Live real-time reporting • 5 day battery life • SMS / e-mail alerts Small GPS Logger The Flash GPS Reporter • 90-Day Swappable, Rechargeable Battery * Most Storage with Over 240,0000 locations recorded • Speed Notification Reports * Works with Google Earth http://www.lightninggps.com/ Predictive Policing is on the Rise California Police Praise Predictive Policing http://www.officer.com/video/10604045/california-police-praise-predictive-policing See also Samuel Greengard's "Policing the Future" in the March 201 2 Communications of the ACM Lots of Interested Parties Federal, State, and Local Law Enforcement Insurance Companies * Note that we aren't stating that these specific entities are necessarily interested in Automated Law Enforcement, but in general we argue such types of entities are. http://en.wikipedia.org/wiki/Oregon_State_Police htl] i ! i it | i i i I ice-bad http://en.wikipedia.0rg/wiki/File:BIA_Police_Officer_Badge.jpg Well Intentioned Officials New York City Soda Ban The Daily Show, 31 May 2012 http://eater.com/uploads/jon-stewart-daily-show-soda-ban.jpg Well Intentioned Law Enforcement. ©EJ\ EBI»...™..,™..„0 Communities Against Terrorism Potential Indicators of Terrorist Activities Related to Internet Cafe lea on Computer indicate; u>\\:::u. \,: Ili.L .1.1. ill Ikil. I I'll I -. .1.. iiiiiiik;LlKHisusiiiL- VOIE'iir^ Liiikiliiip iIiiuli.-Ii ;l EV l-iiiw Joint Regional Intelligence Center i.JRICi www.jrfc.org (888) 705-JRIC (5742) mention "Tripwire" Willi out drawing attention to yourself /// Provides configured video device. /// /// /// The property provides configured video device if user confirmed /// the dialog using "OK" button. If user canceled the dialog, the property is /// set to . /// public VideoCaptureDevice VideoDevice { get { return videoDevice; } } Source Code from iSpy Camera Security Project http://ispysoftware.svn. sourceforge.net/viewvc/ispysoftware/iSpyApplication/VideoSource. cs?revision=343&content-type=text%2Fplain Confinement The New Generation III Alcohol and Marijuana Sensing House Arrest Ankle Bracelet With Active GPS Is Now Available for only $3.00 per day! http://www. Confinement The New Generation III Alcohol and Marijuana Sensing House Arrest Ankle Bracelet With Active GPS Is Now Available for only $3.00 per day! Electronic Monitoring Just Got Better! http://www. Execution http://robocoparchive.com/old/lobby5.JPG Subject 1 aw Fnfnrrpmpnt LOW 1 — 1 1 1 U 1 1 1 IC 1 11 Agency Judicial System Advantages • Safer more secure society • Reduced Bias • Safer more secure society • Community financial incentives •Increased and more effective enforcement • Potential efficiency gains • Safer more secure society • Potential efficiency gains w CD £ > "D CC CO b • False positives •Chilling effect •Societal harm •"Perfect" enforcement •Misidentification •Enforcement of minor infractions •Creating infrastructure for abuse •Security of ALE systems • False positives •Loss of discretion •Lost jobs •Less compliant populace •Loss of "professional courtesy" •Sensor data as legal defense • False positives •Potential overload Advantages and Disadvantages More Secure Society http://en.wikipedia.org/wiki/File:Flughafenkontrolle.jpg More Secure Society http://en.wikipedia.org/wiki/File:Security_spikes_1 .jpg Safer Society CAREFUL THE BEST SAFETY DEVICE IS A CAREFUL WORKER, GET THE SAFETY HABIT http ://www. stolaf.edu/com m ittees/saf ety/ Increased Efficiency http://www.bamesandnoble.eom/w/stepping-stones-to-police-efficiency-wa-gayer/1102723835 Financial Incentives http:/A\ w\\ .ij.org/policinL: ■for-profit -rhc-ahuse-ot' civil .asset torteiture Reduce Bias Respect the badge He earned it with his blood Fear the gun Your sentence may be death because I am THE LAW! You won't f**k around no more I am THE LAW! I judge the rich, I judge the poor lam THE LAW! Commit a crime, I'll lock the door I am THE LAW! Because in Mega-City lam THE LAW! - Anthrax (Judge Dredd/ 1 am the Law) http://en.wikipedia.org/wiki/File:2000AD1 68.jpg Protection from Abuse Dallas set to pay $1 M for two Separate Police Abuse Cases http://cityhallblog.dallasnewsxom/2012/06/dallas-set-to-pay-1m-for-two-separate-police-abuse-cases.html/ Increased Opportunity for Abuse McCarthy-era Propaganda Comic Book http://en.wikipedia.org/wiki/McCarthyism Increased Opportunity for Abuse http://www.fugly.com/media/IMAGES/Funny/speed_trap_ahead.jpg False Positives Please put down your weapon, you have 20 seconds to comply... You now have 1 5 seconds to comply I am now authorized to use physical force ED-209 Dick, I am very disappointed... Sir, I'm sure its only a glitch... http://www.youtube.com/watch?v=A9l9wxGFI4k False Negatives Misidentification Terror List Snag Nearly Grounded Ted Kennedy "If they have that kind of difficulty with a member of Congress, how in the world are average Americans, who are getting caught up in this thing, how are they going to be treated fairly and not have their rights abused?" - Senator Ted Kennedy ]ittn://sans;]h';]d(ir.uscmhassy.go\/lcdkcnncdy _500\500_articulo.jp!j http://www.usatoiUiy.coni/ncw s/wasli inglon/20( )-l -ON- 1 <-> kcnnedy-list_x.htm Societal Harm http://commons.wikimedia.org/wiki/File:Skrzy1_Skin1 .jpg Chilling Effects http://t'ilipspai;ii(ili.t'ilo'i.\\ORlpro'is.coni/200y/()l/free-speech2.jpg Loss of Human Discretion Equal and Fair Enforcement of Law Risk of Unproportional Response http://de-motivational-posters.com/images/texas-speed-trap-one-of-the-reasons- why-you-don-39-t-mess-with-texas.jpg Equal and Fair Enforcement of Law Crack Cocaine http://en.wikipedia.org/wiki/Cocaine http://en.wikipedia.org/wiki/Crack_cocaine Perfect Enforcement of the Law No Jaywalking http://en.wikipedia.org/wiki/File:Singapore_Road_Signs_-_Restrictive_Sign_-_NoJaywalking. svg Social Control ht(| ci ikipedia.oi ?1 i 1 . i il ontro] http://en.wikipedia.0rg/w iki/Fde: Summerfairesign.JPG Loss of Power Montgomery County Maryland Police "Photographed Speeding Past Camera with Extended Middle Finger" http://thecafebelle.com/forum/topics/dont-black-and-drive-in-texas http://www.theblaze.com/stories/police-stop-batman-for-improper-car-tags/ http://www.aclu.org/racial-justice/driving-while-black-racial-profiling-our-nations-highways Lost Jobs II NYC PBA > • f PATROLMEN'S BeNGVOLGNT ASSOCIATION W What's New Benefits *> Contract Employment HelD Save Jhbbw 1 1 1 PBA in the News Political Action Li liana PBA Press Releases Outside Links ■b^LEhI PBA Publications Photo Gallery M — Complaint Procedure HR-21B Amendment LOD Prescription Proa ram PBA in the Community Offers & Discounts ETTTTTW? 1 H From Pat Lynch World Trade Center BaiULaOfeeW. W* General Counsel In Memoriam y<* If? Widows & Children's Fund Contact Us || re — 1 Pension Tan Info | Health/Safety Violation Form FM LA — Amendments http://www.nycpba.org/index-flash.html Overflowing the System Case Studies http://en.wikipedia.org/wiki/File:Hwy_11_Ontario_Winter.JPG Driving Case Study Speeding Windshield wipers without lights in rain Foglights without fog algorithmic specifications for laws built in time constraints, warning periods, higher limits? driving the speed limit will get you run off the road in most major metropolitan areas license plate identification emergency override? remote disable by police?, on star, manufacturer, change programming of engine chip, corvette with key to disable 4 of eight cylinders requirement to broadcast laws? (like marking w/speed zones, school zones with signs, "speed zone ahead" security vulnerabilities in ALE systems, failure modes, possibilities for misuse ignorance of law is no excuse internal sensors (limited knowledge of local laws? centrally fed? updated? validated? external sensors (like a speed camera) may have localize knowledge? law DB, updates? validated how? what legal loopholes/defenses will emerge for some areas technology isn't ready to enforce laws, but in some areas it is rental cars (you don't own it, therefore you rights are less) road use tolls (in news) role of easy pass like systems role of onstar Case Law / Legal Defenses Legal defenses against ALE systems (look to radar guns, red light cameras, speed cameras) Calibration Radar vs GPS tracking device http://www.csmonitor.com/USA/2009/091 1/p02s01 -usgn.html Windshield Wiper Before Headlights Mainstream Media Questions Necessary to Confront Automated Law Enforcement Method of Implementation Control Discretion Perfection of Enforcement Legal Integration of Algorithms Preemptive / Post Hoc Enforcement System Error / Malfunction Administrative Burden Procedural Safeguards The Social Cost of Automation Analysis Procedural Safeguards Due Process Privacy Freedom of Expression The Necessity Defense Transparency http://www.freerepublic.com/focus/f-chat/2749860/posts Countermeasures For more see our HOPE 9 talk Detect Monitor Bypass Shield Exploit sampling rate Scattering Disable Destroy Jam Spoof Camouflage DOS and more. Competing Sensors Algorithmic Laws? 15 Time, minutes Emerging Brain Computer Interfaces See... • Brain-Computer Interfaces - Dennis McFarland and Jonathan Wolpaw. "Brain-Computer Interfaces for Communication and Control." Communications of the ACM, May 2011, Vol. 54, No. 5, pp.60-66. • Feed by M. T. Anderson • My Second Implant by Estragon, 2600, Summer 2010 The OODA Loop Applies Observe Orient Decide Implicit Guidance & Control Act 1 Feed Action Forward (Test) John Boyd s OODA Loop http://en.wikipedia.org/wiki/File:OODA.Boyd.svg Law Resistant Systems • Please include a manual override Security of ALE Systems • Vulnerabilities in Car Electronics • CALEA wiretap infrastructure abuses • http://www.forbes.com/201 0/02/03/hackers- networking-equipment-technology-security- cisco.html (probably should cite Tom's BH talk instead) • Whit Diffie, Susan Landau http://queue.acm.org/detail.cfm?id=1 61 31 30 Research Agenda • Metrics to Assess Risk/Benefit • Design for Transparency • Design for Accountability This isn't Going to Happen Overnight Three Points (redux) 1 . Networked technologies have the potential to automate enforcement of the law. 2. Reckless and pervasive automation can have disastrous and unintended consequences. 3. This audience is in a unique position to defend society and appropriately confront law enforcement systems. For More Information... Lisa Shay, Dominic Larkin, John Nelson, and Gregory Conti. "A Framework for Analysis of Quotidian Exposure in an Instrumented World." IEEE International Carnahan Conference on Security Technology, October 2012. (accepted, to be published) Lisa Shay and Greg Conti. "Countermeasures: Proactive Self-Defense Against Ubiquitous Surveillance." HOPE 9, July 2012. Lisa Shay, Woodrow Hartzog, John Nelson, Dominic Larkin and Gregory Conti. "Confronting Automated Law Enforcement." We Robot, April 2012. Greg Conti. "Our Instrumented Lives: Sensors, Sensors, Everywhere." DEFCON 18, July 2010. We'd also like to thank John Nelson and Dominic Larkin for their support of this work. For More Information... Lisa Shay, Dominic Larkin, John Nelson, and Gregory Conti. "A Framework for Analysis of Quotidian Exposure in an Instrumented World." IEEE International Carnahan Conference on Security Technology, October 2012. (accepted, to be published) Lisa Shay and Greg Conti. "Countermeasures: Proactive Self-Defense Against Ubiquitous Surveillance." HOPE 9, July 2012. Lisa Shay, Woodrow Hartzog, John Nelson, Dominic Larkin and Gregory Conti. "Confronting Automated Law Enforcement." We Robot, April 2012. Greg Conti. "Our Instrumented Lives: Sensors, Sensors, Everywhere." DEFCON 18, July 2010. We'd also like to thank John Nelson and Dominic Larkin for their support of this work. Questions? http://www.lobsterhelp.com/images/cookinglobster5.jpg