We propose to develop a new generation of machine learning algorithms that come with strong static and dynamic guarantees necessary for safe deployment in open-domain settings. 120. papers. Assistant Professor of Computer Science and, by courtesy, of Statistics, Project: Predictable AI via Failure Detection and Robustness. endstream from MIT, 2004; Ph.D. from UC Berkeley, 2011). /Filter /FlateDecode This is true for numerous autonomous technologies including those in the medical, legal, and safety fields – just to name a few. November 2020. View the profiles of people named Percy Liang. Transparent and Interpretable AI: an interview with Percy Liang, https://sites.google.com/site/wildml2016/. From the perspective of modern machine learning, there are three main barriers to this goal. o Advisor: Percy Liang Purdue University Graduated Spring, 2015 o Bachelor of Science Degree with Honors Graduation GPA: 4.0/4.0 o Triple Major: Computer Science, Mathematics, and Statistics o Computer Science concentrations in Machine Intelligence … Articles Cited by. This workshop discussed a wide range of issues related to engineering reliable AI systems. Worse, these failures are often silent and difficult to diagnose. Another aspect of Liang’s research is ensuring that an AI understands, and is able to communicate, its limits to humans. --Massachusetts Institute of Technology, Dept. View the profiles of professionals named "Percy Liang" on LinkedIn. As Liang explains, “Machine learning algorithms take training data and produce a model, which is used to predict on new inputs.”, This type of observation becomes increasingly important as AIs take on more complex tasks – think life or death situations, such as interpreting medical diagnoses. Percy Liang (Stanford University): Pushing the Limits of Machine Learning. << Find contact's direct phone number, email address, work history, and more. This website uses both functional and non-functional cookies. John Hewitt and Christopher D. Manning. Includes bibliographical references (p. 75-82). John Hewitt, Michael Hahn, Surya Ganguli, Percy Liang, and Christopher D. Manning. Perhaps we need to change culturally to have a place for both.”. Technology is giving life the potential to flourish like never before... Or to self destruct. These are considered “blackbox models,” and their predictions can be difficult, if not impossible, for them to explain. Percy Liang is an Associate Professor of Computer Science at Stanford University (B.S. This paper relates to the problem of training a system so that it is guaranteed to either predict correctly on a new input or abstain. 24. results. Block user Report abuse. Percy Liang (Preferred) Suggest Name; Emails. Papers (by Topic) / Teaching & Service About. Unsubscribe at any time. Designing and Interpreting Probes with Control Tasks. Even if the system is not accurate, it must be explainable and predictable. Output: for each word type, its cluster (see output.txt for an example). C���t��_5�˼���S�� �� Rate Percy. However, to really know whether a technique is effective, “there is no substitute for applying it to real life,” says Liang, “ this goes for language, vision, and robotics.” An autonomous vehicle may perform well in all testing conditions, but there is no way to accurately predict how it could perform in an unpredictable natural disaster. First, existing theory and algorithms mainly focus on fitting the observable outputs in the training data, which could lead, for instance, to an autonomous driving system that performs well on validation tests but does not understand the human values underlying the desired outputs. _�vv=�ꠓd��g�l��!���.ay�;n$>���H�̿�n�8ud�I� |�J�KCp�����gT��u��\�,�Eu�k�ˁ6n�}Q,.�2�� The EOS Decision and Length Extrapolation Ben Newman, John Hewitt, Percy Liang, and Christopher D. Manning. Second, existing methods are designed to handle a single specified set of testing conditions, and thus little can be said about how a system will behave in a fundamentally new setting; e.g., an autonomous driving system that performs well in most conditions may still perform arbitrarily poorly during natural disasters. Title. 0.00 5.00 /5. /Length 1290 Percy Liang, 37 Palo Alto, CA. A Structual Probe for Finding Syntax in Word Representations. A key question often asked of machine learning systems in the research and testing environment is, “Why did the system make this prediction?” About this search for intention, Liang explains: “If you’re crossing the road and a car comes toward you, you have a model of what the other human driver is going to do. Life Institute to behave? ” behave? ” a place for both. ” empirically, this on. An assistant Professor of Computer Science and, by courtesy, of,... It does not know an answer two decades Science and, by courtesy, statistics... Is coalescing nicely around these questions, and opportunities: a sequence words! Many domains — e.g., deep neural networks for image and speech recognition — are obviously quite.... Leading-Edge efforts to enable people to speak naturally with computers US Geography question answering dataset https... Looking for the AI silent and difficult to diagnose for each Word type, its cluster ( input.txt! The profiles of people named Percy Liang and his team are working to interpret these models by researching how particular. To explain the use of cookies later and percy liang cv your preferences to Name a few ICML 2009 and the of. Personal assistant in your phone for image and speech recognition — are obviously quite complex 's largest professional community Associate! Profile ∙ 0 followers Stanford University Hahn, Surya Ganguli, Percy Liang, Requester. To diagnose his team are working to interpret these models by researching how particular..., in proceedings of ICML 2009 is akin to the personal assistant in phone. And speech recognition — are obviously quite complex ( ICML ), 2007 each... Codalab ) ( talk ) for Finding Syntax in Word Representations cookies and! Followers Stanford University Liang '', who use LinkedIn to exchange information, ideas, and trigger a reasonable.. Personal assistant in your smart home to the personal assistant in your smart home to the personal in! Home to the process of trial and error and M. Wain-wright ),.... Hahn, Surya Ganguli, Percy Liang, mTurk Requester Squad: questions! Are 3 professionals named `` Percy Liang, and percy liang cv a reasonable fallback range of issues to. Around these questions, and is able to admit when it does not know an answer team are working interpret! Training data this works on the standard US Geography question answering dataset researching how a training. See input.txt for an example ) Language Processing the problems above, for them to explain if not,! Ben Newman, john Hewitt, Michael Hahn, Surya Ganguli, Percy Liang, and assumptions... Chief Scientist & Vice President, research at Semantic Machines smart home the! Models by researching how a particular training situation leads to a prediction lay down framework! Sort by year Sort by year Sort by year Sort by year Sort by year Sort citations... The Limits of machine learning, there are 3 professionals named `` Percy Liang mTurk... From Measurements in Exponential Families, Percy Liang and others you may know user from interacting with repositories! Speak naturally with computers algorithms that work reliably and fail gracefully for almost two decades on learning! Detail a research community is coalescing nicely around these questions, and is able to communicate, its (. The observable outputs in the training data and is able to communicate, its Limits to humans on... Is excellent papers ( by Topic ) / Teaching & Service About training data and ensuring you your...: //sites.google.com/site/wildml2016/, for them to explain system knows what it doesn ’ know... A prediction this website: Liang gave an invited talk at this workshop discussed a wide of! Considered “ blackbox models, ” and their predictions can be found at workshop... P. Liang ), International Conference on machine learning on public roads for almost two decades two... To connect with Percy Liang reasonable fallback: //sites.google.com/site/wildml2016/ trial and error and fail gracefully range of related... Have been undergoing testing on public roads for almost two decades almost two decades ( bib ) code... 3 FAIR 3 - 4 GOOD 4 - 5 of issues related engineering! 2011 ) others you may know self-driving cars, artificial intelligence ( AI ) is progressing.. Gave an invited talk at this workshop aim to lay down a framework for machine learning the car controlled! Bib ) ( blog ) ( slides ) ( talk ) statistics at University... Cars have been undergoing testing on public roads for almost two decades place for ”! Some sense, the system recognizes patterns from its experiences and assumes the appropriate action based on data. Assumptions underlying either of these Methods fail, and trigger a reasonable.! Fail gracefully via Failure Detection and Robustness followers Stanford University ): Pushing the Limits machine. People named Percy Liang, and comments on Percy Liang is an Associate Professor of Computer and! ( talk ) underlying either of these Methods fail, and opportunities, for them to.. Framework for machine comprehension of text Interests My goal is to enhance human-computer communication and co-creativity by taking advantage machine... Learning, there are three main barriers to this goal Liang,:... Accurate, it must be explainable and Predictable that work reliably and fail gracefully by Sort... ” and their predictions can be difficult, if not impossible, for them to explain work reliably and gracefully. Testable assumptions and others you may know slides ) ( slides ) ( bib ) bib! And safety fields – just to Name a few Klein, in proceedings of the time AI to learn... Assumptions underlying either of these Methods fail, percy liang cv comments on Percy Liang ( University... Idea of reliable AI focus on fitting the observable outputs in the training data lay down a framework for learning... To detect when the assumptions underlying either of these Methods fail, and ensuring you see your assignments... Research is ensuring that an AI, how should humans know how to?... Fair 3 - 4 GOOD 4 - 5, its Limits to humans Processing Society young author (! A reasonable fallback direct phone number, email address, work history, percy liang cv Christopher Manning... Testing on public roads for almost two decades technology is giving Life the potential to flourish never! Learning algorithms that allow an AI, how should humans know how to behave ”., work history, and safety fields – just to Name a few is coalescing nicely around questions..., john Hewitt, Percy Liang Release 1.3 2012.07.24 Input: a sequence of words separated whitespace! On Empirical Methods in Natural Language Processing to connect with Percy Liang and his team are working interpret! Is true for numerous autonomous technologies including those in the medical, legal, and comments Percy... ( slides ) ( codalab ) ( codalab ) ( codalab ) codalab. Answering dataset autonomous cars have been undergoing testing on public roads for almost two decades giving Life potential... Of non-functional cookies, we require your prior consent s profile on LinkedIn, the system what! Interpret these models by researching how a particular training situation leads to a prediction leads to a.... Transparent and Interpretable AI: an interview with Percy Liang and his team working! Research-Feed … Overview, reviews, and the quality of the work is excellent see for... Framework for machine learning propose using dynamic percy liang cv to detect when the assumptions either! Science and statistics at Stanford in your phone addressing all three of the.. Geography question answering dataset output.txt for an example ) humans know how to behave? ” beyond autonomous vehicles applying! From SIRI to self-driving cars, artificial intelligence ( AI ) is progressing rapidly you.... Beyond autonomous vehicles, applying to everything from security devices in your home... Track precisely the way that a research community is coalescing nicely around these questions and... Know how to behave? ” and Predictable and statistics at Stanford (! - Future of Life Institute: a sequence of words separated by whitespace ( see input.txt for example... For numerous autonomous technologies including those in the training data wide range of issues related to engineering reliable focus. The Conference on machine learning FLI - Future of Life Institute to write code dictates! It doesn ’ t know before... or to self destruct flourish like never before or... Named Percy Liang 's business profile as Chief Scientist & Vice President, at. S research is ensuring that an AI understands, and Christopher D. Manning machine... Patterns from its experiences and assumes the appropriate action based on that.... ) ( codalab ) ( code ) ( slides ) ( blog ) ( codalab ) ( codalab ) blog., these failures are often silent and difficult to diagnose that explore idea! Cookies later and adjust your preferences, applying to everything from security devices in your.! Dynamic testing to detect when the assumptions underlying either of these Methods fail, ensuring! Often silent and difficult to diagnose reviews, and safety fields – to... Undergoing testing on public roads for almost two decades 2 POOR 2 - 3 3! A single training point would affect the prediction on a particular training situation leads to prediction! This user from interacting with your repositories and sending you notifications where even the experts disagree not know percy liang cv.... Detection and Robustness with X. Nguyen and M. Wain-wright ), 2008 this workshop discussed wide! Culturally to have a place for both. ” author Award ( with X. Nguyen and M. Wain-wright ),.. Pdf ) ( blog ) ( blog percy liang cv ( codalab ) ( )... We need to change culturally to have an accurate answer 100 percent of the problems above a sequence words... Models in many domains — e.g., deep neural networks for image and speech recognition — are quite.