Speaker Schedule (2023)

Date	Speaker	Talk Information	Recommended Reading
4/5	Sylvia Herbert, UCSD	Guaranteeing safe control Value functions have been used extensively for generating safe control policies for robots and other nonlinear systems. The output of the function provides the current “safety level” of the system, and its gradient informs the allowable control inputs to maintain safety. Two common approaches for value functions are control barrier functions (CBFs) and Hamilton-Jacobi (HJ) reachability value functions. Each method has its own advantages and challenges. HJ reachability analysis is a constructive and general method that struggles from computational complexity. CBFs are typically much simpler, but are challenging to find, often resulting in conservative or invalid hand-tuned or data-driven approximations. In this talk I will discuss our work in exploring the connections between these two approaches in order to blend the theory and tools from each. I’ll introduce the “control barrier-value function,” and show how we can refine CBF approximations to recover the maximum safe set and corresponding control policy for a system.
4/12	Sam Bowman, NYU, Anthropic	What’s the deal with AI safety? Motivations and open problems Over the last few years, a research community has been forming to study questions about the impacts of future AI systems with broadly human-level capabilities. This community was initially largely separate from academic ML, with deeper roots in philosophy departments and industry labs. This has started to change, though, with AI safety researchers increasingly focusing on questions about progress in large language models, and with safety-related motivations increasingly steering investments in NLP at large labs like OpenAI and DeepMind. This talk presents the basic goals and projects of the AI safety research community, with a focus on alignment research, large language models, and connections to concerns about present-day deployed language technology.	A recent blog post of Sam’s corresponding fairly closely to the talk, with sources linked The Alignment Problem from a Deep Learning Perspective: A recent paper covering similar topics
4/19	Dewey Murdick, CSET	A few lessons for “AI Development Teams” from a policy analysis shop AI development teams include people with a rich set of skills and experiences that include researchers, system architects, software developers, designers, product managers, business-minded team members, and more. This talk will introduce Georgetown University’s Center for Security and Emerging Technology (CSET) and will highlight relevant insights (for the AI development team) in areas such as AI workforce and education, system design principles, AI system characterization and assessments, AI applications, and policy and national competition issues.	Shaping the Terrain of AI Competition Poison in the Well: Securing the Shared Resources of Machine Learning Reducing the Risks of Artificial Intelligence for Military Decision Advantage Trusted Partners: Human-Machine Teaming and the Future of Military AI Decoupling in Strategic Technologies From Satellites to Artificial Intelligence
4/26	Nigam Shah, Stanford Healthcare	Beyond the model: Considerations in Responsible AI in Healthcare As artificial intelligence (AI) moves from being a luxury to a necessity, it is clear that the benefit obtained from using AI models to prioritize care interventions is an interplay of the model’s performance, the capacity to intervene, and the benefit/harm profile of the intervention. We will discuss the experience from practitioners of AI and data science, who work to ensure that AI tools are useful, reliable, and fair so that they lead to cost-effective solutions that meet health care’s needs.
5/3 [VIRTUAL]	Shai Shalev-Shwartz, Mobileye	On Responsibility-Sensitive-Safety and the ethics of building safe self-driving cars How fast should we drive on a residential road? Driving slower is safer, but hurts the normal flow of traffic. Such safety-usefulness tradeoffs are at the heart of the design of useful self-driving cars. The RSS model aims at providing a formal mathematical language to discuss the safety and ethics of self-driving cars. Time permitting, I’ll discuss the broader “assume-guarantee” approach to safety of AI systems.	On a Formal Model of Safe and Scalable Self-driving Cars
5/10	Sanmi Koyejo, Stanford	Are Emergent Abilities of Large Language Models a Mirage? Recent work claims that large language models display emergent abilities, abilities not present in smaller-scale models that are present in larger-scale models. What makes emergent abilities intriguing is two-fold: their sharpness, transitioning seemingly instantaneously from not present to present, and their unpredictability, appearing at seemingly unforeseeable model scales. Here, we present an alternative explanation for emergent abilities: that for a particular task and model family, when analyzing fixed model outputs, one can choose a metric which leads to the inference of an emergent ability or another metric which does not. Thus, our alternative suggests that existing claims of emergent abilities are creations of the researcher’s analyses, not fundamental changes in model behavior on specific tasks with scale. We present our explanation in a simple mathematical model, then test it in three complementary ways: we (1) make, test and confirm three predictions on the effect of metric choice using the InstructGPT/GPT-3 family on tasks with claimed emergent abilities, (2) make, test and confirm two predictions about metric choices in a meta-analysis of emergent abilities on BIG-Bench; and (3) show how similar metric decisions suggest apparent emergent abilities on vision tasks in diverse deep network architectures (convolutional, autoencoder, transformers). In all three analyses, we find strong supporting evidence that emergent abilities may not be a fundamental property of scaling AI models.
5/17	Aylin Caliskan, UW	Artificial Intelligence, Bias, and Ethics Although ChatGPT attempts to mitigate bias, when instructed to translate the gender-neutral Turkish sentences “O bir doktor. O bir hemşire” to English, the outcome is biased: “He is a doctor. She is a nurse.” In 2016, we have demonstrated that language representations trained via unsupervised learning automatically embed implicit biases documented in social cognition through the statistical regularities in language corpora. Embedding associations in language, vision, and multi-modal language-vision models reveal that large-scale sociocultural data is a source of implicit human biases regarding gender, race or ethnicity, skin color, ability, age, sexuality, religion, social class, and intersectional associations. The study of gender bias in language, vision, language-vision, and generative AI has highlighted the sexualization of women and girls in AI, while easily accessible generative AI models such as text-to-image generators amplify bias at scale. The ethics of AI bias has significant implications for human cognition, society, justice, and the future of AI. Thus, it is crucial to advance our understanding of the depth and prevalence of bias in AI to mitigate it both in machines and society.	Semantics Derived Automatically from Language Corpora Contain Human-like Biases Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biases Gender Bias in Word Embeddings: A Comprehensive Analysis of Frequency, Syntax, and Semantics
5/24	Emma Brunskill, Stanford
5/31	Mac Schwager, Stanford	Safe Robot Planning with Neural Environment Representations New developments in computer vision and deep learning have led to the rise of neural environment representations: 3D maps that are stored as deep networks that spatially register occupancy, color, texture, and other physical properties. These environment models can generate photo-realistic synthetic images from unseen view points, and can store 3D information in exquisite detail. In this talk, I investigate the questions: How can robots use neural environment representations for perception, motion planning, manipulation, and simulation? How can we preserve safety guarantees while using such representations? I will show recent work from my lab in which we build a robot navigation pipeline using a Neural Radiance Field (NeRF) map of an environment. We develop a trajectory optimization algorithm that interfaces with the NeRF model to find dynamically feasible, collision-free trajectories for a robot moving through a NeRF world. We also develop an optimization-based state estimator that uses the NeRF model to give full dynamic state estimates for a robot from only on board images. I will discuss our algorithms for dexterous manipulation using NeRF object models, and will describe our development a differentiable physics simulator that operates directly on NeRF object models. I will conclude with future opportunities and challenges in integrating neural environment representations into the robot autonomy stack.
6/7	Chuchu Fan, MIT	Neural certificates in large-scale autonomy design Learning-enabled control systems have demonstrated impressive empirical performance on challenging control problems in robotics, but this performance comes at the cost of reduced transparency and lack of guarantees on the safety or stability of the learned controllers. In recent years, new techniques have emerged to provide these guarantees by learning certificates alongside control policies — these certificates provide concise, data-driven proofs that guarantee the safety and stability of the learned control system. These methods not only allow the user to verify the safety of a learned controller but also provide supervision during training, allowing safety and stability requirements to influence the training process itself. In this talk, we present an overview of this rapidly developing field of certificate learning. We hope that this talk will serve as an accessible introduction to the theory and practice of certificate learning, both to those who wish to apply these tools to practical robotics problems and to those who wish to dive more deeply into the theory of learning for control.

4/5

Guaranteeing safe control
Value functions have been used extensively for generating safe control policies for robots and other nonlinear systems. The output of the function provides the current “safety level” of the system, and its gradient informs the allowable control inputs to maintain safety. Two common approaches for value functions are control barrier functions (CBFs) and Hamilton-Jacobi (HJ) reachability value functions. Each method has its own advantages and challenges. HJ reachability analysis is a constructive and general method that struggles from computational complexity. CBFs are typically much simpler, but are challenging to find, often resulting in conservative or invalid hand-tuned or data-driven approximations.
In this talk I will discuss our work in exploring the connections between these two approaches in order to blend the theory and tools from each. I’ll introduce the “control barrier-value function,” and show how we can refine CBF approximations to recover the maximum safe set and corresponding control policy for a system.

4/12

Sam Bowman, NYU, Anthropic

What’s the deal with AI safety? Motivations and open problems

Over the last few years, a research community has been forming to study questions about the impacts of future AI systems with broadly human-level capabilities. This community was initially largely separate from academic ML, with deeper roots in philosophy departments and industry labs. This has started to change, though, with AI safety researchers increasingly focusing on questions about progress in large language models, and with safety-related motivations increasingly steering investments in NLP at large labs like OpenAI and DeepMind. This talk presents the basic goals and projects of the AI safety research community, with a focus on alignment research, large language models, and connections to concerns about present-day deployed language technology.

A recent blog post of Sam’s corresponding fairly closely to the talk, with sources linked

The Alignment Problem from a Deep Learning Perspective: A recent paper covering similar topics

4/19

Dewey Murdick, CSET

A few lessons for “AI Development Teams” from a policy analysis shop
AI development teams include people with a rich set of skills and experiences that include researchers, system architects, software developers, designers, product managers, business-minded team members, and more.
This talk will introduce Georgetown University’s Center for Security and Emerging Technology (CSET) and will highlight relevant insights (for the AI development team) in areas such as AI workforce and education, system design principles, AI system characterization and assessments, AI applications, and policy and national competition issues.

Shaping the Terrain of AI Competition

Poison in the Well:
Securing the Shared Resources of Machine Learning

Reducing the Risks of Artificial Intelligence for Military Decision Advantage

Trusted Partners:
Human-Machine Teaming and the Future of Military AI

Decoupling in Strategic Technologies
From Satellites to Artificial Intelligence

4/26

Nigam Shah, Stanford Healthcare

Beyond the model: Considerations in Responsible AI in Healthcare
As artificial intelligence (AI) moves from being a luxury to a necessity, it is clear that the benefit obtained from using AI models to prioritize care interventions is an interplay of the model’s performance, the capacity to intervene, and the benefit/harm profile of the intervention. We will discuss the experience from practitioners of AI and data science, who work to ensure that AI tools are useful, reliable, and fair so that they lead to cost-effective solutions that meet health care’s needs.

5/3
[VIRTUAL]

Shai Shalev-Shwartz,
Mobileye

On Responsibility-Sensitive-Safety and the ethics of building safe self-driving cars
How fast should we drive on a residential road? Driving slower is safer, but hurts the normal flow of traffic. Such safety-usefulness tradeoffs are at the heart of the design of useful self-driving cars. The RSS model aims at providing a formal mathematical language to discuss the safety and ethics of self-driving cars. Time permitting, I’ll discuss the broader “assume-guarantee” approach to safety of AI systems.

On a Formal Model of Safe and Scalable Self-driving Cars

5/10

Sanmi Koyejo, Stanford

Are Emergent Abilities of Large Language Models a Mirage?
Recent work claims that large language models display emergent abilities, abilities not present in smaller-scale models that are present in larger-scale models. What makes emergent abilities intriguing is two-fold: their sharpness, transitioning seemingly instantaneously from not present to present, and their unpredictability, appearing at seemingly unforeseeable model scales. Here, we present an alternative explanation for emergent abilities: that for a particular task and model family, when analyzing fixed model outputs, one can choose a metric which leads to the inference of an emergent ability or another metric which does not. Thus, our alternative suggests that existing claims of emergent abilities are creations of the researcher’s analyses, not fundamental changes in model behavior on specific tasks with scale. We present our explanation in a simple mathematical model, then test it in three complementary ways: we (1) make, test and confirm three predictions on the effect of metric choice using the InstructGPT/GPT-3 family on tasks with claimed emergent abilities, (2) make, test and confirm two predictions about metric choices in a meta-analysis of emergent abilities on BIG-Bench; and (3) show how similar metric decisions suggest apparent emergent abilities on vision tasks in diverse deep network architectures (convolutional, autoencoder, transformers). In all three analyses, we find strong supporting evidence that emergent abilities may not be a fundamental property of scaling AI models.

5/17

Aylin Caliskan, UW

Artificial Intelligence, Bias, and Ethics
Although ChatGPT attempts to mitigate bias, when instructed to translate the gender-neutral Turkish sentences “O bir doktor. O bir hemşire” to English, the outcome is biased: “He is a doctor. She is a nurse.” In 2016, we have demonstrated that language representations trained via unsupervised learning automatically embed implicit biases documented in social cognition through the statistical regularities in language corpora. Embedding associations in language, vision, and multi-modal language-vision models reveal that large-scale sociocultural data is a source of implicit human biases regarding gender, race or ethnicity, skin color, ability, age, sexuality, religion, social class, and intersectional associations. The study of gender bias in language, vision, language-vision, and generative AI has highlighted the sexualization of women and girls in AI, while easily accessible generative AI models such as text-to-image generators amplify bias at scale. The ethics of AI bias has significant implications for human cognition, society, justice, and the future of AI. Thus, it is crucial to advance our understanding of the depth and prevalence of bias in AI to mitigate it both in machines and society.

Semantics Derived Automatically from Language Corpora Contain Human-like Biases

Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biases

Gender Bias in Word Embeddings: A Comprehensive Analysis of Frequency, Syntax, and Semantics

5/24

Emma Brunskill, Stanford

5/31

Mac Schwager, Stanford

Safe Robot Planning with Neural Environment Representations
New developments in computer vision and deep learning have led to the rise of neural environment representations: 3D maps that are stored as deep networks that spatially register occupancy, color, texture, and other physical properties. These environment models can generate photo-realistic synthetic images from unseen view points, and can store 3D information in exquisite detail. In this talk, I investigate the questions: How can robots use neural environment representations for perception, motion planning, manipulation, and simulation? How can we preserve safety guarantees while using such representations? I will show recent work from my lab in which we build a robot navigation pipeline using a Neural Radiance Field (NeRF) map of an environment. We develop a trajectory optimization algorithm that interfaces with the NeRF model to find dynamically feasible, collision-free trajectories for a robot moving through a NeRF world. We also develop an optimization-based state estimator that uses the NeRF model to give full dynamic state estimates for a robot from only on board images. I will discuss our algorithms for dexterous manipulation using NeRF object models, and will describe our development a differentiable physics simulator that operates directly on NeRF object models. I will conclude with future opportunities and challenges in integrating neural environment representations into the robot autonomy stack.

6/7

Chuchu Fan, MIT

Neural certificates in large-scale autonomy design
Learning-enabled control systems have demonstrated impressive empirical performance on challenging control problems in robotics, but this performance comes at the cost of reduced transparency and lack of guarantees on the safety or stability of the learned controllers. In recent years, new techniques have emerged to provide these guarantees by learning certificates alongside control policies — these certificates provide concise, data-driven proofs that guarantee the safety and stability of the learned control system. These methods not only allow the user to verify the safety of a learned controller but also provide supervision during training, allowing safety and stability requirements to influence the training process itself. In this talk, we present an overview of this rapidly developing field of certificate learning. We hope that this talk will serve as an accessible introduction to the theory and practice of certificate learning, both to those who wish to apply these tools to practical robotics problems and to those who wish to dive more deeply into the theory of learning for control.

Support Hours