3rd Workshop on Semantic Reasoning and Goal Understanding in Robotics (SemRob)
Robotics Science and Systems Conference (RSS 2026)
π Location: UTS Faculty of Engineering Building (Building 11; CB11.B3.101) at UTS
(Enter Building 11, find an elevator that allows you to select "B3"; on level-B3, find room 101/102
Zoom: https://cmu.zoom.us/j/94968259029?pwd=7dMBhpqfpbMTO44kMWmBp6FjoByZnl.1
| Time | Info | |
|---|---|---|
| 08:10 | Organizers: Introductory Remarks | |
| 08:20 |
|
Keynote: William Zhi University of Sydney Title: Imitation Learning for High-Dimensional Robotic Systems
As robots move beyond simple manipulation settings, imitation learning must
contend with richer embodiments, longer horizons, complex control interfaces,
and coordination across multiple agents. This talk presents approaches for
structuring imitation learning across memory, shared-control teleoperation, and
multi-robot interaction. It includes methods for bimanual mobile manipulation and
for learning coordinated behaviours across multiple robots through staged and
structured imitation. Together, these works show how complex robotic systems can
acquire reliable and compositional behaviours from demonstration while remaining
practical to operate and deploy.
|
| 08:45 |
|
Keynote: Hang Zhao Tsinghua University + Galaxea AI Title: Autoregressive VLA: Reasoning and Acting in One Autoregressive Stream
Can a robot reason about what to do and execute how to do it within a single
autoregressive process? In this talk, I will present our work on
autoregressive vision-language-action models, where high-level reasoning and
low-level action generation are modeled in one unified token stream. This
formulation allows semantic understanding, task decomposition, and continuous
control to share the same modeling interface and training objective, rather
than being handled by separately designed action experts. I will discuss the
evolution of our action tokenizers and models, including FASTer, ActionCodec
and Galaxea G0.5.
|
| 09:10 |
|
Keynote: Jean Oh Carnegie Mellon University + Lavoro AI Title: Safe Physical AI under the Curse of Rarity: From Self-Driving to Aviation
Abstract coming soon.
|
| 09:35 |
|
Keynote: Jesse Zhang University of Washington Title: Learn Less, Borrow More: Generalization doesnβt have to be re-learned β it can be borrowed
Modern performant foundation models, often able to demonstrate impressive
generalization capabilities, are driven by scalable sources of supervision during
both pre- and post-training. Robot policies, however, rely on expensive human
demonstrations during pre-training and costly environment interaction during
post-training, making them difficult to scale. In this talk, I demonstrate how,
instead of trying to learn to generalize effectively from data and interaction, we
can formulate policies which can borrow generalization capabilities from other
sources for both pre-training and post-training for better performance without
needing the same level of robotics supervision as used for training foundation
models in text and vision domains.
|
| 10:00 |
|
Keynote: Andreea Bobu Massachusetts Institute of Technology Title: Reading Between the Lines: Using Language Models to Amplify Human Data in Robot Learning
Human-in-the-loop robot learning faces a fundamental data challenge that
general machine learning doesn't: unlike settings where we can collect massive
offline datasets, robots must learn from limited, real-time human interactions.
This creates a critical bottleneck: we need methods that can make the most of
limited human input, or, in other words, that can learn a lot from a little.
The challenge is that humans are imperfect communicators of their own intent:
language instructions are often ambiguous, demonstrations can be incomplete or
overfit to a specific setting, and physical corrections have multiple valid
interpretations. Our key insight is that human feedback is shaped by context in
predictable ways, and that modeling that context turns ambiguous, incomplete
feedback into a rich signal about underlying intent. In this talk, I will
discuss three sources of such structure: 1. human feedback modalities are
incomplete in complementary ways, and that complementarity is itself a source
of signal; 2. human feedback is only interpretable relative to how humans
represent the world, and learning that representation is as important as
learning the reward; and 3. behind any specific human input lies a higher-level
intent that generalizes far beyond the situation in which it was expressed.
Together, these directions show that understanding the structure of human
communication β rather than simply collecting more of it β is the key to
efficient, generalizable, human-aligned robot learning.
|
| 10:25 | Spotlight talks: Paper IDs 11,8,17,26 | |
| 11:00 | Coffee Break + Poster session | |
| 11:30 | Panel discussion Panelists: William Zhi, Hang Zhao, Jean Oh, Jesse Zhang, Andreea Bobu |
|
| 12:25 | Organizers: Closing Remarks, Best paper |
Papers incoming ...
18 June 2026
, 23:59 AOE.