2nd Workshop on Semantic Reasoning and Goal Understanding in Robotics (SemRob)

Robotics Science and Systems Conference (RSS 2025)
Olin Hall of Engineering (OHE) #122, USC Campus, June 21, Los Angeles, USA

[Zoom] [Submit Debate Questions]


About
Semantic understanding of the world is essential for robots to make safe and informed decisions, to adapt to changing environmental conditions, and to enable efficient interactions with other agents. In pursuit of semantic understanding, agents must be able to (i) interpret and represent high-level goals, agnostic of their physical morphology and despite irrelevant aspects of their environments; they must be able to (ii) reason, i.e., to extract abstract concepts from observations in the real-world, logically manipulate these concepts, then leverage the results for inference on downstream tasks; and they must be able to (iii) execute morphology-, environment-, and socially-appropriate behaviors towards those high-level goals.

Despite substantial recent advancements in the use of pre-trained, large-capacity models (i.e., foundation models) for difficult robotics problems, methods still struggle in the face of several practical challenges that relate to real-world deployment, e.g., cross-domain generalization, adaptation to dynamic and human-shared environments, and lifelong operation in open-world contexts. This workshop intends to sponsor discussion of new hybrid methodologies—those that combine representations from foundation models with modeling mechanisms that may prove useful for semantic reasoning and abstract goal understanding, including neural memory mechanisms, procedural modules (e.g., cognitive architectures), neuro-symbolic representations (e.g., knowledge/scene graph embeddings), chain-of-thought reasoning mechanisms, robot skill primitives and their composition, 3D scene representations (e.g., 3DGS), etc.

Intended audience. We aim to bring together engineers, researchers, and practitioners from different communities to enable avenues for interdisciplinary research on methods that could facilitate the deployment of semantics-aware and generalizable embodied agents in unstructured and dynamic real world environments. In addition to the organizers, the presenters, panelists, and technical program committee are drawn from the following communities: Robot Learning, Embodied AI, Planning + Controls, Cognitive Robotics, Neuro-Symbolism, Natural Language Processing, Multimodal Machine Learning, Computer Vision, and Digital Twins. We likewise intend to attract an audience from these diverse sub-communities to contribute to compelling discussions.
Event Information This is a primarily in-person workshop, held at the 2025 Robotics Science and Systems conference (RSS), in Los Angeles, United States on 21 June 2025, starting at 8:30am PT.

The workshop location is room #122, in the Olin Hall of Engineering (OHE), at the University of Southern California (USC).

Poster session location: Epstein Family Plaza, right behind OHE.

Here is a map that includes both OHE (workshop location) and Epstein Family Plaza (poster session location): https://roboticsconference.org/images/local2025/workshops.png

RSS SemRob 2025 is Workshop #12, on the official RSS schedule.


Schedule

Time
08:30 Organizers
Introductory Remarks
08:40 Keynote 1: Jesse Thomason
Embracing Language as Grounded Communication
09:00 Keynote 2: Manolis Savva
Towards Realistic & Interactive 3D Simulation for Embodied AI
09:20 Keynote 3: Dorsa Sadigh
Human-Aligned Robot Learning: manipulation policies via preferences, RLHF, and VLM feedback
09:40 Spotlight Talks: #1, #4, #13, #17
10:00 Keynote 4: Yonatan Bisk
Semantics? Reasoning? Can we define either of those terms?
10:20 Keynote 5: Ted Xiao
Full-stack Robotics Foundation Models: From Embodied Reasoning to Dexterity
10:40 Keynote 6: Benjamin Alt
Semantic Digital Twins for Robust and Flexible Robot Behavior
10:50 Coffee Break, Socializing, Posters
11:30 Keynote 7: Lerrel Pinto
On Building General-Purpose Home Robots
11:50 Debate: Implicit/Data-emergent Reasoning Capabilities versus Explicit Reasoning Mechanisms?
Panelists: Jesse Thomason, Ted Xiao, Manolis Savva, Lerrel Pinto, Yonatan Bisk, Benjamin Alt
12:30 Organizers
Closing Remarks


Call for Papers
Targeted Topics
In addition to the RSS subject areas, we especially invite paper submissions on various topics, including (but not limited to):

  • Neural architectures leveraging demonstrations as prompts
  • Goal understanding through few-shot demonstrations
  • Novel abstractions, representations, and mechanisms for few-shot learning
  • Retrieval-augmentation mechanisms used in learning and task-execution
  • Agentic frameworks for failure reasoning, self-guidance, test-time adaptation, etc.
  • LLM-based action models for robot control; large action models
  • Semantic representations for generalizable policy learning
  • World models used for reasoning and optimization
Submission Guidelines
RSS SemRob 2025 suggests 4+N or 8+N paper length formats — i.e., 4 or 8 pages of main content with unlimited additional pages for references, appendices, etc. However, like RSS 2025, we impose no strict page length requirements on submissions; we trust that authors will recognize that respecting reviewers’ time is helpful to the evaluation of their work.

Submissions are handled through CMT: https://cmt3.research.microsoft.com/SEMROB2025

(Required acknowledgement: the Microsoft CMT service was used for managing the peer-reviewing process for this conference. This service was provided for free by Microsoft and they bore all expenses, including costs for Azure cloud services as well as for software development and support.)


We will accept the official LaTeX or Word paper templates, provided by RSS 2025.

Our review process will be double-blind, following the RSS paper submission policy for Science/Systems papers.

All accepted papers will be invited for poster presentations; the highest-rated papers, according to the Technical Program Committee, will be given spotlight presentations. Accepted papers will be made available online on this workshop website as non-archival reports, allowing authors to also submit their works to future conferences or journals. We will highlight the Best Paper Award during the closing remarks at the workshop event.

Important Dates
  • Submission deadline:1 June 2025, 23:59 AOE.
  • Author Notifications: 10 June 2025.
  • Camera Ready: 15 June 2025.
  • Workshop: 21 June 2025, 08:30-12:30 PT

Accepted Papers

Congratulations to Paper #13 (WoMAP: World Models For Embodied Open-Vocabulary Object Localization) for winning the Best Paper Award and for Paper #1 (Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Models) for winning the Best Paper Runner-up!




Acknowledgement
This workshop is supported by the Research Initiative FAME (Future-oriented cognitive Action Modelling Engine) and the European Network of Excellence Centers in Robotics euROBIN.

Contact and Information

Direct questions to semrob.workshop+general@gmail.com. Subscribe to our mailing list to stay updated.