University of Linkoping, Sweden
University of Hyderabad, India
In the context of collaborative robotics, both distributed planning and task allocation, and acquisition of situation awareness are essential for supporting goal achievement, collective intelligence, and decision support in teams of robots and human agents. This is particularly important in applications pertaining to emergency rescue and crisis management. Given a high-level mission specification provided by a member of a rescue team, human or robotic, one then requires a mechanism for generating and executing complex, multi-agent distributed plans and tasks. The proper task representation is essential for both the generation and execution of complex multi-agent distributed tasks. Task Specification Trees have been proposed for this purpose and a Delegation Framework is used for distributed task allocation. Additionally, during operational missions, data and knowledge is gathered incrementally and in different ways by teams of heterogeneous robots and humans. We describe this as the formation and management of Hastily Formed Knowledge Networks (HFKN). The resulting distributed knowledge structures can then be queried by individual agents for decision support. These structures are represented as RDF graphs, and graph synchronization techniques are introduced to retain the consistency of the collective knowledge of a team. Flexible human interaction with teams of robots is also an essential component in emergency rescue. Integrating LLMs into the interaction process provides a new way to think about interaction.
In this talk, I will present both the HFKN and Delegation Frameworks, their integration, and in addition describe various field robotic experiments with UAVs which use the overall system. I will also show some initial work that uses LLMs in the interaction process. If time allows, I will also discuss a Swedish national project where this framework has been used by both industrial and academic partners in large public safety scenarios using UAVs, USVs, and AUVs in maritime and sea rescue scenarios.
Patrick Doherty is a Professor of Computer Science at the Department of Computer and Information Sciences (IDA), Linköping University, Sweden. He leads the Artificial Intelligence Lab at IDA. He is an ECCAI/EurAI fellow, a AAIA fellow, and a member of ACM and AAAI. He previously served as Editor-in-Chief of the Artificial Intelligence Journal. He has over 30 years of experience in areas such as knowledge representation and reasoning, automated planning, intelligent autonomous systems, and multi-agent systems. A major area of application is with Unmanned Aircraft Systems (UAS). He has over 200 refereed scientific publications in his areas of expertise and has given numerous keynote and invited talks at leading international conferences.
Navigation involves guiding a robot, drone, or any autonomous system through an unknown dynamic and complex environment by understanding its location and surroundings. "Perception" involves gathering and interpreting data from the environment through various sensors. This can include cameras, LIDAR, RADAR, ultrasonic sensors, GPS, and IMUs (Inertial Measurement Units). Thus the goal of "Perception" is to create a comprehensive understanding of the environment. Thus, in this talk we discuss the technology stack used to create such maps of a complex and dynamic environment while simultaneously tracking the location of a device within that environment. This talk will not cover "Autonomy", which involves making decisions and taking actions based on the information provided by the "Perception" system. Thus localisation and mapping are fundamental in understanding the environment for navigation and interaction. Visual Simultaneous Localization and Mapping (vSLAM) is one such technology stack and this talk aims to cover core algorithms of vSLAM.
There are many situations of dynamic and complex environments across different domains, where in the talk we highlight the specific challenges and technological solutions required to cope with them. Some examples are like Urban Traffic for Self-Driving Cars, Autonomous Robots in Manufacturing, Autonomous Agriculture (e.g., Harvesting Robots) etc.
Arun Agarwal completed his BTech (Electrical Engineering, IIT Delhi, India, 1979) and PhD (Computer Science, IIT Delhi, India, 1989). He joined University of Hyderabad in 1984 as a Lecturer and superannuated as a Senior Professor of Computer and Information Sciences in 2022. He also served as a Dean of the School (2015-2018). He was also a Pro-Vice-Chancellor-1 (2018-2021) and for a brief period Vice-Chancellor (2021) of University of Hyderabad.
He was a Visiting Scientist at The Robotics Institute, Carnegie-Mellon University, USA (1986) and Research Associate at Sloan School of Management, Massachusetts Institute of Technology, USA (1993-94). He has visited many other Universities and Institutes like: Monash and Melbourne University in Australia; National Center for High Performance Computing, Hsinchu, Taiwan; Chinese Academy of Sciences, Beijing, China; San Diego Supercomputing Centre USA; Mahasarakham University and NECTEC, Thailand; KISTI, South Korea; etc.
He is an elected Fellow of IETE (2003), elected Fellow of Telangana Akademi of Sciences (2011), and Senior Member of IEEE, USA (1998); He was Chairman of IEEE Hyderabad Section for the years 2001 and 2002. He is also recipient of the IEEE Region 10 Outstanding Volunteer Award in 2009 in recognition of his dedications and contributions. Awarded Outstanding Reviewer Status for Journal of Pattern Recognition, Elsevier, November 2015. He was felicitated by Indian Society for Rough Sets in recognition of Academic Excellence and promotion of Rough Set Activities 2017.
He has served on the technical program committee of numerous conferences in the area of Pattern Recognition and Artificial Intelligence. He was also on the Steering Committee of PRAGMA 2004-2015.
His areas of interest are in Computer Vision, Image Processing, Neural Networks, Grid and Cloud Computing. He has guided 18 PhD theses and more than 125 post-graduate dissertations and has published more than 100 papers. He had several projects and consultancy with several industry/research laboratories. Currently, he is Advisor to Zen Technologies Ltd, Hyderabad.