Abstract
—Coordination is an essential characteristic of
any task-achieving multi-robot system (MRS), whether it is
accomplished through an explicit or implicit coordination
mechanism. There is currently little formal work addressing
how various MRS coordination mechanisms are related, how
appropriate they are for a given task, what capabilities they
require of the robots, and what level of performance they
can be expected to provide. Given a MRS composed of
homogeneous robots, we present a method for automated
controller construction such that the resulting controller
makes use of internal state and no explicit inter-robot
communication, yet is still capable of correctly executing
a given task. Understanding the capabilities and limitations
of a MRS composed of robots not capable of inter-robot
communication contributes to the understanding of when
and why inter-robot communication becomes necessary and
when internal state alone is suf cient to achieve the desired
coordination. We validate our method in a multi-robot
construction domain. more
Adaptive internal state space construction method for reinforcement learning of a real-world agent
Abstract
One of the difficulties encountered in the application of the reinforcement learning to real-world problems is the construction of a discrete state space from a continuous sensory input signal. In the absence of a priori knowledge about the task, a straightforward approach to this problem is to discretize the input space into a grid, and to use a lookup table. However, this method suffers from the curse of dimensionality. Some studies use continuous function approximators such as neural networks instead of lookup tables. However, when global basis functions such as sigmoid functions are used, convergence cannot be guaranteed. To overcome this problem, we propose a method in which local basis functions are incrementally assigned depending on the task requirement. Initially, only one basis function is allocated over the entire space. The basis function is divided according to the statistical property of locally weighted temporal difference error (TD error) of the value function. We applied this method to an autonomous robot collision avoidance problem, and evaluated the validity of the algorithm in simulation. The proposed algorithm, which we call adaptive basis division (ABD) algorithm, achieved the task using a smaller number of basis functions than the conventional methods. Moreover, we applied the method to a goal-directed navigation problem of a real mobile robot. The action strategy was learned using a database of sensor data, and it was then used for navigation of a real machine. The robot reached the goal using a smaller number of internal states than with the conventional methods. more
Research on Internal State-Based Systems
A new methodology for evaluating utility preferences using internal state information is attracting much attention within the robotics community. This methodology is based on on-going research in the fields of biology, psychology, and cognitive science and attempts to capture preference information through the use of artificial emotions, drives, and motivations.
Traditional state-based systems focus on external state information, such as the number and type of percepts, etc. when using the current state to influence decisions. External state-based systems scan the environment and then react or deliberate using the information gathered. Internal state-based systems monitor the external state, but these systems also include internal variables such as emotions, motivations, and feelings when making decisions. The internal variables are derived from dynamic internal processes and from associations and recollections pulled from long-term memory. more
Lecture Series on Robotics - Internal State Sensors
Lecture Series on Robotics by Prof.C.Amarnath, Department of Mechanical Engineering,IIT Bombay.
Lecture - 10 Internal State Sensors
Lecture Series on Robotics - External State Sensors
