annotate slide.md @ 5:2470f2f93db3

Fix
author Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
date Fri, 03 Jun 2016 04:51:52 +0900
parents bc12c88b3766
children 8b5af40f3a04
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
1 title: Service robot system with an informationally structured environment
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
2 author: Tatsuki IHA, Nozomi TERUYA
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
3 profile: Kono lab
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
4 lang: English
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
5 code-engine: coderay
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
6
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
7 # 1. Introduction
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
8 - aging of the population is a common problem in modern societies, and rapidly aging populations and declining birth rates have become more serious in recent years
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
9 - for instance, the manpower shortage in hospitals and elderly care facilities has led to the deterioration of quality of life for elderly individuals
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
10 - robot technology is expected to play an important role in the development of a healthy and sustainable society
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
11 - in particular, daily life assistance for elderly individuals in hospitals and care facilities is one of the most urgent and promising applications for service robots
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
12
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
13 # 1. Introduction
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
14 - for a service robot, information about its surrounding, such as the positions of objects, furniture, humans, and other robots is indispensable for safely performing proper service tasks
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
15 - however, current sensing technology, especially for cases of robots equipped with external sensors, is not good enough to complete these tasks satisfactorily
2
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
16 - for example, a vision system is susceptible to changes in lighting conditions and the appearances of objects. moreover, the field of vision is rather narrow
0
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
17
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
18 # 1. Introduction
2
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
19 - although occlusions can be partly solved by sensors on a mobile robot, background changes and unfavorable vibrations of a robot body make processes more difficult
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
20 - in addition, the payload of a robot is not so high and computer resources are also limited
0
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
21
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
22 # 1. Introduction
2
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
23 - fixed sensors in an environment are more stable and can more easily gather information about the environment
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
24 - if a sufficient number of sensors can be embedded in the environment in advance, occlusion is no longer a crucial problem
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
25 - information required to perform tasks is acquired by distributed sensors and transmitted to a robot on demand
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
26 - the concept of making an environment smarter rather than the robot is referred to as an informationally structured environment
0
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
27
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
28 # 1. Introduction
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
29 - an informationally structured environment is a feasible solution for introducing service robots into our daily lives using current technology
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
30 - several systems that observe human behavior using distributed sensor systems and provide proper service tasks according to requests from human or emergency detection, which is triggered automatically, have been proposed
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
31 - several service robots that act as companions to elderly people or as assistants to humans who require special care have been developed
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
32
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
33 # 1. Introduction
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
34 - we also have been developing an informationally structured environment for assisting in the daily life of elderly people in our research project, i.e., the robot town project
2
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
35 - the goal of this project is to develop a distributed sensor network system covering a townsize environment consisting of several houses, buildings, and roads, and to manage robot services appropriately by monitoring events that occur in the environment
0
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
36
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
37 # 1. Introduction
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
38 - events sensed by an embedded sensor system are recorded in the town management system (TMS)
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
39 - and appropriate information about the surroundings and instructions for proper services are provided to each robot
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
40
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
41 # 1. Introduction
2
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
42 - we also have been developing an informationally structured platform in which distributed sensors and actuators are installed to support an indoor service robot
0
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
43 - objects embedded sensors and rfid tags, and all of the data are stored in the TMS database
2
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
44 - a service robot performs various service tasks according to the environmental data stored in the TMS database in collaboration with distributed sensors and actuators, for example, installed in a refrigerator to open a door
0
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
45
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
46 # 1. Introduction
2
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
47 - we herein introduce a new town management system called the ROS-TMS
0
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
48 - in this system, the robot operating system (ROS) is adopted as a communication framework between various modules, including distributed sensors, actuators, robots, and databases
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
49
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
50 # 1. Introduction
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
51 - thanks to the ROS, we were able to develop a highly flexible and scalable system
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
52 - adding or removing modules such as sensors, actuators, and robots, to or from the system is simple and straightforward
2
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
53 - parallelization is also easily achievable
0
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
54
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
55 # 1. Introduction
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
56 - we herein report the followings
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
57 - introduction of architecture and components of the ROS-TMS
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
58 - object detection using a sensing system of the ROS-TMS
2
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
59 - fetch-and-give task using the motion planning system of the ROS-TMS
0
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
60
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
61 # 1. Introduction
2
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
62 - the remainder of the present paper is organized as follows
0
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
63 - section 2 : presenting related research
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
64 - section 3: we introduce the architecture and components of the ROS-TMS
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
65 - section 4: we describe the sensing system of the ROS-TMS for processing the data acquired from various sensors
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
66 - section 5: describes the robot motion planning system of the ROS-TMS used to design the trajectories for moving, gasping, giving, and avoiding obstacles using the information on the environment acquired by the sensing system
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
67 - section 6: we present the experimental results for service tasks performed by a humanoid robot and the ROS-TMS
2
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
68 - section 7: concludes the paper
0
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
69
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
70 # 2. Related research
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
71 - a considerable number of studies have been performed in the area of informationally structured environments/spaces to provide human-centric intelligent services
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
72 - informationally structured environments are referred to variously as home automation systems, smart homes, ubiquitous robotics, kukanchi, and intelligent spaces, depending on the field of research and the professional experience of the researcher
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
73
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
74 # 2. Related research
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
75 - home automation systems or smart homes are popular systems that centralize the control of lighting, heating, air conditioning, appliances, and doors, for example, to provide convenience, comfort, and energy savings
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
76 - the informationally structured environment can be categorized in this system, but the system is designed to support not only human life but also robot activity for service tasks
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
77
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
78 # 2. Related research
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
79 - hashimoto and Lee proposed an intelligent space in 1996
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
80 - intelligent spaces (iSpace) are rooms or areas that are equipped with intelligent devices, which enable spaces to perceive and understand what is occurring within them
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
81 - these intelligent devices have sensing, processing, and networking functions and are referred to as distributed intelligent networked devices (DINDs)
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
82 - one DIND consists of a CCD camera to acquire spatial information and a processing computer, which performs data processing and network interfacing
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
83 - these devices observe the position and behavior of both human beings and robots coexisting in the iSpace
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
84
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
85 # 2. Related research
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
86 - the concept of a physically embedded intelligent system (PEIS) has been introduced in 2005
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
87 - PEIS involves the intersection and integration of three research areas: artificial intelligence, robotics, and ubiquitous computing
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
88 - anything that consists of software components with a physical embodiment and interacts with the environment through sensors or actuators/robots is considered to be a PEIS, and a set of interconnected physically embedded intelligent systems is defined as a PEIS ecology
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
89 - tasks can be achieved using either centralized or distributed approaches using the PEIS ecology
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
90
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
91 # 2. Related research
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
92 - Ubiquitous robotics involves the design and deployment of robots in smart network environments in which everything is interconnected
3
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 2
diff changeset
93 - define three types of Ubibots
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 2
diff changeset
94 - software robots (Sobots)
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 2
diff changeset
95 - embedded robots (Embots)
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 2
diff changeset
96 - mobile robots (Mobots)
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 2
diff changeset
97 - can provide services using various devices through any network, at any place and at any time in a ubiquitous space (u-space)
0
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
98
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
99 # 2. Related research
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
100 - Embots can evaluate the current state of the environment using sensors, and convey that information to users
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
101 - Mobots are designed to provide services and explicitly have the ability to manipulate u-space using robotic arms
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
102 - Sobot is a virtual robot that has the ability to move to any location through a network and to communicate with humans
2
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
103 - The present authors have previously demonstrated the concept of a PIES using Ubibots in a simulated environment and u-space
0
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
104
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
105 # 2. Related research
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
106 - RoboEarth is essentially a World Wide Web for robots, namely, a giant network and database repository in which robots can share information and learn from each other about their behavior and their environment
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
107 - the goal of RoboEarth is to allow robotic systems to benefit from the experience of other robots
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
108
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
109 # 2. Related research
2
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
110 - the informationally structured environment/space (also referred to as Kukanchi, a Japanese word meaning interactive human-space design and intelligence) has received a great deal of attention in robotics research as an alternative approach to the realization of a system of intelligent robots operating in our daily environment
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
111 - human-centered systems require, in particular, sophisticated physical and information services, which are based on sensor networks, ubiquitous computing, and intelligent artifacts
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
112 - information resources and accessibility within an environment are essential for people and robots
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
113 - the environment surrounding people and robots should have a structured platform for gathering, storing, transforming, and providing information
0
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
114 - such an environment is referred to as an informationally structured space
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
115
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
116 # 2. Related research
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
117 - in section 5, we present a coordinate motion planning technique for a fetch-and-give including handing over an object to a person
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
118 - the problem of handing over an object between a human and a robot has been studied in HumanRobot Interaction (HRI)
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
119
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
120 # 2. Related research
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
121 - the work that is closest to ours is the one by Dehais et al
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
122 - in their study, physiological and subjective evaluation for a handing over task was presented
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
123 - the performance of hand-over tasks were evaluated according to three criteria: legibility, safety and physical comfort
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
124 - these criteria are represented as fields of cost functions mapped around the human to generate ergonomic hand-over motions
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
125 - although their approach is similar to our approach, we consider the additional criteria, that is, the manipulability of both a robot and a human for a comfortable and safety fetch-and-give task
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
126
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
127 # 2. Related research
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
128 - the problem of pushing carts using robots has been reported in many studies so far
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
129 - the earlier studies in pushing a cart were reported using a single manipulator mounted on a mobile base
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
130
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
131 # 2. Related research
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
132 - the problem of towing a trailer has also been discussed as an application of a mobile manipulator and a cart
2
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
133 - this work is close to the approach in this paper, however, a pivot point using a cart is placed in front of the robot in our technique
0
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
134
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
135 # 2. Related research
2
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
136 - the work that is closest to ours is the one by Scholz et al
0
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
137 - they provided a solution for real time navigation in a cluttered indoor environment using 3D sensing
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
138
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
139 # 2. Related research
2
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
140 - many previous works focus on the navigation and control problems for movable objects
0
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
141 - On the other hand, we consider the problem including handing over an object to a human using a wagon, and propose a total motion planning technique for a fetch-and-give task with a wagon
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
142
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
143 # 3. Overview of the ROS-TMS
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
144 - in the present paper, we extend the TMS and develop a new Town Management System called the ROS-TMS
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
145 - This system has three primary components
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
146 - real-world
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
147 - database
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
148 - cyber-world
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
149
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
150 <div style="text-align: center;">
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
151 <img src="./images/fig3.svg" alt="message" width="600">
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
152 </div>
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
153
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
154 # 3. Overview of the ROS-TMS
2
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
155 - events occurring in the real world, such as user behavior or user requests, and the current situation of the real world are sensed by a distributed sensing system
0
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
156 - the gathered information is then stored in the database
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
157
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
158 <div style="text-align: center;">
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
159 <img src="./images/fig3.svg" alt="message" width="600">
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
160 </div>
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
161
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
162 # 3. Overview of the ROS-TMS
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
163 - appropriate service commands are planned using the environmental information in the database and are simulated carefully in the cyber world using simulators, such as choreonoid
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
164 - service tasks are assigned to service robots in the real world
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
165
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
166 <div style="text-align: center;">
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
167 <img src="./images/fig3.svg" alt="message" width="600">
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
168 </div>
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
169
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
170 # 3. Overview of the ROS-TMS
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
171 - the following functions are implemented in the ROS-TMS
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
172 1. Communication with sensors, robots, and databases
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
173 2. Storage,revision,backup,and retrieval of real-time information in an environment
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
174 3. Maintenance and providing information according to individual IDs assigned to each object and robot
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
175 4. Notification of the occurrence of particular predefined events, such as accidents
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
176 5. Task schedule function for multiple robots and sensors
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
177 6. Human-system interaction for user requests
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
178 7. Real-time task planning for service robots
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
179
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
180 # 3. Overview of the ROS-TMS
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
181 - ROS-TMS has unique features, as described below
4
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 3
diff changeset
182 - Modularity
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 3
diff changeset
183 - ROS-TMS consists of 73 packages categorized into 11 groups and 151 processing nodes.
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 3
diff changeset
184 - Re-configuration of structures, for instance adding or removing modules such as sensors, actuators, and robots, is simple and straightforward owing to the high flexibility of the ROS architecture
0
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
185 - Scalability
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
186 - ROS-TMS is designed to have high scalability so that it can handle not only a single room but also a building and a town
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
187 - Diversity
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
188 - diversity: The ROS–TMS supports a variety of sensors and robots
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
189 - for instance, Vicon MX (Vicon Motion Systems Ltd.), TopUrg (Hokuyo Automatic), Velodyne 32e (Velodyne Lidar), and Oculus Rift (Oculus VR) are installed in the developed informationally structured platform
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
190 - Safety
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
191 - data gathered from the real world is used to perform simulations in the cyber world in order to evaluate the safety and efficiency of designed tasks
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
192
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
193 # 3. Overview of the ROS-TMS
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
194 - ROS-TMS has unique features, as described below
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
195 - Privacy protection
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
196 - one important restriction in our intelligent environment is to install a small number of sensors to avoid interfering with the daily activity of people and to reduce the invasion of their privacy as far as possible
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
197 - we do not install conventional cameras in the environment
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
198 - Economy
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
199 - sensors installed in an environment can be shared with robots and tasks, and thus we do not need to equip individual robots with numerous sensors
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
200 - in addition, most sensors are processed by low-cost single-board computers in the proposed system
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
201 - this concept has an advantage especially for the system consisting of multiple robots since robots can share the resources in the environment
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
202
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
203 # 3. Overview of the ROS-TMS
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
204 - some features such as modularity, scalability, and diversity owe much to ROS’s outstanding features
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
205 - on the other hand, economical or processing efficiency strongly depends on the unique features of ROS-TMS, since various information gathered by distributed sensor networks is structured and stored to the database and repeatedly utilized for planning various service tasks by robots or other systems
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
206
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
207 # 3. Overview of the ROS-TMS
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
208 - ROS-TMS is composed of five components
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
209 - User
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
210 - Sensor
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
211 - Robot
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
212 - Task
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
213 - Data
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
214 - components are also composed of sub-modules
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
215 - such as the User Request sub-module for the user component
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
216
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
217 <div style="text-align: center;">
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
218 <img src="./images/fig4.svg" alt="message" width="450">
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
219 </div>
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
220
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
221 # 4. Sensing system
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
222 - sensing system (TMS_SS) is a component of the ROS-TMS that processes the data acquired from various environment sensors
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
223 - TMS_SS is composed of three sub-packages
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
224 - Floor sensing system (FSS)
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
225 - Intelligent cabinet system (ICS)
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
226 - Object detection system (ODS)
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
227
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
228 # 4.1 Floor sensing system(FSS)
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
229 - current platform is equipped with a floor sensing system to detect objects on the floor and people walking around
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
230 - this sensing systems is composed of a laser range finder located on one side of the room and a mirror installed along another side of the room
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
231 - this configuration allows a reduction of dead angles of the LRF and is more robust against occlusions
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
232
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
233 <div style="text-align: center;">
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
234 <img src="./images/fig6.svg" alt="message" width="600">
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
235 </div>
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
236
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
237 # 4.1 Floor sensing system(FSS)
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
238 - people tracking is performed by first applying static background subtraction and then extracting clusters in the remainder of the measurements
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
239 - this system can measure the poses of the robot and movable furniture such as a wagon using tags, which have encoded reflection patterns optically identified by the LRF
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
240
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
241 <div style="text-align: center;">
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
242 <img src="./images/fig6.svg" alt="message" width="600">
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
243 </div>
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
244
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
245 # 4.2. Intelligent cabinet system (ICS)
83569495824e first commit
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents:
diff changeset
246 - the cabinets installed in the room are equipped with RFID readers and load cells to detect the types and positions of the objects in the cabinet
1
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
247 - every object in the environment has an RFID tag containing a unique ID that identifies it
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
248 - this ID is used to retrieve the attributes of the object, such as its name and location in the database
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
249
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
250 # 4.2. Intelligent cabinet system (ICS)
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
251 - using the RFID readers, we can detect the presence of a new object inside the cabinet
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
252 - the load cell information allows us to determine its exact position inside the cabinet
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
253
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
254 <div style="text-align: center;">
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
255 <img src="./images/fig7.svg" alt="message" width="1200">
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
256 </div>
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
257
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
258 # 4.3. Object detection system (ODS)
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
259 - available for detecting objects such as those placed on a desk, the object detection system using a RGB-D camera on a robot is provided in this platform
2
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
260 - in this system, a newly appeared object or movement of an object is detected as a change in the environment
1
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
261
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
262 <div style="text-align: center;">
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
263 <img src="./images/fig8.svg" alt="message" width="600">
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
264 </div>
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
265
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
266 # 4.3. Object detection system (ODS)
2
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
267 - the steps of the change detection process are as follows
1
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
268 1. Identification of furniture
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
269 2. Alignment of the furniture model
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
270 3. Object extraction by furniture removal
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
271 4. Segmentation of objects
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
272 5. Comparison with the stored information
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
273
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
274 # 4.3.1. Identification of furniture
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
275 - possible to identify furniture based on the position and posture of robots and furniture in the database
2
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
276 - using this information, robot cameras determine the range of the surrounding environment that is actually being measured
1
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
277 - the system superimposes these results and the position information for furniture to create an updated furniture location model
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
278
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
279 # 4.3.1. Identification of furniture
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
280 - the point cloud (Fig. 9a) acquired from the robot is superimposed with the furniture’s point cloud model (Fig. 9b)
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
281 - After merging the point cloud, the system deletes all other points except for the point cloud model for the furniture and limits the processing range from the upcoming steps
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
282
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
283 <div style="text-align: center;">
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
284 <img src="./images/fig9.svg" alt="message" width="800">
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
285 </div>
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
286
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
287 # 4.3.2. Alignment of the furniture model
2
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
288 - We scan twice for gathering point cloud datasets of previous and current scene
1
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
289 - in order to detect the change in the newly acquired information and stored information, it is necessary to align two point cloud datasets obtained at different times because these data are measured from different camera viewpoints
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
290
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
291 # 4.3.2. Alignment of the furniture model
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
292 - in this method, we do not try to directly align the point cloud data, but rather to align the data using the point cloud model for the furniture
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
293 - the reason for this is that we could not determine a sufficient number of common areas by simply combining the camera viewpoints from the two point cloud datasets and can also reduce the amount of information that must be stored in memory
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
294 - using the aligned point cloud model, it is possible to use the point cloud data for objects located on the furniture, without having to use the point cloud data for furniture from the stored data
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
295 - alignment of the furniture model is performed using the ICP algorithm
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
296
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
297 # 4.3.3. Object extraction by furniture removal
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
298 - after alignment, all points corresponding to furniture are removed to extract an object
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
299 - the system removes furniture according to segmentation using color information and three-dimensional positions
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
300 - more precisely, the point cloud is converted to a RGB color space and then segmented using a region-growing method
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
301
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
302 # 4.3.3. Object extraction by furniture removal
2
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
303 - each of the resulting segments is segmented based on the XYZ space
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
304 - system then selects only those segments that overlap with the model and then removes these segments
1
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
305
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
306 <div style="text-align: center;">
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
307 <img src="./images/fig10.svg" alt="message" width="800">
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
308 </div>
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
309
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
310 # 4.3.4. Segmentation of objects
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
311 - after performing the until now processing, only the points associated with objects placed on furniture remain
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
312 - these points are further segmented based on XYZ space
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
313 - the resulting segments are stored in the database
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
314
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
315 # 4.3.5. Comparison with the stored infomation
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
316 - finally, the system associates each segment from the previously stored information with the newly acquired information
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
317 - system finds the unmatched segments and captures the movement of objects that has occurred since the latest data acquisition
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
318 - segments that did not match between the previous dataset and the newly acquired dataset, reflect objects that were moved, assuming that the objects were included in the previously stored dataset
2
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
319 - segments that appear in the most recent dataset, but not in the previously stored dataset, reflect objects that were recently placed on the furniture
1
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
320
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
321 # 4.3.5. Comparison with the stored infomation
2
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
322 - set of segments that are included in the association process are determined according to the center position of segments
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
323 - for the segments sets from the previous dataset and the newly acquired dataset, the association is performed based on a threshold distance between their center positions, considering the shape and color of the segments as the arguments for the association
1
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
324
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
325 # 4.3.5. Comparison with the stored infomation
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
326 - use an elevation map that describes the height of furniture above the reference surface level to represent the shape of the object
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
327 - reference surface level of furniture is, more concretely, the top surface of a table or shelf, the seat of a chair
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
328 - elevation map is a grid version of the reference surface level and is a representation of the vertical height of each point with respect to the reference surface level on each grid
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
329
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
330 <div style="text-align: center;">
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
331 <img src="./images/fig11.svg" alt="message" width="800">
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
332 </div>
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
333
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
334 # 4.3.5. Comparison with the stored infomation
2
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 1
diff changeset
335 - comparison is performed on the elevation map for each segment, taking into consideration the variations in size, the different values obtained from each grid, and the average value for the entire map
1
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
336 - the color information used to analyze the correlation between segments is the hue (H) and saturation (S)
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
337 - Using these H-S histograms, the previous data and the newly acquired data are compared, allowing the system to determine whether it is dealing with the same objects
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
338
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
339 <div style="text-align: center;">
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
340 <img src="./images/fig11.svg" alt="message" width="800">
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
341 </div>
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
342
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
343 # 4.3.5. Comparison with the stored infomation
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
344 - bhattacharyya distance BC(p, q) within H-S histograms(p,q) is used for determining the similarity between histograms and is calculated according to Eq. (1)
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
345 - once distance values are calculated, the object can be assumed to be the same as for the case in which the degree of similarity is equal to or greater than the threshold value
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
346
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
347 <div style="text-align: center;">
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
348 <img src="./images/eq1.svg" alt="message" width="800">
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
349 </div>
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
350
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
351
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
352 # 4.3.5. Comparison with the stored infomation
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
353
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
354 <div style="text-align: center;">
5
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 4
diff changeset
355 <img src="./images/fig12.svg" alt="message" width="600">
1
Tatsuki IHA <e125716@ie.u-ryukyu.ac.jp>
parents: 0
diff changeset
356 </div>