Published papers

This study proposes a descent control system for a space lander with a large amount of fuel, to protect the lander from overturning when landing on microgravity objects. As sloshing is a major problem during descent, it is desirable to construct a control system that considers the sloshing dynamics. We derive a model of the lander considering fuel dynamics, determine the fuel-optimal trajectory for guidance, design a fuel state estimator for navigation, and design a tracking controller. The effectiveness of the proposed method is evaluated using a physical simulator. The proposed method achieves simultaneous stabilization of the lander and fuel, and is shown to be effective in preventing overturning.

This paper describes a domestic service robot (DSR) that fetches everyday objects and carries them to specified destinations according to free-form natural language instructions. Given an instruction such as 'Move the bottle on the left side of the plate to the empty chair,' the DSR is expected to identify the bottle and the chair from multiple candidates in the environment and carry the target object to the destination. Most of the existing multimodal language understanding methods are impractical in terms of computational complexity because they require inferences for all combinations of target object candidates and destination candidates. We propose Switching Head-Tail Funnel UNITER, which solves the task by predicting the target object and the destination individually using a single model. Our method is validated on a dataset based on a standard dataset for Vision-and-Language Navigation with object manipulation tasks. The results show that our method outperforms the baseline method in terms of language comprehension accuracy. Furthermore, we conduct physical experiments in which a DSR delivers standardized everyday objects in a standardized domestic environment as requested by instructions with referring expressions. The experimental results show that the object grasping and placing actions are achieved with success rates of more than 90 %.

This paper describes a method to adjust the natural frequency of the Vehicle Manipulator of the ITER Blanket Remote Handling System (BRHS). The natural frequency was adjusted to avoid the seismic resonance by using the redundancy of degree of freedom of the Vehicle Manipulator. The Vehicle Manipulator was modeled and analyzed with lumped mass model with joint stiffness. The analysis has shown that the natural frequencies of the Vehicle Manipulator can be changed from 0.5 Hz to 12 Hz in the gripping posture of the First Wall in the vacuum vessel, suggesting that the dominant seismic frequency around 7Hz-9 Hz can be avoided.

© 2018 Elsevier B.V. Background: Stair climbing is the task first affected in patients with knee osteoarthritis (OA); therefore, the precise measurement of time required to climb stairs is important to identify mobility limitations, particularly in the early phase of knee OA. Research question: This study aimed to examine the test-retest reliability, measurement error, and concurrent validity of the stopwatch-based stair-climb test (SCT) in adults with pre-radiographic to mild knee OA. Methods: Fifty-nine participants (mean age, 59.1 [range, 50–69] years; 72.9% female) with Kellgren and Lawrence grade ≤2 disease underwent an 11-step SCT (11-SCT) in accordance with the Osteoarthritis Research Society International recommended method while wearing pressure sensor-mounted standard shoes that is used as a gold standard procedure. Test-retest reliability, measurement errors, and the concurrent validity of the stopwatch-based 11-SCT were evaluated. Results: The test-retest reliability of the stopwatch-based 11-SCT was excellent (intra-class correlation coefficient 1,1 [ICC 1,1 ], 0.952; 95% confidence interval [CI], 0.560 to 0.985; p < 0.001) and the minimal detectable change 95 was 0.102 s. Concurrent validity was excellent (ICC 2,1 : 0.957; 95% CI: 0.661 to 0.986; p < 0.001). Significance: The stopwatch-based 11-SCT had high test-retest reliability and high concurrent validity, which justify its clinical use for identifying mobility limitations in individuals with pre-radiographic to mild knee OA. A difference of 0.2 s in the stopwatch-based 11-SCT time would be considered a true difference beyond a 95% measurement error.

© 2019, International League of Associations for Rheumatology (ILAR). Introduction: This study aimed to examine the interaction effect between ambulatory physical activity (PA) and varus thrust on knee pain in individuals with knee osteoarthritis (OA). Method: Subjects (n = 207; mean age: 73.1 years, 71.5% women) in orthopedic clinics with diagnosed knee OA (Kellgren/Lawrence grade ≥ 1) were enrolled in this 12-month observational cohort study. Participants underwent gait observation for varus thrust assessment and pedometer-based ambulatory PA measurements at baseline and 12-month follow-up. Knee pain intensity was assessed using the Japanese Knee Osteoarthritis Measure pain subscale as a primary outcome measure. Multiple linear regression analyses were performed to evaluate ambulatory PA–thrust interaction on knee pain intensity. Results: Ninety-two subjects (mean age, 73.4 years; 68.5% women) completed the 12-month follow-up assessment. Baseline ambulatory PA–thrust interaction was significant (P = 0.017) in the cross-sectional analysis, adjusting for covariates, which yielded R2 = 0.310. Subgroup analysis showed that varus thrust was significantly associated with worse knee pain in subjects walking ≥ 5000 steps/day adjusting for covariates (beta: 7.94; 95% CI: 3.82, 12.1; P < 0.001) with a higher predictive ability (R2 = 0.664). In contrast, ambulatory PA–thrust interaction in the longitudinal analysis showed no significant association with knee pain changes. Conclusions: Ambulatory PA interacted with varus thrust in the association with knee pain, as coexisting high ambulatory PA and varus thrust had the strongest association with higher knee pain. Maximal pain relief effects might be achieved when both ambulatory PA and varus thrust are treated simultaneously, rather than treating each separately.

Recent research on Flight-deck Interval Management (FIM), a modern technology for increasing safety and improving airspace and runway utilisation through self-spacing, has led to the development of a new rule-based logic for FIM, namely Interval Management - Speed Planning (IM-SP). In an initial benchmark study, IM-SP showed good spacing performance with a significant reduction in speed commands, a major area of concern with previous FIM logics, resulting in a lower burden on the flight crew during FIM operation. Nevertheless, there remains scope for improvement in other aspects, such as fuel burn. In this study, the internal cost function of IM-SP is further analysed and optimised using speed-constrained multi-objective particle swarm optimisation to improve the performance of IM-SP under the multiple objectives of FIM. The optimisation renders new settings that address the problem areas, improve the speed commands and enhance the overall quality of IM-SP. Two distinctive solutions, viz. a spacing performance optimised setting and a fuel burn optimised setting, are further analysed and discussed, and directions for follow-up research are explored.

To achieve the tasks provided by a user, it is necessary for robots to have a plan that fully exploits their functionalities in an environment. The objective of this study is to realize robot task planning in real space for effectively use of the robot's functions The plan is formed by deriving a feasible action sequence by interpreting the instructions within the scope of the action possibilities of the robots and the changes in them. In this paper, we first propose an action graph as a novel environmental representation approach to facilitate the understanding of the robot's action possibility in real space. In the action graph, the action possibility is represented by nodes, which describe the spatial position to perform each feasible action, and edges, which describe the feasible actions, based on the subsystem-level affordance and the arrangement of objects in the environment. We also propose an action-based spatio-temporal robot navigation (ASTRON), which focuses on robot navigation tasks. ASTRON enables the robots to determine a feasible action sequence that utilizes their functions by interpreting the instructions based on the action graph. The effectiveness of the proposed method was evaluated through simulations and actual machine experiments in a coffee shop environment. In the actual machine experiments, the proposed method was applied to robots with different subsystem configurations. The experimental results demonstrated that the proposed method could plan the feasible action sequence to complete the tasks by considering the environmental state and the subsystem configurations of the robot.

Pressure ulcers are caused by tissue damage, which occurs when the blood supply to an area of skin is diminished because of sustained or concentrated pressure on a patient from the mattresses of a bed. One action that concentrates the pressure is a bed head elevation operation. Recently, a control system known as the alternative pressure mattress has been proposed to prevent the concentration of pressure. However, the challenge with this controlled system is that it has a slow response because it uses air cells. The slow response causes controlling the air cells to lag behind the change in pressure during the bed head elevation operation. Therefore, this system cannot adequately prevent the concentration of pressure on a patient. To address this issue, we need to predict the change in pressure during the bed head elevation operation and control the pressure based on this prediction. In the current work, we focus on predicting the pressure distribution. This prediction is high-dimensional. We regarded the pressure distribution as an image and predicted the change in pressure by applying a deep neural network. To the best of our knowledge, this paper is the first to propose the body pressure predictive model.

The goal state of the robot motion involved in an action is determined by the action model based on the effect of the action and the scene state. This study focuses on serving food as a robot action; the action model of serving food is obtained based on demonstrations. The arrangement position is determined by considering the spatial relations between the object being served and the multiple selected reference objects. Therefore, it is necessary to learn the importance of the relations for the reference and the spatial relations between objects based on demonstrations. In this paper, bottom-up action modeling is proposed as an action modeling approach based on the demonstration. In this method, the spatial relations between labeled objects and the importance of the relations are learned through spatial factorization of the scene into partial relations. This approach enables the robot to determine the arrangement position intended by the demonstrator, even in unknown scenes. The effectiveness of the proposed method was evaluated by simulation, which the action of serving food.

As a vast amount of data with respect to the moon and Mars is collected, exploration missions are shifting to the next step, the aim of which is a precise landing on a predetermined target. A promising technology for precision landing is terrain relative navigation (TRN), which collates landmarks detected from images and maps of landmarks. Crater detection is one of the essential technologies for TRN. A problem in detecting craters is the apparent change in craters due to illumination conditions. Another problem is the change in shape due to crater degradation. We propose a novel crater detection method based on combining a support vector machine (SVM) and a convolutional neural network (CNN) to make detection performance robust against apparent change. In the linear SVM, gradient images of a crater image dataset are learned. The learned classifier is then used to calculate the objectness score for region proposal. Next, the CNN identifies the image of the proposed region as to whether or not it is a crater. Our results show that the proposed method can detect craters in a wide range of illumination and shape conditions, and has better average precision than traditional crater detectors.

Recently, the demand for agricultural autonomous robots has been increasing. Using the technology of vision-based robotic environmental recognition, they can generally follow farmers to support their work activities, such as conveyance of the harvest. However, a major issue arises in that dynamic objects (including humans) often enter images that the robots rely on for environmental recognition tasks. These dynamic objects degrade the performance of image recognition considerably, resulting in collisions with crops or ridges when the robots are following the worker. To address the occlusion issue, generative adversarial network (GAN) solutions can be adopted as they feature a generative capability to reconstruct the area behind dynamic objects. However, precedented GAN methods basically presuppose paired image datasets to train their networks, which are difficult to prepare. Therefore, a method based on unpaired image datasets is desirable in real-world environments, such as a farm. For this purpose, we propose a new approach by integrating the state-of-the-art neural network architecture, CycleGAN, and Mask R CNN. Our system is trained with a human-tracking dataset collected by an agricultural autonomous robot in a farm. We evaluate the performance of our system both qualitatively and quantitatively for the task of human removal in images.

Recent image-based robotic systems use predicted future state images to control robots. Therefore, the prediction accuracy of the future state image affects the performance of the robot. To predict images, most previous studies assume that the camera captures the entire scene and that the environment is static. However, in real robot applications, these assumptions do not always hold. For example, if a camera is attached to a mobile robot, its view changes from time to time. In this study, we analyzed the relationship between the performance of the image prediction model and the robot's behavior, controlled by an image-based navigation algorithm. Through mobile robot navigation experiments using front-faced and omni-directional cameras, we discussed the capabilities of the image prediction models and demonstrated their performance when applied to the image-based navigation algorithm. Moreover, to adapt to the dynamic changes in the environment, we studied the effectiveness of directing the camera to the ceiling. We showed that robust navigation can be achieved without using images from cameras directed toward the front or the floor, because these views can be disturbed by moving objects in a dynamic environment.

Background: Footsteps and walking trajectories during the Timed Up and Go test (TUG), and their relationships with spatiotemporal gait parameters during turning in people with Parkinson's disease (PD) and older people have not been clarified. Research question: We investigated the footsteps, walking trajectories, and spatiotemporal parameters during the TUG in PD, healthy older (HO), and healthy young (HY) subjects and the associations between the parameters of footsteps or walking trajectories and the spatiotemporal parameters during turning. Methods: Twenty-six PD, 20 HO, and 20 HY subjects participated. They performed the TUG at self-selected speed and underwent evaluations of spatiotemporal parameters in the forward, turning, and return phases and parameters of footsteps and walking trajectories by a system using laser range sensors to measure the two-dimensional distance data of the legs. Foot placement for each foot was measured as the estimated position when the leg speed of movement reached the minimum between foot-contact and foot-off. We calculated the minimum distance from a TUG marker to footsteps and the maximum anterior distance from the start to the footsteps. Step length was calculated using a method for non-linear walking. Results: The PD subjects showed significantly smaller step lengths in all phases. The minimum distance from the marker to the footsteps in PD subjects was significantly smaller than in HO subjects and was significantly positively correlated only with the turning-phase step length. The maximum anterior distance from the start to footsteps in HO subjects was significantly larger than in PD and HY subjects and was not correlated with any spatiotemporal parameters. Significance: This study demonstrated that people with PD turn close to the TUG marker, and this turning strategy may be associated with the decreased step length when turning. These findings could help in providing instructions which prevent the exaggeration of step-length reduction when turning.

When we perform a task, we select the action by imagining its future consequences. Hence, the ability to predict future states would also be an essential feature for robotic agents because it would allow them to plan effective actions to accomplish given tasks. In this research, we explore an action-conditioned future image prediction model considering its application to navigation tasks for a mobile robot. We investigate the image prediction performance of deep neural network architectures and training strategies with two different camera systems. One camera system is a conventional front-facing camera that has a narrow field of view with high definition, and the other is an omni-directional camera that has a wide field of view with low definition. We compare the performances of prediction models for these two camera systems, and propose using an image prediction model with the omni-directional camera for the navigation tasks of a robot. We evaluate the prediction performance of each camera system through experiments conducted in a complex living room-like environment. We demonstrate that models with an omni-directional camera system outperform models with a conventional front-facing camera. In particular, the model comprising a combination of action-conditioned long short-term memory successfully predicts future images for states of more than 100 steps ahead in both simulation and real-world scenarios. Further, by integrating the proposed system into image-prediction-based navigation algorithm, we demonstrate that navigation based on a model with an omni-directional camera can successfully navigate the robot in cases where one with a conventional front-facing camera fails.

Recently, the demand for service robots that move autonomously in a crowded environment, such as stations, airports, and commercial facilities has increased. Such robots are required to move to a destination without hindering the progress of the surrounding pedestrians. Previous papers proposed collision avoidance methods based on the direction and distance of the destination of the robot, and the position and velocity of pedestrians in the vicinity. However, in a crowded environment, the behavior of a robot may affect other pedestrians, or the behavior of a pedestrian facing the robot may affect other pedestrians. Moreover, considering the motion strategy used by a pedestrian in a crowded environment, the impact on other pedestrians can be reduced by the robot following the pedestrians moving in the direction in which the robot wants to move. This navigation method has not been proposed thus far. Therefore, in this paper, we propose a method that considers the impact of the robot on surrounding pedestrians and the impact of those pedestrians on other pedestrians. The robot determines the avoidance or following action based on the traveling direction of the surrounding pedestrians.

Simultaneous measurement of spatial, temporal, and kinetic gait parameters has become important for monitoring patients with neurological disorders in a hospital. As spatial sensing technologies, laser range sensors (LRSs), which obtain highly accurate two-dimensional distance data over a wide range, have been employed to detect/track both legs during walking. However, when walking on curved trajectories, a continuous occlusion occurs over several sampling steps and produces false tracking. To address this issue in combination with temporal and kinetic sensing, this paper presents a fusion system of LRSs and force-sensing insoles and an occlusion compensation based on probabilistic motion models. First, the system pre-tracks a target person during walking along a straight line and curved paths under different curvatures/directions in a situation without occlusion. Gait cycles of each walking type are divided by foot-grounding obtained from the insoles. Relationships between leg trajectories and traveling directions during the gait cycle are then learned using user-specific Gaussian mixture models (GMMs). When an occlusion occurs in post-tracking, a maximum likelihood (ML) GMM is identified using a joint probability of both legs' trajectories, in accordance with biomechanics that both legs move in a coordinated manner. The ML GMM then compensates for the traveling direction and position of the occluded leg, and the system interpolates/re-tracks its positions to correct the states during occlusion. Experimental results demonstrated that the proposed method (fusion with the insoles, occlusion compensation, and interpolation/re-tracking) significantly enhanced tracking performance during occlusion and estimated leg positions with valid accuracy (errors of under 60 mm).

In Japan, the reduction in agricultural work population and increase in abandoned cultivated land are major social issues. To overcome these issues, we have proposed a small agricultural robot that supports the transportation of harvested crops while it follows the user. To move in the furrowed field, ridges and furrows where the robot can move must be detected using robot-mounted sensors and follow the ridge while avoiding the ridges and crop rows. However, when the robot follows the user ahead of it, it has difficulty detecting the ridges because of occlusion by the user. In addition, the robot requires to follow the user and move in and out of furrows of different widths in the farm field. To achieve smooth and safe harvesting support in the field with different width furrows, we propose a method for ridge detection and tracking by measuring the furrow width and interpolation using an RGB-D camera and a method for human-following control while avoiding ridges according to the ridge observation state using fuzzy set theory. To verify the proposed methods, experiments in the farm are carried out.

In this study, obstacle avoidance in the collaborative robotics field, where a human operator moves the robot manually through direct contact, is investigated. If an operator is distracted or is not used to control the robot, an accident can happen during operation, which might harm the operator. For the purpose of assisting the human operator in his/her task, the integration of a damping control is proposed. Damping is a coefficient that influences the oscillations of a system. By modifying the damping of the robot, an artificial stiffness is created. The information is transmitted to the user through his/her haptic return, making it easier for him/her to avoid the obstacles in the workspace. In this study, a damping control is developed and implemented. Then, through experimentation, the efficiency of the haptic sense and the general damping control are demonstrated, and thus, an efficient, comfortable, and safe method is formulated for the operator to avoid obstacles during operations.

Several approaches have been proposed to assist humans in co-manipulation and teleoperation tasks given demonstrated trajectories. However, these approaches are not applicable when the demonstrations are suboptimal or when the generalization capabilities of the learned models cannot cope with the changes in the environment. Nevertheless, in real co-manipulation and teleoperation tasks, the original demonstrations will often be suboptimal and a learning system must be able to cope with new situations. This paper presents a reinforcement learning algorithm that can be applied to such problems. The proposed algorithm is initialized with a probability distribution of demonstrated trajectories and is based on the concept of relevance functions. We show in this paper how the relevance of trajectory parameters to optimization objectives is connected with the concept of Pearson correlation. First, we demonstrate the efficacy of our algorithm by addressing the assisted teleoperation of an object in a static virtual environment. Afterward, we extend this algorithm to deal with dynamic environments by utilizing Gaussian Process regression. The full framework is applied to make a point particle and a 7-DoF robot arm autonomously adapt their movements to changes in the environment as well as to assist the teleoperation of a 7-DoF robot arm in a dynamic environment.

To prevent the occurrence of the kidnapped robot problem, it is vital to evaluate the likelihood of each particle considering an environmental change. Moving objects are one of the leading causes of environmental change, and each object has its own movability. For example, a chair has high movability because it is designed to move and often interacts with humans. However, walls or shelves have low movability because they are designed not to move, and they interact less often with humans. Therefore, in this study, we define classes of objects and their movability. We propose a localization approach that focuses on the association between sensor information obtained from objects whose movability is low and prior map by considering classes and movability, to prevent the occurrence of the kidnapped robot problem.