Published papers

Autonomous mobile robots (AMRs) perform tasks in various industrial applications to enhance productivity and reduce labor costs. To ensure the safety of humans sharing spaces with robots in operation, most AMRs use visual sensors to avoid collision by detecting humans and obstacles. However, when humans are outside the robot's field of vision, the robot should mitigate the risk of collision by sensing humans' presence before they enter the robot's visual range. In this study, we propose a non-line-of-sight sound source localization algorithm that can be applied to collision avoidance for AMRs. The range of human detection for robots can be increased using auditory sensors. To localize the position of a human in a non-line-of-sight area, such as behind a corner of a corridor, we use direction-of-arrival (DOA) estimation of reflection sounds and the ray tracing method. Performance evaluations in real life were performed to verify the possibility of its implementation in practical scenes.

Patrol robots in hospital wards are expected not only to ensure the safety of their movements, but also to perform gait measurement as part of their monitoring functions. In this study, we present a system that enables a patrol robot to perform a 5-meter walk test (5mWT) on passing pedestrians in a corridor. The pedestrian’s position and velocity are measured sequentially, and the robot’s trajectory is generated based on the results using a nonlinear programming method, followed by Model Predictive Control tracking. This allows for adaptive gait measurement during passing interactions, taking into account arbitrary pedestrian trajectories. Furthermore, a parameter estimation method tailored to the characteristics of the acquired data, namely whole-body, continuous and multi-view measurements, is proposed. Through simulation and experiments, we confirmed the feasibility of the measurement operation and accuracy of the derived gait parameters. The proposed method can not only be used anywhere, but also measure gait over a long distance with high accuracy using a single sensor. Moreover, it could contribute to the realization of gait measurement without pedestrian’s tension.

In recent years, the workload of healthcare workers in hospitals has increased, and there has been a demand for autonomous patrolling instead of nurses and gait measurements of patients in hospital wards. In this study, a novel motion-planning method is proposed to enable a mobile robot to measure the gait of passing pedestrians while patrolling wards. In nonlinear programming, the feasibility of the gait measurement is evaluated as an object function, and constraints on the robot kinematics, environment, collision avoidance, and target position and arrival time are considered in the constraints. The optimization variables are the trajectory of the robot’s position, velocity, and acceleration. This schematization guarantees that the control input, which is the solution to this problem, satisfies all constraints and is optimized to realize gait measurement. The results of the simulation and experiment demonstrated the efficiency and evolvability of the proposed method.

Task planning is a pivotal component of autonomous mobile robots, requiring them to formulate a sequence of actions to achieve a predefined final goal. This requires a semantic understanding of the scene, enabling the robot to comprehend its surroundings and identify objects available for interaction. While some systems rely on environmental sensing for accurate estimation of the obstacles, they can only achieve this under controlled environments and is not flexible nor scalable. In this paper, we address these problems by introducing a new concept of onboard semantic mapping for action graphs. We create an environment map relying solely on an onboard RBG-D camera sensor and generate a graph representing all action sequences that the robot can perform over time. In our method, we integrate a 3D object detector with a visual SLAM pipeline to construct a semantic map, which contains only essential information for the action graph generation. The map is systematically fed into the action graph generator, which computes the potential actions of the robot and the subsequent states of the environment. We evaluate our approach on real-world scenarios in a qualitative and quantitative manner.

In recent years, some countries have embarked on missions to the Moon and Mars to explore the possibility of life and to expand the range of human activities. Simultaneously, several missions are being planned to land on celestial bodies with microgravity and return samples back to Earth. There are two unique challenges in landing and staying on a celestial body with microgravity. The first is that the restoring force due to gravity is small and difficult to balance. Especially, when landing on a slope, there is a risk of large rebound, making it difficult to continue the mission. The second is that the actuator (thruster) can only thrust upward when landing to prevent regolith from scattering. This constraint makes pinpoint landings difficult, because control begins after the rebound and horizontal motion can no longer be controlled independently. To solve this problem, this paper proposes a guidance and control system that enables safe and pinpoint landing. The first step is to model the lander that can represent rebound by explicitly considering the ground. Next, landing reference trajectory design and optimal reference trajectory selection are performed using an optimization algorithm. Finally, the desired control is achieved by tracking the reference trajectory using nonlinear model predictive control while considering actuator constraints. The effectiveness of the proposed method was verified through numerical analysis using a simulator created with MATLAB Simscape Multibody.

In a limited space, such as pressurized rovers, insufficient personal space is a source of stress for astronauts. The aim of this study is to mitigate effects caused by personal space invasions. The proposed system for that problem is using a spotlight system that automatically adjusts based on vital sign feedback. In this study, experiments with human subjects to test the effects of spotlight exposure are conducted. Specifically, respiration signal was measured to investigate the relationship between respiration, distance and the spotlight in a limited space. The results suggest that changes in respiration may reflect the effects of spatial distance, suggesting that respiration may serve as an indicator of personal spatial discomfort. Furthermore, the usefulness of spotlight lighting in confined spaces was demonstrated through its effects on breathing, demonstrating its potential to enhance comfort by adjusting lighting based on vital sign.

The growing prevalence of autonomous driving is expected to shift passengers’ attention from driving, increasing the demand for enhanced ride comfort. Studies addressing ride comfort have prominently explored active suspension control with recent research on preview suspension control using on-board sensors. The proposed systems often include LiDAR deployment at the front for high-precision road surface profiles. However, these systems often involve costly sensors such as LiDAR, making it impractical for on-board installation. Nonetheless, in recent autonomous vehicles, LiDAR tend to be mounted on the roof. It would be beneficial to leverage this LiDAR for preview control, the point cloud obtained from the roof has insufficient density to accurately perceive the unevenness on the road surface. To overcome the low-density issue in point cloud data obtained from less channels LiDAR, this study applies a supervised machine learning model, developed for autonomous driving, to estimate road surface profiles and enhance the precision of these estimations.

The progressive decline in pulmonary function contributes to the risk of chronic respiratory diseases and reduction in quality of life. However, the conventional pulmonary function test, Spirometry, is only available after the patient acknowledges the symptoms. In case of structural damage to the lungs, it is irreversible, and complete treatment is difficult. Therefore, we aimed to establish a simple pulmonary function monitoring system using the photoplethysmography (PPG) sensor on the patch-type multimodal sensor. As a pilot study, we conducted trend analysis of the proposed sensor during simulated hypoxemia and comparison based on smoking history. From hypoxia simulation, respiratory PPG signals are likely to be affected not only by physical respiratory dynamics, but also by internal pulsatile indices. The degree of deformation in signal amplitude and intensity differed by smoking history. In addition, it showed a mild correlation with poor pulmonary function. Although further evaluation with a broader participant pool and clinical pulmonary evaluation is necessary, these findings suggest potential application for early pulmonary function monitoring with the proposed sensor.Clinical Relevance - This suggests the potential use of the PPG signal from the patch-type multimodal sensor for early screening of pulmonary function deterioration, for which many patients remain undiagnosed due to its asymptomatic symptoms.

Net-based debris capture is one of the countermeasures against space debris. This paper considers the system which consists of flexible net with actuators attached to the corner nodes to release multiple debris into transitioned orbit. The debris releasing from a net system is assumed, and the net-debris composite system is attributed to nonlinear programming. The nonlinear dynamics of net’s flexibility and contact between the net and the debris are explicitly considered so that the debris can realize the target speed and angle of velocity, and the optimized control input history of the net system actuators is derived accordingly. The validity of derived actuator inputs is verified by numerical simulations.

Study Design. Cross-sectional study. Objective. This study aimed to objectively evaluate spastic gait and reveal its novel characteristics via analysis of gait in patients with cervical myelopathy (CM) using the Timed Up and Go (TUG) test with a laser range sensor. Summary of Background Data. Among patients with CM, spastic gait is a common diagnostic symptom; thus, objective assessments of spastic gait would be useful for the diagnosis of CM and recognition of disease status. Although spastic gait has been objectively evaluated in previous studies, the methods employed in those studies are not suitable for clinical settings. Methods. In total, 37 and 24 participants were recruited for a control group and CM group, respectively. CM was diagnosed by spine surgeons. We developed a laser TUG test, in which the position and velocity of both the legs were captured. The parameter values for both groups were statistically compared, and odds ratios were calculated using logistic regression analyses. Results. The total TUG-test time, time to stand up, time to first step, number of steps, and trajectory error for the CM group were significantly higher than those for the control group, whereas the average velocity and average stride length for the CM group were significantly lower than those for the control group. There was a significant independent association between the total TUG-test time and CM. The optimal cutoff point of the total test time for CM risk was approximately 9seconds. Conclusion. Through the use of the laser TUG test, we were able to identify characteristics of spastic gait, which leads to difficulty in standing and taking the first step, wobbling while walking, and an increased risk of falling. We found that the risk of CM was higher if the individual took longer than 9seconds to complete the TUG test.

Three-dimensional ground reaction forces (3D GRFs) in various gait patterns (e.g., walking and turning) provide essential information for clinical assessment. Previous scholars proposed to estimate GRFs from insoles with few pressure sensors using machine learning techniques. However, estimating GRFs during turning using a model learned from walking proved difficult because relationships between plantar pressure and GRFs vary according to the gait pattern. In this study, GRFs were estimated based on gait pattern recognition. GRF estimation models were learned from various gait patterns in advance. These models represented relationships between the insole measurements and GRFs using a Gaussian process regression (GPR). The GRFs were estimated from the insole measurements using a maximum likelihood (ML) model recognized as the current gait pattern. The ML model had the largest time frames, in which its standard deviation (SD) of the probabilistically estimated vertical GRF was the smallest among all the models (i.e., its training data were closest to the inputs), in the initial and terminal stance phases (subphases). In addition, the time frames of the contralateral leg were considered to enhance the recognition accuracy based on interlimb coordination. Experiment results showed that the proposed system, which used a common model learned from all the gait patterns for the vertical GRF and the ML model in the subphases of both legs for the horizontal GRFs, estimated 3D GRFs during walking and turning with different curvatures and directions with higher accuracy. The system can be widely applied for clinical walking tests and monitoring in daily life.

Gait measurement in clinical settings is often limited to qualitative evaluations despite the availability of various measurement devices. Although marker-less motion capture techniques, such as Kinect, have been attracting attention recently, they have rarely been introduced in the field because of the complexity of measurement operations. Therefore, in this letter, we propose a gait measurement algorithm that uses the light detection and ranging (LiDAR) camera of an iPad, a widely used mobile device. First, as part of the floor was hidden by the walking foot, the missing points in the floor point cloud were detected as feet. Subsequently, we placed the potentials at the detected foot positions, summed them over a series of frames, and obtained the local maximum value to obtain the ground position during walking. Spatial gait parameters, such as step length and width, were obtained from the ground position. In an accuracy verification experiment with 13 elderly subjects, the average absolute error of the step length was 28 mm, and that of the step width was 19 mm. These results indicated that the values estimated using the proposed method are useful for evaluating clinical tasks.

Recently, collaborative robots, such as collaborative delivery robots, have been expected to improve the work efficiency of users. For natural human-robot collaboration, it is necessary to infer the appropriate goal position to transport instruments, where the user’s convenience and the surrounding environment are considered. In conventional research, the goal is inferred by demonstrating the user’s desired positions, but position demonstration requires many trials to obtain the inference model, which is burdensome for the user. Therefore, we focus on the user’s correction of the robot position and generate multiple position samples from the user’s corrective path. In addition, these position samples are weighted based on the implicit intention of the correction to learn both the desired and undesired positions. Consequently, the robot improves goal inference in fewer trials. The effectiveness of the proposed method was evaluated by experiment that simulated human-robot collaborative environments.

Patients with knee osteoarthritis show low stair climbing ability, but a diagnosis of stair performance time is not enough to identify the early stages of knee osteoarthritis. Therefore, we developed an indicator named range of the knee joint trajectory (RKJT) as a kinematic parameter to express more detailed characteristics than stair performance time. To achieve this, we used our developed “IR-Locomotion”, a markerless measurement system that can track the knee joint trajectory when climbing stairs. This study aimed to test whether the RKJT effectively identifies patients with early knee osteoarthritis even after controlling stair performance time. Forty-seven adults with moderate to severe knee pain (mean age 59.2 years; 68.1% women) underwent the radiographic examination (Kellgren and Lawrence grade) of both knees and a stair climbing test on 11 stairs. The RKJT during the stair climbing test was calculated by “IR-Locomotion”. A generalized linear mixed model was used to evaluate the discriminative capability of RKJT on early knee osteoarthritis (i.e., Kellgren and Lawrence grade of 1). As expected, patients with early knee osteoarthritis showed larger RKJT than non-radiographic controls (95% confidence interval: 1.007, 1.076). Notably, this finding was consistent even after adjusting stair performance time.

Image-based robot action planning is becoming an active area of research owing to recent advances in deep learning. To evaluate and execute robot actions, recently proposed approaches require the estimation of the optimal cost-minimizing path, such as the shortest distance or time, between two states. To estimate the cost, parametric models consisting of deep neural networks are widely used. However, such parametric models require large amounts of correctly labeled data to accurately estimate the cost. In real robotic tasks, collecting such data is not always feasible, and the robot itself may require collecting it. In this study, we empirically show that when a model is trained with data autonomously collected by a robot, the estimation of such parametric models could be inaccurate to perform a task. Specifically, the higher the maximum predicted distance, the more inaccurate the estimation, and the robot fails navigating in the environment. To overcome this issue, we propose an alternative metric, “task achievability” (TA), which is defined as the probability that a robot will reach a goal state within a specified number of timesteps. Compared to the training of optimal cost estimator, TA can use both optimal and non-optimal trajectories in the training dataset to train, which leads to a stable estimation. We demonstrate the effectiveness of TA through robot navigation experiments in an environment resembling a real living room. We show that TA-based navigation succeeds in navigating a robot to different target positions, even when conventional cost estimator-based navigation fails.

Objective: To summarize the level of knowledge regarding the effects of microcurrent therapy (MCT) on musculoskeletal pain in adults. Data Sources: The PubMed, Physiotherapy Evidence Database, Cumulative Index to Nursing Allied Health Literature, Cochrane Central Register of Controlled Trials, and Igaku Chuo Zasshi database were searched from the time of their inception to December 2020. Study Selection: Randomized controlled trials (RCTs) investigating the effects of MCT on musculoskeletal pain were included. Additionally, non-RCTs were included to assess the adverse events. Data Extraction: The primary outcomes were pain and adverse events related to MCT. To assess the reproducibility of MCT, we evaluated the completeness of treatment description using the Template for Intervention Description and Replication (TIDieR) checklist. We also assessed the quality of evidence using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE). Data Synthesis: A comprehensive assessment of 4 RCTs and 5 non-RCTs that met the inclusion criteria revealed that MCT significantly improved shoulder pain (1 study, 40 patients) and knee pain (1 study, 52 patients) compared with sham MCT without any severe adverse events. MCT has clinically significant benefits for knee pain. This study also revealed a clinically significant placebo response in treating knee pain. This evidence highlights the substantial effect of placebo response in clinical care. These treatment effects on knee pain are further supported by the high quality of evidence in GRADE with high reproducibility in TIDieR. Conclusions: The findings of this meta-analysis highlight the effect of placebo response in treating knee pain. MCT is a potential, core nonpharmacologic treatment option in clinical care with minimal adverse events and should be further investigated. This study proposes a framework for the future investigation of the effect of MCT on musculoskeletal pain to enhance the study quality and reproducibility.

Task allocation and path planning considering changes in the mobility of robots in the environment allows the robots to efficiently execute tasks with smaller travel times. A lunar base construction is one of the situations in which robots can more efficiently accomplish its goal by taking such environment changes into account when performing tasks. For the construction, we assumed that when a robot executes a task of building a road, the environment changes such that aisles that were unusable before the task become usable post execution. If such changes in environment are considered in advance, the robot can efficiently plan to wait until the environment changes and can move before executing the task. However, previous studies have not considered such changes, resulting in inefficient planning. To solve this problem, we developed a multi-agent action graph that consists of multiple layers and expresses the environment changes associated with task execution in terms of changes in these layers. In this graph, task allocation and path planning are formulated as a combinatorial optimization problem and are optimized using mixed-integer programming. Multi-agent action graphs and the proposed formulation enable efficient planning considering changes in the robots' mobility in advance. Through simulations, we confirmed that the proposed method completed the construction of the lunar base approximately 16.4% earlier than the conventional method, while consuming approximately 16.0% less total energy of the robots.

Recently, the agricultural population is decreasing, and the demand for autonomous agricultural robots has increased. For a robot to successfully harvest fruits in diverse environments, various obstacles must be pushed away from its path. During the pushing motion, however, gravity acts on the fruits, causing them to move back and forth and return to their original state after being pushed away. This results in fruit-to-fruit contact, and fruit damage may occur. Therefore, a method to avoid obstacles while reducing the risk of fruit damage is required. However, the direct consideration of fruit damage is difficult because it requires many parameters that are difficult to observe. Herein, we propose a motion planning method by considering the potential energy of fruits as an evaluation index, which could be calculated based on observable parameters; this consideration could also suppress the reciprocating motion of fruits, thus preventing fruit damage. The proposed method was evaluated through simulations, and its effectiveness was verified.

This paper proposes a control method of Active Mass Damper (AMD) for Building Mass Damper (BMD). The parameters of the soft story of BMD must be higher than the optimal values to prevent excessive displacement of the soft story. The AMD makes the behavior of the non-optimal BMD close to that of the optimal BMD and improves damping performance by the proposed control method, which is based on Model Predictive Control (MPC). Through the results of numerical analysis and shaking table test, it is demonstrated that the proposed method can mitigate the responses of the non-optimal BMD.

For the proper functioning of mobile manipulator-type autonomous robot performing complicated tasks in a human-robot coexistence environment, tasks and motions must be planned simultaneously. In such environments, a human and robot should collaborate with each other. Therefore, the robot must act in accordance with the human and avoid useless actions duplicated with those of humans. However, any action undertaken by a human has uncertainty, and thus, predicting them correctly is challenging. This study proposed probabilistic task and motion planning considering both deterministic and probabilistic environment changes caused by robot and human actions temporarily and spatially, respectively. First, the environmental changes were modeled, where the robot is capable of recognizing the possibility of environmental changes. Second, in task planning, the probabilities of each environmental change owing to human actions was minimized. Finally, in motion planning, a movement path connecting each task in a planned order was planned, thereby enabling the robot to perform actions not duplicated with those by a human. Furthermore, the plans generated were compared without considering possibility of human actions and the effectiveness of the proposed method was verified. Consequently, the proposed method was confirmed to reduce the time required for finishing the tasks.