Published papers

Summary: Deep inferior epigastric perforator flaps are commonly used for breast reconstruction using autologous tissue. For such free flaps, the internal mammary artery provides stable blood flow as the recipient for anastomosis. We report a novel dissection method of the internal mammary artery. First, the perichondrium and costal cartilage of the sternocostal joint are dissected with electrocautery. Then, the incision on the perichondrium is extended along the cephalic and caudal ends. Next, this C-shaped superficial layer of perichondrium is elevated from the cartilage. The cartilage is incompletely fractured with electrocautery, with the deep layer of perichondrium intact. Then, the cartilage is completely fractured by leverage and removed. The remaining deep layer of perichondrium is incised at the costochondral junction and shifted aside, revealing the internal mammary artery. The preserved perichondrium creates a rabbet joint to protect the anastomosed artery. This method not only enables a more reliable, safer dissection of the internal mammary artery, but also allows reusage of the perichondrium as underlayment in the setting of anastomosis, and coverage for the incised rib edge, protecting the anastomosed vessels.

To create anatomical educational materials that can be viewed in three dimensions using stereo photographs and photogrammetry, multiple photographs must be taken from different directions. In this process, shadows and reflections from different positions in each photograph are undesirable for creating three-dimensional (3D) anatomy educational materials. Although a ring flash eliminates shadows, allowing light to enter from all directions, reflections cannot be eliminated. In particular, Thiel-embalmed cadavers, which are widely used in clinical anatomy, are highly wet and exhibit strong specular highlights. In this study, a straight polarization filter was attached to a handheld camera lens and ring flash, and shooting was performed using cross-polarization photography. Consequently, even in Thiel-embalmed cadavers, the details lost due to the effects of reflections and shadows can be recovered, and good results can be obtained when taking stereo photos or creating a 3D model using photogrammetry.

Background: With the standardization of surgical techniques and continuous development of plate fixators, medial open-wedge high tibial osteotomy (OWHTO) has become an option for the treatment of medial knee osteoarthritis. However, it is also associated with several complications. To reduce the risk of these complications, it is essential for surgeons to accurately understand the three-dimensional (3D) anatomical structures, including the surroundings of the surgical field. Gross dissection using a cadaver provides a large surgical field and is the best learning method for studying anatomical structures in three dimensions; however, it is not available at all medical institutions. Methods: One female cadaveric knee, fixed using the Thiel method, was used in this study, and OWHTO was performed. A series of photographs was taken from approximately 60 different angles at each stage of the autopsy for digital photogrammetry to ensure that there was at least a 60% overlap between photographs. Results: Each 3D object was created to be faithful to the original specimen and displayed on an augmented reality headset. It was created to capture the subtle nuances of irregular shapes and structures, highlighting the unique ability of photogrammetry to capture pathology. This method allows surgeons to visualize the location of neurovascular injuries in the intraoperative field of view. Conclusions: Using a technique that applies digital photogrammetry to create 3D objects from handheld photographs, this reports for the first time an educational tool that can reproduce the anatomy related to high tibial osteotomy in three dimensions.

Occlusion-free video generation is challenging due to surgeons’ obstructions in the camera field of view. Prior work has addressed this issue by installing multiple cameras on a surgical light, hoping some cameras will observe the surgical field with less occlusion. However, this special camera setup poses a new imaging challenge since camera configurations can change every time surgeons move the light, and manual image alignment is required. This paper proposes an algorithm to automate this alignment task. The proposed method detects frames where the lighting system moves, realigns them, and selects the camera with the least occlusion. This algorithm results in a stabilized video with less occlusion. Quantitative results show that our method outperforms conventional approaches. A user study involving medical doctors also confirmed the superiority of our method.

Purpose: The venous plexus of Rektorzik (VPR), first described by Rektorzik in 1858, is a venous plexus around the internal carotid artery in the carotid canal. However, the VPR has never been investigated using the recently developed imaging modalities. In this study, we analyzed the VPR using computed tomography-digital subtraction venography (CT-DSV). Methods: This study included 253 patients who had undergone head CT-DSV. The presence or absence of the right and left VPRs and their connecting veins were visually examined by two researchers. Results: The VPR was observed in 60 patients (24%), 39 of which showed VPR only on the right side, 10 only on the left side, and 11 on both sides. VPR was significantly more common on the right side (p = 0.0002) and was observed more frequently around the horizontal segment of the internal carotid artery than around the vertical segment. The most common veins identified as distal and proximal VPR connections were the cavernous sinus (63/71, 89%) and the anterior condylar confluence (27/71, 38%), respectively. The mean age was significantly lower in patients with the VPR than in those without (53 vs. 57 years, p = 0.02). Conclusion: The VPR was significantly more frequent on the right side and in younger patients but was not a radiographically constant structure. In most cases, the VPR connected the cavernous sinus and anterior condylar confluence. Preoperative evaluation of VPR may lead to refined surgical procedures.

Objectives: Information regarding the depth of lymphatic vessel is important for lymphatic surgeons because rapid identification of functional lymphatic vessels and veins is necessary to perform good lymphaticovenular anastomosis, which is a surgical procedure for lymphedema cases. Photoacoustic lymphangiography (PAL) may be useful for such identification because it allows the assessment of the depth of lymphatic vessels. Thus, we aimed to measure the lymphatic vessel depth using images obtained by PAL. Methods: This study included healthy individuals and patients with lymphedema. In all participants, indocyanine green dissolved in dextrose was injected subcutaneously into the first and fourth webs of the foot and the lateral malleolus, and PAL was performed on the medial side of the lower leg. The lymphatic vessel depth was measured from the ankle joint, 10 cm above the medial malleolus, and 20 cm above the medial malleolus on PAL in the cross-sectional view and was compared between the participant groups. Results: The healthy group (mean age, 43.3 ± 12.9 years) included 21 limbs of 4 male and 16 female healthy individuals (bilateral limbs of 1 patient were considered). The lymphedema group (mean age, 62.0 ± 11.7 years) included 17 limbs of 3 male and 14 female patients with lymphedema. The average lymphatic vessel depths from the ankle joint, 10 cm above the medial malleolus, and 20 cm above the medial malleolus were 2.6, 4.7, and 5.6 mm in the healthy group and 3.6, 7.3, and 7.4 mm in the lymphedema group, respectively. Lymphatic vessels were significantly deeper in the lymphedema group than in the healthy group at all measurement locations. Conclusions: Using PAL, we determined the lymphatic vessel depth in living bodies. By searching for the lymphatic vessels based on our findings, even surgeons who are relatively inexperienced with lymphatic surgery may be able to identify functional lymphatic vessels more efficiently.

Multi-camera multi-person (MCMP) tracking and re-identification (ReID) are essential tasks in safety, pedestrian analysis, and so on; however, most research focuses on outdoor scenarios because they are much more complicated to deal with occlusions and misidentification in a crowded room with obstacles. Moreover, it is challenging to complete the two tasks in one framework. We present a trajectory-based method, integrating tracking and ReID tasks. First, the poses of all surgical members captured by each camera are detected frame-by-frame; then, the detected poses are exploited to track the trajectories of all members for each camera; finally, these trajectories of different cameras are clustered to re-identify the members in the operating room across all cameras. Compared to other MCMP tracking and ReID methods, the proposed one mainly exploits trajectories, taking texture features that are less distinguishable in the operating room scenario as auxiliary cues. We also integrate temporal information during ReID, which is more reliable than the state-of-the-art framework where ReID is conducted frame-by-frame. In addition, our framework requires no training before deployment in new scenarios. We also created an annotated MCMP dataset with actual operating room videos. Our experiments prove the effectiveness of the proposed trajectory-based ReID algorithm. The proposed framework achieves 85.44% accuracy in the ReID task, outperforming the state-of-the-art framework in our operating room dataset.

Mounting multi-view cameras within a surgical light is a practical choice since somecameras are expected to observe surgery with few occlusions. Such multi-view videos must bereassembled for easy reference. A typical way is to reconstruct the surgery in 3D. However, thegeometrical relationship among cameras is changed because each camera independently moves everytime the lighting is reconfigured (i.e., every time surgeons touch the surgical light). Moreover, featurematching between surgical images is potentially challenging because of missing rich features. Toaddress the challenge, we propose a feature-matching strategy that enables robust calibration ofthe multi-view camera system by collecting a set of a small number of matches over time whilethe cameras stay stationary. Our approach would enable conversion from multi-view videos to a3D video. However, surgical videos are long and, thus, the cost of the conversion rapidly grows.Therefore, we implement a video player where only selected frames are converted to minimizetime and data until playbacks. We demonstrate that sufficient calibration quality with real surgicalvideos can lead to a promising 3D mesh and a recently emerged 3D multi-layer representation. Wereviewed comments from surgeons to discuss the differences between those 3D representations on anautostereoscopic display with respect to medical usage.

Recording surgery in operating rooms is a crucial task for both medical education and evaluation of medical treatment. In this paper, we propose a method for visualizing surgical areas that are occluded by the heads or hands of medical professionals in various surgical scenes. To recover the occluded surgical areas, we utilize a surgery recording system equipped with multiple cameras embedded in the surgical lamp, with the aim of ensuring that at least one camera can capture the surgical area without occlusion. We propose the application of a transformer-based Neural Radiance Field (NeRF) model, originally proposed for normal scenes, to surgery scenes, and demonstrate through experimentation that it is feasible to generate occluded surgical areas. We believe this research has the potential to make our multi-camera recording system practical and useful for physicians.

Recording surgery in operating rooms is one of the essential tasks for education and evaluation of medical treatment. However, recording the fields which depict the surgery is difficult because the targets are heavily occluded during surgery by the heads or hands of doctors or nurses. We use a recording system which multiple cameras embedded in the surgical lamp, assuming that at least one camera is recording the target without occlusion. In this paper, we propose Conditional-BARF (C-BARF) to generate occlusion-free images by synthesizing novel view images from the camera, aiming to generate videos with smooth camera pose transitions. To the best of our knowledge, this is the first work to tackle the problem of synthesizing a novel view image from multiple images for the surgery scene. We conduct experiments using an original dataset of three different types of surgeries. Our experiments show that we can successfully synthesize novel views from the images recorded by the multiple cameras embedded in the surgical lamp.

The portable document format (PDF) is a globally recognized standard file format of Word documents. The PDF supports the embedding of three-dimensional objects. This three-dimensional PDF can contribute to new ways of presenting and disseminating medical information in future medical journals.

Lymphaticovenular anastomosis (LVA) is a widely performed surgical procedure for the treatment of lymphedema. For good LVA outcomes, identifying lymphatic vessels and venules is crucial. Photoacoustic lymphangiography (PAL) is a new technology for visualizing lymphatic vessels. It can depict lymphatic vessels at high resolution; therefore, this study focused on how to apply PAL for lymphatic surgery. To visualize lymphatic vessels, indocyanine green was injected as a color agent. PAI-05 was used as the photoacoustic imaging device. Lymphatic vessels and veins were visualized at 797-and 835-nm wavelengths. First, it was confirmed whether the branching of the vasculature as depicted by the PAL was consistent with the actual branching of the vasculature as confirmed intraoperatively. Second, to use PAL images for surgical planning, preoperative photoacoustic images were superimposed onto the patient limb through augmented reality (AR) glasses (MOVERIO Smart Glass BT-30E). Lymphatics and venule markings drawn using AR glasses were consistent with the actual intraoperative images obtained during LVA. To anastomose multiple lymphatic vessels, a site with abundant venous branching was selected as the incision site; and selecting the incision site became easier. The anatomical morphology obtained by PAL matched the surgical field. AR-based marking could be very useful in future LVA.

Background: Dermal backflow (DBF), which refers to lymphatic reflux due to lymphatic valve insufficiency, is a diagnostic finding in lymphedema. However, the three-dimensional structure of DBF remains unknown. Photoacoustic lymphangiography (PAL) is a new technique that enables the visualization of the distribution of light-absorbing molecules, such as hemoglobin or indocyanine green (ICG), and can provide three-dimensional images of superficial lymphatic vessels and the venous system. This study reports the use of PAL to visualize DBF structures in the extremities of patients with lymphedema after cancer surgery. Methods: Patients with a clinical or lymphographic diagnosis of lymphedema who previously underwent surgery for cancer at one of two participating hospitals were included in this study. PAL was performed using the PAI-05 system. ICG was administered subcutaneously in the affected hand or foot, and ICG fluorescence lymphography was performed using a nearnfrared camera system prior to PAL. Results: Between April 2018 and January 2019, 21 patients were enrolled and examined usng PAL. The DBF was composed of dense, interconnecting, three-dimensional lymphatic vessels. It was classified into three patterns according to the composition of the lymphatic vessels: A linear structure of lymphatic collectors (pattern 1), a network of lymphatic capillaries and lymphatic collectors in an underlying layer (pattern 2), and lymphatic capillaries and precollectors with no lymphatic collectors (pattern 3). Conclusions: PAL showed the structure of DBF more precisely than ICG fluorescence lymphography. The use of PAL to visualize DBF assists in understanding the pathophysiology and assessing the severity of cancer-related lymphedema.

外科医が直視下で行う手術(open surgery)では、手術室スタッフがカメラを細かく調整する必要があるうえ、術中にカメラと術野の間に外科医の頭や身体が入り込むため、術野の撮影が困難で、全録画も不可能であった。こうした課題に対応するため、5台のカメラを搭載した「マルチカメラ搭載型無影灯」を開発しており、これまでopen surgeryの全録画が可能であることを実証してきた。今回、1)マルチカメラ搭載型無影灯映像の質について、外科医20名による検証を行うとともに、2)口蓋裂の患者に対して1歳頃に行う口蓋形成術の患者説明に同映像を用い、その評価を3名の患者家族へのアンケート調査から検討した。1)については、5台のカメラが撮影した5視点の映像を、AIを用いて生成した切り替え映像が、同時に撮影した医療用のヘッドマウントカメラ(「ナノカム」)と統計学的に劣らないQoV(映像の質)を示し、Stability(映像の安定性)、Line of Sight(術野が動画の中心に映っているか)については有意に優れているとの評価を得た。また、2)の患者説明における手術動画の活用については、患者家族より「動画の視聴により手術のイメージができた」「不安の解消に役立った」といった評価を得た。

撮影による手術映像の記録は手術技能の共有に効果的である一方,それらは長時間に渡る点,また管理のコストや労力の点からあまり実用されない現状にある.この問題の解決手段として機械学習を用いた手術映像の工程分類による自動要約が挙げられるが,形成外科手術においては空間的な撮影難度が原因で行われていなかった.本研究ではマルチカメラ搭載型無影灯を用いた形成外科手術野の撮影による動画データセットの作成を行った上で,それらを用いた形成外科手術の工程分類を行う際の多視点映像利用の有効性の検証を行う.(著者抄録)

© 2019 IEEE. Recording medical surgery operations is important for sharing the various operating techniques. In most operating rooms, fixed surgery cameras are already installed, but it is almost impossible to capture the surgical field because of occlusion by the surgeon's head and body. In order to capture the surgical field in real surgery operations, we proposed the installation of multiple cameras in a surgical lighting system, so that at least one camera can capture the target surgical field even when the surgeon's head and body occlude other cameras. In this paper, we present a method for automatic viewpoint selection from multi-view surgical videos, so that the surgical field can always be recorded in the output video. We employ a method for learning-based object detection from videos for automatic evaluation of the surgical field area from multiple input videos. By selecting the viewpoint with the largest area of the surgical field, we can virtually reduce the area of the surgeon's head and hands. In general, frequent camera switching degrades the video quality of view (QoV). Therefore, we apply the Dijkstra method widely used in the shortest path problem as a combinatorial optimization method for this problem. Our camera scheduling method is that the camera switching is not performed for a certain period of time, and the surgical field observed in the entire video is maximized.

© 2020, Springer Nature Switzerland AG. Recording surgery in operating rooms is an essential task for education and evaluation of medical treatment. However, recording the desired targets, such as the surgery field, surgical tools, or doctor’s hands, is difficult because the targets are heavily occluded during surgery. We use a recording system in which multiple cameras are embedded in the surgical lamp, and we assume that at least one camera is recording the target without occlusion at any given time. As the embedded cameras obtain multiple video sequences, we address the task of selecting the camera with the best view of the surgery. Unlike the conventional method, which selects the camera based on the area size of the surgery field, we propose a deep neural network that predicts the camera selection probability from multiple video sequences by learning the supervision of the expert annotation. We created a dataset in which six different types of plastic surgery are recorded, and we provided the annotation of camera switching. Our experiments show that our approach successfully switched between cameras and outperformed three baseline methods.

Copyright © 2020 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved. Recording surgery is important for sharing various operating techniques. In most surgical rooms, fixed surgical cameras are already installed, but it is almost impossible to capture the surgical field because of occlusion by the surgeon’s head and body. In order to capture the surgical field, we propose the installation of multiple cameras in a surgical lamp system, so that at least one camera can capture the surgical field even when the surgeon’s head and body occlude other cameras. In this paper, we present a method for automatic viewpoint switching from multi-view surgical videos, so that the surgical field can always be recorded. We employ a method for learning-based object detection from videos for automatic evaluation of the surgical field from multi-view surgical videos. In general, frequent camera switching degrades the video quality of view (QoV). Therefore, we apply Dijkstra’s algorithm, widely used in the shortest path problem, as an optimization method for this problem. Our camera scheduling method works so that camera switching is not performed for the minimum frame we specified, and therefore the surgical field observed in the entire video is maximized.

Recording surgery is an important technique for education and the evaluation of medical treatments. However, capturing targets such as the surgical field, surgical tools, and the surgeon's hands, is almost impossible since these targets are heavily occluded by the surgeon's head and body during a surgery. We used a recording system in which multiple cameras are installed in the surgical lump, supposing at least one camera would capture the target without occlusion. As this system records multiple video sequences, we address the task to select a best view camera automatically. Recently, learning-based approaches in a fully supervised manner have been proposed for this task, but these previous approaches completely rely on manual annotation of the training data. In this paper, we focus on the eye tracker mounted on the surgeon's head, which can capture the recording targets without occlusion. Employing this first-person-view video synchronized with multiple videos of the surgical lump, we propose a novel camera selection approach using a self-supervised learning framework. In experiments, we created a dataset composed of four different breast surgery. Our extended experiments showed that our approach successfully switched to the best camera view without manual annotation and achieved competitive accuracy compared with conventional supervised methods. Also, our approach yielded effective visual representations comparable to state-of-the-art self-supervised learning frameworks.

Detecting surgical tools is an essential task for the analysis and evaluation of surgical videos. However, in open surgery such as plastic surgery, it is difficult to detect them because there are surgical tools with similar shapes, such as scissors and needle holders. Unlike endoscopic surgery, the tips of the tools are often hidden in the operating field and are not captured clearly due to low camera resolution, whereas the movements of the tools and hands can be captured. As a result that the different uses of each tool require different hand movements, it is possible to use hand movement data to classify the two types of tools. We combined three modules for localization, selection, and classification, for the detection of the two tools. In the localization module, we employed the Faster R-CNN to detect surgical tools and target hands, and in the classification module, we extracted hand movement information by combining ResNet-18 and LSTM to classify two tools. We created a dataset in which seven different types of open surgery were recorded, and we provided the annotation of surgical tool detection. Our experiments show that our approach successfully detected the two different tools and outperformed the two baseline methods.