Published papers

In this paper, we propose a system for the intuitive control of fluid behavior through direct manipulation. The proposed system recognizes specific hand gesture commands to manage underlying 3D fluid simulation and adjust the parameters of fluid behavior. Thus, users are allowed to control the dynamics of fluid easily without detailed knowledge of physical simulation. The proposed system is expected to bring the benefits of 3D fluid simulations to a broader range of users.

A key compositional technique, leading lines guide the observer's gaze across visual scenes. This study presents a novel approach to the visual analysis of leading lines in artworks by applying discrete Morse theory to saliency maps refined through persistent homology. The saliency map’s gradient field is abstracted into a structured representation using the maximum graph, enabling the identification of both explicit and implicit leading lines. To enhance the analysis, texture gradient cues are incorporated to embed depth information into the saliency map. Our visual analysis system also features semiautomatic thresholding and tools to depict the direction of visual guidance. An experimental study with participants from an art university validated the system’s feasibility by analyzing variousWestern paintings. This research contributes to the field of digital and public humanities and computational art analysis by offering an environment in which to explore visual composition.

Virtual reality (VR) live performances offer a novel style of experience through the deployment of three-dimensional cartoonlike performers and through spatiotemporally asynchronous participation. However, such experiences often lack the presence and atmospheric richness of physical live concerts. To address this gap, prior research has proposed audience non-player characters (NPCs) that emulate the collective characteristics of real-world audiences to enhance viewer engagement. In this study, we extend existing approaches by incorporating individual motion diversity to synthesize audience NPCs that form a heterogeneous and lifelike crowd. In addition to collective characteristics, such as music-driven motion changes and gradual synchronization, our modeling approach introduces agent-level diversity, even within the same motion categories. To assess the psychological impact of the proposed method, we developed a VR concert scenario featuring approximately 1,500 audience NPCs and conducted an empirical evaluation of the user experience. The findings indicate that enhanced motion heterogeneity can increase a viewer's sense of presence, engagement, perceived unity, and co-existence in a VR concert environment.

In the present work, we attempted at expressing human "two-facedness" by the "two-sidedness" of the 3D structures. In the field of geometry, there exists a two-facedness called “duality,” which represents the relationship between diagrams that switch between faces (cells) and vertices. Delaunay diagram and Voronoi diagram give a typical example. We have developed an algorithm to generate a Poisson-disk distribution in an arbitrary shape to make sculptures with those diagrams. The Delaunay diagram generated by Poissondisk distribution point cloud makes up the uniform tetrahedral mesh (except for the surface). Since those cells are close to a regular tetrahedron, Voronoi vertices located on the circumcenters also have an almost uniform distribution.

Even when generated with the same weapon, a　wound's appearance would be affected by its use and the assaulter’s physique. In contrast, even if wounds look similar in shape and/or color, they could have a different wounding history. In this work, we strive to extract the features of shape and color from a given wound image and build on the mathematical model of meaning to create the associated semantics of the criminal act. We demonstrate that a system called THEMIS (theoretical estimation of the meaning of insults) provides a context-sensitive visual similarity analysis for wound imagery in computational forensics. The THEMIS system can help forensic doctors and e-court stakeholders potentially determine important aspects of a case through a comparison with wounds of corpses in past cases, in a way that is not currently possible.

Flow effects such as flames, smoke, and liquids play an important role in activating illustrations, but drawing these effects requires artistic expertise as well as a great deal of effort. In this paper, we propose a method for adding stereoscopic effects to character illustrations using various shapes of 3D pottery wheel-type transparent canvases. One approach to designing a flow effect to decorate a character relies on simple curved geometry to beautify its flowin an organized composition.We extend this approach to present a drawing system—SpiCa (spinning canvas), which enables users to use rotatable transparent canvases to design 3D flow effects. User evaluations show that users can create such effects more easily and effectively and reduce their workload with SpiCa in comparison with an existing 2D illustration tool.

Virtual space can eliminate the physical constraints of real space. Three-dimensional (3D) digital twins enable us to experience manipulation of cultural assets that are inaccessible in real space. However, most 3D models obtained using current reconstruction techniques are static; they cannot be manipulated dynamically. In this study, we reproduce a dynamic 3D model of a simple articulated object with a rotating joint from a static point cloud only with reference to a video of the motion and a manually added rotation axis. The reconstructed actual cultural asset is used for first-person experiences in an augmented reality environment.

Accomplished artists often incorporate leading lines into their compositions to guide the observer’s attention. Although saliency maps are typically employed to locate attractive regions in still images, such scalar features do not express the trajectory of an observer’s gaze. In this study, we propose a method for the visual analysis and approximate identification of leading lines based on maximum graphs, sparse subsets of Morse–Smale complexes, extracted from saliency maps. We provide empirical evidence substantiating the feasibility of our method through a comparison with actual observers’ eye tracking results. Further, we investigate the limitations of our approach by employing it to analyze a variety of artworks with diverse styles.

Virtual reality (VR) live concerts have recently attracted much attention because of their various possibilities for future music entertainment in cyberworlds. However, it has been reported that current VR live experiences do not reach the levels of enthusiasm observed at actual live concerts, and achieving a sense of enthusiasm in VR live concerts is a challenge to offer enough engagement. According to studies in the field of social psychology, psychological interactions with the surrounding audience are crucial to generating a sense of enthusiasm at actual live events. In response, this paper proposes a method for synthesizing enthusiastic collective non-player characters (NPCs) for VR live concerts. The proposed method computes the psychological interactions among the audience members, including the viewer, according to two psychological parameters. The synthesized audience NPCs are affected by both the music and their psychological interactions according to the surrounding audience, including the viewer, and they exhibit the features of an enthusiastic collective crowd. As such, an empirical study was conducted to explore the psychological influence of the proposed audience NPCs on VR live experiences. The results indicate that the proposed method could give the impression of an enthusiastic crowd at VR live experiences. In addition, it was suggested that social sensations between the audience NPCs and the viewer should be enhanced to increase the level of enthusiasm portrayed during VR live experiences. What was further suggested is the need for a certain degree of individual difference and diversity in appearance and motions for a natural collective audience.

In this paper, a novel compositor to realize illumination-aware digital image compositing is proposed. Digital image compositing is a technique to re-synthesize a single image from multiple visual elements. Assembling separated visual elements leads to highly efficient content creation at risk of losing visual coherence. Visual coherence has two types: positional and illumination. Compositing while maintaining illumination coherence is considered a challenging task, because light transport information is needed to reproduce global illumination effects—color bleeding, soft shadows, caustics, and so on. The pipeline of the proposed compositor incorporates shape reconstruction, as well as global illumination simulation and relighting; thus, the compositor can reproduce background-to-foreground color bleeding and soft shadows. It is empirically evaluated that the proposed compositor compares favorably with rendering both in terms of quality and speed. Because it is in principle impossible to render the foreground and background elements together, the proposed compositor is inevitably selected to generate images with the two types of visual coherence. This leads to efficient construction of photorealistic cyberworlds.

Metamorphopsia is a symptom caused by abnormalities in the retina, and people with metamorphopsia experience distortions in their field of view. To compensate for this disorder, computer-based distortion assessment and compensation methods have been proposed. In the state-of-theart method, compensation is carried out by dynamically deforming the image with a manipulation map, which is bound to the symptom of an individual user with metamorphopsia, according to the user’s gaze data captured by an eye tracker. However, this method suffers from the instability and inaccuracy of the eye tracker, which leads to a poor compensation effect. In this paper, we propose a novel method for metamorphopsia compensation without using an eye tracker. The proposed method generates a compensation image by simultaneously applying multiple manipulation maps to the input image. To evaluate the effectiveness of the proposed method, a preliminary subjective experiment using a reading task and involving 10 participants with normal vision was conducted. The evaluation results show that the proposed method can compensate for visual distortion during reading tasks.

Autopsy reports are imperative for both medical and legal science. Medical examiners (MEs) and diagnostic radiologists (DRs) cross-reference autopsy findings, while judicial personnel derive legal documents. In a prior study, we proposed a visual analysis system named FORSETI (forensic autopsy system for e-court instruments) with x-LMML (extended legal medicine markup language) for MEs and DRs to author and review e-autopsy reports. In this paper, we outline our extended work in progress to introduce a provenance infrastructure for forensic data accountability to FORSETI, which can be characterized by two technical essences. The first is a provenance management mechanism that combines the forensic autopsy workflow management system (FAWfMS) and lmmlgit (a version control system for x-LMML files), allowing a large amount of provenance information about e-autopsy reports and their documented autopsy processes to be individually parsed. The second is authority management, which ensures the confidentiality of e-autopsy reports by deploying strict syntax-guided workflow controls and a custom-tailored tool.

Human beings perceive scenes by enlarging an area of interest (AOI), whereas an ordinal camera captures the entire scene evenly. Therefore, it is not easy to share subjective impressions with others effectively only by sharing normal photos. Such challenges exist in cyberworlds as well. In this study, we design Digitus View Finder (DVF), a system that automatically outputs images close to the scenes a user perceives. A preliminary user study indicated that the DVF would be able to estimate a plausible scale factor of the AOI as a function of three parameters: the number of subjects in the AOI, the distance to the subjects, and a personal difference as a correction term.

The digitization of material objects is rapidly advancing, progressively offering people an unprecedented level of convenience and accessibility. However, studies have shown that people feel a lesser attachment to digital contents than to material objects. In this paper, we focus on shelves, thought to be hubs for cultivating individual relationships with material objects in daily life. We thus propose HomeShelf, an application that accelerates interactions with digital contents and fosters deeper attachments to them. Further, we discuss the features to be added using new technologies and the impact on digitized material objects.

People with metamorphopsia suffer from perceiving things in a distorted way. Various methods for examining metamorphopsia have been suggested in the current literature, with the most advanced techniques demonstrating the ability to yield quantitative measurements. However, these cutting-edge methods necessitate extended examination durations and impose challenging manipulations on patients. In this study, our objective is to enhance the time efficiency of the inspection process and alleviate the burden placed on the user. We propose a novel user-friendly quantitative inspection system which utilizes interactive reinforcement learning. Instead of having users directly operate the system, we ask them to evaluate the stimuli generated by the system. Based on their evaluations, the system gradually refines the deformation map representing the distortion perceived by the user. The reinforcement learning scheme is implemented using relevance feedback approach based on optimum-path forest classifier. To evaluate the effectiveness of the proposed system, subjective evaluation experiments involving simulated and real metamorphopsia participants were conducted in this study. The experimental findings reveal that, when compared to the state-of-the-art method, our proposed system yields comparable inspection out-comes while significantly reducing both the inspection duration and the mental workload.

In recent years, ray tracing has begun to be used for real-time rendering. However, the number of rays that can be emitted in real time is limited due to the high time complexity of ray tracing. In this paper, we propose affine-transformed ray alignment as a novel method of reducing time complexity. Using affine transformations, the proposed method aligns multiple rays to form a single ray while maintaining the intersection points obtained from the scene, thereby speeding up ray tracing without introducing errors. We also propose application of our method to the problems of primary ray traversal, stereo rendering, and multi-frame rendering and empirically prove that time complexity can be reduced by these simple implementations.

People with Homonymous Hemianopia (HH) suffer from losing ipsilateral half side of visual field in both eyes, which results in failing to obtain visual information in the lost field. Making using of the remaining of the visual field, the state-of-the-art studies proposed Overlaid Overview Window (OOW) and Edge Indicator (EI) on the basis of Augmented-Reality (AR) glasses for compensation. However, experiments conducted in these studies investigate user performance with tasks involving events in lost field or remaining field singly. On the other hand, both studies recruited normal individuals for mock experiment, while their way to simulate HH, which requiring the participants to fix their view angles, were not practical to real HH patients. In this study, we conduct a contextual information experiment to investigate the user performance involving in the task requiring the information across both the visible and invisible sides of HH, with the compensation of OOW and Flicker-based EI (FEI). At the same time, we also recruit volunteers with normal vision for mock experiment, while the participants in our study are allowed to move their gaze freely, because we simulate the invisible field of HH on AR glasses with eye tracking. The experiment results showed that OOW is better for the task that related to move something from the remaining FoV to the lost FoV, while FEI is better for moving something from the lost FoV to the remaining FoV.

People who suffer from color vision deficiency (CVD) can face difficulties when communicating with others by failing to identify target objects referred by their color names. While most existing studies on CVD compensation have focused on the issue of color contrast loss. Although there are approaches can provide clues of color name to users, these techniques either require training, or cannot protect users' privacy, i.e., the fact of having CVD. In this paper, based on augmented reality (AR) and deep learning technologies, we propose a novel system to provide supporting information to users affected by CVD for color communication assistance. The state-of-the-art deep neural network (DNN) model for referring segmentation (RS) is adopted to generate supporting information, and AR glasses are utilized for information presentation. To improve the performance of the proposed system further, a new dataset is constructed based on a novel concept called Color-Object Noun Pair. The results of evaluation experiments show that the new dataset can enhance the performance of the adopted DNN model, and the proposed system can help users affected by CVD successfully identify target objects by their color names.

Exploring data features using visual clustering is a significant challenge of big data analytics. In this vision paper, we focus primarily on the relationship among visual data clustering, the discovery of universalities, and the design of an evolutionary database to propose an inter-disciplinary method for scientific data management. The feasibility of the proposed method is empirically proven through application to a practical visual analytics environment for time-varying multi-dimensional datasets of blazar observations.

Shell mapping is a method of representing microstructures on surfaces by mapping a three-dimensional texture space bijectively into a shell space. This paper introduces a volumetric texture to the texture space for the purpose of representing implicit surfaces and translucent materials. However, conventional approaches often generate artifacts at the boundaries of neighboring tetrahedra due to piecewise-linear approximate mapping. To achieve smooth mapping, we present an approach of mapping rays nonlinearly into the texture space, and we also address efficient sampling by using hierarchical volumetric textures.