Published papers

Optical polarization provides important clues to the magnetic field in blazar jets. It is easy to find noteworthy patterns in the time-series data of the polarization degree (PD) and position angle (PA). On the other hand, we need to see the trajectory of the object in the Stokes QU plane when the object has multiple polarized components. In this case, ironically, the more data we have, the more difficult it is to gain any knowledge from it. Here, we introduce TimeTubes, a new visualization scheme to explore the time-series data of polarization observed in blazars. In TimeTubes, the data is represented by tubes in 3D (Q, U, and time) space. The measurement errors of Q and U, color, and total flux of objects are expressed as the size, color, and brightness of the tubes. As a result, TimeTubes allows us to see the behavior of six variables in one view. We used TimeTubes for our data taken by the Kanata telescope between 2008 and 2014. We found that this tool facilitates the recognition of the patterns in blazar variations; for example, favored PA of flares and PA rotations associated with a series of flares.

Blazars are celestial bodies of high interest to astronomers. In particular, through the analysis of photometric and polarimetric observations of blazars, astronomers aim to understand the physics of the blazar's relativistic jet. However, it is challenging to recognize correlations and time variations of the observed polarization, intensity, and color of the emitted light. In our prior study, we proposed TimeTubes to visualize a blazar dataset as a 3D volumetric tube. In this paper, we build primarily on the TimeTubes representation of blazar datasets to present a new visual analytics environment named TimeTubesX, into which we have integrated sophisticated feature and pattern detection techniques for effective location of observable and recurring time variation patterns in long-term, multi-dimensional datasets. Automatic feature extraction detects time intervals corresponding to well-known blazar behaviors. Dynamic visual querying allows users to search long-term observations for time intervals similar to a time interval of interest (query-by-example) or a sketch of temporal patterns (query-by-sketch). Users are also allowed to build up another visual query guided by the time interval of interest found in the previous process and refine the results. We demonstrate how TimeTubesX has been used successfully by domain experts for the detailed analysis of blazar datasets and report on the results.

In modern society, the digital signage installed in many large-scale facilities supports daily life. However, owing to their limited screen size, it is difficult to simultaneously provide different types of information to many viewers at varying distances from the screen. Therefore, in this study, we extend the existing research on the use of hybrid images for tiled displays. To facilitate smoother information selection, a new interactive display method is proposed that incorporates touch-activated widgets as high-frequency parts of hybrid images. These widgets are novel because they are more visible to viewers near the display. We developed an authoring tool called the hybrid image display resolution optimizer (HYDRO), which features two types of control functions to optimize the visibility of touch-activated widgets in terms of placement and resolution. The effectiveness of the proposed method is demonstrated empirically through a quantitative user study and an eyetracking-based qualitative evaluation.

In this paper, we introduce a novel method for particle-based fluid surface reconstruction that incorporates mesh shaders for the first time. This approach eliminates the need to store triangle meshes in a GPU's global memory, resulting in a significant reduction in the amount of memory needed. Furthermore, our method employs a bidirectional two-level uniform grid, which not only accelerates the computationally expensive stage of surface cell detection but also effectively addresses the issue of vertex overflow among the mesh shaders. Experimental results prove that our method outperforms the state-of-the-art method, achieving both acceleration and memory reductions simultaneously, without sacrificing quality. The method is highly practical, with no major limitations other than requiring a GPU that supports the mesh shaders.

Nowadays, three-dimensional (3D) reconstruction techniques are becoming increasingly important in the fields of architecture, game development, movie production, and more. Due to common issues in the reconstruction process, such as perspective distortion and occlusion, traditional 3D reconstruction methods face significant challenges in achieving high-precision results, even when dense data are used as inputs. With the advent of neural radiance field (NeRF) technology, high-fidelity 3D reconstruction results are now possible. However, high computational resources are usually required for NeRF computations. Recently, a few data inputs are used to ensure the highest quality. In this paper, we propose an innovative low- to highfrequency-guided NeRF (L2H-NeRF) framework that decomposes scene reconstruction into coarse and fine stages. For the first stage, a low-frequency enhancement network based on a vision transformer is proposed, where the low-frequency-based globally coherent geometric structure is recovered, with the dense depth restored in a depth completion way. In the second stage, a high-frequency enhancement network is incorporated, where the high-frequency-related detail is compensated by robust feature alignment across adjacent views using a plug-and-play feature extraction and matching module. Experiments demonstrate that both the accuracy of the geometric structure and the feature detail of the proposed L2H-NeRF outperforms state-of-the-art methods.

Exploring hidden structures in subsequences extracted from long-term time-series data is one of the primary tasks of time-series data analysis. Clustering is one of the most commonly used techniques in that context; however, various existing issues must be addressed, such as the way to extract less-overlapping subsequences and the definition of the inter-subsequence similarity. Especially in multi-dimensional data analysis, correlations among variables should also be emphasized. To boost users’ exploration of the universalities of subsequences, we incorporate multi-dimensional time-series subsequence clustering methods and visual clustering analysis interface into TimeTubesX, which is an integrated visual analytics environment for multi-dimensional time-dependent observation datasets of blazars. TimeTubesX extracts and filters subsequences with various lengths according to the characteristics of the data and clusters them automatically in consideration of correlations among observed attributes. And then, it allows users to visually examine the clustering results in terms of the cluster features, intercluster transitions, and temporal distributions of clusters. Through the application to two practical case studies, we demonstrate how the enhanced TimeTubesX enables users to see not only instances but also universalities (i.e., time-series motifs or cluster prototypes) in time-series observations of blazars.

Weathering, an expression of degradation caused by rain and wind, is essential for photorealistic computer graphics. One of the commonest targets of weathering is metal, which is omnipresent in reality. However, for the realistic reproduction of scenes, many of which display degradation, the application of rust-proof paint to metal surfaces cannot be ignored. In our study, we propose a weathering method for coated films on metal objects, which are modeled using a three-dimensional (3D) triangular polygon mesh and deformed by combining two kinds of simulations: static simulation, for determining fractures based on the balance of the internal forces, and position-based bend simulation for moving vertices according to geometric constraints. Our method can digitally reproduce the deterioration of coated films using complex 3D deformation, which is difficult to achieve by material manipulation only.

This paper proposes a stable method for reconstructing spatially varying appearances (SVBRDFs) from multiview images captured under casual lighting conditions. Unlike flat surface capture methods, ours can be applied to surfaces with complex silhouettes. The proposed method takes multiview images as inputs and outputs a unified SVBRDF estimation. We generated a large-scale dataset containing the multiview images, SVBRDFs, and lighting appearance of vast synthetic objects to train a two-stream hierarchical U-Net for SVBRDF estimation that is integrated into a differentiable rendering network for surface appearance reconstruction. In comparison with state-of-the-art approaches, our method produces SVBRDFs with lower biases for more casually captured images.

Opal, a renowned gemstone, exhibits an optical phenomenon known as “play of color,” which is classified as a kind of structural color. This unique property makes opal a highly valuable material from the natural science, industry, and humanity perspectives. Opal has a complex internal structure called colloidal polycrystalline, and thus, representations of opal remain unestablished in computer graphics. In this study, to approximate the complex internal structure of opal, we imitate its formation process using the three-dimensional additively weighted Voronoi tessellation and percolation model. Further, we apply path tracing to compute the diffraction patterns of visible light inside opal by utilizing the Ewald construction employed in X-ray diffraction theory. Finally, we achieve a visually plausible output through spectral rendering.

Metamorphopsia patients experience distortions in their views. The number of metamorphopsia patients is arising with the increase in the elderly population. For early detection of metamorphopsia, examination approaches have been proposed. However, none of the existing methods can be employed to precisely assess the perceptions of patients. We propose LineM as a novel method to precisely measure the distorted views of metamorphopsia patients based on interactive line manipulation task involving the patients. To explore the potential of LineM, a case study which uses the measured results from LineM to visualize the distorted views of metamorphopsia patients is presented. Furthermore, a novel technique for compensating the distorted views of individual metamorphopsia patients using the results from LineM is also presented. Considering that elderly individuals account for the majority of metamorphopsia patients, an intuitive and user-friendly prototype system has been implemented. The effectiveness and usability of LineM and the compensation method were demonstrated via a qualitative and subjective experiment that involved metamorphopsia patients and elderly participants.

Homonymous hemianopia (HH) is a visual field defect in which the same side of both eyes is blind. In this paper, we propose an out-of-visual-field information compensation system for HH patients using an optical see-through head-mounted display. In the proposed system, in order to reduce the obstruction in the remaining visual field, indicators are placed at the boundary between the remaining visual field and the defective part to notify the HH patient about the events occurring at the blinded side of the visual field. Two different kinds of indicators, namely brightness-based and flicker-based, are designed to visualize the spatial information of the events. We conducted experiments with simulated HH participants to verify the effectiveness of the proposed system and compare the two designed indicators, as well as an existing method that makes use of the overlay overview window. The experimental results proved that the proposed system can react more intuitively and is faster than the existing method in the part near the central visual field. Moreover, it can alleviate the obstruction problem of the existing method.

Visual field defects (VFDs) can be caused by numerous diseases, some of those (e.g.,glaucoma) are the main causes of blindness in humans. The present perimetry, which is defined as the measurement of visual field function, has high requirements for patients such as long-term fixation, reliable interaction with the system. In this pilot study, we combined the videos and eye tracking techniques in a free-watching task, explored the different eye movement behaviors of people with several types of artificial VFDs, such as hemianopia, altitudinal VFDs, and tunnel vision. We carried out the task in gaze-contingent modality, 38 participants with normal vision were recruited to watch a group of videos monocularly. The eye gaze data were recorded by the tracker during the task. We hypothesized that people with simulated VFDs will produce a more active eye movement coping mechanism to compensate for their visual field disadvantage. A new measurement called eye movement amount (MA) was proposed to describe the amount of eye movement toward a specific direction. Statistically significant differences caused by artificial VFDs were observed by comparing the MA values between the VFD groups and the control group. In addition, we found artificial hemianopia and inferior VFD lead participants to produced increased MA in horizontal and vertical directions respectively. Artificial tunnel vision, on the contrary, induces a decreased MA in both horizontal and vertical directions. The proposed metrics can be used as potential biomarkers for distinguishing VFDs in video free-watching task.

In forensic autopsy, medical examiners (MEs) and diagnostic radiologists (DRs) cooperate with each other to perform an autopsy of the corpse. Effective computational assistance tools are imperative for facilitating the intricate collaborative work involved in the autopsy. In this paper, we present an integrated visual analysis environment named FORSETI (forensic autopsy system for e-court instruments), whose technical essence is two-fold. The first is to be designed on the basis of an extended version of legal medicine mark-up language for authoring reports on physical autopsy (PA) as well as on virtual autopsy (VA). The second lies in autopsy juxtaposition, which seamlessly assists the MEs and DRs in referring to the VA and PA works, respectively. A fictitious case with the Visible Female Dataset is used to demonstrate the effectiveness of an initial prototype of the FORSETI system.

Autopsy reports play a pivotal role in forensic science. Medical examiners (MEs) and diagnostic radiologists (DRs) cross-reference autopsy results in the form of autopsy reports, while judicial personnel derive legal documents from final autopsy reports. In our prior study, we presented a visual analysis system called the forensic autopsy system for e-court instruments (FORSETI) with an extended legal medicine markup language (x-LMML) that enables MEs and DRs to author and review e-autopsy reports. In this paper, we present our extended work to incorporate provenance infrastructure with authority management into FORSETI for forensic data accountability, which contains two features. The first is a novel provenance management mechanism that combines the forensic autopsy workflow management system (FAWfMS) and a version control system called lmmlgit for x-LMML files. This management mechanism allows much provenance data on e-autopsy reports and their documented autopsy processes to be individually parsed. The second is provenance-supported immersive analytics, which is intended to ensure that the DRs’ and MEs’ autopsy provenances can be viewed, listed, and analyzed so that a principal ME can author their own report through accountable autopsy referencing in an augmented reality setting. A fictitious case with a synthetic wounded body is used to demonstrate the effectiveness of the provenance-aware FORSETI system in terms of data accountability through the experience of experts in legal medicine.

We present a novel compositing framework for full-length human figures that maintains their surface details and appends the localized nature of light and shadow, thereby synthesizing composite results with high visual coherence. The framework is extended from the compositing pipeline proposed in our previous study so that it deploys five stages for photometric information estimation, as well as for 3D reconstruction, global illumination simulation, lighting transfer, and compositing. Based on the interpretation that a sense of coexistence can be achieved through visual coherence, we demonstrate that the proposed framework functions properly as a group portrait compositor. The composite results that the proposed framework composed the images separately rendered 3D human models compared favorably with the results which rendered multiple avatars together. Based on this empirical evaluation, the proposed framework is expected as a new means of fostering a sense of coexistence in remote societies and of efficiently generating highly photorealistic cyberworlds.

Creating realistic materials is essential in the construction of immersive virtual environments. While existing techniques for material capture and conditional generation rely on flash-lit photos, they often produce artifacts when the illumination mismatches the training data. In this study, we introduce DiffMat, a novel diffusion model that integrates the CLIP image encoder and a multi-layer, cross-attention denoising backbone to generate latent materials from images under various illuminations. Using a pre-trained StyleGAN-based material generator, our method converts these latent materials into high-resolution SVBRDF textures, a process that enables a seamless fit into the standard physically based rendering pipeline, reducing the requirements for vast computational resources and expansive datasets. DiffMat surpasses existing generative methods in terms of material quality and variety, and shows adaptability to a broader spectrum of lighting conditions in reference images.

Game creators always use prefabricated models as non-player characters (NPCs) to create an atmosphere for game scenes. However, players are likely to see different NPCs with the same 3D models because creators usually cannot prepare a high number of NPC models for big game scenes. In this article, we present an anime-like character customization system, where each customizing parameter can adjust the shape or color for the corresponding part of the character model. Based on this system, we propose an improved approach for generating a rich variety of 3D anime-like NPCs including body models and clothing items in different styles. We introduce a neural network to control the facial appearances, Gaussian mixture models to control the colors of hair and clothes, and a Bayesian network to control the outfits of clothing items. We demostrate that the proposed approach can maintain the variety and stability of the generated characters.

We propose a visual analysis approach for exploring the compositional process of composers in the Spectral School, with a particular focus on its sub-processes of sound analysis and synthesis. Spectral music has been one of the significant trends in contemporary music since the 1970s. Composers in the Spectral School use the acoustic properties of sounds as the basis of their compositional materials. One of the representative software systems they use is AudioSculpt, developed by IRCAM. We develop an accompanying system with AudioSculpt to manage the history of creating sounds with spectral analysis of sound materials. The salient feature of our system lies in its tessellated spectrogram space whose axes represent the elapsed time of the sound and the progress of composition. On the pixel-oriented, spatial substrate, the system makes it possible for users to analyze the compositional processes by using dedicated interactive manipulations. The present system can also be regarded as an initial attempt at managing the provenance of time-series events in the music visualization field. Our approach is intended to pave the way for composers and musicologists to analyze, develop, and share the compositional methodologies.

Various attempts at exploiting information visualization for sports have recently been reported in the literature, although it is still challenging to analyze continuous ball matches. In this paper, we propose a novel visual analytics system, called TideGrapher, to track the transition of tactile situations in a rugby match. With a particular focus on the side position of the ball, we designed a dedicated spatial substrate based on the spatio-temporal trajectory of the ball and provided a set of basic interactions. Quantitative analysis was strengthened by adding a new index, called initiative, to commonly used possession (ball occupation) and territory (dominance of territory). The feasibility of the proposed visual analytics system was proven empirically through application to datasets from real amateur and professional matches.