Published papers

Many methods using eye tracking have been proposed for realizing interfaces for everyday device including digital signage and HMD. Such visual interfaces have a potential to be utilized as audio interface. We herein intend to allow the viewer to select the audio data by detecting his/her gaze with a single webcam. As a result, we will be able to provide the viewer with an immersive environment where he/she can focus on the object easily. Also, we present a preliminary design of the interface with motivation for application to a museum or an art gallery.

We propose a visual analysis approach to explore the compositional process of composers in Spectral School, with a particular focus on the sub-processes of sound analysis and synthesis. Spectral music is one of the significant trends in contemporary music since the 1970s. Composers in Spectral School use the acoustic properties of sound spectra as the basis of their compositional materials. One of the representative software systems they use is AudioSculpt, which has been developed by IRCAM. We have developed an accompanying system with AudioSculpt to represent the history of creating sounds with spectral analysis of sound materials and processing of sound spectra using the various filters. The salient feature of the system lies in its stacked spectrogram space whose axes represent the elapsed time of the sound and the progress of composition. On the pixeloriented spatial substrate, the system makes it possible for users to analyze the compositional processes by dedicated interactive manipulations. As such, the users are allowed to examine the compositional processes of musical pieces, whereas primary targets of most previous studies were only completed pieces. The present system can also be regarded as an initial attempt at managing the provenance of timeseries events in music visualization field. Our approach is intended to open a door for the composers to develop and share the compositional methodologies.

Computer visualization is widely used to analyze various phenomena in many areas. It is useful to explore the target dataset which has a few physical fields, but it is hard to understand multiple fields simultaneously. Especially, in flow fields, since most datasets have vector field and its associated multiple scalar fields, we are faced with the above problem as long as we rely only on visualization. In this paper, we focus on a multisensory realization method that combines haptization with visualization for the analysis of flow fields. We propose the use of adaptive visual transfer functions which are based on the topological structure of a target scalar field. Haptic transfer functions, which produce three-dimensional forces, are designed to analyze the vector field in a haptic manner. We evaluate the effectiveness of our method using a time-varying three-dimensionalwake turbulence dataset.

Nowadays a variety of interaction technologies have been developed and some of them allow a user to interact in non-contact ways such as through body gestures. Such non-contact interactions become to be utilized for wide purposes, including communication with digital signage, which is one of the new types of media and often used as advertisements in public. However, current utilization of non-contact interactions for digital signage advertisements lacks consideration of its effects from the aspect of consumer behaviors. In this paper, we study the effects of non-contact interactions on digital signage advertisements through hypothesis testing in the light of consumer behaviors. Our results suggest the pros and cons of non-contact interactions for digital signage advertisements. We discuss how we can capitalize the strengths of non-contact interactions for digital signage advertisement.

This paper proposes the methods for creating a human-body model that reflects individual victim characteristics and obtaining position information on the human-body model for the purpose of forensic autopsy data visualization. Although modern imaging technologies, such as CT and MRI, make it possible to obtain fine volumetric human-body model, these technologies are still not widely used in the real forensic autopsy sites due to the cost. To enable a forensic specialist to easily create 3D models of victims, we propose a new method which uses an affordable depth-sensing camera and requires few user interactions. To visualize the autopsy data directly on a 3D model, we also propose a method for specifying a position on the 3D model without impeding the progress of the autopsy procedure.

In this paper, we propose a simple interface for novices to design the shape of 2D fluid intuitively through direct manipulation, where parameters of fluid simulation are specified with hand motions to be captured. Recognizing specific hand motions as hand gesture commands allows the users to control the dynamics of 2D fluid at will, without detailed knowledge of fluid simulation.

Making realistic yarn objects, a common material for textile, without losing details is a challenging issue. Recent methods for modeling yarn can be classified into two categories: volume-based methods and fiber-based ones. Among them, one of the latest fiber-based methods attempts to reconstruct twisted fibers from CT (Computed Tomography) images to improve the details of yarn models. While this method dramatically improves the details, mainly because of the limited resolution of the CT scanner, the fluff and fuzz of the yarn still cannot be obtained. Our goal is to realize the fluff and fuzz of yarn. To that end, we developed an extension method to attach microfibers to each fiber of the yarn. The method approximates the distribution of microfibers by recursively using statistical data. In this paper, we discuss the preliminary design of the method.

The design of automatic transfer functions for volume rendering is a perennial problem in volume visualization. For the last two decades, a variety of design methodologies has been proposed. However, sensitive adjustment of related control parameters remains entrusted to users, because rendering conditions such as the size of a target dataset and sampling distance of ray casting differ on a caseby- case basis. Our group proposed one-dimensional transfer functions to accentuate topological changes in the scalar field of the target dataset. However, the method forces us to determine the actual control parameter values for the transfer functions in an empirical manner. In this extended abstract, we propose a supplementary mechanism to judiciously define an appropriate profile of opacity values. In our approach, the height and width of the hat functions to accentuate feature isosurfaces are determined according to the number of voxels belonging to the relevant topologically equivalent scalar field interval. The feasibility of the proposed method is evaluated by application to three kinds of volume dataset.

デジタルサイネージやヘッドマウントディスプレイなど, さまざまな日常生活レベルのデバイスのインタフェースとして視線追跡を利用する手法が数多く研究されている. 一方で, 視覚インタフェースはオーディオとの融合にも大いに可能性があると考えられる. 本研究では, 一般的なカメラを1 台だけ用いて得られたユーザの視線方向によって, 再生するオーディオデータを選択可能にする手法を提案する. 視線追跡をベースとしたオーディオインタフェースを開発することで, ユーザが対象物に集中しやすい環境を作り出し, 臨場感を提供する ことを目的とする. また，美術館における予備実験や博物館のオーディオガイドへの適用を前提とした予備実験を行う.

運動中の呼吸データをキャプチャし，そのデータを適用したアニメーションを作成する．本研究では，モーションデータに合わせて実際に運動し，その時の呼吸動作をキャプチャする．この2 つのデータを，3D アバタへの前段階として，肺・横隔膜モデルを付与した拡張ボーンモデルに適用し，呼吸データを適用していないボーンモデルに比べて，力感や間合いの表現が向上したアニメーションを作成する．さらに，そのアニメーションを3D メッシュモデルへ拡張する方法を提案する．

吹奏楽活動のなかには，演奏者が演奏しながら定められた経路に従って歩くことにより隊形を作る，マーチングという演奏行為が存在する．本ポスタでは，市販のビデオカメラにより撮影されたマーチングの動画像から，演奏者の移動経路の精度を視覚的に解析するシステムを提案した．本システムは，動画像に映っている演奏者個人の移動経路を，物体追跡アルゴリズムを用いて同定し，さらにその演奏者が歩くべき経路と実際の経路を照らし合わせることにより，誤差を表示する．演奏者の選択や，移動経路が可視化された動画像の再生は，専用ユーザインタフェースを介して容易に実行できる．対象とする演奏者が他の演奏者や楽器により遮蔽された場合でも，その演奏者が動画像に再び現れるまで，それまでの移動ベクトルを用いて位置を推測することにより頑健に追跡を続ける．本システムにより演奏者は自分の改善点を知ることができ，その結果マーチング中の隊形の完成度を高めることが可能となった．

頭髪の色を変えるヘアカラーリングは, 施術後の髪色を事前に見ることができないため, 想像していたものとはかけ離れた髪色になることが多いという問題が生じる. 本研究では, ヘアカラーリングのなかでも毛髪の色を脱色するブリーチという方法を対象とし, ユーザがブリーチを行う時間を指定することで, 施術後の髪色を3D頭髪モデルにより提示することを目的とする. 毛髪の色が2種類のメラニン色素の量により決定する事実に着目し, そのメラニン量を表すパラメタを物理ベースの頭髪シェーディングモデルに組み込み, このパラメタ値により頭髪の色を操作可能にした. そして, 実際の毛髪のブリーチ実験による髪色変化量データをもとにメラニンパラメタ値を変化させることで, ブリーチによる頭髪の色変化を写実的に表現するビジュアルシミュレーションを実現した.

直接操作によって2次元流体を簡易に作成できるインタフェースを提案した．本ポスタでは，流体シミュレーションおよび流体制御に用いるパラメータ調整を特定のモーションデータに対応させた直接操作によって，直感的な流体制御を試みた．これにより，流体シミュレーションの知識を必要とせず，2 次元流体を簡易に形状設計することが可能となった．モーションデータの取得には，Leap Motion を用いた．特定のモーションデータをコマンドとして識別し，2次元空間上でコマンドに応じた流体シミュレーションを実行した．

In this study, we take various features inherent to rugby into account to propose a novel pixel-oriented visualization method that helps spectators immediately grasp the transition of tactile situations in a match. We also deploy a set of indices to quantitatively analyze the strategic advantage that a team gains in the match. We demonstrate the effectiveness of our visual analysis by applying it to a real match: South Africa vs. Japan in the Rugby World Cup 2015.

Groove, the sense of rhythmic feel or musical swing, is one of the most essential factors of a good musical performance. Groove visualization can help players/listeners acquire groove sensation and share impressive expressions of music, regardless of personal hearing ability or musical sense. In this paper, we present SeeGroove2 as an extension of our previous system SeeGroove, by featuring a new groove visualization scheme relying on an orbit metaphor. Rhythm patterns are visually interpreted as various orbital shapes by plotting each played note as an orbiting circle and smoothly interpolating them. Thanks to the adoption of a module-oriented and multi-threaded architecture, the system can change the orbit shapes interactively as the music performance progresses, and thereby the user can feel groove transitions in the music on the fly.

When a musical instrument player performs music, the accompanying visual information can have a significant effect on the performance. For example, several players in a jam session may change their style of playing immediately by closely examining the co-players’ expressions and behaviors and predicting their emotion and intention on the fly. In this paper, we propose a system that generates videos in response to the impromptu performance of a single musical instrumental player. The system evaluates the input signals in an affective way and generates a corresponding video based on the results of the evaluation. The player tends to change his/her performance while being inspired by the generated video and giving further triggers for the system to modify the video. The system aims to establish such an affective loop, where it is expected to act as a “co-player” who continues having an influence on the performance of the real player. The final goal of this study is to improve the quality of the player’s performing experience through such interactions between the player and the system. By conducting a user evaluation, it was proven that we were able to inspire an amateur guitarist as the subject through the affective video generation and provide a cyberworld where he is allowed to experience a better performance than playing alone.

本講演は，2016年8月26日に北海道函館市公立はこだて未来大学で開催された画像電子学会第278回研究会（共催：映像情報メディア学会映像表現&コンピュータグラフィックス研究会）の招待講演である．ここでは，具体的なビジュアル情報処理の研究事例を参照し，視覚データの生産から消費までパイプラインを拡大するとともに，20年周期の研究スパイラルを意識することの学術的意義について平易に解説した．

Justice needs efficient tools to exact an accurate and fair judgment. Our research aims at developing a subset of those tools for storage and utilization of autopsy data (autopsy being the medical examination to identify the cause of death, through LMML, the Legal Medicine Mark-up Language, a mark-up language whose syntax and semantics are designed according to the operative procedure certified by the Japanese Society of Legal Medicine. Using an interface we developed, the user (i.e. a medical examiner, a prosecutor or a lawyer) can manage, create, edit LMML files and output autopsy visualizations. The scope of this poster was focused on how injuries are described within the framework.

Blazars are active galactic nuclei whose relativistic jets ejected from the central black hole are pointing toward the Earth. Astronomers have attempted to classify blazars, but analyzing the time-dependent multivariate datasets with conventional visualization methods, such as scatter plot matrices, is difficult. This paper presents TimeTubes, a new visualization scheme that allows astronomers to analyze dynamic changes in and feature causality among the multiple time-varying variables. We target six representative time-varying variables from the originals, including two polarization-related parameters and their corresponding errors, intensity, and color. The four polarization parameters with a common time stamp are transformed to an ellipse, and a series of such ellipses are aligned in parallel along the time line to form a volumetric tube in 3D space. The resulting tube is then colorized by the observed intensities and colors of the blazar. We designed a designated interface with nine functions to control the view of the tube interactively. The usability of TimeTubes was discussed with feedback from astronomers.

Fighting against crime is paramount to any society, maybe more today than ever before. Tools to fight and elucidate crime are rooted in forensic science. Through the autopsy of a body, we can answer a whole range of questions as to how death happened and come up with explanations and counter-measures so that the same dire circumstance does not happen again. Now, because the reports collecting the data are written manually, the recording of the data collected through traditional autopsy still is a cumbersome, time-consuming task. Our framework, based on a mark-up language (that we dubbed ''LMML'') to store, describe and arrange forensic data, aims at overcoming those issues. Our contribution is twofold: the design of the syntax and semantics of LMML, and the conception of an interface to create, edit, analyse or query files written in that language. Thus, this framework allows quicker, smoother input of forensic data, for better automation and visualization thereof, so that they can be used by medical examiners, investigators, as well as judicial courts.