Published papers

Although some coarse grained reconfigurable arrays (CGRAs) have a function to unify multiple processing elements (PEs) to enhance the energy efficiency, it sometimes causes propagation of glitches widely resulting in the power increases. We propose a dynamic power model considering glitches and an optimization technique using it for CGRAs. The model aims to estimate the energy consumption from the switching counts of a PE array approximately. The model and optimization were applied to a real chip of the low power CGRA called the VPCMA (Variable Pipeline Cool Mega Array). Compared with the energy estimation with a post-layout simulation, the model could estimate it with more than 10000 times faster with smaller error from the results of the real chip measurement. The optimized pipeline structure using the proposed method achived better energy consumption compared to fixed pitch pipeline structures in most cases.

The design of the network topology of a large-scale parallel computer system can be represented as an order/degree problem in the graph theory. To solve the order/degree problem, we have to obtain all-pairs-shortest-path (APSP) for the graph. A conventional APSP algorithm for GPUs is based on the adjacency matrix (ADJ-APSP). When focusing on low-degree and unweighted graphs, most of the matrix elements are zero in the first few iterations of the algorithm. We will further speed up the APSP algorithm by treating the adjacency matrix as a sparse matrix in the first iterations of the algorithm. Evaluation results show that our proposed algorithm on a single GPU (NVIDIA GeForce RTX 3080) reduces the execution time by up to 32.7 % compared to the conventional algorithm.

Recently proposed irregular networks can reduce the latency for both on-chip and off-chip systems with a large number of computing nodes and thus can improve the performance of parallel application. However, these networks usually suffer from deadlocks in routing packets when using a naive minimal path routing algorithm. To solve this problem, we focus attention on a lately proposed theory that generalizes the turn model to maintain the network performance with deadlock-freedom. The theorems remain a challenge of applying themselves to arbitrary topologies including fully irregular networks. In this paper, we advance the theorems to completely general ones. To apply the idea of the turn model to arbitrary topologies, we introduce a concept of regions that define continuous directions of channels on an n-dimensional space. Moreover, we provide a feasible implementation of a deadlock-free routing method based on our advanced theorem. To reduce the latency and the number of required Virtual Channels (VCs) with this method, a heuristic approach is introduced to reduce the number of prohibited turns between channels. Experimental results show that the routing method based on our proposed theorem can improve the network throughput by up to 138 % compared to a conventional deterministic minimal routing method. Moreover, it can reduce the latency by up to 2.9 % compared to another fully adaptive routing method.

Although convolutional neural networks (CNNs) have plenty of parallelism, traditional layer-by-layer task division designs for multi-FPGA systems have the following problems: (1) The computational load of each layer is different from each other, so the execution time is dominated with the heaviest one. (2) Each FPGA must be designed independently, it means that we must design, generate and manage various configuration files. To address this problem, we propose a horizontal division method that enables us to use of a single design for each FPGA. All layers are divided horizontal direction of the target CNN, and a set of layers is implemented on an FPGA. It reduces the time of design as well as management costs for the execution. Also, since the weight data can be separated, the usage of local memory can be reduced. The apparent disadvantage of this method is that it requires all-to-all data communication between FPGA boards, and so it is not suitable to traditional multi-FPGA systems with a simple linear network. Here, we tried to apply the method to FiC (Flow-in-Cloud) which has a powerful network to enable efficient broadcasting. A simple CNN LeNet and a matrix multiplication for more practical fully connected layer is implemented on the FiC prototype. As a result of the evaluation, LeNet using 8 FP-GAs achieved 7.5 times faster than that with a single FPGA, and achieved 12.6 times faster than the optimized software of a high-end CPU.

A multi-FPGA system, Flow-in-Cloud (FiC) system is currently being developed as a server for Multi-access Edge Computing (MEC), one of the core technologies of 5G. FiC system is composed of mid-range FPGAs directly connected by highspeed serial links and works virtually as a single FPGA with huge resources. Since the applications of MEC are sometimes timing- critical, a Static Time Division Multiplexing (STDM) network has been built on the FiC system. However, the STDM network suffers from the extended latency and low usage of the network resource especially when the network traffic is light. Here, we propose a hybrid router that allows packet switching to use empty slots of the STDM. The evaluation results from a real system appear that packet switching is 2.42 times faster than STDM with 8 boards FFT. Also, we propose and evaluate a dynamic allocation method that changes the switching mode according to the network load.

Pattern matching is a versatile task which has a variety of applications including genome sequencing as a major application. During the analysis, short read mapping technique is used where short DNA sequences are mapped relative to a known reference sequence. This paper discusses the use of reconfigurable hardware to accelerate the short read mapping problem. The proposed design is based on the FM-index algorithm. Although several pattern matching techniques are available, FM-index based pattern search is perfectly suitable for genome sequencing due to the fastest mapping from known indices. In order to make use of inherent parallelism, a multi-FPGA system called Flow-in-Cloud (FiC) is used. FiC consists of multiple boards, mounting middle scale Xilinx’s FPGAs and SDRAMs, which are tightly coupled with high speed serial links. By distributing the input data transfer with I/O ring network and broadcasting I-Table, C-Table and Suffix-Array with the board-to-board interconnection network, about 10 times performance improvement was achieved when compared to the software implementation. Since the proposed method is scalable to the number of boards, we can obtain the required performance by increasing the number of boards.

In order to cope with computation cost and energy required for the recent deep learning technology, domain specific systems have been used in the cloud computing so as to be shared by many application developers efficiently. Although GPU(Graphics Processing Unit)s and more specialized systems like TPU (Tensor Processing Unit)s have been popularly utilized, FPGAs have been receiving an attention especially for their power efficiency and flexibility. Since energy efficiency is one of the most important issues in recent cloud computing, a lot of researches to use FPGAs in the cloud have been asserted[3], and commercial systems including Amazon's F1 instance are available. However, the performance improvement in the FPGA is limited by the upper limit of the FPGA resource, even an FPGA in the cloud. Thus, in order to implement a large deep layer learning application, we must use an expensive high-end FPGA or adopt a lightweight algorithm sacrificing throughput and accuracy. To deal with this problem, in the project 'Power-saving AI engine and platform with heterogeneous engine integrated cloud' supported by NEDO started to develop a large-scale AI system called Flow-in-Cloud (FiC)[4]. FiC is consisting of a number of middle-scale economical FPGAs interconnected with high communication bandwidth network. From an HLS (High Level Synthesis) programmer, a lot of FPGAs can be handled as if they were a single large FPGA. The programmer can implement a large-scale deep learning model without caring resources on a single FPGA. FiC is managed by the Flow-OS, and shared efficiently by many users. Although FiC is designed to build a heterogeneous computing system, the current prototype is consisting of multiple FPGA boards called 'FiC-SW' each of which provides both the switching and computing capabilities. Here, as a case study of such an energy efficient multi-FPGA board computing, we implemented the inference part of Long Short Term Memory (LSTM) [1].

A multi-FPGA System called Flow-in-Cloud (FiC) is currently being developed as an accelerator of Multi-access Edge Computing (MEC), which is one of the core 5G technologies. FiC is a system that consists of multiple mid-range FPGAs tightly connected by high-speed serial links. Since timing-critical jobs are assumed to be executed in MEC, a circuit-switched network with static time-division multiplexing (STDM) switches has been implemented on FiC. In this paper, we investigate the methods to enhance the interconnection performance of FiC. We proposed the link aggregation and the slot distribution for making the use of multiple lanes naturally supported with Aurora-IP and economical FireFly cables. We evaluated combinations of the techniques by using three practical applications on a real FiC prototype with 24 boards. When the number of slots is large and the number of packets is small, the slot distribution is sometimes more effective, while the link aggregation is superior for other most cases. As a result, the communication performance could be improved up to 3.01 times by using four lanes. Furthermore, we found that application designers can use enough resources even using four lanes and the link aggregation is better than the slot distribution.

A multi-FPGA system Flow-in-Cloud (FiC) consists of nodes with a mid-class cost-efficient FPGA and Raspberry Pi3B connected with high speed serial links. It aims to implement a large scale AI applications which is difficult to be implemented on a single FPGA by dividing the target into a number of boards. On the other hand, overlay domain specific architecture receives attention for implementing complicated application programs easily on the FPGA. Here, we focus on an open-source AI compiler framework Apache TVM, and its implementation for FPGA, VTA(Versatile Tensor Accelerator). In order to use FiC through the TVM, we implemented it on FiC and executed the ResNet-18 inference benchmark. The preliminary evaluation results showed that VTA on FiC-SW achieved up to 10 times performance compared to the execution on ARM Cortex-A54 software.

SUMMARY Multi-FPGA systems have gained attention because of their high performance and power efficiency. A multi-FPGA system called Flow-in-Cloud (FiC) is currently being developed as an accelerator of multi-access edge computing (MEC). FiC consists of multiple mid-range FPGAs tightly connected by high-speed serial links. Since time-critical jobs are assumed in MEC, a circuit-switched network with static time-division multiplexing (STDM) switches has been implemented on FiC. This paper investigates techniques of enhancing the interconnection performance of FiC. Unlike switching fabrics for Network on Chips or parallel machines, economical multi-FPGA systems, such as FiC, use Xilinx Aurora IP and FireFly cables with multiple lanes. We adopted the link aggregation and the slot distribution for using multiple lanes. To mitigate the bottleneck between an STDM switch and user logic, we also propose a multi-ejection STDM switch. We evaluated various combinations of our techniques by using three practical applications on an FiC prototype with 24 boards. When the number of slots is large and transferred data size is small, the slot distribution was sometimes more effective, while the link aggregation was superior for other most cases. Our multi-ejection STDM switch mitigated the bottleneck in ejection ports and successfully reduced the number of time slots. As a result, by combining the link aggregation and multi-ejection STDM switch, communication performance improved up to 7.50 times with few additional resources. Although the performance of the fast Fourier transform with the highest communication ratio could not be enhanced by using multiple boards when a lane was used, 1.99 times performance improvement was achieved by using 8 boards with four lanes and our multi-ejection switch compared with a board.

Network throughput has become an important issue for big-data analysis on Warehouse-Scale Computing (WSC) systems. It has been reported that randomly-connected inter-switch networks can enlarge the network throughput. For irregular networks, a multi-path routing method called k-shortest path routing is conventionally utilized. However, it cannot efficiently exploit longer-than-shortest paths that would be detour paths to avoid bottlenecks. In this work, a novel routing method called k-optimized path routing to achieve high throughput is proposed for irregular networks. We introduce a heuristic to select detour paths that can avoid bottlenecks in the network to improve the network throughput. Experimental results show that the proposed k-optimized path routing can improve the throughput by up to 133 % compared to the conventional k-shortest path routing. Moreover, it can improve the network bandwidth while maintaining the low network latency.

Body bias control is a fundamental technique widely used to provide an efficient tradeoff between leakage power and performance in ultralow-power systems. Therefore, a lot of research about power optimization which provides optimal power supply and body bias voltages has been carried out. However, considering the actual voltage sources, the conventional approaches suffer from limited performance/power control granularity and may lead to degradation in terms of the energy efficiency. Therefore, in this paper, a power optimization method that improves the performance/power control granularity is proposed and evaluated with real processor chips. In the proposed optimization, the body biases for nMOSFET and pMOSFET are controlled independently, while the conventional methods control them uniformly. This increases the number of possible voltage combinations and allows finer target frequency selection leading to lower power consumption than the conventional methods at the cost of the optimization complexity. In order to ease this complexity, the proposed optimization is based on simple power and delay models. The model-based optimization does not require brute force search in the phase of real chip testing; thus, the testing time and cost can be significantly reduced. Since the coefficients of the models are extracted with real chip measurements, the error of the model can be suppressed to a few percent in average. The proposed approach is validated by real chips implemented with a 65-nm fully depleted silicon on insulator technology. The evaluation results show that the proposed optimization is an efficient mean of power reduction for a leakage current dominant chip. In fact, when compared with the conventional method, the proposed approach achieves 9.617% of average power reduction reaching up to 22.77% in the case of the V850 microcontroller.

Body Biasを用いてディジタルシステムの性能を自動チューニングする方法の提案

リアルタイム制御を考慮してボディバイアスを最適化する方法の提案

HotSpot is an open source analysis tool for estimating temperature of both 2D and 3D chip stacks. It counts both the primary path of the heat transfer through the heat-sink and the secondary path through the print circuit board. It can estimate the temperature of chips. However, HotSpot-6.0 cannot treat a complicated chip stacking such as the castle of chips (CoC) with inductive wireless coupling through-chip interface (TCI). Therefore, in this report we have extended the HotSpot simulator and updated it in order to evaluate CoC. Compared to the original HotSpot, the extended version of HotSpot had an average execution time increase of about 6% when evaluating the same vertical stacking. The execution time of CoC with the same number of chips is shorter than when executing vertical stacking, but when the number of layers is the same, the execution time is almost equal. Moreover, our design considers high productivity, e.g. it can easily set air cooling, oil cooling, water cooling evaluation and chip rotation setting.

Since deep learning workloads perform a large number of matrix operations on training data, GPUs (Graphics Processing Units) are efficient especially for the training phase. A cluster of computers each of which equips multiple GPUs can significantly accelerate the deep learning workloads. More specifically, a back-propagation algorithm following a gradient descent approach is used for the training. Although the gradient computation is still a major bottleneck of the training, gradient aggregation and optimization impose both communication and computation overheads, which should also be reduced for further shortening the training time. To address this issue, in this paper, multiple GPUs are interconnected with a PCI Express (PCIe) over 10Gbit Ethernet (10GbE) technology. Since these remote GPUs are interconnected with network switches, gradient aggregation and optimizers (e.g., SGD, AdaGrad, Adam, and SMORMS3) are offloaded to FPGA-based 10GbE switches between remote GPUs; thus, the gradient aggregation and parameter optimization are completed in the network. The proposed FPGA-based 10GbE switches with the four optimizers are implemented on NetFPGA-SUME board. Their resource utilizations are increased by PEs for the optimizers, and they consume up to 56% of the resources. Evaluation results using four remote GPUs connected via the proposed FPGA-based switch demonstrate that these optimizers are accelerated by up to 3.0x and 1.25x compared to CPU and GPU implementations, respectively. Also, the gradient aggregation throughput by the FPGA-based switch achieves up to 98.3% of the 10GbE line rate.

Artificial Intelligence(AI) has achieved unprecedented success in various fields including image/speech recognition which is useful for edge computing. Most of AI systems are implemented on power-hungry devices like GPU, high-end FPGA, or even TPU to process data with high performance. However, these energy budgets are often not affordable to edge computing. Low-end FPGA taking advantage of high energy-efficiency is a desirable platform to meet the requirements of image recognition working on small autonomous vehicles. In this paper, we propose the design methodology and implementation to adapt a neural network system to a low-end FPGA board using HLS description. The whole design consists of algorithm-level downscaling and hardware optimization. The former emphasizes the model downscale by considering accuracy. The latter applies various HLS design techniques to speed-up the application running on the target board. In the case study of tiny YOLO (You Only Look Once) v3, the model running on PYNQ-Z1 presents up to 22× acceleration comparing with the PYNQ ARM CPU. Energy efficiency also achieves 3× better than Xeon E5-2667.

An inductive-coupling link for 3-D network-on-chips (NoC) is presented. Inductively coupled coils allow high-speed wireless communication between stacked chips. 35-bit parallel input data are serialized and transmitted. Silicon measurements from test chips implementing transceiver circuits, in 65 nm SOI CMOS technology demonstrate 875 Mb/s operation.

The performance prediction for a NoC-based Chip Multi-Processor (CMP) is one of the main design concerns. Generally, there is a trade-off between accuracy and time overhead on the performance prediction of computer systems. In particular, the time overhead is proportional or exponential to the number of cores when using a cycle-accurate full-system simulation, such as gem5. In this study, we propose an accurate and scalable method to predict the influence of design NoC parameters on its performance. Our method counts the number of execution cycles when employing the target NoC based on the statistics of one-time execution of a full-system simulation using a fully-connected NoC. To evaluate the accuracy and execution time overhead, we use the case that randomly generates allocations of processors with 3D mesh topology NoC. Its Mean Absolute Percentage Error of the estimated cycles is about 4.7%, and the Maximum Absolute Percentage Error is about 8.5%.

The building block computing system can build several systems by stacking small chips with inductive coupling wireless chip-to-chip connection TCI (Through Chip Interface). TCI IP (Intellectual Property) was developed by using the Renesas 65nm SOTB process, and several family chips have been developed with the IP. Although all of the chips worked alone without problems, when they were stacked to construct a system, problems were found on some combinations of chips that TCI did not work as designed. This paper analyzes the resistances of power grid of chips with IPs from their layout to investigate the reason of the problems. Then, the voltage drop is estimated with the model built from the real chip evaluation. The analysis results appeared that in some chips, the resistances of the power grid are more than double of the target value, and because of the voltage drop, the supply voltage given to the IP of the largest chip SMTT is about the half of given to the power pad. Although it comes from the limitation of the I/O pads for chip stacking and fixed location of TCI IPs, we must take special care of the design of the transmitter power grid.