Published papers

Standard Cell based Memory (SCM) is drawing attention as a technique to use the standard digital design flow to realize embedded memory macros. One of the strong points of SCM is that it correctly operates at such low voltage that SRAM macros provided by vendors usually do not work. This paper describes a design of energy-efficient SCM using Silicon-on-Thin-BOX (SOTB). We present automatic layout methodology for optimal body-bias separation (BBS) for SCM, which enables to apply different body bias voltages to latches and to other peripheral circuits within SCM. Results from simulations and chip measurements have demonstrated effectiveness of this approach.

A non-volatile flip-flop (NVFF) introducing MTJ has many strong points in high endurance and read/write performance, and hence is very attractive as a component to be used for power gating of sequential circuits. However, large write-energy to MTJ becomes a big obstacle in achieving low energy dissipation. This paper proposes a NVFF circuit enabling to verify the success of a store operation to MTJ and retry it by prolonging the store time. We designed a NVFF circuit with this feature and applied it to 20,000 flip-flops in a dynamically reconfigurable processor (DRP). We conducted simulations considering write time variations caused by various factors such as process variations and thermal fluctuations. The results demonstrated that the proposed approach reduces store energy by 35-36% at four image-processing applications and the break-even time (BET) for non-volatile power gating is 2.0-2.9us at the 0.004% write error rate, at which no failures occur for the total number of NVFFs in the DRP.

In this study, a second-generation coarse-grained reconfigurable array with non-volatile flip-flops (NVFFs), known as the non-volatile cool mega array with multi-context (NVCMA/MC), is proposed. Similar to the previous NVCMA, verify-and-retriable NVFFs (VR-NVFFs) are provided for their configuration memory, constant memory, data memory, and instruction memory. The dedicated instructions for controlling the store, verify, and restore operations of the NVFFs are provided to the microcontroller in addition to power gating functions. Based on experience of the NVCMA, four hardware contexts are introduced to maintain the configuration data for four tasks, without the sacrifice of memory leakage. The array size is expanded, and pipeline registers are introduced to reduce the trade-off between the performance and power consumption. This study mainly focuses on the energy-saving effect of the VR-NVFFs and the multi-context facility of the NVCMA/MC, including the measurement of the break-even point. The evaluation of a real chip implemented with 40 nm MTJ/MOS hybrid process technology demonstrates that the store energy is reduced by 65% with the two-step store control of the VR-NVFFs. Moreover, applications that run intermittently for intervals as short as approximately 3 μs can benefit from the multi-context power gating.

—While the semiconductor manufacturing process is shrinking, the Bytes per Flop (B/F) ratio on recent machines is becoming lower. Particle-based computational fluid dynamics (CFD) methods such as Moving Particle Simulation (MPS) require a higher B/F ratio than that of stencil-based CFD methods. Techniques to reduce the B/F ratio by exploiting temporal parallelism is becoming popular in stencil-based CFD methods on CPU and GPU. It is also reported that a technique to combine temporal blocking with stencil buffer is suitable for FPGA and can outperform CPU and GPU. On the other hand, it has been considered that temporal parallelism cannot be exploited in the particle-based CFD methods. This is because the number of particles in each bucket, a three-dimensional grid covering a computational domain, changes every time-step. In this paper, we propose a technique to exploit temporal parallelism in MPS method, a particle-based CFD method for incompressive fluid. The key idea is that the buckets in MPS can be considered as stencils in stencil-based CFD. This is because the maximum number of particles in a bucket can be assumed empirically in the case of an incompressible fluid. To the best of our knowledge, this is the first research which exploits temporal parallelism in the particle-based incompressible fluid method. We implemented the proposed technique with a degree of temporal parallelism of three. We also optimized it on Intel Arria10 FPGA in Intel HLS, and measured the performance and resource consumption. The result shows that the optimized implementation with a degree of temporal parallelism of three achieved 2.1 times speedup compared with implementation without exploiting temporal parallelism on CPU.

Recurrent neural networks (RNNs) have been proven effective for sequence-based tasks thanks to their capability to process temporal information. In real-world systems, deep RNNs are more widely used to solve complicated tasks such as large-scale speech recognition and machine translation. However, the implementation of deep RNNs on traditional hardware platforms is inefficient due to long-range temporal dependence and irregular computation patterns within RNNs. This inefficiency manifests itself in the proportional increase in the latency of RNN inference with respect to the number of layers of deep RNNs on CPUs and GPUs. Previous work has focused mostly on optimizing and accelerating individual RNN cells. To make deep RNN inference fast and efficient, we propose an accelerator based on a multi-FPGA platform called Flow-in-Cloud (FiC). In this work, we show that the parallelism provided by the multi-FPGA system can be taken advantage of to scale up the inference of deep RNNs, by partitioning a large model onto several FPGAs, so that the latency stays close to constant with respect to increasing number of RNN layers. For single-layer and four-layer RNNs, our implementation achieves 31x and 61x speedup compared with an Intel CPU.

The communication latency is a primary concern for designing Network-on-Chips (NoCs) since it significantly affects the parallel application performance on a many-core computer system. To reduce the communication latency, we propose an on-chip router that (de)compresses the contents of an incoming packet before completing switch arbitration. The compression router thus has no latency penalty for the compression operation, whereas it shortens a packet length that decreases the network injection-and-ejection latency. Evaluation results show that the compression router improves 7.7% of the parallel application performance (IS, CG, FT, and TSP) and 49% of the effective network throughput by 1.8 compression ratio on NoC. The drawback is that the router area and its energy consumption per bit increase by 0.12mm2 and 1.4 times compared to the conventional virtual-channel router.

Recent IoT devices are required to be an extremely low power in idle time, while a certain performance is required in active time. NVCMA (Non-volatile Cool Mega Array) is a coarse-grain reconfigurable accelerator (CGRA) providing memory elements with spin transfer torque type non-volatile memory technology to keep data when the power gating is applied. Here, in order to reduce the energy for storing data into non-volatile Flip Flops (NVFF)s, a verification mechanism is introduced. The data are written with a storing time much shorter than usual. If the verification result is not correct, the next trial is done until all results are verified. This approach can omit unnecessary energy for using long writing time with margin considering variation and temperature. For NVCMA, we propose a power manager which controls store, restore, verification, sleep-down and wake-up by extending micro-instructions of the original CMA. Sophisticated power management both for storing data into NVFFs and power gating is integrated into the application program, and the content of data memory can be stored in NVFFs as required as well as configuration data and micro-instructions. Total NVFFs are divided into 10 independent domains which can be controlled independently. Power management instructions are designed so as to reduce the number of instructions as possible by using the bit-map registers and compound instructions. By using the short store with verification, the energy for storing data was reduced by 30% in average. The ever-on leakage power is just about 4% of the total leakage which can be saved by the power gating. Compared with the hard-wired implementation, the proposed power manager increases the area by 17.7%, yet the total area overhead is only 4%.

Recent IoT devices require extremely low standby power consumption, while a certain performance is needed during the active time, and Coarse Grain Reconfigurable Arrays (CGRAs) are suitable because of their high energy efficiency. However, even in CGRAs, the leakage power for its configuration memory must be reduced. Although the power gating is a popular technique, the data in flip-flops and memory are lost so they must be retrieved after the wake-up. Recovering everything requires numerous state transitions and considerable overhead both on its execution time and energy. To address the problem, Non-volatile Cool Mega Array (NVCMA), a CGRA providing non-volatile flip-flops (NVFFs) with spin transfer torque type non-volatile memory (NVM) technology has been developed. However, in general, non-volatile memory technologies have problems with reliability. Some NVFFs are stacked-at-0/1, and cannot store the data in a certain possibility. To improve the chip yield, we propose a mapping algorithm to avoid faulty processing elements of the CGRA caused by the erroneous configuration data. Then, we also propose a method to add an error-correcting code (ECC) mechanism to NVFFs used for the configuration and constant memory. The proposed method was applied to NVCMA to evaluate the availability rate and reduction of write time. By using both methods, the 99.4% availability ratio is achieved with 0.1% probability of faulty FFs, while almost no chips are available without using them. The energy for storing data becomes about 2.28 times because of the hardware overhead of ECC but the proposed method can save 11.1% of the storing energy on average.

This paper proposes a novel configuration data compression technique for coarse-grained reconfigurable architectures (CGRAs). The proposed technique is based on a multicast configuration technique called RoMultiC, which reduces the configuration time by multicasting the same data to multiple PEs(Processing Elements) with two bit-maps. Scheduling algorithms for an optimizing the order of multicasting have been proposed. In general, configuration data for CGRAs can be divided into some fields like machine code formats. The proposed scheme confines a part of fields for multicasting so that the possibility of multicasting more PEs can be increased. This paper analyzes algorithms to find a configuration pattern which maximizes the number of multicasted PEs. We implemented the proposed scheme to CMA (Cool Mega Array), a straight forward CGRA as a case study. Experimental results show that the proposed method achieves 40.0% smaller configuration for an image processing application at maximum. Furthermore, it achieves 35.6% reduction of the power consumption for the configuration with a negligible area overhead.

A novel configuration data compression technique for coarse-grained reconfigurable architectures (CGRAs) is proposed. Reducing the size of configuration data of CGRAs shortens the reconfiguration time especially when the communication bandwidth between a CGRA and a host CPU is limited. In addition, it saves energy consumption of configuration cache and controller. The proposed technique is based on a multicast configuration technique called RoMultiC, which reduces the configuration time by multicasting the same data to multiple PEs (Processing Elements) with two bit-maps. Scheduling algorithms for an optimizing the order of multicasting have been proposed. However, the multicasting is possible only if each PE has completely the same configuration. In general, configuration data for CGRAs can be divided into some fields like machine code formats of general perpose CPUs. The proposed scheme confines a part of fields for multicasting so that the possibility of multicasting more PEs can be increased. This paper analyzes algorithms to find a configuration pattern which maximizes the number of multicasted PEs. We implemented the proposed scheme to CMA (Cool Mega Array), a straight forward CGRA as a case study. Experimental results show that the proposed method achieves 40.0% smaller configuration than a previous method for an image processing application at maximum. The exploration of the multicasted grain size reveals the effective grain size for each algorithm. Furthermore, since both a dynamic power consumption of the configuration controller and a configuration time are improved, it achieves 50.1% reduction of the energy consumption for the configuration with a negligible area overhead.

Recently proposed irregular networks can reduce the latency for both on-chip and off-chip systems with a large number of computing nodes and thus can improve the performance of parallel applications. However, these networks usually suffer from deadlocks in routing packets when using a naive minimal path routing algorithm. To solve this problem, we focus attention on a lately proposed theory that generalizes the turn model to maintain the network performance with deadlock-freedom. The theorems remain a challenge of applying themselves to arbitrary topologies including fully irregular networks. In this paper, we advance the theorems to completely general ones. Moreover, we provide a feasible implementation of a deadlock-free routing method based on our advanced theorem. Experimental results show that the routing method based on our proposed theorem can improve the network throughput by up to 138 % compared to a conventional deterministic minimal routing method. Moreover, when utilized as the escape path in Duato’s protocol, it can improve the throughput by up to 26.3 % compared with the conventional up*/down* routing.

We propose a partition method to improve the performance of convolutional neural networks (CNN) on a multi-FPGA system called Flow-in-Cloud (FiC) and implement the 2nd layer of AlexNet on FiC. As a result, our implementation is slightly more energy-efficient than the CPU and the GPU with an optimized machine learning framework.

FPGA (Field Programmable Gate Array) based accelerators are attracting significant interest in cloud computing systems. Combining multi-FPGA systems with cloud computing brings a new perspective to the reconfigurable computing research. However, the multi-tenancy of a multi-FPGA system has not been fully discussed in the previous researches. In this paper, we propose a multi-tenant resource management system, named FiC-RM, for a multi-FPGA cloud system. FiC-RM provides users with a set of FPGA resources according to their requirements and allows them to exclusively access FPGA boards and the interconnection network. To achieve this, we propose a placement algorithm which is a key to efficiently share the limited resources. We demonstrate FiC-RM controls a practical scale multi-FPGA system. Moreover, Our simulation study shows that our placement algorithm achieved 3 to 4% improvement in the average resource usage and a 20-second reduction in the response time, compared to other existing naive algorithms.

One of the benefits of wireless inductive through chip interface is that it allows to build a shared bus just by placing a coil on the same location in each chip. Since providing dedicated channels for arbitration is expensive, carrier sense multiple access with collision detection (CSMA/CD) is hopeful as the bus control method. Here, a rotationpersistent collision avoidance algorithm dedicated for the wireless inter-chip bus is proposed. In the method, after the carrier sense, only a specific chip can instantly send the data, and others must wait for a random time interval. It improved the throughput of 1-persistent CSMA/CD by 33% on average.

A building block computing system with inductive coupling Through Chip Interface (TCI) consists of 3-D chip stack, each of which is small dedicated chips. By changing the combination of stacked chips, various types of systems can be built. A MIPS R3000 compatible processor GeyserTT, a neural network accelerator SNACC and the shared memory for building the twin-Tower of chips SMTT have been developed with a Renesas 65nm low leakage CMOS process. They provide the TCI IP (Intellectual Property), and an escalator network is built just by stacking them. This paper shows each chip evaluation results and performance estimation of stacking them with the RTL simulator. The performance of the single-Tower and twin-Tower configuration is estimated by RTL simulation when a part of Alexnet is implemented. The evaluation results showed that the single-Tower configuration with GeyserTT+SNACC achieved about twice performance as the case with GeyserTT. Also, experimental results using each of the single real chip showed that all of them work at least 50MHz with extremely low power consumption. The twin-Tower configuration achieved about 2x of the single-Tower, that is about 6x of GeyserTT. The power consumption was about 276mW for the single-Tower and 496mW for the twin-Tower.

This paper proposes a multi-FPGA programming environment based on NEC's integrated design tool CyberWorkBench (CWB) for a multi-FPGA system FiC (Flow-in-Cloud). Programmers describe their program in SystemC as small modules connected with FIFO channels, then verify the operation with the behavioral simulation considering parallel execution. After the high-level synthesis (HLS) is done with CWB, modules distributed to each board are decided, and the interface module is inserted. The cycle accurate simulation is applied to ensure the operation and estimate the performance. Finally, generated Verilog HDL code for each board is implemented with Xilinx's Vivado just like the traditional design and configuration is obtained. As an example, a simple convolutional neural network LeNet is described and implemented on a real system using the tool. Although the cycle accurate simulation takes 105.34sec, the estimated cycles are only 2.2% difference from the real boards execution result. Since the example CNN LeNet is too small, it can be implemented into a single board with a traditional design tool. However, considering the pipeline execution, parallel execution with two boards can distribute the input and output into different FPGAs, and relax the bottleneck.

In this paper, we propose a novel optimization method for indirect Simultaneous Localization And Mapping(SLAM). This method avoids error-intended map points using a random subset of extracted feature points. As a result, this method making the calculation time short at the cost of some accuracy. The published code of ORB-SLAM2 has longer calculation time compared to other systems such as Semi-direct odometry(SVO). This is due to feature extraction time of an indirect method, which makes the indirect methods robust. However, considering CPU performance on edge devices like MAVs, a lightweight system is welcome. ORB-SLAM2 is widely known as a robust and rapid visual Simultaneous-Localization-And-Mapping (vSLAM) method than before. It extracts FAST feature points from a given image and optimizes using all of these points for camera-posing estimation. However, some points disturb accurate estimation causing large reprojection error after optimization. Although these points are eliminated as outliers after optimization, they may be an obstacle for accurate pose estimation. To make this system more robust and more rapid, We suggest an improved method to estimate camera-posing, where we use a subset of extracted points for estimation, avoiding mixing large-reprojection-error points in bundle adjustment. The experimental results on the SLAM benchmark dataset KITTI odometry demonstrated that the proposed method outperformed the original implementation of ORB-SLAM2 from the viewpoint of calculation time.

FPGAs can be a promising accelerator used for MEC (Multi-Access Edge Computing) which provides timing critical services for a number of terminals at the base stations near from edges. Although a high-end FPGA can support a fixed latency computation with a relatively small power consumption, they are expensive and the available acceleration circuits are limited into a size of single FPGA. FiC (Flow-in-Cloud) has been developed for building a virtual large FPGA from a number of middle-range economical FPGAs connected with high speed serial links. Although the current target of FiC is cloud computing, it is more suitable for the future MEC, because huge hardware resource can be supported with small cost. One of the problem to use such multi-FPGA systems for timing critical computation is network uncertainty. With a common packet switching, the computation speed is influenced with the network traffic. That is, the fixed latency computation which could be supported by a single FPGA is hard to be supported with multi-FPGA systems using common packet switching networks. In order to address this problem, we introduced STDM (Static Time Division Multiplexing) switch in the FiC system. Since the STDM always supports a constant communication latency, transfer time can be estimated beforehand. Through the implementation of the STDM switch on the FPGA board for FiC, it appeared that the utilization ratio of the LUTs for the STDM switch is smaller than 14%. The required number of slots is less than 16 even for a system with 256 nodes. We implemented the Conjugate Gradient method, which includes all-To-All communication, on 4x2 FiC system. It achieved 17.9 times performance improvement of Intel E5-2667 2.90GHz CPU with 6 cores.

In this work, we present a low-overhead performance monitor which can emulate the maximum operational frequency of a target system by utilizing a delay chain so as to achieve efficient adaptive voltage control. The proposed monitor can be fully built by logic cells provided by general PDKs; thus, an automatic cell-based design flow can be used for its implementation. In addition, interconnect delay behaviors can also be imitated by exploiting wires which are automatically routed. In order to validate our concept, the proposed monitor is fabricated with a 65-nm Fully Depleted Silicon on Insulator (FD-SOI) technology. Real chip experiments reveal that the automated layout design can achieve the reasonable ability to delay emulation. Indeed, when the maximum operational frequency of a CNN accelerator is emulated, the proposed SDM achieved several percents of the performance tracking error. Also, its power overhead is only few percents.

Three-dimensional stack memory which provides both high-bandwidth access and large capacity is a promising technology for next-generation computer systems. While a large number of memory cubes increase the aggregate memory capacity, the communication latency and power consumption increase significantly owing to its low-radix large-diameter packet network. In this context, we propose a memory-cube network called Diagonal Memory Network (DMN). A diagonal network topology, its floor layout, and its lightweight router were designed for low-latency and low-voltage memory-read communication. DMN routing efficiently avoids deadlocks of packets, although it allows each packet transmitted to a processor to use both bypassing and original datapaths. Our evaluation results show that the DMN router decreases the use of hardware resources by more than 31% compared with a conventional virtual channel router. The DMN router reduces energy consumption by 13% and 67% to transit a packet along with the original datapath and bypassing datapath, respectively. Furthermore, using flit-level discrete event simulation, a DMN topology achieves high throughput and latency that is lower than that of existing network topologies using conventional packet routers.