Published papers

Due to the limitations of cloud computing on latency, bandwidth and data confidentiality, edge computing has emerged as a novel location-aware paradigm to provide the capacity constrained portable terminals with more processing capacity to improve the performance and quality of service (QoS) in several typical domains of the human activity in smart society, such as social networks, medical diagnosis, transportation, Internet of Things (IoT), etc. These domains often handle a vast collection of entities with various relationships which can be naturally represented by graph data structures. Graph processing is a powerful tool to model and optimize the complex problems where the graph-based data is involved. In view of the relatively insufficient resource provision of the edge devices, for the first time to our knowledge, we propose a reliable edge-cloud collaboration framework facilitating the shortest path search (SPS) operations based on a lightweight interactive graph processing library. One practical case is presented as well to show the typical application scenarios of our graph processing strategy. Experimental evaluation is conducted to indicate that the proposed strategy is able to efficiently implement the relevant application tasks with considering user-friendliness, low-latency response, collaborations between edge and cloud, interactions among edge devices, and privacy protection at an acceptable overhead.

Advanced flow control mechanisms employed by modern on-chip networks are the reasons of large energy footprint and long per-hop latency. On the other hand, dated and simpler flow controls such as circuit switching can draw far less power and offer an end-to-end latency analogous to wire delay. In this paper, we present a hybrid flow control mechanism, which mixes both virtual channels and circuit-switching, to provide a latency-competitive and energy-efficient on-chip network design. Contrary to existing hybrid switching designs, our proposal is based on local traffic so that circuits are formed without the knowledge of end-to-end traffic. When compared to on-chip networks with virtual channels, our proposal achieves a very competitive latency per flit, for up to 4% lower, while also dramatically suppressing the energy per flit by up to 18%.

SpMV is a very common algorithm in linear algebra, which is widely adopted by machine learning applications nowadays. Especially, fully-connected MLP layers dominate many SpMV tasks that play a critical role in diverse services, and therefore a large fraction of data center cycles are spent. Despite exploiting sparse matrix storage techniques such as CSR/CSC, SpMV still suffers from limited memory bandwidth during data transferring because of the architecture of modern computing systems. However, we find that both integer type and floating-point type data used in matrix-vector multiplications are handled plainly without any necessary pre-processing. We added compression and decompression pre-processing between the main memory and Last Level Cache (LLC) which may dramatically reduce the memory bandwidth consumption. Furthermore, we also observed that convergence speed in some typical scientific computation benchmarks will not be degraded when adopting compressed floating-point data instead of the original double type. Based on these discoveries, in this paper, we propose a simple yet effective compression approach that can be implemented in general computing architectures and HPC systems preferably. When adopting this technique, a performance improvement of 1.92x is made in the best case.

As the feature size of transistors continues to scale down, process variations have been a critical issue for large scale computer systems, such as high-performance or warehouse-scale computers, to realize energy efficient operations. With process variations, it is very hard to effectively utilize the given power budget for such systems as many components inside them simply draw different amount of power for the same performance, even if they have the same hardware specifications. To allow such computer systems to operate in an accurate and energy-efficient manner, we demonstrate how calibrations, modelling and load balancing are realized to address process variations with an simple in-house DSL. With our case studies and evaluations, we prove the effectiveness of our tools and approaches as the impacts from process variations in our experimental system is minimized with considerably smaller amount of effort, which leads to highly energy efficient executions of parallel applications and dramatic reductions in profiling overheads.

Federated learning has attracted much interest and attention as a solution to collaboratively learn a prediction model without sharing the training data of users. Existing federated learning approaches usually develop a single independent local model for each client to train their privacy-sensitive data, afterward adopt a single centralized global model to exchange the trained parameters of clients that participate in federated training. However, given the diverse characteristics of local data and the heterogeneity across participating clients, the conventional federated learning paradigm may not achieve uniformly good performance over all users. In this work, we propose a novel federated learning mechanism which suggests using a Multi-Local and Multi-Global (MLMG) model aggregation to train the non-IID user data with clustering methods. Then a Matching algorithm is introduced to derive the appropriate exchanges between local models and global models. The new federated learning mechanism helps separate the data and user with different characteristics, thus makes it easier to capture the heterogeneity of data distributions across the users. We choose the latest on-device neural network for anomaly detection to evaluate the proposal, and experimental results based on several benchmark datasets demonstrate better detection accuracy (up to 2.83% accuracy improvement) of the novel paradigm compared with a conventional federated learning approach.

Modern Energy Harvesting Wireless Sensor Nodes (EHWSNs) need to intelligently allocate their limited and unreliable energy budget among multiple tasks to ensure long-term uninterrupted operation. Traditional solutions are ill-equipped to deal with multiple objectives and execute a posteriori tradeoffs. We propose a general Multi-objective Reinforcement Learning (MORL) framework for Energy Neutral Operation (ENO) of EHWSNs. Our proposed framework consists of a novel Multi-objective Markov Decision Process (MOMDP) formulation and two novel MORL algorithms. Using our framework, EHWSNs can learn policies to maximize multiple task-objectives and perform dynamic runtime tradeoffs. The high computation and learning costs, usually associated with powerful MORL algorithms, can be avoided by using our comparatively less resource-intensive MORL algorithms. We evaluate our framework on a general single-task and dual-task EHWSN system model through simulations and show that our MORL algorithms can successfully tradeoff between multiple objectives at runtime.

IoT embedded systems have multiple objectives that need to be maximized simultaneously. These objectives conflict with each other due to limited resources and tradeoffs that need to be made. This requires multi-objective optimization (MOO) and multiple Pareto-optimal solutions are possible. In such a case, tradeoffs are made w.r.t. a user-defined preference. This work presents a general Multi-objective Reinforcement Learning (MORL) framework for MOO of IoT embedded systems. This framework comprises a general Multi-objective Markov Decision Process (MOMDP) formulation and two novel low-compute MORL algorithms. The algorithms learn policies to tradeoff between multiple objectives using a single preference parameter. We take the energy scheduling problem in general Energy Harvesting Wireless Sensor Nodes (EHWSNs) as a case example in which a sensor node is required to maximize its sensing rate, and transmission performance as well as ensure long-term uninterrupted operation within a very tight energy budget. We simulate single-task and dual-task EHWSN systems to evaluate our framework. The results demonstrate that our MORL algorithms can learn better policies at lower learning costs and successfully tradeoff between multiple objectives at runtime.

Semi-supervised anomaly detection is an approach to identify anomalies by learning the distribution of normal data. Backpropagation neural networks (i.e., BP-NNs) based approaches have recently drawn attention because of their good generalization capability. In a typical situation, BP-NN-based models are iteratively optimized in server machines with input data gathered from the edge devices. However, (1) the iterative optimization often requires significant efforts to follow changes in the distribution of normal data (i.e., concept drift), and (2) data transfers between edge and server impose additional latency and energy consumption. To address these issues, we propose ONLAD and its IP core, named ONLAD Core. ONLAD is highly optimized to perform fast sequential learning to follow concept drift in less than one millisecond. ONLAD Core realizes on-device learning for edge devices at low power consumption, which realizes standalone execution where data transfers between edge and server are not required. Experiments show that ONLAD has favorable anomaly detection capability in an environment that simulates concept drift. Evaluations of ONLAD Core confirm that the training latency is 1.95x∼6.58x faster than the other software implementations. Also, the runtime power consumption of ONLAD Core implemented on PYNQ-Z1 board, a small FPGA/CPU SoC platform, is 5.0x∼25.4x lower than them.

To design exascale HPC systems, power limitation is one of the most crucial and unavoidable issues; and it is also necessary to optimize the power-performance of user applications while keeping the power consumption of the HPC system below a given power budget. For this kind of power-performance optimization for HPC applications, it is indispensable to have enough information and good understanding about both the system specifications (what kind of hardware resources are included in the system, which component can be used as a “power-knob”, how to control the power-knob, etc.) and user applications (which part of the application is CPU-intensive, memory-intensive, and so on). Because this situation forces both the users and administrators of power-constrained HPC systems pay much effort and cost, it has been highly demanded to realize a simple framework to automate a power-performance optimization process, and to provide a simple user interface to the framework. To tackle these concerns, we propose and implement a versatile framework to help carry out power management and performance optimization on power-constrained HPC systems. In this framework, we also propose a simple DSL as an interface to utilize the framework. We believe this is a key to effectively utilize HPC systems under the limited power budget.

A building block computing system with inductive coupling Through Chip Interface (TCI) consists of 3-D chip stack, each of which is small dedicated chips. By changing the combination of stacked chips, various types of systems can be built. A MIPS R3000 compatible processor GeyserTT, a neural network accelerator SNACC and the shared memory for building the twin-Tower of chips SMTT have been developed with a Renesas 65nm low leakage CMOS process. They provide the TCI IP (Intellectual Property), and an escalator network is built just by stacking them. This paper shows each chip evaluation results and performance estimation of stacking them with the RTL simulator. The performance of the single-Tower and twin-Tower configuration is estimated by RTL simulation when a part of Alexnet is implemented. The evaluation results showed that the single-Tower configuration with GeyserTT+SNACC achieved about twice performance as the case with GeyserTT. Also, experimental results using each of the single real chip showed that all of them work at least 50MHz with extremely low power consumption. The twin-Tower configuration achieved about 2x of the single-Tower, that is about 6x of GeyserTT. The power consumption was about 276mW for the single-Tower and 496mW for the twin-Tower.

Cloud computing has established a convenient approach for computing offloading, where the data produced by edge devices is gathered and processed in a centralized server. However, it results in critical issues related to latency. Recently, a neural network-based on-device learning approach is proposed, which offers a solution to the latency problem by relocating processing data to edge devices; even so, a single edge device may face insufficient training data to train a high-quality model, because of its limited available processing capabilities and energy resources. To address this issue, we extend the work to a federated learning system which enables the edge devices to exchange their trained parameters and update local models. However, in federated learning for anomaly detection, the reliability of local models would be different. For example, a number of trained models are likely to contain the features of anomalous data because of noise corruption or anomaly detection failure. Besides, as the communication protocol amongst edges could be exploited by attackers, the training data or model weights may have potential risks of being poisoned. Therefore, when we design a federated training algorithm, we should carefully select the local models that participate in model aggregation. In this work, we leverage an observed dataset to compute prediction errors, so that the unsatisfying local models can be excluded from federated training. Experimental results show that the federated learning approach improves anomaly detection accuracy. Besides, the proposed model aggregation solution achieves obvious improvement compared with the popular Federated Averaging method.

In the real world, normal and abnormal behavior patterns vary depending on a given environment, which means that the abnormal behavior detection model should be customized. To address this issue, in this paper, we employ OS-ELM (Online Sequential Extreme Learning Machine) and Autoencoder for adaptive abnormal behavior detection. First, state-transition probability tables of a target during an initial learning period are learned as normal behaviors. Then, Autoencoder-based anomaly detection is performed for the state-transition probability tables of subsequent time frames. The abnormal behavior detection model is updated by using OS-ELM algorithm every time a new probability table or behavior comes. The number of abnormal behavior detection instances is dynamically tuned to reflect the recent normal patterns or modes. Also, the table is compressed to reduce the computation cost. Evaluation results using a driving dataset of cars show that the proposed abnormal behavior detection accurately identifies normal and aggressive driving patterns with the optimal number of the abnormal behavior detection instances.

Hardware overprovisioned systems have recently been proposed as a viable alternative for a power-efficient design of next-generation supercomputers. A key challenge for such systems is to determine the degree of overprovisioning, which refers to the number of extra nodes that need to be installed under a given power constraint. In this paper, we first show that the degree of overprovisioning depends on dynamic parameters, such as the job mix as well as the global power constraint, and that static decisions can result in limited system throughput. We then study an exhaustive combination of adaptive resource management strategies that span three job scheduling algorithms, four power capping techniques, and three node boot-up mechanisms to understand the trade-off space involved. We then draw conclusions about how these strategies can adaptively control the degree of overprovisioning and analyze their impact on job throughput and power utilization.

A building block computing system is consisting of multiple chips connecting with inductive coupling wireless through chip interconnect. Like building Lego blocks, various types of systems can be built by stacking different types of chips. In order to develop such systems, several techniques are investigated in the project: that is, inductive coupling wireless through chip interface, low power circuit technologies, autonomous interconnection network architectures, thermal dissipation and building block operating system. Here, the overview of the project and a prototype system are introduced.

Virtual channel (VC) flow control is the de facto choice for modern networks-on-chip (NoCs) to allow better utilization of the link bandwidth through buffering and packet switching (PS), which are also the sources of large power footprint and long per-hop latency. However, bandwidth can be plentiful for parallel workloads under VC flow control. Thus, dated but simpler mechanisms, such as circuit switching (CS), can help improve the energy efficiency of modern NoCs. In this paper, we propose to apply CS to part of the link bandwidth so that a considerable amount of traffic can be transmitted bufferlessly without routing. Evaluations reveal that this proposal leads to a reduction of energy per flit by up to 32% while also provides very competitive latency when compared to networks under VC flow control.

With the wide spread of artificial intelligence (AI) technologies, many applications using AI are increasingly deployed in many fields. Specially anomaly detection is one of the key applications of AI. Among several targets, detecting anomaly behavior of drivers or vehicles has been attracting due to the growing demand of safety driving. It is crucial to study and evaluate techniques for anomaly driving detection with AI technologies. The Online Sequential Extreme Learning Machine (OS-ELM) is a recently attracting neural network model that has high memory efficiency and can perform highspeed sequential learning with streaming data. Though OSELM is known to be effective for anomaly detection, it has not yet been verified for non-stationary time series data such as driving sensor data. In this paper, we study the effectiveness of OS-ELM based anomaly driving behavior detector using sensor data of vehicles and compared the performance of it with a Hidden Markov Model (HMM) based and traditional Long Short-Term Memory (LSTM) based methods. Since the existing driving behavior benchmark data is not enough for evaluating anomaly driving, we also create a new dataset with a powered wheelchair. Throughout the evaluation, we show that the OS-ELM based anomaly driving detector has almost the same or even better accuracy in anomaly driving detection with much faster sequential learning speed compared with the HMM or LSTM based detector.

Autoencoder, a neural-network based dimensionality reduction algorithm has demonstrated its effectiveness in anomaly detection. It can detect whether an input sample is normal or abnormal by just training only with normal data. In general, Autoencoder is built on backpropagation-based neural networks (BP-NNs). When BP-NNs are implemented in edge devices, they are typically specialized only for prediction with weight matrices precomputed offline due to the high computational cost. However, such devices cannot be immediately adapted to time-series trend changes of input data. In this paper, we propose an FPGA-based unsupervised anomaly detector, called OS-ELM-FPGA, that combines Autoencoder and an online sequential learning algorithm OS-ELM. Based on our theoretical analysis of the algorithm, the proposed OS-ELM-FPGA completely eliminates matrix pseudoinversions while improving the learning throughput. Simulation results using open-source datasets show that OS-ELM-FPGA achieves favorable anomaly detection accuracy compared to CPU and GPU implementations of BP-NNs. Learning throughput of OS-ELM-FPGA is 3.47x to 27.99x and 5.22x to 78.06x higher than those of CPU and GPU implementations of OS-ELM. It is also 3.62x to 36.15x and 1.53x to 43.44x higher than those of CPU and GPU implementations of BP-NNs.

Power consumption is a first class design constraint for developing future exascale computing systems. To achieve exascale system performance with realistic power provisioning of 20-30MW, we need to improve power-performance efficiency significantly compared to today's supercomputer systems. In order to maximize effective performance within a power constraint, investigating how to optimize power resource allocation to each hardware component or each job submitted to the system is necessary. We have been conducting research and development on a software framework for code optimization and system power management for the power-constraint adaptive systems. We briefly introduce the research efforts for maximizing application performance under a given power constraint, power-aware resource manager, and power-performance simulation and analysis framework for future supercomputer systems.

Energy Harvesting Wireless Sensor Nodes (EHWSNs) require adaptive energy management policies for uninterrupted perpetual operation in their physical environments. Contemporary online Reinforcement Learning (RL) solutions take an unrealistically long time exploring the environment to converge on working policies. Our work accelerates learning by partitioning the state-space for simultaneous exploration by multiple agents. We achieve this by using a novel coordinated e-greedy method and implement it via Distributed RL (DiRL) in an EHWSN network. Our simulation results show a four-fold increase in state-space penetration and reduction in time to achieve optimal operation by an order of magnitude (50x). Moreover, we also propose methods to reduce instances of disastrous outcomes associated with learning and exploration. This translates to reducing the downtimes of the nodes in simulations corresponding to a real-world scenario by one thirds.

Due to the recent advances in Deep Neural Network (DNN) technologies, recognition and inference applications are expected to run on mobile embedded systems. Developing high-performance and power-efficient DNN engines becomes one of the important challenges for embedded systems. Since DNN algorithms or structures are frequently updated, flexibility and performance scalability to deal with various types of networks are crucial requirement of the DNN accelerator design. In this paper, we describe the architecture and LSI design of a flexible and scalable CNN accelerator called SNACC (Scalable Neuro Accelerator Core with Cubic integration) which consists of several processing cores, on-chip memory modules, and ThruChip Interface (TCI).