RESEARCH PAPERS OF MR NABIL LITAYEM

Transcription

1 RESEARCH PAPERS OF MR NABIL LITAYEM Published during his PHD entitled METHODOLOGICAL CONTRIBUTIONS TO THE DESIGN AND OPTIMIZATION OF EMBEDDED SYSTEMS

2 International Journal 1. N Litayem, M Kolhar, I Mhadhbi, S M. Abd El-atty, S Ben Saoud. Hashing Based Authentication for Ultra-Low Cost Low Power SCADA Application Using MSP430 Microcontroller, IJETAE Volume 3, Issue 12, December N Litayem, B Jaafar, S Ben Saoud. Embedded Microprocessor Performance Evaluation Case Study Of The Leon3 Processor, JESTEC Vol. 7, No. 5 (2012) N Litayem, AB Achballah, SB Saoud. Building XenoBuntu Linux Distribution for Teaching and Prototyping Real-Time Operating Systems, IJACSA Vol. 2, No.2, February N Litayem, S Ben Saoud. Impact of the Linux Real-time Enhancements on the System Performances for Multi-core Intel Architectures, IJCA 2011, Number 2 - Article N Litayem,S Ben Saoud Rapid Hardware-In-the-Loop Implementation for FPGA Based Embedded Controller for More Reliable Electrical Traction Systems, JCSCS 2010 Volume 3, Issue 2. International conference 1. I Mhadhbi, N Litayem, S Ben Othmen, and SBen Saoud. "DSC Performance Evaluation and Exploration, Case of TMS320F335."International Conference on Control, Engineering & Information Technology (CEIT 13) Proceedings Engineering & Technology - Vol.1, N Litayem, M Ghrissi, AK Ben Salem, S Ben Saoud. Designing and building embedded environment for robotic control application, IECON 2009 Porto, Portugal, November 3-5, AK Ben Salem, S Ben Othman, S Ben Saoud, N Litayem. Servo Drive System Based on Programmable SoC Architecture, IEEE IECON 2009 Porto, Portugal, November 3-5, N Litayem, S Ben Saoud. Impact of real-time enhancements on the computation performances for multi-core intel architectures, JTEA 2010, Hammamet, Tunisia, April N Litayem, M Ghrissi, S Ben Saoud. Etude de l Influence de la Communication Processeur Mémoire sur la Performance d un SoC, à base de circuit FPGA XILINX, JTEA 08, Hammamet, Tunisia, April M Ghrissi, S Ben Saoud, N Litayem. Correction des erreurs systématiques de l odomètre et suivi de trajectoires sur un robot mobile industriel type tricycle, JTEA 08, Hammamet, Tunisia, April N Litayem, M Ghrissi, S Ben Saoud. Etude Comparative des moyens de communications inter processeurs dans les architectures MPSoC, GEI 08, Sousse, Tunisia, March N Litayem, M Ghrissi, S Ben Saoud. Embedded Microprocessor Systems Hardware Performance Evaluation and Benchmarking, ICESCA 08, Gammarth, Tunisia, May 2008.

3 International Journal of Emerging Technology and Advanced Engineering Website: (ISSN , ISO 9001:2008 Certified Journal, Volume 3, Issue 12, December 2013) Hashing Based Authentication for Ultra-Low Cost Low Power SCADA Application Using MSP430 Microcontroller Nabil Litayem 1, Manjur Kolhar 2, Imene Mhadhbi 3, Saied M. Abd El-atty 4, Slim Ben Saoud 5 1,2,4 Computer Science and Information, Salman Bin Abdulaziz University, Wadi College of Arts and Science, Kingdom of Saudi Arabia 3,5 LSA Laboratory, INSAT-EPT, University of Carthage, TUNISIA Abstract Nowadays SCADA (Supervisory Control and Data Acquisition) systems became widely used technology. This fact is directly related to the ubiquity of smart systems using a wide range of technologies in control and supervision applications. MCU technology are a very promising technology in this field especially, with the emergence of safety critical MCU, cost/power reduction and wireless connectivity. Due to the ubiquitous use of such technologies, security considerations must be considered. In this paper, we propose a RTU (Remote Terminal Unit) authentication solution that is based on a lightweight hashing algorithm. Proposed solution is suitable for SCADA systems using MSP430 ultra low power low cost MCU. This work is seen as a proof of concept of using such technology with freely available tools to add reliable authentication functionality of our previously designed SCADA systems. Keywords SCADA, MSP430, RTU, Security, hash-based authentication. I. INTRODUCTION Earlier SCADA system were based on an event-driven operating system and basic serial communications. This kind of solution does not have any security threats because complete physical isolation SCADA devices from any external intrusion. Thanks to Moor Law, SCADA Supervisory Control and Data Acquisition [1] applications become cost effective and ubiquitous. Such solution are based on standard hardware, open source software and open protocols. SCADA applications are nowadays used in power distribution monitoring, nuclear simulators, military data acquisition, health care applications and many thousands of various applications [2], [3] furthermore, they are considered as a part of Internet of Things ecosystem. The ubiquity requires many careful security considerations to ensure confidentiality, integrity and availability of such systems. Any compromise in SCADA system security can have serious consequences [4]. During this last decade, many research works have studied the security of such system and proposed innovative solutions, [5], [6], and [7]. In this study, we introduce an authentication solution using a hashing algorithm for MSP430 microcontroller for SCADA RTU. The proposed solution has the authentication system or algorithm using various profiles of Quark [8] hashing algorithm, which are chosen after qualitative and quantitative surveys that are presented in this paper. The remainder of this work is organized as follows: Section 2, gives a presentation of MSP430 development platform, followed by a survey about SCADA applications and their availability solutions and applications. In section 4, we present the choice and execution of the hashing algorithm. Finally, section 5 concludes this contribution. II. HARDWARE PLATFORM A. Introduction to MSP430 Microcontroller Known for its low power consumption, MSP430 from Texas Instruments is a family of 16-bit microcontrollers commonly used in wireless sensors/actuator network and metering applications [8]. The utilization of these MCU becomes too broad due to the introduction of new innovative features,- apart from low cost and low power. The main features of MSP430 microcontroller are summarized in Table

4 International Journal of Emerging Technology and Advanced Engineering Website: (ISSN , ISO 9001:2008 Certified Journal, Volume 3, Issue 12, December 2013) Feature Instruction sets Registers Memory Addressing modes Peripherals Frequency Electric Power On-Chip Memory TABLE 1. MAIN CHARACTERISTICS OF THE MSP430 MCU Description 27 RISC instructions 12 general purpose registers 16 Bit Word or Bytes Addressing Register direct, register indexed, register indirect and register indirect USART, SPI, I²C, 10/12/14/16-bit ADCs, internal oscillator, timer, PWM, watch dog, brownout reset circuitry, comparators, on-chip op-amps, 12-bit DAC, LCD driver, hardware multiplier, USB, and DMA 1Mhz- 25Mhz <1µA in IDLE mode 256KB Flash, 16 KB RAM B. MSP430 family and development tools Texas instrument has a wide range of MSP430 flavours designed for diverse applications, such as smart metering, wireless communication, motor control, personal health care, etc. For each applications of MSP430 flavour Texas Instrument has a development or evaluation board. The most successful development boards are MSP- EXP430F5529, ez430 Chronos [9] and MSP430 Launchpad [10]. MSP430 has the advantage of complete software ecosystem ranging from powerful development environment such as IAR, Code Composer Studio and Energia to very appropriate software stack such as SimpliciTI [11] or Capacitive Touch sense library. In the other hand, TI MCU solutions are also very cost effective and scalable. The wide variety of available TI MCU offers the possibility to easily change from one TI MCU to another. C. Launchpad Board Since 2010, Texas Instrument has expanded MSP430 portfolio by introducing MSP430 Value Line shown in Figure 1. This new low cost family starting at 0.25$, is essentially intended to replace the old 8-bits MCU. Fig 1. Functional Block Diagram, MSP430G2x53 Fig 2. Launchpad Board To promote this new family, TI has introduced the MSP- EXP430G2 LaunchPad showed in Figure 2. This evaluation board is a low cost very valuable evaluation platform with a price of 4$30. Launchpad can be used to develop applications for the overall Value Line MSP430 microcontrollers. 501

5 International Journal of Emerging Technology and Advanced Engineering Website: (ISSN , ISO 9001:2008 Certified Journal, Volume 3, Issue 12, December 2013) III. SCADA SYSTEMS A. Introduction SCADA Systems Nowadays, SCADA systems (Supervisory Control and Data Acquisition) are almost available in a wide range of electronic devices and applications such as steel making, electric grid, healthcare devices and chemistry. On the other hand, SCADA has become vital to drive critical experiments such as nuclear fusion. SCADA is not specific to a precise technology, but a type of application. Any application that gets data about a system in order to control that system is a SCADA application. Furthermore, SCADA are computer-based systems that introduce various advanced and innovative supervisory functionalities. This leads to automate traditional complex industrial processes where human control is impractical. Critical infrastructures and industries are nowadays requiring excessive use of this kind of technologies. Client Client Dedicated Server Data Server Network Network Controller Controller Controller Fig 3. SCADA System Architecture [12] As shown in Figure 3, typical SCADA application control systems, collect field and sensor data, processes and displays the collected data, and send commands to the controlled systems. In industrial control system, geographic location is the main classification criteria between SCADA and DCS (Distributed Control Systems), since DCSs are used within a single processing or generating plant or over a small geographic area [12] and SCADA systems are used for large geographically dispersed distribution operations. If we consider nuclear power plant, DCS can be used in power production and SCADA in power distribution. Nowadays, with the emergence of Smart Grid and Internet of Things Concepts, SCADA systems more considered. Our work is based on SCADA systems, but it may be extended to DCS. 502 B. SCADA system Architecture A SCADA system has three main basic components [13]: Remote Terminal Unit (RTU) is an intelligent part connected to the controlled process. RTU is responsible for reading inputs, make a smart decision, provide outputs signal, take new orders and provide real time feedback to the HMI. Human Machine Interface (HMI) is the interface between user and SCADA system. HMI must provide intelligible data about the physical controlled process. Communication infrastructure is used to connect various components to the SCADA system. Communication infrastructure is responsible to handle various communication protocols and provide some bridging capabilities between RTU network and corporate network. C. Security of SCADA systems Earlier SCADA systems were not designed for public access considerations. The only possible security threat was the physical destruction. Due to the interconnection of modern SCADA systems to public networks, several security considerations must be seriously considered. Many researchers proved the existences of the threats to the SCADA system by simulations and real systems [14]. Because of the security flaws present in the SCADA system many academicians and various organizations are putting efforts to make SCADA safe from the threats [15].Sandia National Laboratories (USA), National Infrastructure Security Co-ordination Centre (UK) and British Columbia Institute of Technology (Canada) are the most influential organizations working in this field. Attack Denial of service Unauthorized changes Wrong information sends Control system software modification Interference TABLE 2 COMMON ATTACKS FOR SCADA SYSTEMS Impact on the system Delaying or blocking the stream of data through control networks Modification of programs instructions in RTUs at remote sites, resulting in damage to equipment, precipitate closure of processes, or even disabling control equipment Used to control system operators to disguise unauthorized changes or to initiate inappropriate actions by system operators Producing unpredictable results Interfere with the operation of safety systems.

6 International Journal of Emerging Technology and Advanced Engineering Website: (ISSN , ISO 9001:2008 Certified Journal, Volume 3, Issue 12, December 2013) D. Studied SCADA RTU Our designed system has the ability to supervise and control various greenhouse signals such as temperature, humidity and pressure. This system can also be remotely controlled to initiate or receive critical alarms. Our studied SCADA RTU is a PID thermal process controller with supervision capability using USCI interface. This intelligent part of this RTU is based on MSP430G2553 MCU. Fig 6. Designed SCADA RTU The system illustrated by Figure 6 is designed to emulate greenhouse temperature regulation with some local supervision features such as LCD and LED interface. In the other hand the supervision interface is designed using Visual Basic language. Through this interface we can tune the PID regulator, fix the temperature consign and supervise the evolution of the temperature. Figure 4, and Figure 5 show some views of this interface. Fig 4. Supervision interface Fig 5. Configuration Interface IV. CHOICE AND ADAPTATION OF LIGHTWEIGHT HASHING ALGORITHM The goal of this section is to review the available hashing algorithm in order to adopt an appropriate one as an authentication solution for our SCADA system. The choice of the hashing algorithm will be made according to security level, computational complexity and memory footprint. The two last criteria are primordial since our hardware MSP430G2553 microcontroller uses16kbyte of Flash memory, 512 Byte of RAM memory, and can t go over 16MHz in frequency. A. Introduction hashing Algorithm Hashing algorithms [16] are commonly used in computing, their main purpose is to map a variable message length to a fixed length message. It consists of applying H (hashing function) to x (message) to produce H (x) called the message hash. On the other hand, finding y as H (y) =H (x) must be computationally infeasible. This behavior can be used in the following fields: 503

7 International Journal of Emerging Technology and Advanced Engineering Website: (ISSN , ISO 9001:2008 Certified Journal, Volume 3, Issue 12, December 2013) Authentication algorithms Password storage mechanisms Digital Signature Standard (DSS) Transport Layer Security (TLS) Internet Protocol Security (IPSec) Random number generation algorithms In our application, hashing algorithm will be used to protect the authenticity of transmitting information and to offer reliable authentication mechanism. B. Review of lightweight hashing algorithm Hashing algorithms are widely used for a broad type of applications. Nowadays, many hashing algorithms are available. Each algorithm can be more adapted for specific fields such as a powerful video platform, FPGA platforms, low performance platforms etc. Table 3, review these algorithms to deliver a big view about these algorithms. Based on this review, we will take the right algorithm to be suitable for our application. Algorithm SHA family MD4 MD5 MD6 Quark CubeHash Grøstl Lane Shabal Spectral Hash Keccak-f TABLE 3 CANDIDATE HASHING FUNCTIONS Presentation Secure Hash Algorithms are a family of Hash Algorithms published by NIST since SHA has many derivative standards such as SHA-0, SHA-1, SHA-3 Message-Digest Algorithm is a family of broadly used cryptographic hash function developed by Ronald Rivest that produces a 128-bit for MD4 and MD5, 256-bit for MD6 Family of cryptographic functions designed for resource-constrained hardware environments. A very simple cryptographic hash function designed in University of Illinois at Chicago, Department of Computer Science Hashing algorithm designed by a team of cryptographers from Technical University of Denmark (DTU) and TU Graz Cryptographic hash function suggested in the NIST SHA-3 competition by the COSIC research group Cryptographic hash function submitted by the France funded research project Saphir to NIST s Cryptographic hash function family based on the discrete Fourier submitted to the NIST hash function competition Cryptographic hash function submitted to the NIST SHA-3 hash function competition 504 Whirlpool UHASH SPONGENT Photon dm-present SQUASH C. Quark Hashing Algorithm Whirlpool is a cryptographic hash function recommended by the NESSIE project, adopted by the ISO and IEC as part of the ISO/IEC standard. UHASH is a keyed hash function, specified in RFC4418. The primary application of this algorithm is in UMAC message authentication code. Lightweight hash-function family, known for their small footprint for hardware implementation A lightweight hash - function designed for very constrained devices Ultra-lightweight block cipher designed for RFID applications Not collision resistant, suitable for RFID applications As noted in [21], designers of lightweight cryptographic algorithms or protocols have trade-off between two opposite design philosophies. The first one consists in creating new schemes from scratch, whereas the second consists in reusing available schemes and adapting them to system constraints. The main features of Quark are separating digest length, security level and working with shift registers. D. Adoption of Quark hashing algorithm to the MSP430G2553 microcontroller In our SCADA system the execution of the hashing algorithm is just used during new supervision node connection, then this algorithm can have a middle complexity level since during the authentication the system does not have any notable load. V. PERFORMANCE EVALUATION OF VARIOUS QUARK PROFILES RUNNING UNDER MSP430G2553 E. Obtained results After adapting the various Quark algorithm profiles for MSP430G2553, we did some performance evaluation according execution time detailed in Table 4 and algorithm footprint detailed in Table 5.

8 International Journal of Emerging Technology and Advanced Engineering Website: (ISSN , ISO 9001:2008 Certified Journal, Volume 3, Issue 12, December 2013) TABLE 4. EXECUTION TIME OF VARIOUS PROFILES OF UQUARK ALGORITHM Alghorithm Execution time (ms) UQUARK DQUARK SQUARK CQUARK TABLE 5. FOOTPRINT OF VARIOUS PROFILES OF UQUARK ALGORITHM Alghorithm Footprint (Byte) UQUARK 4057 DQUARK 4188 SQUARK 4230 CQUARK 4376 F. Results analysis The obtained result reflects the good performance of Quark algorithm, the lighter version can be very appropriate for wireless sensor network applications. On the other hand, the overhead of complete profile is acceptable and we think that this interpretation can be more adapted for modern SCADA applications. We would like to emphasize that this outcome is obtained with 1 MHz MCU frequency, which can be easily improved by increasing the frequency of the MCU since the adopted MCU can run up to 16 MHz or by switching to higher MCU family. VI. CONCLUSION AND PERSPECTIVES This study focused on SCADA system security but since the boundaries between SCADA systems, DCS, WSN, WSAN and IoT become bluer, this work can be extended to encompass such areas. In the other hand, Texas Instruments introduced a large brand of Launchpad board for various MCU platforms. These platforms offer various types of features. Investigating these platforms can be an interesting extension to this work. Actually, we plan to expand this work by adding an appropriate lightweight cryptography algorithm [22] to our platform. Such solution can be very interesting in distributed SCADA applications. Acknowledgment I would like to thank the Salman Bin Abdulaziz University for the continued and positive support for scientific researches, and thank the anonymous reviewers for their valuable comments. 505 REFERENCES [1] WHAT IS SCADA?, A. Daneels, CERN, Geneva, Switzerland W.Salter, CERN, Geneva, Switzerland, International Conference on Accelerator and Large Experimental Physics Control Systems, 1999, Trieste, Italy [2] Efficient SCADA Module for Improving Medical Information Monitoring and Reliable Medical Service in Hospital Networks Randy S. Tolentino1), Sungwon Park2), Journal of Security Engineering 2010 [3] Study of Wireless Sensor Network in SCADA System for Power Plant, U. S. Patil, International Journal of Smart Sensors and Ad Hoc Networks (IJSSAN) ISSN No (Print) Volume-1, Issue-2, 2011 [4] Vulnerability Assessment of Cybersecurity for SCADA Systems, Chee-Wooi Ten, Student Member, IEEE, Chen-Ching Liu, Fellow, IEEE, and Govindarasu Manimaran, Member, IEEE, IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 23, NO. 4, NOVEMBER 2008 [5] Wang, Yongge. "sscada: Securing SCADA infrastructure.( 2012 ) arxiv: communications."arxiv preprint [6] A Testbed for Secure and Robust SCADA Systems, Annarita Giani, Gabor Karsai, Tanya Roosta, Aakash Shah, Bruno Sinopoli, Jon Wiley [7] Secure SCADA framework for the protection of energy control systems, Cristina Alcaraz1, Javier Lopez1, Jianying Zhou2 and Rodrigo Roman1, Concurrency Computat.: Pract. Exper. 2011; 23: [8] Aumasson, Jean-Philippe, et al. "Quark: A lightweight hash." Cryptographic Hardware and Embedded Systems, CHES Springer Berlin Heidelberg, [9] Yoo, Seong-eun. "A Wireless Sensor Network-Based Portable Vehicle Detector Evaluation System." Sensors 13, no. 1 (2013): [10] Chernbumroong, Saisakul, Anthony S. Atkins, and Hongnian Yu. "Activity classification using a single wrist-worn accelerometer." Software, Knowledge Information, Industrial Management and Applications (SKIMA), th International Conference on. IEEE, [11] Nikitin, Pavel V., Shashi Ramamurthy, and Rene Martinez. "Simple Low Cost UHF RFID Reader." [12] Friedman, Larry. "SimpliciTI: simple modular RF network specification." Update (2007): [13] Daneels, Axel, and Wayne Salter. "What is SCADA." International Conference on Accelerator and Large Experimental Physics Control Systems [14] Daneels, A., & Salter, W. (1999, October). What is SCADA. In International Conference on Accelerator and Large Experimental Physics Control Systems (pp ). [15] Davis, C. M., Tate, J. E., Okhravi, H., Grier, C., Overbye, T. J., & Nicol, D. (2006, September). SCADA cyber security testbed development. In Power Symposium, NAPS th North American (pp ). IEEE. [16] Igure, V. M., Laughter, S. A., & Williams, R. D. (2006). Security issues in SCADA networks. Computers & Security, 25(7),

9 International Journal of Emerging Technology and Advanced Engineering Website: (ISSN , ISO 9001:2008 Certified Journal, Volume 3, Issue 12, December 2013) [17] Jianwei, L., & Huijie, C. (2013). A Dynamic Hashing Algorithm Suitable for Embedded System. TELKOMNIKA Indonesian Journal of Electrical Engineering, 11(6). [18] Balasch, J., Ege, B., Eisenbarth, T., Gérard, B., Gong, Z., Güneysu, T& Von Maurich, I. (2013). Compact implementation and performance evaluation of hash functions in attiny devices. In Smart Card Research and Advanced Applications (pp ). Springer Berlin Heidelberg. [19] Aumasson, J. P., Henzen, L., Meier, W., & Naya-Plasencia, M. (2013). Quark: A lightweight hash. Journal of cryptology, 26(2), [20] Guo, J., Peyrin, T., & Poschmann, A. (2011). The PHOTON family of lightweight hash functions. In Advances in Cryptology CRYPTO 2011 (pp ). Springer Berlin Heidelberg. [21] Bogdanov, A., Knežević, M., Leander, G., Toz, D., Varıcı, K., & Verbauwhede, I. (2011). SPONGENT: A lightweight hash function. In Cryptographic Hardware and Embedded Systems CHES 2011 (pp ). Springer Berlin Heidelberg. [22] Eisenbarth, T., & Kumar, S. (2007). A survey of lightweightcryptography implementations. Design & Test of Computers, IEEE, 24(6),

10 Journal of Engineering Science and Technology Vol. 7, No. 5 (2012) School of Engineering, Taylor s University EMBEDDED MICROPROCESSOR PERFORMANCE EVALUATION CASE STUDY OF THE LEON3 PROCESSOR NABIL LITAYEM 1,2, *, BOCHRA JAAFAR 2, SLIM BEN SAOUD 1 1 LECAP, EPT-INSAT, Centre urbain nord, BP 676 Tunis cedex, Tunisia 2 Computer Science and Information Department, King Salman Bin Abdulaziz University, Wadi Addwaser, Saudia Arabia *Corresponding Author: nabil.litayem@instructor.net Abstract In this paper we propose a performance evaluation methodology based on three complementary benchmarks. This work is motivated by the fact that embedded systems are based on very specific hardware platforms. Measuring the performance of such systems becomes a very important task for any embedded system design process. In a classic case hardware performance is a basic result reported by the hardware manufacturer. The personalization of hardware configuration is one of the fundamental task of FPGA based embedded systems designer. They must measure the hardware performance himself. This paper will focus on hardware performance analysis of FPGA based embedded system using freely available benchmarks to reflect various performance aspects. We used in our study two embedded systems (Mono-processor and Bi-Processor) based on LEON3MMU processor and ecos RTOS. Keywords: Embedded Systems, Performance, Benchmark. 1. Introduction The human activity becomes more and more reliant to embedded systems that are actually present in many products like PDA, camera, telephones etc. Designing this kind of systems can take many approaches depending on the used platform. In a classic approach, General Purpose processor (GPP), Application Specific Processor (ASIP) or Application Specific Integrated Circuit (ASIC) can be used as a heart of the embedded system. Each one of precedent solutions has its advantages and weaknesses. Actually we assist to the Field Programmable Gate Array (FPGA) based embedded systems emergence. This kind of solution can allow rapid embedded 574

11 Embedded Microprocessor Performance Evaluation Case Study Nomenclatures CPI MFLOPS MIPS MOPS Cycles per instruction Million Floating Point Operations Per Second Million Instructions Per Second Millions of Operations Per Second Abbreviations AMBA ASIC ASIC ASIP EEMBC FPGA GNU GPL GPP HAL HDL MMU PDA POSIX RISC RTEMS RTOS SPEC VAX Advanced Microcontroller Bus Architecture Application-Specific Integrated Circuit Application-Specific Integrated Circuit Application-Specific Instruction-set Processor EDN Embedded Microprocessor Benchmark Consortium Field-Programmable Gate Array Gnu's Not Unix General Public License General Purpose Processor Hardware Abstraction Layer Hardware Description Language Memory Management Unit Personal Digital Assistant Portable Operating System Interface for Unix Reduced Instruction Set Computer Real-Time Executive for Multiprocessor Systems Real-Time Operating System Standard Performance Evaluation Corporation Virtual Address extension systems generation [1], easy personalization of hardware configuration using pre-designed Hardware IPs [2], future evolution of embedded system and cost reduction. To design an FPGA based embedded system we have to choose an embedded processor and embedded operating system. Embedded processor can be Hard-Core (built in silicon level) or Soft-Core (netlist or as HDL source). Hardcore embedded processor has the advantage of computing performance but limit the system in terms of portability and scalability. Soft-Core embedded processor offer less computing possibility if the final platform is an FPGA, but is greatly enhanced in term of configurability, portability, customization and scalability [3, 4]. Embedded operating system can be shared time or real-time, proprietary or open source. The adoption of an embedded operating system depends on the available memory, developers' strategy, real-time requirements, operation fields certification etc. On the other hand, the hardware resources limitation for embedded software requires that the developer must have a clear idea about the hardware computing performance. Actually, many studies focus on hardware performance evaluation or estimation of customized architecture [5]. In this paper we present an approach to measure hardware performance of FPGA based embedded system using freely available benchmark solutions. This work is divided into six parts. The first part is a survey about the performance evaluation. The Journal of Engineering Science and Technology October 2012, Vol. 7(5)

12 576 N. Litayem et al. second part presents an overview about the used platform. In the third part we will present the adopted benchmarks. This is followed by the experimental condition and environment preparation presentation. In the fifth part experimental results are presented. The last section summarizes our position and future works. 2. Overview of Performance Evaluation Tools And Techniques Evaluating performance in computer system [6] will always be a true challenge for designer of this kind of systems due to the constant evolution of such systems, especially for embedded system field where the architecture tend to be more and more complex [7]. This kind of activity is actually related to the performance engineering field which tries to propose tools and methods to quantify nonfunctional requirements of computer systems. The performance analysis of an embedded system may have various aspects depending on the application for which the embedded system is designed. Several design decisions are a direct result of performance evaluation. The first evaluation method was based on the number of operations per second. This approach quickly became deprecated and traditional benchmark was developed and adopted to measure special performance aspects. This aspect can be one of the various computer system criteria. Actually we have too many solutions to measure hardware performance. The most part of solutions are based on standard algorithms that are executed and used to report a number which reflects the performance of the hardware speed in a special field. In generally the MIPS unit in its literal meaning millions of instructions per second is used to measure the processor hardware performance. This unit became insignificant when RISC computer architectures appeared since the performed instruction by one CISC computer cycle requires several RISC instructions. Which lead to the MIPS redefinition as VAX MIPS which is the factor for a given machine relative to the performance of a VAX 11/780. Other redefinitions are later proposed such as Peak MIPS, and EDN MIPS. Due to the fuzziness of performance unit definition, several benchmarks are proposed that can be executed under various architectures to report their execution speed. The most recognized solutions are Dhrystone which report the performance of the architecture in Dhrystone MIPS, Stanford which computes different algorithms and report the performance of the architecture in every computation field covered by the benchmark, and Paranoia who is able to report the characteristics of the floating point unit. Nowadays, there are several benchmarks that include the previously presented ones combined with other widely known benchmarks such as Whestone and Linpac. Each one of these benchmarks tries to reflect the most of the hardware performance aspects. Mibench [8] is the most popular implementation of this combination since it can measure the performance of a studied architecture in several application fields. We can also find other commercial benchmarking solutions more efficient and more specialized like SPEC (Standard Performance Evaluation Corporation) which cover different computing field or EEMBC (Embedded Microprocessor Benchmark Consortium) designed especially for embedded systems. Journal of Engineering Science and Technology October 2012, Vol. 7(5)

13 Embedded Microprocessor Performance Evaluation Case Study In this paper we will present a hardware performance analysis of monoprocessor and bi-processor embedded systems using three benchmarks, each one cover one side of computing system. Our platform is based on two open source components (LEON3 Processor and ecos RTOS) allowing us to be independent to any FPGA or RTOS vendor. 3. Presentation of the Used Platform 3.1. Overview of LEON3 microprocessor LEON3 [9] presented in Fig. 1 is a synthesizable VHDL model of a 32-bit processor compliant with the SPARC V8 [10] architecture. The model is highly configurable, and particularly suitable for system-on-a-chip (SOC) designs. The full source code is available under the GNU GPL (General Public License), allowing free and unlimited use for research and education. LEON3 is also available under a low-cost commercial license, allowing it to be used in any commercial application for a fraction of the cost of comparable IP cores. On the other hand Gaisler research offers a fault tolerant version of the LEON3 for a very competitive cost. The LEON3 processor is distributed as a part of the GRLIB IP library, allowing simple integration into complex SoC designs. GRLIB also includes a configurable LEON multi-processor design, with up to 16 CPU's attached to AHB bus, and a large range of on-chip peripheral blocks. The GRLIB library contains template designs and bitfiles for various FPGA boards from Actel, Altera, Latice and Xilinx. IRQ 15 Interrupt Control 3-Port Regfile MUL32 MAC 16 DIV 32 7-Stage Integer Pipeline Instruction Cache Data Cache IEEE 754 Floating- Point Unit Co-Processor Debug Interface Debug I/F MMU Trace Buffer AMBA AHB Interface 32 Minimum Configuration Optional Blocks Co-Processors Fig. 1. Overview of LEON3 Architecture [9] Overview of ecos RTOS ecos is an abbreviation of Embedded Configurable Operating System [11]. It is an open source, royalty-free, real-time operating system intended for deeply embedded applications. The highly configurable nature of ecos allows the operating system to be customized to precise application requirements, delivering the best possible run-time performance and an optimized hardware resource footprint. A thriving net community has grown up around the operating system Journal of Engineering Science and Technology October 2012, Vol. 7(5)

14 578 N. Litayem et al. ensuring on-going technical innovation and wide platform support. Figure 2 shows the layered architecture of ecos. Application Libraries Compatibility RedBootR OM Monitor Math C POSIX µitron Web Kernel Hardware Abstraction Layer Interrupts Virtual Vectors Exceptions Ethernet Serial Flash Target Hardware Networking Stack Server File System Device Driver Fig. 2. Overview of ecos Architecture. The main components in ecos architecture are the HAL (Hardware Abstraction Layer) and ecos Kernel. The purpose of ecos HAL is to allow the application to be independent of hardware target. It can manipulate the hardware layer using the HAL API. This HAL is also used by others upper OS layer which make porting ecos to a new hardware target a simple task consisting of developing the HAL of the new target. ecos kernel is the core of ecos system, it includes the most part of modern operating system components: scheduling, synchronization, interrupt, exception handling, counters, clocks, alarms, timers, etc. It is written in C++ language allowing application written in this language to interface directly to the kernel resources. The ecos kernel also supports interfacing to standard µitron and POSIX compatibility layers Combination ecos LEON3 The choice of these components allows us to be independent from any FPGA constructor or RTOS vendor since ecos is available for a wide range of embedded processors and LEON is ported for XILINX, ALTERA, ACTEL and LATICE FPGA. In the other hand ecos allows a smooth migration to embedded Linux. We can also use OpenRISC [12] as processor. RTEMS [13] or embedded Linux [14] can be adopted as OS. The performance measure will be presented using the three benchmarks that we will present, and the hardware platform will be simulated using tsim-leon3 for a mono-processor architecture and grsim-leon3 for the bi-processor platform in SMP (synchronous multi-processing) configuration. These two simulation tools are able to represent very closely the LEON3 architecture with many other very important enhanced features for system prototyping. Journal of Engineering Science and Technology October 2012, Vol. 7(5)

15 Embedded Microprocessor Performance Evaluation Case Study Used Benchmarks In our work we have chosen to adopt three complementary benchmarks that can mirror a complementary performance side of the studied embedded system. Each one of the obtained results of these benchmarks can be used to evaluate a special performance aspect of the hardware platform Dhrystone Dhrystone [15] is a synthetic benchmark developed in 1984 by Reinhold P. Weicker in ADA, Pascal and C languages. It is intended to be representative of integer system performances. Dhrystone is constituted from 12 procedures included in one measuring loop with 94 statements. The Dhrystone grew to become representative of general processor (CPU) performance until it was outdated by the CPU89 benchmark suite from the Standard Performance Evaluation Corporation [16], today known as the "SPECint" suite Stanford The Stanford Benchmark Suite [17] is a small benchmark suite that was assembled by John Hennessy and Peter Nye around the same time period of the MIPS R3000 processors. The benchmark suite contains ten applications, eight integer benchmarks and two floating-point benchmarks. The original suite measured the execution time in milliseconds for each benchmark in the suite. The Stanford Small Benchmark Suite includes the following programs: Perm: A tightly recursive permutation program. Towers: the canonical Towers of Hanoi problem. Queens: The eight Queens Chess problem solved 50 times. Integer MM: Two 2-D integer matrices multiplied together. FP MM: Two 2-D floating-point matrices multiplied together. Puzzle: a compute bound program. Quicksort: An array sorted using the quicksort algorithm. Bubblesort: An array sorted using the bubblesort algorithm. Treesort: An array sorted using the Treesort algorithm. FFT: A floating-point Fast Fourier Transform program. This kind of benchmark is very interesting in terms of exploration of various architecture behaviours Paranoia The Paranoia benchmark [18] is designed by William Kahan as a C programable to characterize floating-point behaviour of computer system. Paranoia does the following test: Small integer operations. Search for radix and precision. Check if rounding is done correctly. Check for sticky bit. Journal of Engineering Science and Technology October 2012, Vol. 7(5)

16 580 N. Litayem et al. Test if 2 X = X for a number of integers. If it will pass monotonicity. If it is correctly rounded or chopped. i Testing power Z, for small Integers Z and i. Searching for underflow threshold and smallest positive number. Q Testing power Z at four nearly extreme values. Searching for overflow threshold and saturation. Tries to compute 1/0 and Experimental Set-Up Gaisler research offer all the required tools to begin developing application using Leon processor combined with a wide range of supported RTOS such ecos. In this section, we present the ecos configuration phases, the benchmarks compilation and their respective execution environment ecos configuration The ecos RTOS installation and configuration can be divided into four steps. In our case we have chosen to use Windows host but similar approach can be followed with Linux Environment installation Since ecos and their associated tools require Linux-like environment, we must begin by installing Linux emulation environment. For this purpose there are two candidates which are Cygwin and MingW. We chose to use MingW due to its lightweight. It must be noted that make, mpfr, sharutils, tcltk, wget, automake and gcc packages must be installed with the emulation environment Cross compiler installation In order to produce Leon3 executable we must have a Leon cross compiler installed in our host. The recommended cross-compiler for ecos is sparc-elf-gcc. This one is available in the Gaisler web site [19] and must be downloaded, decompressed in the /opt directory and installed using export PATH=/opt/sparc-elf mingw/bin:$PATH command Source code and configuration tool In this stage we must install the ecos source code and their configuration tool. These resources are also available in the Gaisler web site [19]. Source code must be downloaded and decompressed in a chosen directory that can be used in the configuration phase. The configuration tool is available in different versions that differ according to their software dependencies. For our case we used the native Windows version that does not need additional library. Journal of Engineering Science and Technology October 2012, Vol. 7(5)

17 Embedded Microprocessor Performance Evaluation Case Study ecos configuration ecos is one of the most architecture free RTOS. The choice of a specific target is done using the configuration tool GUI that offer a wide range of hardware platform and software configurations. In this step we must run the ecos configuration tool and select the target platform LEON3 processor as shown in Fig. 3. In the same configuration window we can chose a predefined setting that can be customised later by selecting specific networking stack, debugging interface or any other specific software component. In our case default packages can be enough. Fig. 3. Hardware Platform and Used Package. The build tool and Cygwin binary tools directories must be selected in the configuration tool as shown in Figs. 4 and 5. In the case of using MingW, these ones are located at C:\MinGW\msys\1.0\bin. Fig. 4. ecos Repository. Fig. 5. ecos Build Tools. Journal of Engineering Science and Technology October 2012, Vol. 7(5)

18 582 N. Litayem et al. Finally we must save the configuration file and starting the building process using build item in the configuration tool GUI Benchmarks compilation One of the most valuable tools offered by Gaisler research is the eclipse based IDE. This one offers various wizards for project creation that generates project template for ecos shared library, static library or executable. We used this wizard as showed in Figs. 6 and 7 to generate ecos project template. Fig. 6. ecos Project Generation. Fig. 7. ecos Example Code Generation. The ecos ECOS_INSTALL_DIR environment variable must then be configured in the project setting according the previously chosen ecos install directory. To build adequately the three previously presented benchmarks, we created three projects in which we substitute the generated code with the benchmark source code. In the build configuration panel math library and ecos must be selected to be used during the link process Benchmarks execution Gaisler research offers two simulation platforms. The first one is Tsim-leon3 designed to simulate mono-processor Leon3 based architecture, the second one is grsim-leon3 who is designed to simulate bi-processor Leon3 based architecture. These two simulators have nearly the same usage syntax. We can then load and execute the appropriate benchmark in their respective architecture. 6. Performance Measurement and Analysis In the following section we present the performance evaluation and their interpretation according the flowchart showed in Fig. 8. Journal of Engineering Science and Technology October 2012, Vol. 7(5)

19 Embedded Microprocessor Performance Evaluation Case Study After preparing the environment in terms of configuring, building ecos for LEON3MMU architecture and testing some applications examples running under ecos and executing their benchmarks from SDRAM. We build the three benchmarks for our two hardware configurations described in Table 1. Table 1. Hardware Configuration. Mono-processor Configuration Bi-processor Configuration SDRAM 16 Mbyte in 1 bank 16 Mbyte 1 bank ROM 2048 Kbyte 2048 Kbyte Instruction Cache 1*4 Kbytes, 16 bytes/line 1*4 Kbytes, 16 bytes/line Data cache 1*4 Kbytes, 16 bytes/line 1*4 Kbytes, 16 bytes/line Start Setup evaluation environment Monoprocessor integer performance evaluation Bioprocessor integer performance evaluation Monoprocessor domain specific performance evaluation Monoprocessor floating point characterisation Bioprocessor domain specific performance evaluation Bioprocessor floating point characterisation Monoprocessor performance results Bioprocessor performance results Choose appropriate configuration Stop Fig. 8. Performance Evaluation Flowchart Obtained results using Dhrystone benchmark After executing Dhrystone benchmark under our platforms simulators we have the reported values in Fig. 9 and summarized in Table 2. These results show the Journal of Engineering Science and Technology October 2012, Vol. 7(5)

20 584 N. Litayem et al. performance of studied architecture in term of Dhrystone MIPS. The gain in performance is about 33% with the bi-processor configuration. Fig. 9. Performance in Dhrystones MIPS of the Two Architectures. Table 2. Results Obtained with the Two Platform Simulators for Dhrystone Benchmark. Mono-processor Architecture Bi-processor Architecture Cycles Instructions Overall CPI CPU performance (50.0 MHz) MOPS (31.66 MIPS, 0.00 MFLOPS) MOPS (41.43 MIPS, 0.00 MFLOPS) 6.2. Obtained results using Stanford benchmark After executing Stanford in our platform simulator we collected the performance report plotted in Figs. 10 and 11 and summarised in Table 3. Fig. 10. Execution Time in Millisecond of the Ten Algorithms Included in Stanford Benchmark. Journal of Engineering Science and Technology October 2012, Vol. 7(5)