Greening Clouds and High Performance Computing: Challenges and Initiatives

Transcription

1 Greening Clouds and High Performance Computing: Challenges and Initiatives Jean-Patrick Gelas, Olivier Glück, Laurent Lefèvre, Mehdi Diouri, François Rossigneux, Ghislain Landry Tsafack Chetsa INRIA AVALON / LIP École Normale Supérieure de Lyon GreenTouch, ECOCITY sept Nantes, France

2 Context : HPC and Cloud computing (briefly ) HPC (High Performance Computing) and Grid computing: Grid5000, Egee, Datagrid, Cloud computing: Amazon, Google, Microsoft Azur, Rackspace, Saleforce, - 2

3 Energy : 1 st limiting factor for large scale systems (Datacenter, Grids, Clouds, Internet)? Future exascale/datacenters platforms -> systems from 20 to 100MW (current 1-18 MW) Top500 : 1-Tianhe-2, 3M cores, 33 pflops (1,8 Gflops/W); 11 Total (100k cores, 2 MW, 2 pflops, 1Gflops/W). Green500 : 1 2,5 Gflops/W. How to build such systems and make them energy sustainable/responsible? - Hardware will help (component by component) : «At a fixed computing load, the amount of battery you need will fall by a factor of two every year and a half.» -- Jonathan Koomey Software must be adapted to be scalable but also more energy efficient Usage must be energy aware - 3

4 Power demand and Green IT explosion IT 2-5% of CO 2 emissions / 10% electricity Green IT è reducing electrical consumption of IT equipments - CO 2 impact depends on countries Focus on usage : fighting un-used/overprovisioned plugged resources Researchers are launched : GreenIT scientific events (tracks/issues greenit-conferences.org) - 4

5 We are part of climate changing! Or at least of enormous electricity usage as IT users/designers «Greenpeace reports » High Performance Datacenter has a bigger role to play - 5

6 In the Wattmeters jungle Eaton, Schleifenbauer, OmegaWatt, Dell idrac6, WattsUp, Zimmer LMG450, M. Diouri, M. Dolz, O. Glück, L. Lefevre, P. Alonso, S. Catalan, R. Mayo, E. Quintan-Orti. "Solving some Mysteries in Power Monitoring of Servers: Take Care of your Wattmeters! ", EE-LSDS 2013 : Energy Efficiency in Large Scale Distributed Systems conference, Vienna, Austria, April 22-24,

7 «If you can not measure it, you can not improve it.» Lord Kelvin - 7

8 If external monitoring is not enough : looking inside with dedicated equipment... Example : Powermon 2 : from RENCI ilab: ilab.renci.org/powermon Daniel Bedard, Min Yeol Lim, Robert Fowler, and Allan Porterfield PowerMon: Fine-grained and Integrated Power Monitoring for Commodity Computer Systems. 8 measurement channels connects directly to ATX supply - 8

9 Limits :-( Difficult to deploy at large scale! - 9

10 Profiling applications (old measures) Profiling the energy consumption of applications - 10

11 Profiling applications and Node s power usage Profiling the energy consumption of applications : more than watts? Dell R610 - Zimmer LMG450-11

12 - 12

13 - 13

14 - 14

15 Understanding measurements : take care of your wattmeters! Frequency / precision M. Diouri, M. Dolz, O. Glück, L. Lefevre, P. Alonso, S. Catalan, R. Mayo, E. Quintan-Orti. "Solving some Mysteries in Power Monitoring of Servers: Take Care of your Wattmeters! ", EE- LSDS 2013 : Energy Efficiency in Large Scale Distributed Systems conference, Vienna, Austria, April 22-24,

16 Homogeneity (in energy consumption) does not exist! Depends on technology Same flops but not same flops per watt Idle / static cost CPU : main responsible Mohammed el Mehdi Diouri, Olivier Gluck, Laurent Lefevre and Jean-Christophe Mignot. "Your Cluster is not Power Homogeneous: Take Care when Designing Green Schedulers!", IGCC2013 : International Green Computing Conference, Arlington, USA, June 27-29, - 16

17 Collecting and exposing l The Green Grid'5000 l French experimental testbed l 7400 cores l 10 sites l External energy sensors l Full site monitoring - 17

18 - 18

19 Introducing 4 methods to improve energy consumption 1. By Knowing applications and services 2. Without knowledge of applications 3. Taking account user requirements 4. Smartgrid : adapting power supply to power demand - 19

20 1. Reduce consumption knowing applications and services 4 steps: Service analysis, Measurements, Calibration, Estimation. Examples (exascale services) : Resilience & Data Broadcasting Helping users make the right choices depending on context and parameters M. Diouri, Olivier Glück, Laurent Lefevre, and Franck Cappello. "ECOFIT: A Framework to Estimate Energy Consumption of Fault Tolerance Protocols during HPC executions", CCGrid2013, the 13th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, Delft, the Netherlands, May 13-16,

21 2. Without knowledge on applications Irregular usage of resources Phase detection, characterisation Power saving modes deployment Landry Tsafack, Laurent Lefevre, Jean-Marc Pierson, Patricia Stolf, and Georges Da Costa. "A runtime framework for energy efficient HPC systems without a priori knowledge of applications", ICPADS 2012 : 18th International Conference on Parallel and Distributed Systems, Singapore, December

22 3. Taking into account users requirements Reservation Matching the user requirements Providing homogeneous nodes Including energy usage in billing Selecting the greenest hosts Using power saving modes - 22

23 Taking into account users requirements (Kwapi) - 23

24 4. Exascale : a big energy consumer => consume better With a x MW customer: exchange framework is mandatory (SESAMES) M. Diouri, O. Gluck, and L. Lefevre. "Towards a novel Smart and Energy-Aware Service-Oriented Manager for Extreme-Scale applications, First Workshop for Power Grid-Friendly Computing (PGFC'12), San Jose, USA, June 2012

25 Conclusion GreenIT is not just a buzz word! Measuring, analyse, understand at large scale is not easy It will helps to position VM and Data, Understand applications energetic profile. Objective : Providing the best without being intrusive. Optimizations requires multi-disciplinary approach (thermic, electronic, social, ) - 25

26 Thanks to : L.Lefèvre, M. Diouri, G. Tsafack, O. Glück, J.C. Mignot, F. Cappello, F. Rossigneux, J.- P. Gelas, J.-M. Pierson, P. Stolf, G. DaCosta Questions? Laurent.lefevre@inria.fr Jean-patrick.gelas@univ-lyon1.fr Olivier.gluck@univ-lyon1.fr

27 Titre : Propositions pour améliorer l'efficacité énergétique des Data Centers et des Clouds Mots clés: Cloud, efficacité énergétique, Green IT Les technologies de l'information sont aujourd'hui responsables de près de 2% des émissions de CO 2 (soit l'équivalent de celles du transport aérien). 23% de ces émissions sont provoquées par les centres de calcul de plus en plus grands et nombreux (en 2007 ils consommaient 623 milliards de kwh). La consommation des centres de calcul comme ceux de Amazon, Google, Facebook ou Microsoft peuvent consommer chacun l'équivalent de foyers. Depuis quelques années, avec l engouement pour le GreenIT, les centres investissent dans différentes solutions pour réduire leur consommation et donc leur facture électrique (sans oublier d'améliorer leur image de marque auprès du grand public). Dans la pratique beaucoup de serveurs sont sous utilisés. Pour tirer meilleure partie de leur plein potentiel une solution très classique aujourd'hui est l'usage de la virtualisation qui permet à plusieurs applications de fonctionner sur un même serveur physique de manière isolée. Cette technique est à l origine des infrastructures de type Cloud. Aujourd'hui, si le Cloud était un pays, il aurait la 5ème position des pays qui consomment le plus d'électricité (La France 9ème (447milliards kwh) consomme moins!). Cette présentation débutera donc par une introduction sur l impact énergétique des technologies de l information sur notre planète en appuyant notre discours sur des résultats obtenus non seulement dans la littérature scientifique mais également à partir des travaux de recherche et de mesures expérimentales de consommation énergétique que nous menons au laboratoire LIP au sein de l équipe INRIA Avalon depuis quelques années. Puis nous proposerons de présenter et de discuter un ensemble de solutions, pour améliorer l'efficacité énergétique des centres de calcul et des Clouds et donc de minimiser leur impact écologique sur la planète. Sources et liens : Koomey: Data Center Energy Use is Moderating How clean is your cloud? (Greenpeace) Kwapi in action :