Overview Performance of Distributed Systems (COMP40550/COMP47070) ASE module, 1-5 December 2008 ! ! Practicals: start each afternoon at 2pm, B1.06 (unless otherwise notified…) Dr. John Murphy Room B2.15, B2.15, ee-mail: j.murphy@ucd.ie, j.murphy@ucd.ie, tel: 716 2928 2928 Lectures: start each morning at 9:30am, Computer Science Lecture Theatre (should be finished by 12:30…) ! Recommended (not required) Textbook: ! Prof. Liam Murphy 2002, ISBN 0-201-72229-1 Room B2.22, B2.22, ee-mail: liam.murphy@ucd.ie liam.murphy@ucd.ie,, tel: 716 2914 Head Demonstrator: John Bergin e-mail: john.bergin @ucd.ie ie john.bergin@ucd. ! ! Contacting us: email preferred… Website: http://www.csi.ucd.ie/staff/lmurphy/home/ase_perf-eng.htm How to Pass this Module ! ! ! ! Attend Lectures Read the Lecture Notes, & understand them – ask questions if you don’t understand ! 1 short In-class test, worth 25%, on Friday 9:30am, to make sure you do this! Attend Practicals and ask the demonstrators questions Assignment Report (due after this module is over…) ! Worth 75% of the marks in this module ! ! We reserve the right to interview any student about their report – so don’t plagiarise code/text/… No Final Exam Performance Solutions: A Practical Guide to Creating Responsive, Scalable Software, Connie U. Smith and Lloyd G. Williams, pub. Addison Wesley, Textbook ! ! Connie Smith’s company’s website: http://www.perfeng.com/ However, our lecture notes (on the web) will be your primary source material. What does performance mean? Performance of Distributed Systems (COMP40550/COMP47070) ! ! A Brief Introduction to Performance Engineering ! Typical performance engineering tasks ! Specifying system performance requirements ! Evaluating design alternatives ! Comparing the performance of 2 or more systems ! Determining optimal parameter value(s) ! Finding performance bottleneck(s) ! Characterising the load on the system ! Capacity planning – system sizing for the future ! Forecasting performance at future loads/workmixes/etc A measure of how a system operates ! Examples: response time; throughput; efficiency; users’ QoE; … Possible goals of performance engineering: ! Achieve the best performance for a given cost ! Achieve acceptable performance at minimum cost ! Determine performance/cost of each of a number of alternatives ! Find performance problem(s) and determine how to fix them ! Determine when and how the system should be modified/upgraded so that its performance remains acceptable ! and many others… Performance Engineering is an “art” ! requires detailed knowledge of system and its environment, and choice of appropriate evaluation techniques & tools ! BUT: there are some general principles… Why is performance evaluation difficult? ! Lots of mistakes can be made! ! No goals for the performance study; or goals are unclear, or biased ! Unsystematic approach e.g. arbitrary parameter and workload values ! Inappropriate performance metrics e.g. easily computed but irrelevant ! Unrealistic workload ! Inappropriate level of detail in the model ! No analysis; or incorrect analysis e.g. simulation run too short ! Ignoring variability, or future possibilities ! Inappropriate presentation of results ! …and many many others… Misleading performance results ! Could be deliberate e.g. vendor benchmark results, or incorrect analysis ! Example: ratio game Average Throughputs: System Workload 1 A 20 B 10 Throughput measurements: System Workload 1 Workload 2 A 20 10 B 10 20 Systematic approach to performance evaluation ! State goals of the study and define system scope ! Define system services and possible outcomes of user requests ! Select performance metric(s) ! Identify system parameters that affect performance ! Workload 2 Average 10 15 20 15 Average Throughput wrt B: System Workload 1 Workload 2 Average A 2 0.5 1.25 B 1 1 1 Average Throughput wrt A: System Workload 1 Workload 2 Average A 1 1 1 B 0.5 2 1.25 A and B have the same performance “A is better than B” “B is better than A” ! Select which parameters are variable Select evaluation technique(s) ! Analytical modelling ! Simulation/emulation ! Experimental (real system/prototype) ! Select workload(s) to be studied ! Design experiments ! Analyse and interpret the data (statistical techniques) ! Present the results in an understandable manner