Life on the Edge with ESI Kit Chan(kichan@yahoo-inc.com) Yahoo! Life on the Edge Source: http://img.shockblast.net/2012/01/ShockBlast-4-803109.jpg Agenda      Introduction Use Cases Performance Characteristics Future Q&A Agenda      Introduction Use Cases Performance Characteristics Future Q&A Edge Computing/CDN Source: http://www.excitingip.com/wp-content/uploads/2011/01/contentdeliverynetworkblockdiagram.jpeg CDN in Yahoo! User Location Latency - 100K file in US Through Y! Edge California, USA 0.2s 0.02s Singapore 2s 0.02s Korea 1.3s 0.02s Ireland 1.2s 0.2s Switzerland 1.0s 0.2s - Pages: avg 30.2 Gbps send, 28.48 Gbps recv, 20.2% cache ratio - Assets: avg 38.7 Gbps send, 4.1 Gbps recv. 96.7% cache hit ratio Edge Computing in Yahoo! User Location Latency - 100K file in US Through Y! Edge California, USA 0.2s 0.02s Singapore 2s 0.02s Korea 1.3s 0.02s Ireland 1.2s 0.2s Switzerland 1.0s 0.2s - Pages: avg 30.2 Gbps send, 28.48 Gbps recv, 20.2% cache ratio - Assets: avg 38.7 Gbps send, 4.1 Gbps recv. 96.7% cache hit ratio ESI – Edge Side Includes Syntax Example Markup in Response <esi:include src=http://xyz.com/content.html/> Syntax Example Can handle response error and timeout <esi:try><esi:attempt> <esi:include src=http://xyz.com/content.html/ > </esi:attempt><esi:except> Sorry there is an error! </esi:except></esi:try> ESI Support in Proxy Server – Proven & Popular ESI Support in CDN Vendor Agenda      Introduction Use Cases Performance Characteristics Future Q&A Use Case 1 – Content Assembly Use Case 1 – Content Assembly Use Case 1 – Content Assembly Use Case 1 - Content Assembly Why? Performance! Scenario No server cache + No ESI Origin Server Handling Full Page requests each time and executing every module every time. Server cache + No ESI Origin Server handling full page requests, only executing non cacheable modules FE throughput per box Server latency 45 queries per 140 queries per second second 900 ms per request 80 ms per page ATS Cache + ESI modules Origin Server only serving requests for non cacheable modules 190 queries per second 55 ms per page Why? Availability Why it goes down? - Releases - Outage How ESI Help? - Template cached - Fallback for each ESI module Why? Legacy Module Support Use Case 2 – Combo Handler  Allow us to fetch multiple JS/CSS in one HTTP request  E.g. http://l.yimg.com/zz/combo?d/1.js&d/2.js - It combines http://l.yimg.com/d/1.js and http://l.yimg.com/d/2.js into one response in the server and returns one response  Minimize HTTP requests from browser to increase performance Combo Handler Sequence Diagram Sample ESI Doc from Combo URL Decode Service <esi:try><esi:attempt> <esi:include src=http://l.yimg.com/d/1.js/> </esi:attempt><esi:except> /* error fetching 1.js */ </esi:except></esi:try> <esi:try><esi:attempt> <esi:include src=http://l.yimg.com/d/2.js/ > </esi:attempt><esi:except> /* error fetching 2.js */ </esi:except></esi:try> Why?  ATS with ESI Plugin - Provide sophisticated features for HTTP resource caching - Provide other features such as timeout, collapse forwarding for request - Provide capability of fetching contents simultaneously and assembling them together Use Case 3 - Timely Launch Before Launch After Launch Timely Launch – Before Launch Timely Launch – Before Launch Timely Launch – After Launch Timely Launch – After Launch Usage Pattern ESI Service Client Proxy Server/CDN Origin Server for ESI Include Agenda      Introduction Use Cases Performance Characteristics Future Q&A Concurrent Requests First Byte Flush ESI Support in Varnish No concurrent requests for includes First Byte Flush Support ESI Support in Apache Traffic Server Concurrent requests for includes First Byte Flush Support* Agenda      Introduction Use Cases Performance Characteristics Future Q&A History The spec (ESI 1.0) is published over 10 years ago. - W3C spec - Editor - Mark Nottingham from Akamai Mark Nottingham http://www.mnot.net/ Future – Deep HTTP Integration Origin Server returing ESI Document Client Proxy Server Supporting ESI Origin Server for ESI Include Future – Deep HTTP Integration Origin Server returing ESI Document Client Proxy Server Supporting ESI Origin Server for ESI Include Final response header to user cannot be changed Future – Deep HTTP Integration Request Header, method, timeout cannot be changed for includes Client Origin Server returing ESI Document Proxy Server Supporting ESI Origin Server for ESI Include Final response header to user can be changed Future – Deep HTTP Integration Request Header, method, timeout can be changed for includes Client Origin Server returing ESI Document Proxy Server Supporting ESI Origin Server for ESI Include Final response header to user can be changed Request/Response Header cannot be retrieved as variables Future – Smart Assembly One request for multiple includes (e.g. one Ad call fetching multiple ad position) Future – More Use cases Device Detection Bucket Testing https://doazrm3qeh7n2.cloudfront.net/images/features/quick-a-vs-b-testing.jpg http://illusionmedia.com/wp-content/uploads/2012/03/mobileDevices1300x200.jpg Agenda      Introduction Use Cases Performance Characteristics Future Q&A Q&A Thank you kichan@yahoo-inc.com Reference  ESI Language Specification – http://www.w3.org/TR/esi-lang  Edge Architecture Specification - http://www.w3.org/TR/edge-arch  ATS ESI plugin https://github.com/apache/trafficserver/blob/master/plugins/experimental/esi/README  Varnish ESI - https://www.varnish-cache.org/docs/3.0/tutorial/esi.html  Akamai ESI - http://www.akamai.com/dl/technical_publications/akamai_esi_extensions.pdf  F5 - http://support.f5.com/kb/enus/products/wa/manuals/product/wa9_4policy/WA_Policy_9_4-19-1.html Extra: Moving to the Real Edge? – the Clients  Problem with SEO  Performance - More HTTP requests between clients and edge servers