The Solid Earth and Environment Grid Robert Woodcock, CSIRO 6 September 2005 www.csiro.au Back in 2003… “…It is necessary for open standards and interfaces to be established by communities to be able to interoperate effectively… The Solid Earth and Environment Grid community has been established… to bring together people in the earth, environmental and computing sciences… to address the issues of "transparent access" to data and knowledge about the earth,… and the available and potential technologies offered by the grid… that enhance our ability to explore for and manage our natural and mineral resources.” The Solid Earth and Environment Grid SEE Grid Roadmap Geochemistry WFSand online eXploration Mining 228 Registered in 3 geological Markup surveys Language (XMML) Users • Gplates ML 500-700 unique visitors per Spatial Interoperability • CoalML month Demonstrator• Project Geochemistry Profile APAC Grid for• AssayOver Data1000 visits per month GeosciencesExchange Continued steady growth OGC WFS: Geoserver Australian-UK Open Geospatial Six sub-web communities collaboration OGC Consortium WMS for the(OGC) Exploitation of Grid and Geospatial Standards. SocialCommission Change Online for ….Web Map Composer Geoscience Information Data Model Collaboration Metech – Acquire Natural Resources Fractal Technologies – FracSIS Suite Marine 2005 Globus GridSystems Middleware…. Earth The SEE Grid Community Working together (loosely) to develop a toolkit for interoperability for the Solid Earth and Environmental Sciences  Together… because our information and services need to be shared more easily to achieve our goals  Loosely… because ultimately we are separated by political and economic boundaries  Toolkit… because our World is dynamic and we need tools that can be reconfigured and chained together quickly to answer our questions …in this context we must reduce the barriers to becoming a part of the community The Solid Earth and Environment Grid What is involved in being part of the Community? Interoperability – how? A community shares: What does 2 years of SEE Grid give us?  Community Architecture  Information Models  Information Service Interfaces  Computational Service interfaces  Education …SEE Grid seeks to capture and publish these shared aspects Figures from Software Fortresses by Roger Sessions The Solid Earth and Environment Grid Architecture Open Geospatial Consortium Notional Architecture  Most projects use OGC NA as a starting point  And its information standards…  And its service interfaces…  It can be quite a stack of paper! Key Lessons:  OGC NA breaks problem into specific aspects  Use it to identify necessary services for your application  Reconcile desire for simplicity with realities of the situation  Formal documentation of assumptions and scope  Write Service Profiles - Check if they exist and use them  It pays to get assistance from specialists in the community – use Roadmaps and Guides  OGC is NOT enough  domain specific profiles required  Many components do NOT support this Figures from Software Fortresses by Roger Sessions The Solid Earth and Environment Grid Human face of Systems Architecture Nobody understands all the problem domain, technology and institutions… Architecture allows:  Separation of concerns  a business planning capability:  From an architecture one can develop a “roadmap” to address short term compromises  reconcile user needs with provider capabilities  Not everything is there yet… The Solid Earth and Environment Grid Common Compromises Data isn’t as consistent as you’d hoped  Reduced user functionality  Data remediation/governance improvements  change in business practice! Technology doesn’t quite allow you to jump from what you have to what you need in 1 configurable step…  Components can be improved  SEE Grid attempting to collect and document common requirements to assist community in improving components “Stubbing” of components  Registers in particular though this can’t go on much longer… The Solid Earth and Environment Grid Information Models Not a storage problem…  User View (Data Products)  Custodian’s View (Consistency, Efficiency)  Systems View (bandwidth, query cost) Semantics and structure  GML, GML application schemas  XMML and other domain specific models – available from version controlled repository, automated validation being setup  Community collaboration addressing cross-domain issues (Earth Systems and Geosciences)  Common patterns emerging – Observations and Measurements  These are fed back to standards bodies From one data store, many different products Specific views are “profiles” of a more complete model  SEE Grid community profiles are starting to be documented The Solid Earth and Environment Grid Would be nice to have… Architecture  Catalog of Usage and Management Use Cases – this helps identify community issues Information Models  Governance – by whom?  Catalog of Design Patterns – starting to emerge… Education  GML Application Schema Design Tutorial – getting the ball rolling… Profiles = simplicity, consistency  Geochemistry WFS Profile underway, others exist in OGC but many more required in domain specific areas  Document them and place them on SEE Grid Harmonisation  OGC Web Services, SOAP, Grid, WSRF The Solid Earth and Environment Grid What is expected to be achieved (actually in proposals/plans) •National pre-competitive geoscience data grid  Assays – national demonstrator running (upcoming talk)  Hyper-spectral, geophysics, geology, drill holes •Full B2B cycle for Assay data exchange •Computational Services  Predictive Mineral Discovery – connectivity to Australian Partnership for Advanced Computing Grid for HPC resources. (in progress)  Earthbyte – service chaining (upcoming talk)  Seismic simulation  Mantle convection service (in progress) The Solid Earth and Environment Grid The Sample Application… Service Interactions User Workflow... Login Edit Problem Run Description Simulation WS-Security, SAML, Shibboleth… Authorisation, Authentication & Audit Resource Registry Service Registry Job Monitor Archive Search Local Repository Common Component Gridsphere, JSR 168 Portlets… Job Globus, WS-RF… ebRIM, WSDL, UDDI… Management Portal ebRIM, OGC Catalog Service… Data Management Portal Results Archive SRB, ebRIM… Job Snark Profile, GPML, XMML… OGC WFS, XMML, Service Profiles… Management Service Observational Databases EarthBytes Profile, GPML, XMML… Geology Rock Prop. W.A W.A Geology Rock Prop. S.A N.S.W The Solid Earth and Environment Grid EarthBytes Service Snark Service HPC Repository Conclusion “…The Solid Earth and Environment Grid community has been established…” SEE Grid Roadmap Geochemistry WFSand online eXploration Mining 228 Registered in 3 geological Markup surveys Language (XMML) Users • Gplates ML 500-700 unique visitors per Spatial Interoperability • CoalML month Demonstrator• Project Geochemistry Profile APAC Grid for• AssayOver Data1000 visits per month GeosciencesExchange Continued steady growth OGC WFS Australian-UK Open Geospatial Six sub-web communities collaboration for the SocialConsortium Change Online (OGC) Exploitation of Grid and Web Map Composer Geospatial Standards. Commission for ….Metech – Acquire Information Geoscience Data Model Collaboration Fractal Technologies – FracSIS SuiteResources Natural 2005 Geoserver Marine Globus GridSystems Middleware…. Earth The Solid Earth and Environment Grid CSIRO Exploration and Mining Name Robert Woodcock Title Principal Software Engineer Phone +61 8 6436 8780 Email Robert.Woodcock@csiro.au Web www.seegrid.csiro.au Thank You Contact CSIRO Phone 1300 363 400 +61 3 9545 2176 Email enquiries@csiro.au Web www.csiro.au www.csiro.au