SYSTEM TESTING AND DEPLOYMENT Lecture Seven (Chapter 7, Notes; Chapter 8, Textbook) Chapter 7: System Testing and Deployment System Testing & Deployment after Knowledge Codification Intelligence gathering Capture Tools Programs, books, articles, experts KNOWLEDGE CAPTURE (Creation) Shells, tables, tools, frames maps, rules KNOWLEDGE CODIFICATION TESTING AND DEPLOYMENT DATABASES Explicit Knowledge KNOWLEDGE SHARING KNOWLEDGE TRANSFER KNOWLEDGE BASE 7-2 Chapter 7: System Testing and Deployment Stages of KMSLC Evaluate Existing Infrastructure Form the KM Team Knowledge Capture Iterative Rapid Prototyping Design KM Blueprint Verify and validate the KM System Implement the KM System Manage Change and Rewards Structure Post-system evaluation 7-3 Chapter 7: System Testing and Deployment Quality Assurance  KM system should meet expectations of members represented in the KM team.  Performance depend highly on quality of explicit/tacit knowledge stored in KB.  For expert, quality relates to reasoning process that produces reliable and accurate solutions.  For user, quality relates to system’s ease of use and efficiency.  For knowledge developer, quality relates to how valid the knowledge sources are and how well they are codified into KB. 7-4 Chapter 7: System Testing and Deployment KMS Testing  To control quality, performance and efficiency of both KB and KMS.  Types of Testing  Logical Testing, make sure that the system produces correct results.  User Acceptance Testing, follows logical testing and check system’s behaviour in a realistic environment. 7-5 Chapter 7: System Testing and Deployment Hurdles in KMS Testing  Subjective nature of tacit knowledge.  Lack of reliable specifications to guide testing  How to decide whether knowledge is complete  Human errors  Lack of time  User interface un-friendly 7-6 Chapter 7: System Testing and Deployment Logical Testing Approaches  Knowledge base formation  Structure of knowledge as it relates to circular or redundant errors  Consistency, correctness, and completeness of knowledge base rules  Knowledge base functionality  Confidence  Reliability of knowledge base 7-7 Chapter 7: System Testing and Deployment Logical Errors  Circular errors - contradictory in meaning or logic  Redundancy errors - offer different approaches to the same problem  Unusable knowledge - comes up no matter conditions succeed or fail  Subsumption errors - if one rule is true, one knows the second rule is always true  Inconsistent knowledge - the same inputs yield different results 7-8 Chapter 7: System Testing and Deployment User Acceptance Testing Approach  Select a person or a team for testing  Decide on user acceptance test criteria  Develop a set of test cases unique to the system  Maintain a log on various versions of the tests and test results  Field-test the system 7-9 Chapter 7: System Testing and Deployment Requirements for Test Team  Independent of the design or codification of the system  Understand the knowledge problem and computer technology  Well versed in the organization’s goals  Know how the system being tested will fit into the designated area of operation 7-10 Chapter 7: System Testing and Deployment Content of Test Plan  Items of KM system that require testing  Schedule of user acceptance testing  Description of test methods that are approved by user and company  Guidelines for documenting test results 7-11 Chapter 7: System Testing and Deployment User Acceptance Criteria  Accuracy and correctness of outcome for test items  Adequacy of the solutions  Adaptability to novel input  Ease of use  Face validity or credibility  Robustness  Technical/operational test 7-12 Chapter 7: System Testing and Deployment User Acceptance Test Techniques  Face  Test validation Cases  Sub-systems validation 7-13 Chapter 7: System Testing and Deployment Factors affecting System Deployment Technical Organizational Procedural Behavioural Political Economical 7-14 Chapter 7: System Testing and Deployment Steps in KMS Deployment  Transfer of KM System from knowledge developer to the organization’s operating unit.  Transfer of KM’s skills from knowledge developer to the organization’s operators. 7-15 Chapter 7: System Testing and Deployment Successful KM System Deployment  User has prior experience with computer applications  User has been involved in the building of the KM system  Benefits of the KM system is immediate and measurable  First impression  Management support the system from start  Adequate user training 7-16 Chapter 7: System Testing and Deployment Factors affecting User Training  Primary  The user’s knowledge of knowledge-based system  Complexity of KM system and user interface’s ease of use  Secondary  Trainer’s technical experience/ communication skills  Environment of training venue 7-17 Chapter 7: System Testing and Deployment Pre User Training Session  Define how the KM system agrees with organizational mission  Demonstrate how the system can support organizational goals  Allocate adequate resources for the training project  Advocate positive effects of the system  Explain cost-benefit analysis of the KM system 7-18 Chapter 7: System Testing and Deployment Overcome Resistances to Change  User attitude survey  Communication sessions  Role negotiation 7-19 Chapter 7: System Testing and Deployment Post-Implementation Review  How KM System has changed accuracy/timeliness of decision making  How KM System has affected attitude of end users  Whether the system has caused constructive organizational changes?  Whether the system has benefited the operation cost? 7-20 Chapter 7: System Testing and Deployment Security Issues  The new system should provide password / protocol protection.  Security procedures should be consistently observed.  Access should be restricted regarding update of the KB. 7-21 Chapter 7: System Testing and Deployment Circular Errors  Rule 1:  IF John and Rob are co-owners of company A THEN John and Rob own the same company  Rule 2:  IF John and Rob own the same company THEN John and Rob are co-owners 7-22 Chapter 7: System Testing and Deployment Redundancy Errors  Rule  IF 1: (Salary > 50,000 OR HOME > 100,000) THEN Loan = ‘yes’  Rule  IF 2: (Salary > 60,000) THEN Loan = ‘yes’ 7-23 Chapter 7: System Testing and Deployment Unusable Knowledge  IF humidity is 5% AND heavy_rain = ‘yes’ THEN some actions 7-24 Chapter 7: System Testing and Deployment Subsumption Errors  Rule  IF A AND B, THEN D  Rule  IF 1: 2: A AND B AND C, THEN D 7-25