T17 – Computer-Assisted Diagnostics Prof. Dr. Thomas M. Deserno, né Lehmann Department of Medical Informatics RWTH Aachen University, Aachen, Germany Thomas M. Deserno Computer-Aided Diagnosis Motivation  Feature extraction techniques    Why do Haralick’s texture features work?  Do I need shape descriptors in image-based diagnosis?  Take a supervised or unsupervised classifier?  How to generate reliable ground truth?  Can I compare algorithms without ground truth? Thomas M. Deserno Computer-Aided Diagnosis T17 – 2  Benign moles  Malignant moles Texture-based features Shape-based features Ground truth Example: STAPLE Example  Idefix Thomas M. Deserno Computer-Aided Diagnosis T17 – 3 Motivation  How can I chose appropriate texture analysis approaches? Evaluation   What is “texture” and can it be measured?  Problem: How to Differ?    T17 – 1 Overview  Can you answer? Thomas M. Deserno Computer-Aided Diagnosis T17 – 4 Example Textures Goal  Gray scale textures  Color textures Brodatz image imaging image processing and analysis VisTex, MIT diagnosis patient Thomas M. Deserno Computer-Aided Diagnosis T17 – 5 Thomas M. Deserno Computer-Aided Diagnosis T17 – 6 Medical Textures  Bone structure   Skin melanoma  Photography      Mamma calcification  Cell structures    Mammography Computer-Aided Diagnosis T17 – 7       Co-occurrence matrices (CM) Haralick’s features Markov Random Fields  Function of displacement vector d Without spatial correspondence pixel 1 255 200 Fourier analysis Gabor analysis pixel 1 pixel 2 Computer-Aided Diagnosis   50 T17 – 9 Thomas M. Deserno  Computer-Aided Diagnosis T17 – 10 Idea      Haralick RM, Shanmugam K, Dinstein I Textural Features for Image Classification IEEE Trans System Man Cybern 1973; 3(6): 610-21 Further developments   T17 – 11 Capture texture properties from CM Contrast Coarseness Initial publication  Computer-Aided Diagnosis 50 100 150 200 255 Haralick‘s Texture Features b: [(i,j),(i+1,j)] c: [(i,j),(i,j+1)] d: [(i,j),(i+2,j-2)] e: [(i,j),(i+2,j+1)] Thomas M. Deserno pixel 2 0 Displacement vector  100 0 Co-Occurrence Matrix  150 displacement d Fractal dimension Box count Thomas M. Deserno T17 – 8 Co-occurrence matrix (CM) Fractal (self similarity)   Computer-Aided Diagnosis Signal-Theoretical (deterministic signal)   Thomas M. Deserno representation ambiguous 2nd Order Statistics Statistical (random process)  unique description  Mathematical Texture Analysis  Coarseness Contrast Regularity Direction (rotation (in-)variance) Resolution (macro- / micro-texture)  No Microscopy Thomas M. Deserno  Visually distinguishable patterns Radiography Histology   Texture Properties De-correlation of measures Reduction of measure number Thomas M. Deserno Computer-Aided Diagnosis T17 – 12 Haralick‘s Texture Features Markov Random Fields  h1: Angular Second Moment  h7: Sum Variance  h2: Contrast  h8: Sum Entropy  h3: Correlation  h9: Entropy  h4: Sum of Squares (variance)  h10: Difference Variance  h5: Inverse Different Moment  h11: Difference Entropy  h6: Sum Average  h12 - h14: Information Measures of Correlation   Statistically predict local gray scales  Left: 1  Right: P(1) = 58% P(2) = 42% Depends on the neighborhood system  Clique Christine Hivernat & Xavier Descombes Thomas M. Deserno Computer-Aided Diagnosis T17 – 13 Thomas M. Deserno Fourier Analysis    F (u, v)  Spatial domain Fourier domain Gabor transform     Windowed Foiurier transform Gaussian window Color texture analysis Drawback  T17 – 14 Gabor Analysis f (m, n) Example Computer-Aided Diagnosis H Averaging the spatial domain real S imaginary V original Thomas M. Deserno Computer-Aided Diagnosis T17 – 15 Gabor Color Texture Analysis H  real  Gabor T17 – 16   V No. of self similar objects to cover the original object Box count method   spectrum Self similarity & scale invariance Infinite detail at every position Definition imaginary complex representation spectrum Computer-Aided Diagnosis Mandelbrot set  S decomp. complex representation Fractal Dimension  original decomp. Thomas M. Deserno Use boxes of side length R  Rn Sub-divide boxes into (2m)n smaller boxes Gabor W. Beyer Thomas M. Deserno Computer-Aided Diagnosis T17 – 17 Thomas M. Deserno Computer-Aided Diagnosis T17 – 18 Example: Osteoporosis Detection Shape  Fractal dimension  “filling factor”  Bones modeled as cylinders, range between        Hollow  “Osteoporosis”, no trabeculae,  = 1.0 Normal,   1.7 to 1.8 Solid  “Osteopetrosis”,  = 2.0    Algorithm     Segment bone from CT Select region of interest Compute  Edge-based Perimeter Chain code Curvature scale space Fourier descriptor Region-based analysis    Bounding box Best fitting ellipse Solidity http://www.rad.washington.edu/exhibits/fractal.html Thomas M. Deserno Computer-Aided Diagnosis T17 – 19 Thomas M. Deserno Perimeter     Direction to next boundary pixel Neighborhood definition 7 0 1 6 pos 2 5 4 3  Contour represented by curvature  Shape descriptor  Perimeter  T17 – 20 Curvature Chain code  Computer-Aided Diagnosis Length of chain code  Location of zero crossings of curvature relative to starting point Comparison of shapes by cyclic shifting of starting points Example  Chain code <Contour Len=“11“ StartX=“1“ StartY=“1“> 22344657071 <\Contour>  Perimeter = 11 Thomas M. Deserno Computer-Aided Diagnosis T17 – 21 Curvature Scale Space (CSS)    T17 – 22  Example Smooth contour Gaussian kernel Iteration    Computer-Aided Diagnosis CSS Algorithm  Thomas M. Deserno Repeat until all inflection points vanished Record positions of inflection point image Convergence  Elliptical shape Thomas M. Deserno inflections Computer-Aided Diagnosis T17 – 23 Thomas M. Deserno Computer-Aided Diagnosis T17 – 24 Example: Cilia-driven Mixing  Fourier Descriptors CSS smoothing in each state of progression  Fourier representation of contour    Only main components      Thomas M. Deserno Computer-Aided Diagnosis    Thomas M. Deserno    Standardized photographs       Thomas M. Deserno Computer-Aided Diagnosis T17 – 27   Thomas M. Deserno  Center of gravity Principal axis Area    Solidity  1  solid object Solidity < 1  irregular object or holes contained Definitions area convex area convexity  T17 – 29 solidity=1 convex perimeter perimeter compactness  Computer-Aided Diagnosis T17 – 28 Measure of densitiy solidity  Thomas M. Deserno Computer-Aided Diagnosis Solidity Ellipse equals object in  68 female, 86 male skulls Significant difference (Wilcoxon test, p = 0.0043) Logistic regression model significant (p = 0.0033) Model quality poor (r2 = 5.89%) S alone does not solve the classification problem Only 63% correct classifications male Best Fitting Ellipse  Fourier descriptors Hierarchical smoothing (Gaussians) Convexity index S Analysis  female T17 – 26 Processing  Female: smooth Male: more corner like Data  Computer-Aided Diagnosis Example: Fourier Descriptors Hypothesis: shape of orbit differs in gender  S(0) = ? S(0), S(1) = ? S(0), S(1), S(2), = ? ... Similar to CCS T17 – 25 Example: Fourier Descriptors s(t), (x,y)t=1, (x,y)t=2, ... S(f) = F {s(t)} Thomas M. Deserno solidity=0.592 4  area perimeter 2 Computer-Aided Diagnosis T17 – 30 Example Compactness  Evaluation of Image Analysis L-929 fibroblast in contact with ethanol (toxin)  Example: Segmentation  Rank approaches   0%    5%  10% Thomas M. Deserno  Computer-Aided Diagnosis  Wenzel A, Hintze H: Editorial Review:        Computer-Aided Diagnosis  Computer-Aided Diagnosis Robert M. Haralick The choice of gold standard at SPIE MI 2000: for evaluation tests for caries diagnosis. sometimes it‘s not even [Dentomaxillofaccopper, Radiolit‘s 1999; 28:132-136] plastic! A robust gold standard is a method that    Is itself precise, i.e. reproducible Reflects the patho-anatomical appearance of the disease Is established independently of the diagnostic method under evaluation Example   T17 – 33 Caries diagnostic based on radiography exams Confirmed by histological analysis Thomas M. Deserno Computer-Aided Diagnosis T17 – 34 Example: Manual References Manual references Thomas M. Deserno T17 – 32 Wenzel A, Hintze H: Editorial Review:   Generation of Ground Truth   Caries diagnostic based on radiography exams Confirmed by histological analysis Thomas M. Deserno Computer-Aided Diagnosis Definition: Ground Truth / Gold Standard Is itself precise, i.e. reproducible Reflects the patho-anatomical appearance of the disease Is established independently of the diagnostic method under evaluation Example  Thomas M. Deserno The choice of gold standard for evaluation tests for caries diagnosis. [Dentomaxillofac Radiol 1999; 28:132-136] A robust gold standard is a method that Which one is best? Which one is correct? Ground truth needed T17 – 31 Definition: Ground Truth / Gold Standard Ultrasound partitioning  T17 – 35 Inter-observer mean Thomas M. Deserno Computer-Aided Diagnosis T17 – 36 Evaluation Without Ground Truth  Recap: Medical Image Processing     Evaluation Without Ground Truth  Exactness & robustness required Variety & variability Uncertainty   Which curve is better? Computer-Aided Diagnosis Usage   T17 – 37 STAPLE  Establish ground truth Quality metric for comparison Thomas M. Deserno Computer-Aided Diagnosis T17 – 38 Example: Identification of Dental Fixtures Example  Type            http://www.crl.med.harvard.edu Thomas M. Deserno Computer-Aided Diagnosis T17 – 39 Image Processing Scheme Thomas M. Deserno  digitization optimization APA Ceram Bonefit Branemark Frialit TPS screw ... Computer-Aided Diagnosis T17 – 40 IDeFix: Model for Feature Extraction acquisition calibration Transdental Transossal Subperiossal Enossal … Device   Compute probabilistic estimate of true segmentation  Expectation maximization  Healthy bone tissue Tumorous tissue Soft tissue Background Thomas M. Deserno Given a set of segmentations  Manual  Automatic Segmenting uncertainty  Simultaneous Truth and Performance Level Estimation (STAPLE) registration transformation Measures d1  Length: l  Diameter: d1, d2, d3 Cross section   Implant: AO  Tap hole: AB filtering AB AO l d2 t  Rotation lock: AL surface . reconstr.. feature extraction compression illumination segmentation archiving  No. of turns: t AL retrieval  Cone impact: c = d1 / d2 Slimness: s = l / (d1+d2+d3) d3 shading classification  display Thomas M. Deserno interpretation measurement Computer-Aided Diagnosis communication T17 – 41  Ground truth from vendor specifications Thomas M. Deserno Computer-Aided Diagnosis T17 – 42 IDeFix: Image Processing Chain IDeFix: Evaluation  Collect images  Establish ground truth  Perform experiments   Branemark  How many? Manual class labels Count errors TPS Frialit feature space Thomas M. Deserno Computer-Aided Diagnosis Branemark T17 – 43 Summary  Texture features       Co-occurrence matrices Markov random fields Fourier & Gabor analysis Fractal dimension (box count) Shape features      Evaluation     Example  Perimeter (chain code) Curvature scale space Fourier descriptor Best fitting ellipse Solidity      Thomas M. Deserno Ground truth Gold standard STAPLE algorithm Computer-Aided Diagnosis Acquisition (x-ray) Preprocessing (histogram) Segmentation (thresholding) Feature extraction (shape-based) Classification (kNN) Evaluation T17 – 45 Thomas M. Deserno Computer-Aided Diagnosis T17 – 44