How to apply the simulation extrapolation method in errors-in-variables models? Silvelyn Zwanzig, Uppsala 20@math.uu.se Sunday August 29, 2004 1 OU T LIN E • The main trick • Variations of the method • What can we reach in the best cases? • What did we reach in linear EIV? • Advantages of SIMEX ? • Extension to nonlinear models Cooperation with Jörg Polzehl WIAS, Berlin 2 T HE M AIN IDEA • Set up Y = (y1, ..., yn)T yi ∼ Pθ,i independently , θ ∈ Θ V ar (yi) = σ 2 • Wanted: θ ∈ Θ • Start with an ''unpleasant'' estimator b θ = arg min Q (Y, θ) θ∈Θ • ''unpleasant'' because ¡ ¢ Eb θ = θ + BIAS σ 2 + rest but BIAS (.) ↓ f or σ 2 ↓ 3 AIM • Simulate new observations with smaller variances. BUT • It is impossible to generate new observations with smaller variances. • We can simulate observations with a larger error by adding a pseudo error δ ∗i √ ∗ yi(λ) = yi + λ δ i such that V ar (yi(λ)) = σ 2 + λ • OBS! λ ≥ 0 • DREAM: λ = −σ 2 4 T HE P ROCEDU RE (1) For λ1, ..., λK positive fixed simulate p = yi + λk ε∗ik , i = 1, .., n Y (λk ) = (y1∗ (λk ) , ..., yn (λk ))T yi∗ (λk ) ∗ (2) Calculate the estimators b θ(k) b θ(k) = arg min Q (Y ∗ (λk ) , θ) θ∈Θ (3) Fit a parametric model to b θ(1), .., b θ(K) K ° °2 X ° °b fb = arg min °θ(k) − f (λk )° f ∈F k=1 (4) Extrapolate backwards b θSIM EX = fb(−σ 2) 5 V ARIAN T S (1) Simulation step different design for λ1, ..., λK B repeated simulations for each λk Add the pseudo errors on all observations? (2) Calculate the estimators b θ(k) Which ''unpleasant'' estimators give a good start? (3) Model fit Which model? local linear nonlinear model F,if BIAS(.) ∈ F (4) Extrapolation step variance unknown determine λ∗ : b θSIM EX = fb(λ∗) 6 ORIGIN AL SIM EX EIV-Model: yi = ξ Ti β + σ y εi xi = ξ i + σ xδ i Cook & Stefanski (1994) bnaive. • start with the naive estimator β • Consider each component of the estimator separately. • BIAS is a function of σ 2x. • Adding pseudo errors on the related component of xi only. • Fit a parametric curve bb(.) for the component of the naive estimator, © a ª BIAS (λ) ∈ b+λ , a, b > 0 . • Assume that σ 2x is known and bj,SIM EX := bb(−σ 2 ), j = 1, .., p. define β x 7 SY M EX − symmetrized SIM EX J. Polzehl and S.20 (2003) • Use the implicit model formulation Z = ζ + ε, αT ζ = 0, α = (−1, β 1, ..., β p)T • Start with p + 1 different naive estimators (l) α b naive, l = 0, ..., p with respect to the p + 1 different explicit models. • Carry out the first steps SIMEX type procedure for each l. Add pseudo errors to Z(−l) , l = 0, ..., p. • Fit the model¡ ¢−1 (l) (l) θ, αl (λ, θ) = 1 α(−l) (λ, θ) = M(−l)M(−l) + λI θ p−dimensional The model is linear in θ and nonlinear in λ. All components of the estimator are fitted simultaneously. 8 • Retransform to the parametrization of the explicit models ³ ´ (l) 1 (l) b (λ) = ³ ´ α(−0) λ, b θ , l = 0, ..., p. β (l) α0 λ, b θ • Extrapolation step °2 XX° (m) °b(l) ∗ b (λ)° λ = arg min °β (λ) − β ° λ l m • Define the SYMEX estimator by X (l) 1 b (λ∗) bSY M EX = β β p+1 l For known σ 2x we have ¢ (0) ¡ 2 b b β SIM EX = β −σ x 9 ADAP T IV E SIM EX J. Polzehl and S.20 (2003) bnaive = • Consider only one naive estimator β (0) b β naive • Simulation step same as in SYMEX for l = 0 • Model fit same as in SYMEX for l = 0 • Extrapolation step: Determine the variance estimator by σ e2x = λmin (MZZ ) and set λ∗∗ = −λmin (MZZ ) • Define (0) b b β ASIM EX = β (λ∗∗) 10 W hat can we reach in the best cases? E = simple EIV-Model:   T T   Y = (y1, ..., yn) , X = (x1, x2..., xn) , yi ∼ N (βξ i, 1), xi ∼ N (ξ i, 1), independent   θ = (β, ξ 1, ..., ξ n) ∈ Θ F = SIMEX experiment:   T T Y = (y1, ..., yn) , X = (x11, x21..., xnK ) ind.         yi ∼ N (βξ i, 1), xik ∼ N (ξ i, 1 + λk ),   Cov (Y ) = I, Cov(X) = In ⊗ Σ, Cov(X, Y ) = 0   T     Σ = 1K 1K + diag(λ1, ..., λK ),     θ = (β, ξ 1, ..., ξ n) ∈ Θ 11 Comparing chances − Comparing experiments The deficiency δ (E, F) describes the loss of information. We say δ (E, F) = 0 iff F is less informative than E. F is equivalent to E iff δ (E, F) = 0 and δ (F, E) = 0. Le Cam (1986): For E = {Pθ , θ ∈ Θ} and F = {Qθ , θ ∈ Θ}the deficiency δ (E, F) is exactly 1 δ (E, F) = inf sup kQθ − KPθ k , K θ 2 where the infimum is taken over the sets of all transitions K . 12 It holds • The original experiment is better than the SIMEX experiment δ (E, F) = 0, because Qθ = KPθ . • The SIMEX experiment is really worse. But asymptotically for K → ∞ they are equivalent. 1 0 < δ (F, E) < const. K 13 SIM EX M ethods − the best what we can reach! σ 2x σ 2y known: For all n and for K → ∞ bSY M EX = β bT LS + oP ∗ (1) β bASIM EX = β bT LS + oP ∗ (1) β • SYMEX and ASIMEX are a methods for calculation of TLS. • SYMEX and ASIMEX are approximately best. • SYMEX and ASIMEX are consistent for fixed σ 2. σ 2x known: For all n and for K → ∞ bSIM EX = β bM M E + oP ∗ (1) . β 14 Why should we use SIMEX methods instead of TLS? • Multicollinarity? • Computational advances? • Is it only a theoretical result for justification of SIMEX methods? 15 ASIM EX f or N ON LIN EAR M odels ∗ T ∗ ) by adding , ..., ynk (1) Simulate Yk∗ = (y1k pseudo errors p ∗ yik = yi + λk ε∗ik (2) Calculate b θ(k) = arg min Q (Yk∗, θ) and θ∈Θ ³ ´ ∗ b Qk = Q Yk , θ(k) (3) Fit a parametric model to b θ(1), .., b θ(K) K ° °2 X ° ° θ(k) − f (λk )° fb = arg min °b f ∈F k=1 (4) Fit a parametric model to Q1, ..., QK K X qb = arg min (Qk − q(λk ))2 q∈Q k=1 (5) Extrapolate the criterium function to zero: qb(λ∗∗) = 0 (6) Define the ASIMEX by : b θASIM EX = fb(λ∗∗). 16 Man kan säga SIMEX är en trick för att uppfylla varje statistikers dröm om fler och exaktere observationer. ?????? 17