Weighted Least Squares
======================


.. _wls_notebook:

`Link to Notebook GitHub <https://github.com/statsmodels/statsmodels/blob/master/examples/notebooks/wls.ipynb>`_

.. raw:: html

   
   <div class="cell border-box-sizing code_cell rendered">
   <div class="input">
   <div class="prompt input_prompt">In&nbsp;[1]:</div>
   <div class="inner_cell">
       <div class="input_area">
   <div class="highlight"><pre><span class="kn">from</span> <span class="nn">__future__</span> <span class="kn">import</span> <span class="n">print_function</span>
   <span class="kn">import</span> <span class="nn">numpy</span> <span class="kn">as</span> <span class="nn">np</span>
   <span class="kn">from</span> <span class="nn">scipy</span> <span class="kn">import</span> <span class="n">stats</span>
   <span class="kn">import</span> <span class="nn">statsmodels.api</span> <span class="kn">as</span> <span class="nn">sm</span>
   <span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="kn">as</span> <span class="nn">plt</span>
   <span class="kn">from</span> <span class="nn">statsmodels.sandbox.regression.predstd</span> <span class="kn">import</span> <span class="n">wls_prediction_std</span>
   <span class="kn">from</span> <span class="nn">statsmodels.iolib.table</span> <span class="kn">import</span> <span class="p">(</span><span class="n">SimpleTable</span><span class="p">,</span> <span class="n">default_txt_fmt</span><span class="p">)</span>
   <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">seed</span><span class="p">(</span><span class="mi">1024</span><span class="p">)</span>
   </pre></div>
   
   </div>
   </div>
   </div>
   
   </div>
   <div class="cell border-box-sizing text_cell rendered">
   <div class="prompt input_prompt">
   </div>
   <div class="inner_cell">
   <div class="text_cell_render border-box-sizing rendered_html">
   <h2 id="wls-estimation">WLS Estimation</h2>
   <h3 id="artificial-data-heteroscedasticity-2-groups">Artificial data: Heteroscedasticity 2 groups</h3>
   <p>Model assumptions:</p>
   <ul>
   <li>Misspecification: true model is quadratic, estimate only linear</li>
   <li>Independent noise/error term</li>
   <li>Two groups for error variance, low and high variance groups</li>
   </ul>
   </div>
   </div>
   </div>
   <div class="cell border-box-sizing code_cell rendered">
   <div class="input">
   <div class="prompt input_prompt">In&nbsp;[2]:</div>
   <div class="inner_cell">
       <div class="input_area">
   <div class="highlight"><pre><span class="n">nsample</span> <span class="o">=</span> <span class="mi">50</span>
   <span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">linspace</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">20</span><span class="p">,</span> <span class="n">nsample</span><span class="p">)</span>
   <span class="n">X</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">column_stack</span><span class="p">((</span><span class="n">x</span><span class="p">,</span> <span class="p">(</span><span class="n">x</span> <span class="o">-</span> <span class="mi">5</span><span class="p">)</span><span class="o">**</span><span class="mi">2</span><span class="p">))</span>
   <span class="n">X</span> <span class="o">=</span> <span class="n">sm</span><span class="o">.</span><span class="n">add_constant</span><span class="p">(</span><span class="n">X</span><span class="p">)</span>
   <span class="n">beta</span> <span class="o">=</span> <span class="p">[</span><span class="mf">5.</span><span class="p">,</span> <span class="mf">0.5</span><span class="p">,</span> <span class="o">-</span><span class="mf">0.01</span><span class="p">]</span>
   <span class="n">sig</span> <span class="o">=</span> <span class="mf">0.5</span>
   <span class="n">w</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">ones</span><span class="p">(</span><span class="n">nsample</span><span class="p">)</span>
   <span class="n">w</span><span class="p">[</span><span class="n">nsample</span> <span class="o">*</span> <span class="mi">6</span><span class="o">/</span><span class="mi">10</span><span class="p">:]</span> <span class="o">=</span> <span class="mi">3</span>
   <span class="n">y_true</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">dot</span><span class="p">(</span><span class="n">X</span><span class="p">,</span> <span class="n">beta</span><span class="p">)</span>
   <span class="n">e</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">normal</span><span class="p">(</span><span class="n">size</span><span class="o">=</span><span class="n">nsample</span><span class="p">)</span>
   <span class="n">y</span> <span class="o">=</span> <span class="n">y_true</span> <span class="o">+</span> <span class="n">sig</span> <span class="o">*</span> <span class="n">w</span> <span class="o">*</span> <span class="n">e</span> 
   <span class="n">X</span> <span class="o">=</span> <span class="n">X</span><span class="p">[:,[</span><span class="mi">0</span><span class="p">,</span><span class="mi">1</span><span class="p">]]</span>
   </pre></div>
   
   </div>
   </div>
   </div>
   
   </div>
   <div class="cell border-box-sizing text_cell rendered">
   <div class="prompt input_prompt">
   </div>
   <div class="inner_cell">
   <div class="text_cell_render border-box-sizing rendered_html">
   <h3 id="wls-knowing-the-true-variance-ratio-of-heteroscedasticity">WLS knowing the true variance ratio of heteroscedasticity</h3>
   </div>
   </div>
   </div>
   <div class="cell border-box-sizing code_cell rendered">
   <div class="input">
   <div class="prompt input_prompt">In&nbsp;[3]:</div>
   <div class="inner_cell">
       <div class="input_area">
   <div class="highlight"><pre><span class="n">mod_wls</span> <span class="o">=</span> <span class="n">sm</span><span class="o">.</span><span class="n">WLS</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">X</span><span class="p">,</span> <span class="n">weights</span><span class="o">=</span><span class="mf">1.</span><span class="o">/</span><span class="n">w</span><span class="p">)</span>
   <span class="n">res_wls</span> <span class="o">=</span> <span class="n">mod_wls</span><span class="o">.</span><span class="n">fit</span><span class="p">()</span>
   <span class="k">print</span><span class="p">(</span><span class="n">res_wls</span><span class="o">.</span><span class="n">summary</span><span class="p">())</span>
   </pre></div>
   
   </div>
   </div>
   </div>
   
   <div class="output_wrapper">
   <div class="output">
   
   
   <div class="output_area"><div class="prompt"></div>
   <div class="output_subarea output_stream output_stdout output_text">
   <pre>
                               WLS Regression Results                            
   ==============================================================================
   Dep. Variable:                      y   R-squared:                       0.910
   Model:                            WLS   Adj. R-squared:                  0.909
   Method:                 Least Squares   F-statistic:                     487.9
   Date:                Wed, 27 Apr 2016   Prob (F-statistic):           8.52e-27
   Time:                        23:33:58   Log-Likelihood:                -57.048
   No. Observations:                  50   AIC:                             118.1
   Df Residuals:                      48   BIC:                             121.9
   Df Model:                           1                                         
   Covariance Type:            nonrobust                                         
   ==============================================================================
                    coef    std err          t      P&gt;|t|      [95.0% Conf. Int.]
   ------------------------------------------------------------------------------
   const          5.2726      0.185     28.488      0.000         4.900     5.645
   x1             0.4379      0.020     22.088      0.000         0.398     0.478
   ==============================================================================
   Omnibus:                        5.040   Durbin-Watson:                   2.242
   Prob(Omnibus):                  0.080   Jarque-Bera (JB):                6.431
   Skew:                           0.024   Prob(JB):                       0.0401
   Kurtosis:                       4.756   Cond. No.                         17.0
   ==============================================================================
   
   Warnings:
   [1] Standard Errors assume that the covariance matrix of the errors is correctly specified.
   
   </pre>
   </div>
   </div>
   
   </div>
   </div>
   
   </div>
   <div class="cell border-box-sizing text_cell rendered">
   <div class="prompt input_prompt">
   </div>
   <div class="inner_cell">
   <div class="text_cell_render border-box-sizing rendered_html">
   <h2 id="ols-vs-wls">OLS vs. WLS</h2>
   <p>Estimate an OLS model for comparison: </p>
   </div>
   </div>
   </div>
   <div class="cell border-box-sizing code_cell rendered">
   <div class="input">
   <div class="prompt input_prompt">In&nbsp;[4]:</div>
   <div class="inner_cell">
       <div class="input_area">
   <div class="highlight"><pre><span class="n">res_ols</span> <span class="o">=</span> <span class="n">sm</span><span class="o">.</span><span class="n">OLS</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">X</span><span class="p">)</span><span class="o">.</span><span class="n">fit</span><span class="p">()</span>
   <span class="k">print</span><span class="p">(</span><span class="n">res_ols</span><span class="o">.</span><span class="n">params</span><span class="p">)</span>
   <span class="k">print</span><span class="p">(</span><span class="n">res_wls</span><span class="o">.</span><span class="n">params</span><span class="p">)</span>
   </pre></div>
   
   </div>
   </div>
   </div>
   
   <div class="output_wrapper">
   <div class="output">
   
   
   <div class="output_area"><div class="prompt"></div>
   <div class="output_subarea output_stream output_stdout output_text">
   <pre>
   [ 5.2426  0.4349]
   [ 5.2726  0.4379]
   
   </pre>
   </div>
   </div>
   
   </div>
   </div>
   
   </div>
   <div class="cell border-box-sizing text_cell rendered">
   <div class="prompt input_prompt">
   </div>
   <div class="inner_cell">
   <div class="text_cell_render border-box-sizing rendered_html">
   <p>Compare the WLS standard errors to  heteroscedasticity corrected OLS standard errors:</p>
   </div>
   </div>
   </div>
   <div class="cell border-box-sizing code_cell rendered">
   <div class="input">
   <div class="prompt input_prompt">In&nbsp;[5]:</div>
   <div class="inner_cell">
       <div class="input_area">
   <div class="highlight"><pre><span class="n">se</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">vstack</span><span class="p">([[</span><span class="n">res_wls</span><span class="o">.</span><span class="n">bse</span><span class="p">],</span> <span class="p">[</span><span class="n">res_ols</span><span class="o">.</span><span class="n">bse</span><span class="p">],</span> <span class="p">[</span><span class="n">res_ols</span><span class="o">.</span><span class="n">HC0_se</span><span class="p">],</span> 
                   <span class="p">[</span><span class="n">res_ols</span><span class="o">.</span><span class="n">HC1_se</span><span class="p">],</span> <span class="p">[</span><span class="n">res_ols</span><span class="o">.</span><span class="n">HC2_se</span><span class="p">],</span> <span class="p">[</span><span class="n">res_ols</span><span class="o">.</span><span class="n">HC3_se</span><span class="p">]])</span>
   <span class="n">se</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">round</span><span class="p">(</span><span class="n">se</span><span class="p">,</span><span class="mi">4</span><span class="p">)</span>
   <span class="n">colnames</span> <span class="o">=</span> <span class="p">[</span><span class="s1">&#39;x1&#39;</span><span class="p">,</span> <span class="s1">&#39;const&#39;</span><span class="p">]</span>
   <span class="n">rownames</span> <span class="o">=</span> <span class="p">[</span><span class="s1">&#39;WLS&#39;</span><span class="p">,</span> <span class="s1">&#39;OLS&#39;</span><span class="p">,</span> <span class="s1">&#39;OLS_HC0&#39;</span><span class="p">,</span> <span class="s1">&#39;OLS_HC1&#39;</span><span class="p">,</span> <span class="s1">&#39;OLS_HC3&#39;</span><span class="p">,</span> <span class="s1">&#39;OLS_HC3&#39;</span><span class="p">]</span>
   <span class="n">tabl</span> <span class="o">=</span> <span class="n">SimpleTable</span><span class="p">(</span><span class="n">se</span><span class="p">,</span> <span class="n">colnames</span><span class="p">,</span> <span class="n">rownames</span><span class="p">,</span> <span class="n">txt_fmt</span><span class="o">=</span><span class="n">default_txt_fmt</span><span class="p">)</span>
   <span class="k">print</span><span class="p">(</span><span class="n">tabl</span><span class="p">)</span>
   </pre></div>
   
   </div>
   </div>
   </div>
   
   <div class="output_wrapper">
   <div class="output">
   
   
   <div class="output_area"><div class="prompt"></div>
   <div class="output_subarea output_stream output_stdout output_text">
   <pre>
   =====================
             x1   const 
   ---------------------
   WLS     0.1851 0.0198
   OLS     0.2707 0.0233
   OLS_HC0 0.194  0.0281
   OLS_HC1 0.198  0.0287
   OLS_HC3 0.2003 0.029 
   OLS_HC3 0.207   0.03 
   ---------------------
   
   </pre>
   </div>
   </div>
   
   </div>
   </div>
   
   </div>
   <div class="cell border-box-sizing text_cell rendered">
   <div class="prompt input_prompt">
   </div>
   <div class="inner_cell">
   <div class="text_cell_render border-box-sizing rendered_html">
   <p>Calculate OLS prediction interval:</p>
   </div>
   </div>
   </div>
   <div class="cell border-box-sizing code_cell rendered">
   <div class="input">
   <div class="prompt input_prompt">In&nbsp;[6]:</div>
   <div class="inner_cell">
       <div class="input_area">
   <div class="highlight"><pre><span class="n">covb</span> <span class="o">=</span> <span class="n">res_ols</span><span class="o">.</span><span class="n">cov_params</span><span class="p">()</span>
   <span class="n">prediction_var</span> <span class="o">=</span> <span class="n">res_ols</span><span class="o">.</span><span class="n">mse_resid</span> <span class="o">+</span> <span class="p">(</span><span class="n">X</span> <span class="o">*</span> <span class="n">np</span><span class="o">.</span><span class="n">dot</span><span class="p">(</span><span class="n">covb</span><span class="p">,</span><span class="n">X</span><span class="o">.</span><span class="n">T</span><span class="p">)</span><span class="o">.</span><span class="n">T</span><span class="p">)</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
   <span class="n">prediction_std</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">sqrt</span><span class="p">(</span><span class="n">prediction_var</span><span class="p">)</span>
   <span class="n">tppf</span> <span class="o">=</span> <span class="n">stats</span><span class="o">.</span><span class="n">t</span><span class="o">.</span><span class="n">ppf</span><span class="p">(</span><span class="mf">0.975</span><span class="p">,</span> <span class="n">res_ols</span><span class="o">.</span><span class="n">df_resid</span><span class="p">)</span>
   </pre></div>
   
   </div>
   </div>
   </div>
   
   </div>
   <div class="cell border-box-sizing code_cell rendered">
   <div class="input">
   <div class="prompt input_prompt">In&nbsp;[7]:</div>
   <div class="inner_cell">
       <div class="input_area">
   <div class="highlight"><pre><span class="n">prstd_ols</span><span class="p">,</span> <span class="n">iv_l_ols</span><span class="p">,</span> <span class="n">iv_u_ols</span> <span class="o">=</span> <span class="n">wls_prediction_std</span><span class="p">(</span><span class="n">res_ols</span><span class="p">)</span>
   </pre></div>
   
   </div>
   </div>
   </div>
   
   </div>
   <div class="cell border-box-sizing text_cell rendered">
   <div class="prompt input_prompt">
   </div>
   <div class="inner_cell">
   <div class="text_cell_render border-box-sizing rendered_html">
   <p>Draw a plot to compare predicted values in WLS and OLS:</p>
   </div>
   </div>
   </div>
   <div class="cell border-box-sizing code_cell rendered">
   <div class="input">
   <div class="prompt input_prompt">In&nbsp;[8]:</div>
   <div class="inner_cell">
       <div class="input_area">
   <div class="highlight"><pre><span class="n">prstd</span><span class="p">,</span> <span class="n">iv_l</span><span class="p">,</span> <span class="n">iv_u</span> <span class="o">=</span> <span class="n">wls_prediction_std</span><span class="p">(</span><span class="n">res_wls</span><span class="p">)</span>
   
   <span class="n">fig</span><span class="p">,</span> <span class="n">ax</span> <span class="o">=</span> <span class="n">plt</span><span class="o">.</span><span class="n">subplots</span><span class="p">(</span><span class="n">figsize</span><span class="o">=</span><span class="p">(</span><span class="mi">8</span><span class="p">,</span><span class="mi">6</span><span class="p">))</span>
   <span class="n">ax</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="s1">&#39;o&#39;</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">&quot;Data&quot;</span><span class="p">)</span>
   <span class="n">ax</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y_true</span><span class="p">,</span> <span class="s1">&#39;b-&#39;</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">&quot;True&quot;</span><span class="p">)</span>
   <span class="c1"># OLS</span>
   <span class="n">ax</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">res_ols</span><span class="o">.</span><span class="n">fittedvalues</span><span class="p">,</span> <span class="s1">&#39;r--&#39;</span><span class="p">)</span>
   <span class="n">ax</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">iv_u_ols</span><span class="p">,</span> <span class="s1">&#39;r--&#39;</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">&quot;OLS&quot;</span><span class="p">)</span>
   <span class="n">ax</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">iv_l_ols</span><span class="p">,</span> <span class="s1">&#39;r--&#39;</span><span class="p">)</span>
   <span class="c1"># WLS</span>
   <span class="n">ax</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">res_wls</span><span class="o">.</span><span class="n">fittedvalues</span><span class="p">,</span> <span class="s1">&#39;g--.&#39;</span><span class="p">)</span>
   <span class="n">ax</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">iv_u</span><span class="p">,</span> <span class="s1">&#39;g--&#39;</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s2">&quot;WLS&quot;</span><span class="p">)</span>
   <span class="n">ax</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">iv_l</span><span class="p">,</span> <span class="s1">&#39;g--&#39;</span><span class="p">)</span>
   <span class="n">ax</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="s2">&quot;best&quot;</span><span class="p">);</span>
   </pre></div>
   
   </div>
   </div>
   </div>
   
   <div class="output_wrapper">
   <div class="output">
   
   
   <div class="output_area"><div class="prompt"></div>
   <div class="output_subarea output_stream output_stdout output_text">
   <pre>
   Error in callback &lt;function post_execute at 0x7fb786441500&gt; (for post_execute):
   
   </pre>
   </div>
   </div>
   
   <div class="output_area"><div class="prompt"></div>
   <div class="output_subarea output_text output_pyerr">
   <pre>
   <span class="ansired">---------------------------------------------------------------------------</span>
   <span class="ansired">RuntimeError</span>                              Traceback (most recent call last)
   <span class="ansigreen">/usr/lib/python2.7/dist-packages/matplotlib/pyplot.pyc</span> in <span class="ansicyan">post_execute</span><span class="ansiblue">()</span>
   <span class="ansigreen">    145</span>             <span class="ansigreen">def</span> post_execute<span class="ansiblue">(</span><span class="ansiblue">)</span><span class="ansiblue">:</span><span class="ansiblue"></span>
   <span class="ansigreen">    146</span>                 <span class="ansigreen">if</span> matplotlib<span class="ansiblue">.</span>is_interactive<span class="ansiblue">(</span><span class="ansiblue">)</span><span class="ansiblue">:</span><span class="ansiblue"></span>
   <span class="ansigreen">--&gt; 147</span><span class="ansired">                     </span>draw_all<span class="ansiblue">(</span><span class="ansiblue">)</span><span class="ansiblue"></span>
   <span class="ansigreen">    148</span> <span class="ansiblue"></span>
   <span class="ansigreen">    149</span>             <span class="ansired"># IPython &gt;= 2</span><span class="ansiblue"></span><span class="ansiblue"></span>
   
   <span class="ansigreen">/usr/lib/python2.7/dist-packages/matplotlib/_pylab_helpers.pyc</span> in <span class="ansicyan">draw_all</span><span class="ansiblue">(cls, force)</span>
   <span class="ansigreen">    148</span>         <span class="ansigreen">for</span> f_mgr <span class="ansigreen">in</span> cls<span class="ansiblue">.</span>get_all_fig_managers<span class="ansiblue">(</span><span class="ansiblue">)</span><span class="ansiblue">:</span><span class="ansiblue"></span>
   <span class="ansigreen">    149</span>             <span class="ansigreen">if</span> force <span class="ansigreen">or</span> f_mgr<span class="ansiblue">.</span>canvas<span class="ansiblue">.</span>figure<span class="ansiblue">.</span>stale<span class="ansiblue">:</span><span class="ansiblue"></span>
   <span class="ansigreen">--&gt; 150</span><span class="ansired">                 </span>f_mgr<span class="ansiblue">.</span>canvas<span class="ansiblue">.</span>draw_idle<span class="ansiblue">(</span><span class="ansiblue">)</span><span class="ansiblue"></span>
   <span class="ansigreen">    151</span> <span class="ansiblue"></span>
   <span class="ansigreen">    152</span> atexit<span class="ansiblue">.</span>register<span class="ansiblue">(</span>Gcf<span class="ansiblue">.</span>destroy_all<span class="ansiblue">)</span><span class="ansiblue"></span>
   
   <span class="ansigreen">/usr/lib/python2.7/dist-packages/matplotlib/backend_bases.pyc</span> in <span class="ansicyan">draw_idle</span><span class="ansiblue">(self, *args, **kwargs)</span>
   <span class="ansigreen">   2024</span>         <span class="ansigreen">if</span> <span class="ansigreen">not</span> self<span class="ansiblue">.</span>_is_idle_drawing<span class="ansiblue">:</span><span class="ansiblue"></span>
   <span class="ansigreen">   2025</span>             <span class="ansigreen">with</span> self<span class="ansiblue">.</span>_idle_draw_cntx<span class="ansiblue">(</span><span class="ansiblue">)</span><span class="ansiblue">:</span><span class="ansiblue"></span>
   <span class="ansigreen">-&gt; 2026</span><span class="ansired">                 </span>self<span class="ansiblue">.</span>draw<span class="ansiblue">(</span><span class="ansiblue">*</span>args<span class="ansiblue">,</span> <span class="ansiblue">**</span>kwargs<span class="ansiblue">)</span><span class="ansiblue"></span>
   <span class="ansigreen">   2027</span> <span class="ansiblue"></span>
   <span class="ansigreen">   2028</span>     <span class="ansigreen">def</span> draw_cursor<span class="ansiblue">(</span>self<span class="ansiblue">,</span> event<span class="ansiblue">)</span><span class="ansiblue">:</span><span class="ansiblue"></span>
   
   <span class="ansigreen">/usr/lib/python2.7/dist-packages/matplotlib/backends/backend_agg.pyc</span> in <span class="ansicyan">draw</span><span class="ansiblue">(self)</span>
   <span class="ansigreen">    472</span> <span class="ansiblue"></span>
   <span class="ansigreen">    473</span>         <span class="ansigreen">try</span><span class="ansiblue">:</span><span class="ansiblue"></span>
   <span class="ansigreen">--&gt; 474</span><span class="ansired">             </span>self<span class="ansiblue">.</span>figure<span class="ansiblue">.</span>draw<span class="ansiblue">(</span>self<span class="ansiblue">.</span>renderer<span class="ansiblue">)</span><span class="ansiblue"></span>
   <span class="ansigreen">    475</span>         <span class="ansigreen">finally</span><span class="ansiblue">:</span><span class="ansiblue"></span>
   <span class="ansigreen">    476</span>             RendererAgg<span class="ansiblue">.</span>lock<span class="ansiblue">.</span>release<span class="ansiblue">(</span><span class="ansiblue">)</span><span class="ansiblue"></span>
   
   <span class="ansigreen">/usr/lib/python2.7/dist-packages/matplotlib/artist.pyc</span> in <span class="ansicyan">draw_wrapper</span><span class="ansiblue">(artist, renderer, *args, **kwargs)</span>
   <span class="ansigreen">     59</span>     <span class="ansigreen">def</span> draw_wrapper<span class="ansiblue">(</span>artist<span class="ansiblue">,</span> renderer<span class="ansiblue">,</span> <span class="ansiblue">*</span>args<span class="ansiblue">,</span> <span class="ansiblue">**</span>kwargs<span class="ansiblue">)</span><span class="ansiblue">:</span><span class="ansiblue"></span>
   <span class="ansigreen">     60</span>         before<span class="ansiblue">(</span>artist<span class="ansiblue">,</span> renderer<span class="ansiblue">)</span><span class="ansiblue"></span>
   <span class="ansigreen">---&gt; 61</span><span class="ansired">         </span>draw<span class="ansiblue">(</span>artist<span class="ansiblue">,</span> renderer<span class="ansiblue">,</span> <span class="ansiblue">*</span>args<span class="ansiblue">,</span> <span class="ansiblue">**</span>kwargs<span class="ansiblue">)</span><span class="ansiblue"></span>
   <span class="ansigreen">     62</span>         after<span class="ansiblue">(</span>artist<span class="ansiblue">,</span> renderer<span class="ansiblue">)</span><span class="ansiblue"></span>
   <span class="ansigreen">     63</span> <span class="ansiblue"></span>
   
   <span class="ansigreen">/usr/lib/python2.7/dist-packages/matplotlib/figure.pyc</span> in <span class="ansicyan">draw</span><span class="ansiblue">(self, renderer)</span>
   <span class="ansigreen">   1157</span>         dsu<span class="ansiblue">.</span>sort<span class="ansiblue">(</span>key<span class="ansiblue">=</span>itemgetter<span class="ansiblue">(</span><span class="ansicyan">0</span><span class="ansiblue">)</span><span class="ansiblue">)</span><span class="ansiblue"></span>
   <span class="ansigreen">   1158</span>         <span class="ansigreen">for</span> zorder<span class="ansiblue">,</span> a<span class="ansiblue">,</span> func<span class="ansiblue">,</span> args <span class="ansigreen">in</span> dsu<span class="ansiblue">:</span><span class="ansiblue"></span>
   <span class="ansigreen">-&gt; 1159</span><span class="ansired">             </span>func<span class="ansiblue">(</span><span class="ansiblue">*</span>args<span class="ansiblue">)</span><span class="ansiblue"></span>
   <span class="ansigreen">   1160</span> <span class="ansiblue"></span>
   <span class="ansigreen">   1161</span>         renderer<span class="ansiblue">.</span>close_group<span class="ansiblue">(</span><span class="ansiblue">&apos;figure&apos;</span><span class="ansiblue">)</span><span class="ansiblue"></span>
   
   <span class="ansigreen">/usr/lib/python2.7/dist-packages/matplotlib/artist.pyc</span> in <span class="ansicyan">draw_wrapper</span><span class="ansiblue">(artist, renderer, *args, **kwargs)</span>
   <span class="ansigreen">     59</span>     <span class="ansigreen">def</span> draw_wrapper<span class="ansiblue">(</span>artist<span class="ansiblue">,</span> renderer<span class="ansiblue">,</span> <span class="ansiblue">*</span>args<span class="ansiblue">,</span> <span class="ansiblue">**</span>kwargs<span class="ansiblue">)</span><span class="ansiblue">:</span><span class="ansiblue"></span>
   <span class="ansigreen">     60</span>         before<span class="ansiblue">(</span>artist<span class="ansiblue">,</span> renderer<span class="ansiblue">)</span><span class="ansiblue"></span>
   <span class="ansigreen">---&gt; 61</span><span class="ansired">         </span>draw<span class="ansiblue">(</span>artist<span class="ansiblue">,</span> renderer<span class="ansiblue">,</span> <span class="ansiblue">*</span>args<span class="ansiblue">,</span> <span class="ansiblue">**</span>kwargs<span class="ansiblue">)</span><span class="ansiblue"></span>
   <span class="ansigreen">     62</span>         after<span class="ansiblue">(</span>artist<span class="ansiblue">,</span> renderer<span class="ansiblue">)</span><span class="ansiblue"></span>
   <span class="ansigreen">     63</span> <span class="ansiblue"></span>
   
   <span class="ansigreen">/usr/lib/python2.7/dist-packages/matplotlib/axes/_base.pyc</span> in <span class="ansicyan">draw</span><span class="ansiblue">(self, renderer, inframe)</span>
   <span class="ansigreen">   2322</span> <span class="ansiblue"></span>
   <span class="ansigreen">   2323</span>         <span class="ansigreen">for</span> zorder<span class="ansiblue">,</span> a <span class="ansigreen">in</span> dsu<span class="ansiblue">:</span><span class="ansiblue"></span>
   <span class="ansigreen">-&gt; 2324</span><span class="ansired">             </span>a<span class="ansiblue">.</span>draw<span class="ansiblue">(</span>renderer<span class="ansiblue">)</span><span class="ansiblue"></span>
   <span class="ansigreen">   2325</span> <span class="ansiblue"></span>
   <span class="ansigreen">   2326</span>         renderer<span class="ansiblue">.</span>close_group<span class="ansiblue">(</span><span class="ansiblue">&apos;axes&apos;</span><span class="ansiblue">)</span><span class="ansiblue"></span>
   
   <span class="ansigreen">/usr/lib/python2.7/dist-packages/matplotlib/artist.pyc</span> in <span class="ansicyan">draw_wrapper</span><span class="ansiblue">(artist, renderer, *args, **kwargs)</span>
   <span class="ansigreen">     59</span>     <span class="ansigreen">def</span> draw_wrapper<span class="ansiblue">(</span>artist<span class="ansiblue">,</span> renderer<span class="ansiblue">,</span> <span class="ansiblue">*</span>args<span class="ansiblue">,</span> <span class="ansiblue">**</span>kwargs<span class="ansiblue">)</span><span class="ansiblue">:</span><span class="ansiblue"></span>
   <span class="ansigreen">     60</span>         before<span class="ansiblue">(</span>artist<span class="ansiblue">,</span> renderer<span class="ansiblue">)</span><span class="ansiblue"></span>
   <span class="ansigreen">---&gt; 61</span><span class="ansired">         </span>draw<span class="ansiblue">(</span>artist<span class="ansiblue">,</span> renderer<span class="ansiblue">,</span> <span class="ansiblue">*</span>args<span class="ansiblue">,</span> <span class="ansiblue">**</span>kwargs<span class="ansiblue">)</span><span class="ansiblue"></span>
   <span class="ansigreen">     62</span>         after<span class="ansiblue">(</span>artist<span class="ansiblue">,</span> renderer<span class="ansiblue">)</span><span class="ansiblue"></span>
   <span class="ansigreen">     63</span> <span class="ansiblue"></span>
   
   <span class="ansigreen">/usr/lib/python2.7/dist-packages/matplotlib/axis.pyc</span> in <span class="ansicyan">draw</span><span class="ansiblue">(self, renderer, *args, **kwargs)</span>
   <span class="ansigreen">   1106</span>         ticks_to_draw <span class="ansiblue">=</span> self<span class="ansiblue">.</span>_update_ticks<span class="ansiblue">(</span>renderer<span class="ansiblue">)</span><span class="ansiblue"></span>
   <span class="ansigreen">   1107</span>         ticklabelBoxes, ticklabelBoxes2 = self._get_tick_bboxes(ticks_to_draw,
   <span class="ansigreen">-&gt; 1108</span><span class="ansired">                                                                 renderer)
   </span><span class="ansigreen">   1109</span> <span class="ansiblue"></span>
   <span class="ansigreen">   1110</span>         <span class="ansigreen">for</span> tick <span class="ansigreen">in</span> ticks_to_draw<span class="ansiblue">:</span><span class="ansiblue"></span>
   
   <span class="ansigreen">/usr/lib/python2.7/dist-packages/matplotlib/axis.pyc</span> in <span class="ansicyan">_get_tick_bboxes</span><span class="ansiblue">(self, ticks, renderer)</span>
   <span class="ansigreen">   1056</span>         <span class="ansigreen">for</span> tick <span class="ansigreen">in</span> ticks<span class="ansiblue">:</span><span class="ansiblue"></span>
   <span class="ansigreen">   1057</span>             <span class="ansigreen">if</span> tick<span class="ansiblue">.</span>label1On <span class="ansigreen">and</span> tick<span class="ansiblue">.</span>label1<span class="ansiblue">.</span>get_visible<span class="ansiblue">(</span><span class="ansiblue">)</span><span class="ansiblue">:</span><span class="ansiblue"></span>
   <span class="ansigreen">-&gt; 1058</span><span class="ansired">                 </span>extent <span class="ansiblue">=</span> tick<span class="ansiblue">.</span>label1<span class="ansiblue">.</span>get_window_extent<span class="ansiblue">(</span>renderer<span class="ansiblue">)</span><span class="ansiblue"></span>
   <span class="ansigreen">   1059</span>                 ticklabelBoxes<span class="ansiblue">.</span>append<span class="ansiblue">(</span>extent<span class="ansiblue">)</span><span class="ansiblue"></span>
   <span class="ansigreen">   1060</span>             <span class="ansigreen">if</span> tick<span class="ansiblue">.</span>label2On <span class="ansigreen">and</span> tick<span class="ansiblue">.</span>label2<span class="ansiblue">.</span>get_visible<span class="ansiblue">(</span><span class="ansiblue">)</span><span class="ansiblue">:</span><span class="ansiblue"></span>
   
   <span class="ansigreen">/usr/lib/python2.7/dist-packages/matplotlib/text.pyc</span> in <span class="ansicyan">get_window_extent</span><span class="ansiblue">(self, renderer, dpi)</span>
   <span class="ansigreen">    959</span>             <span class="ansigreen">raise</span> RuntimeError<span class="ansiblue">(</span><span class="ansiblue">&apos;Cannot get window extent w/o renderer&apos;</span><span class="ansiblue">)</span><span class="ansiblue"></span>
   <span class="ansigreen">    960</span> <span class="ansiblue"></span>
   <span class="ansigreen">--&gt; 961</span><span class="ansired">         </span>bbox<span class="ansiblue">,</span> info<span class="ansiblue">,</span> descent <span class="ansiblue">=</span> self<span class="ansiblue">.</span>_get_layout<span class="ansiblue">(</span>self<span class="ansiblue">.</span>_renderer<span class="ansiblue">)</span><span class="ansiblue"></span>
   <span class="ansigreen">    962</span>         x<span class="ansiblue">,</span> y <span class="ansiblue">=</span> self<span class="ansiblue">.</span>get_unitless_position<span class="ansiblue">(</span><span class="ansiblue">)</span><span class="ansiblue"></span>
   <span class="ansigreen">    963</span>         x<span class="ansiblue">,</span> y <span class="ansiblue">=</span> self<span class="ansiblue">.</span>get_transform<span class="ansiblue">(</span><span class="ansiblue">)</span><span class="ansiblue">.</span>transform_point<span class="ansiblue">(</span><span class="ansiblue">(</span>x<span class="ansiblue">,</span> y<span class="ansiblue">)</span><span class="ansiblue">)</span><span class="ansiblue"></span>
   
   <span class="ansigreen">/usr/lib/python2.7/dist-packages/matplotlib/text.pyc</span> in <span class="ansicyan">_get_layout</span><span class="ansiblue">(self, renderer)</span>
   <span class="ansigreen">    350</span>         tmp, lp_h, lp_bl = renderer.get_text_width_height_descent(&apos;lp&apos;,
   <span class="ansigreen">    351</span>                                                          self<span class="ansiblue">.</span>_fontproperties<span class="ansiblue">,</span><span class="ansiblue"></span>
   <span class="ansigreen">--&gt; 352</span><span class="ansired">                                                          ismath=False)
   </span><span class="ansigreen">    353</span>         offsety <span class="ansiblue">=</span> <span class="ansiblue">(</span>lp_h <span class="ansiblue">-</span> lp_bl<span class="ansiblue">)</span> <span class="ansiblue">*</span> self<span class="ansiblue">.</span>_linespacing<span class="ansiblue"></span>
   <span class="ansigreen">    354</span> <span class="ansiblue"></span>
   
   <span class="ansigreen">/usr/lib/python2.7/dist-packages/matplotlib/backends/backend_agg.pyc</span> in <span class="ansicyan">get_text_width_height_descent</span><span class="ansiblue">(self, s, prop, ismath)</span>
   <span class="ansigreen">    227</span>             fontsize <span class="ansiblue">=</span> prop<span class="ansiblue">.</span>get_size_in_points<span class="ansiblue">(</span><span class="ansiblue">)</span><span class="ansiblue"></span>
   <span class="ansigreen">    228</span>             w, h, d = texmanager.get_text_width_height_descent(s, fontsize,
   <span class="ansigreen">--&gt; 229</span><span class="ansired">                                                                renderer=self)
   </span><span class="ansigreen">    230</span>             <span class="ansigreen">return</span> w<span class="ansiblue">,</span> h<span class="ansiblue">,</span> d<span class="ansiblue"></span>
   <span class="ansigreen">    231</span> <span class="ansiblue"></span>
   
   <span class="ansigreen">/usr/lib/python2.7/dist-packages/matplotlib/texmanager.pyc</span> in <span class="ansicyan">get_text_width_height_descent</span><span class="ansiblue">(self, tex, fontsize, renderer)</span>
   <span class="ansigreen">    673</span>         <span class="ansigreen">else</span><span class="ansiblue">:</span><span class="ansiblue"></span>
   <span class="ansigreen">    674</span>             <span class="ansired"># use dviread. It sometimes returns a wrong descent.</span><span class="ansiblue"></span><span class="ansiblue"></span>
   <span class="ansigreen">--&gt; 675</span><span class="ansired">             </span>dvifile <span class="ansiblue">=</span> self<span class="ansiblue">.</span>make_dvi<span class="ansiblue">(</span>tex<span class="ansiblue">,</span> fontsize<span class="ansiblue">)</span><span class="ansiblue"></span>
   <span class="ansigreen">    676</span>             dvi <span class="ansiblue">=</span> dviread<span class="ansiblue">.</span>Dvi<span class="ansiblue">(</span>dvifile<span class="ansiblue">,</span> <span class="ansicyan">72</span> <span class="ansiblue">*</span> dpi_fraction<span class="ansiblue">)</span><span class="ansiblue"></span>
   <span class="ansigreen">    677</span>             <span class="ansigreen">try</span><span class="ansiblue">:</span><span class="ansiblue"></span>
   
   <span class="ansigreen">/usr/lib/python2.7/dist-packages/matplotlib/texmanager.pyc</span> in <span class="ansicyan">make_dvi</span><span class="ansiblue">(self, tex, fontsize)</span>
   <span class="ansigreen">    420</span>                      <span class="ansiblue">&apos;string:\n%s\nHere is the full report generated by &apos;</span><span class="ansiblue"></span>
   <span class="ansigreen">    421</span>                      <span class="ansiblue">&apos;LaTeX: \n\n&apos;</span> <span class="ansiblue">%</span> repr<span class="ansiblue">(</span>tex<span class="ansiblue">.</span>encode<span class="ansiblue">(</span><span class="ansiblue">&apos;unicode_escape&apos;</span><span class="ansiblue">)</span><span class="ansiblue">)</span> <span class="ansiblue">+</span><span class="ansiblue"></span>
   <span class="ansigreen">--&gt; 422</span><span class="ansired">                      report))
   </span><span class="ansigreen">    423</span>             <span class="ansigreen">else</span><span class="ansiblue">:</span><span class="ansiblue"></span>
   <span class="ansigreen">    424</span>                 mpl<span class="ansiblue">.</span>verbose<span class="ansiblue">.</span>report<span class="ansiblue">(</span>report<span class="ansiblue">,</span> <span class="ansiblue">&apos;debug&apos;</span><span class="ansiblue">)</span><span class="ansiblue"></span>
   
   <span class="ansired">RuntimeError</span>: LaTeX was not able to process the following string:
   &apos;lp&apos;
   Here is the full report generated by LaTeX: 
   
   </pre>
   </div>
   </div>
   
   <div class="output_area"><div class="prompt"></div>
   
   
   <div class="output_text output_subarea ">
   <pre>
   &lt;matplotlib.figure.Figure at 0x7fb77374ad10&gt;
   </pre>
   </div>
   
   </div>
   
   </div>
   </div>
   
   </div>
   <div class="cell border-box-sizing text_cell rendered">
   <div class="prompt input_prompt">
   </div>
   <div class="inner_cell">
   <div class="text_cell_render border-box-sizing rendered_html">
   <h2 id="feasible-weighted-least-squares-2-stage-fwls-">Feasible Weighted Least Squares (2-stage FWLS)</h2>
   </div>
   </div>
   </div>
   <div class="cell border-box-sizing code_cell rendered">
   <div class="input">
   <div class="prompt input_prompt">In&nbsp;[9]:</div>
   <div class="inner_cell">
       <div class="input_area">
   <div class="highlight"><pre><span class="n">resid1</span> <span class="o">=</span> <span class="n">res_ols</span><span class="o">.</span><span class="n">resid</span><span class="p">[</span><span class="n">w</span><span class="o">==</span><span class="mf">1.</span><span class="p">]</span>
   <span class="n">var1</span> <span class="o">=</span> <span class="n">resid1</span><span class="o">.</span><span class="n">var</span><span class="p">(</span><span class="n">ddof</span><span class="o">=</span><span class="nb">int</span><span class="p">(</span><span class="n">res_ols</span><span class="o">.</span><span class="n">df_model</span><span class="p">)</span><span class="o">+</span><span class="mi">1</span><span class="p">)</span>
   <span class="n">resid2</span> <span class="o">=</span> <span class="n">res_ols</span><span class="o">.</span><span class="n">resid</span><span class="p">[</span><span class="n">w</span><span class="o">!=</span><span class="mf">1.</span><span class="p">]</span>
   <span class="n">var2</span> <span class="o">=</span> <span class="n">resid2</span><span class="o">.</span><span class="n">var</span><span class="p">(</span><span class="n">ddof</span><span class="o">=</span><span class="nb">int</span><span class="p">(</span><span class="n">res_ols</span><span class="o">.</span><span class="n">df_model</span><span class="p">)</span><span class="o">+</span><span class="mi">1</span><span class="p">)</span>
   <span class="n">w_est</span> <span class="o">=</span> <span class="n">w</span><span class="o">.</span><span class="n">copy</span><span class="p">()</span>
   <span class="n">w_est</span><span class="p">[</span><span class="n">w</span><span class="o">!=</span><span class="mf">1.</span><span class="p">]</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">sqrt</span><span class="p">(</span><span class="n">var2</span><span class="p">)</span> <span class="o">/</span> <span class="n">np</span><span class="o">.</span><span class="n">sqrt</span><span class="p">(</span><span class="n">var1</span><span class="p">)</span>
   <span class="n">res_fwls</span> <span class="o">=</span> <span class="n">sm</span><span class="o">.</span><span class="n">WLS</span><span class="p">(</span><span class="n">y</span><span class="p">,</span> <span class="n">X</span><span class="p">,</span> <span class="mf">1.</span><span class="o">/</span><span class="n">w_est</span><span class="p">)</span><span class="o">.</span><span class="n">fit</span><span class="p">()</span>
   <span class="k">print</span><span class="p">(</span><span class="n">res_fwls</span><span class="o">.</span><span class="n">summary</span><span class="p">())</span>
   </pre></div>
   
   </div>
   </div>
   </div>
   
   <div class="output_wrapper">
   <div class="output">
   
   
   <div class="output_area"><div class="prompt"></div>
   <div class="output_subarea output_stream output_stdout output_text">
   <pre>
                               WLS Regression Results                            
   ==============================================================================
   Dep. Variable:                      y   R-squared:                       0.914
   Model:                            WLS   Adj. R-squared:                  0.912
   Method:                 Least Squares   F-statistic:                     507.1
   Date:                Wed, 27 Apr 2016   Prob (F-statistic):           3.65e-27
   Time:                        23:34:00   Log-Likelihood:                -55.777
   No. Observations:                  50   AIC:                             115.6
   Df Residuals:                      48   BIC:                             119.4
   Df Model:                           1                                         
   Covariance Type:            nonrobust                                         
   ==============================================================================
                    coef    std err          t      P&gt;|t|      [95.0% Conf. Int.]
   ------------------------------------------------------------------------------
   const          5.2710      0.177     29.828      0.000         4.916     5.626
   x1             0.4390      0.019     22.520      0.000         0.400     0.478
   ==============================================================================
   Omnibus:                        4.076   Durbin-Watson:                   2.251
   Prob(Omnibus):                  0.130   Jarque-Bera (JB):                4.336
   Skew:                           0.003   Prob(JB):                        0.114
   Kurtosis:                       4.443   Cond. No.                         16.5
   ==============================================================================
   
   Warnings:
   [1] Standard Errors assume that the covariance matrix of the errors is correctly specified.
   
   </pre>
   </div>
   </div>
   
   </div>
   </div>
   
   </div>

   <script src="https://c328740.ssl.cf1.rackcdn.com/mathjax/latest/MathJax.js?config=TeX-AMS_HTML"type="text/javascript"></script>
   <script type="text/javascript">
   init_mathjax = function() {
       if (window.MathJax) {
           // MathJax loaded
           MathJax.Hub.Config({
               tex2jax: {
               // I'm not sure about the \( and \[ below. It messes with the
               // prompt, and I think it's an issue with the template. -SS
                   inlineMath: [ ['$','$'], ["\\(","\\)"] ],
                   displayMath: [ ['$$','$$'], ["\\[","\\]"] ]
               },
               displayAlign: 'left', // Change this to 'center' to center equations.
               "HTML-CSS": {
                   styles: {'.MathJax_Display': {"margin": 0}}
               }
           });
           MathJax.Hub.Queue(["Typeset",MathJax.Hub]);
       }
   }
   init_mathjax();

   // since we have to load this in a ..raw:: directive we will add the css
   // after the fact
   function loadcssfile(filename){
       var fileref=document.createElement("link")
       fileref.setAttribute("rel", "stylesheet")
       fileref.setAttribute("type", "text/css")
       fileref.setAttribute("href", filename)

       document.getElementsByTagName("head")[0].appendChild(fileref)
   }
   // loadcssfile({{pathto("_static/nbviewer.pygments.css", 1) }})
   // loadcssfile({{pathto("_static/nbviewer.min.css", 1) }})
   loadcssfile("../../../_static/nbviewer.pygments.css")
   loadcssfile("../../../_static/ipython.min.css")
   </script>