@@ -4010,16 +4010,16 @@ <h1 class="title">Processing StatCan Data</h1>
40104010<h1>Setup</h1>
40114011<section id="parameters" class="level2">
40124012<h2 class="anchored" data-anchor-id="parameters">Parameters</h2>
4013- <div id="df6ba651 " class="cell" data-execution_count="1">
4013+ <div id="049424e0 " class="cell" data-execution_count="1">
40144014<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb1"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="im">from</span> pyprojroot <span class="im">import</span> here</span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
40154015</div>
4016- <div id="c119dc1a " class="cell" data-tags="["parameters"]" data-execution_count="2">
4016+ <div id="1bca3f75 " class="cell" data-tags="["parameters"]" data-execution_count="2">
40174017<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb2"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb2-1"><a href="#cb2-1" aria-hidden="true" tabindex="-1"></a>LABOUR_DATA_FILE <span class="op">=</span> here() <span class="op">/</span> <span class="st">"data"</span> <span class="op">/</span> <span class="st">"14100355.csv"</span></span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
40184018</div>
40194019</section>
40204020<section id="libraries" class="level2">
40214021<h2 class="anchored" data-anchor-id="libraries">Libraries</h2>
4022- <div id="e8512537 " class="cell" data-execution_count="3">
4022+ <div id="62f2f395 " class="cell" data-execution_count="3">
40234023<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb3"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a><span class="im">import</span> polars <span class="im">as</span> pl</span>
40244024<span id="cb3-2"><a href="#cb3-2" aria-hidden="true" tabindex="-1"></a><span class="im">import</span> polars.selectors <span class="im">as</span> cs</span>
40254025<span id="cb3-3"><a href="#cb3-3" aria-hidden="true" tabindex="-1"></a><span class="im">from</span> mizani.bounds <span class="im">import</span> squish</span>
@@ -4038,7 +4038,7 @@ <h2 class="anchored" data-anchor-id="read-data">Read data</h2>
40384038<li>Additional <code>YEAR</code>, <code>MONTH</code>, and <code>DATE_YMD</code> columns extracted from <code>REF_DATE</code></li>
40394039<li>Sorted chronologically by year and month</li>
40404040</ul>
4041- <div id="676283c6 " class="cell" data-execution_count="4">
4041+ <div id="e8246c40 " class="cell" data-execution_count="4">
40424042<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb4"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb4-1"><a href="#cb4-1" aria-hidden="true" tabindex="-1"></a>labour <span class="op">=</span> read_labourcan(LABOUR_DATA_FILE)</span>
40434043<span id="cb4-2"><a href="#cb4-2" aria-hidden="true" tabindex="-1"></a>labour.glimpse()</span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
40444044<div class="cell-output cell-output-stdout">
@@ -4080,7 +4080,7 @@ <h2 class="anchored" data-anchor-id="change-per-month">% Change per month</h2>
40804080<li>Age group</li>
40814081</ul>
40824082<p>In the seasonally adjusted dataset, only Industry and Geolocation are provided. The LFC is total employment, the Gender is both, and Age group is all.</p>
4083- <div id="223274e8 " class="cell" data-execution_count="5">
4083+ <div id="97f6773e " class="cell" data-execution_count="5">
40844084<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb6"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb6-1"><a href="#cb6-1" aria-hidden="true" tabindex="-1"></a>labour_processed <span class="op">=</span> (</span>
40854085<span id="cb6-2"><a href="#cb6-2" aria-hidden="true" tabindex="-1"></a> <span class="co"># if we sort acesnding by time, then lag value is the month before</span></span>
40864086<span id="cb6-3"><a href="#cb6-3" aria-hidden="true" tabindex="-1"></a> labour.sort([<span class="st">"Industry"</span>, <span class="st">"YEAR"</span>, <span class="st">"MONTH"</span>])</span>
@@ -4124,7 +4124,7 @@ <h2 class="anchored" data-anchor-id="signed-centered-rank">Signed Centered Rank<
41244124<p>Now we can compute the <code>signed centered rank</code>.</p>
41254125<p>Define <code>centered_rank_expr</code> function which takes a polars series and returns an expression, meaning it can be used in a polars <code>with_columns</code> call, which is nice because it can take advantage of polars lazy-evaluation optimization.</p>
41264126<p>Below is the definition and a test-case.</p>
4127- <div id="7bca1487 " class="cell" data-execution_count="6">
4127+ <div id="953bfd6d " class="cell" data-execution_count="6">
41284128<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb8"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb8-1"><a href="#cb8-1" aria-hidden="true" tabindex="-1"></a><span class="kw">def</span> centered_rank_expr(col):</span>
41294129<span id="cb8-2"><a href="#cb8-2" aria-hidden="true" tabindex="-1"></a> <span class="co">"""</span></span>
41304130<span id="cb8-3"><a href="#cb8-3" aria-hidden="true" tabindex="-1"></a><span class="co"> - Largest negative value gets rank -1</span></span>
@@ -4225,7 +4225,7 @@ <h2 class="anchored" data-anchor-id="signed-centered-rank">Signed Centered Rank<
42254225</div>
42264226</div>
42274227<p>Looks good, so now we can apply to the data:</p>
4228- <div id="d1d899af " class="cell" data-execution_count="7">
4228+ <div id="d3d3bcab " class="cell" data-execution_count="7">
42294229<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb9"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb9-1"><a href="#cb9-1" aria-hidden="true" tabindex="-1"></a>labour_processed <span class="op">=</span> labour_processed.with_columns(</span>
42304230<span id="cb9-2"><a href="#cb9-2" aria-hidden="true" tabindex="-1"></a> centered_rank_across_industry<span class="op">=</span>centered_rank_expr(pl.col(<span class="st">"PDIFF"</span>)).over(</span>
42314231<span id="cb9-3"><a href="#cb9-3" aria-hidden="true" tabindex="-1"></a> [<span class="st">"YEAR"</span>, <span class="st">"MONTH"</span>]</span>
@@ -4248,7 +4248,7 @@ <h2 class="anchored" data-anchor-id="signed-centered-rank">Signed Centered Rank<
42484248</div>
42494249</div>
42504250<p>Check output visually for 1 year 1 month</p>
4251- <div id="2e2a479c " class="cell" data-execution_count="8">
4251+ <div id="e6c84ee2 " class="cell" data-execution_count="8">
42524252<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb11"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb11-1"><a href="#cb11-1" aria-hidden="true" tabindex="-1"></a><span class="co"># check 1 year 1 month</span></span>
42534253<span id="cb11-2"><a href="#cb11-2" aria-hidden="true" tabindex="-1"></a>(</span>
42544254<span id="cb11-3"><a href="#cb11-3" aria-hidden="true" tabindex="-1"></a> labour_processed</span>
0 commit comments