@@ -4018,16 +4018,16 @@ <h1 class="title">Processing StatCan Data</h1>
40184018<h1>Setup</h1>
40194019<section id="parameters" class="level2">
40204020<h2 class="anchored" data-anchor-id="parameters">Parameters</h2>
4021- <div id="8d685586 " class="cell" data-execution_count="1">
4021+ <div id="3d0b5bb8 " class="cell" data-execution_count="1">
40224022<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb1"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="im">from</span> pyprojroot <span class="im">import</span> here</span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
40234023</div>
4024- <div id="f8777108 " class="cell" data-tags="["parameters"]" data-execution_count="2">
4024+ <div id="dbfee5be " class="cell" data-tags="["parameters"]" data-execution_count="2">
40254025<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb2"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb2-1"><a href="#cb2-1" aria-hidden="true" tabindex="-1"></a>LABOUR_DATA_FILE <span class="op">=</span> here() <span class="op">/</span> <span class="st">"data"</span> <span class="op">/</span> <span class="st">"14100355.csv"</span></span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
40264026</div>
40274027</section>
40284028<section id="libraries" class="level2">
40294029<h2 class="anchored" data-anchor-id="libraries">Libraries</h2>
4030- <div id="71b2c36f " class="cell" data-execution_count="3">
4030+ <div id="56200178 " class="cell" data-execution_count="3">
40314031<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb3"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a><span class="im">import</span> polars <span class="im">as</span> pl</span>
40324032<span id="cb3-2"><a href="#cb3-2" aria-hidden="true" tabindex="-1"></a><span class="im">import</span> polars.selectors <span class="im">as</span> cs</span>
40334033<span id="cb3-3"><a href="#cb3-3" aria-hidden="true" tabindex="-1"></a><span class="im">from</span> mizani.bounds <span class="im">import</span> squish</span>
@@ -4046,7 +4046,7 @@ <h2 class="anchored" data-anchor-id="read-data">Read data</h2>
40464046<li>Additional <code>YEAR</code>, <code>MONTH</code>, and <code>DATE_YMD</code> columns extracted from <code>REF_DATE</code></li>
40474047<li>Sorted chronologically by year and month</li>
40484048</ul>
4049- <div id="51eee5b6 " class="cell" data-execution_count="4">
4049+ <div id="628269c4 " class="cell" data-execution_count="4">
40504050<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb4"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb4-1"><a href="#cb4-1" aria-hidden="true" tabindex="-1"></a>labour <span class="op">=</span> read_labourcan(LABOUR_DATA_FILE)</span>
40514051<span id="cb4-2"><a href="#cb4-2" aria-hidden="true" tabindex="-1"></a>labour.glimpse()</span></code></pre></div><button title="Copy to Clipboard" class="code-copy-button"><i class="bi"></i></button></div>
40524052<div class="cell-output cell-output-stdout">
@@ -4088,7 +4088,7 @@ <h2 class="anchored" data-anchor-id="change-per-month">% Change per month</h2>
40884088<li>Age group</li>
40894089</ul>
40904090<p>In the seasonally adjusted dataset, only Industry and Geolocation are provided. The LFC is total employment, the Gender is both, and Age group is all.</p>
4091- <div id="26d5e385 " class="cell" data-execution_count="5">
4091+ <div id="69307e2a " class="cell" data-execution_count="5">
40924092<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb6"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb6-1"><a href="#cb6-1" aria-hidden="true" tabindex="-1"></a>labour_processed <span class="op">=</span> (</span>
40934093<span id="cb6-2"><a href="#cb6-2" aria-hidden="true" tabindex="-1"></a> <span class="co"># if we sort acesnding by time, then lag value is the month before</span></span>
40944094<span id="cb6-3"><a href="#cb6-3" aria-hidden="true" tabindex="-1"></a> labour.sort([<span class="st">"Industry"</span>, <span class="st">"YEAR"</span>, <span class="st">"MONTH"</span>])</span>
@@ -4132,7 +4132,7 @@ <h2 class="anchored" data-anchor-id="signed-centered-rank">Signed Centered Rank<
41324132<p>Now we can compute the <code>signed centered rank</code>.</p>
41334133<p>Define <code>centered_rank_expr</code> function which takes a polars series and returns an expression, meaning it can be used in a polars <code>with_columns</code> call, which is nice because it can take advantage of polars lazy-evaluation optimization.</p>
41344134<p>Below is the definition and a test-case.</p>
4135- <div id="a174a724 " class="cell" data-execution_count="6">
4135+ <div id="60fa8a6a " class="cell" data-execution_count="6">
41364136<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb8"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb8-1"><a href="#cb8-1" aria-hidden="true" tabindex="-1"></a><span class="kw">def</span> centered_rank_expr(col):</span>
41374137<span id="cb8-2"><a href="#cb8-2" aria-hidden="true" tabindex="-1"></a> <span class="co">"""</span></span>
41384138<span id="cb8-3"><a href="#cb8-3" aria-hidden="true" tabindex="-1"></a><span class="co"> - Largest negative value gets rank -1</span></span>
@@ -4233,7 +4233,7 @@ <h2 class="anchored" data-anchor-id="signed-centered-rank">Signed Centered Rank<
42334233</div>
42344234</div>
42354235<p>Looks good, so now we can apply to the data:</p>
4236- <div id="6fefaf9d " class="cell" data-execution_count="7">
4236+ <div id="8a9a4e55 " class="cell" data-execution_count="7">
42374237<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb9"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb9-1"><a href="#cb9-1" aria-hidden="true" tabindex="-1"></a>labour_processed <span class="op">=</span> labour_processed.with_columns(</span>
42384238<span id="cb9-2"><a href="#cb9-2" aria-hidden="true" tabindex="-1"></a> centered_rank_across_industry<span class="op">=</span>centered_rank_expr(pl.col(<span class="st">"PDIFF"</span>)).over(</span>
42394239<span id="cb9-3"><a href="#cb9-3" aria-hidden="true" tabindex="-1"></a> [<span class="st">"YEAR"</span>, <span class="st">"MONTH"</span>]</span>
@@ -4256,7 +4256,7 @@ <h2 class="anchored" data-anchor-id="signed-centered-rank">Signed Centered Rank<
42564256</div>
42574257</div>
42584258<p>Check output visually for 1 year 1 month</p>
4259- <div id="58d4f64e " class="cell" data-execution_count="8">
4259+ <div id="db2f85f8 " class="cell" data-execution_count="8">
42604260<div class="code-copy-outer-scaffold"><div class="sourceCode cell-code" id="cb11"><pre class="sourceCode python code-with-copy"><code class="sourceCode python"><span id="cb11-1"><a href="#cb11-1" aria-hidden="true" tabindex="-1"></a><span class="co"># check 1 year 1 month</span></span>
42614261<span id="cb11-2"><a href="#cb11-2" aria-hidden="true" tabindex="-1"></a>(</span>
42624262<span id="cb11-3"><a href="#cb11-3" aria-hidden="true" tabindex="-1"></a> labour_processed</span>
0 commit comments