Jekyll2023-07-28T19:15:22+00:00https://pxp9.github.io/feed.xmlThe software engineer oasisProgramming and stuffPepe Márquez RomeroFang, async background processing for Rust2022-08-06T00:00:00+00:002022-08-06T00:00:00+00:00https://pxp9.github.io/rust/async-processing<p>Even though the first stable version of Rust was released in 2015, there are still some holes in its ecosystem for solving common tasks. One of which is background processing.</p>
<p>In software engineering background processing is a common approach for solving several problems:</p>
<ul>
<li>Carry out periodic tasks. For example, deliver notifications, update cached values.</li>
<li>Defer expensive work so your application stays responsive while performing calculations in the background</li>
</ul>
<p>Most programming languages have go-to background processing frameworks/libraries. For example:</p>
<ul>
<li>Ruby - <a href="https://github.com/mperham/sidekiq">sidekiq</a>. It uses Redis as a job queue.</li>
<li>Python - <a href="https://github.com/Bogdanp/dramatiq">dramatiq</a>. It uses RabbitMQ as a job queue.</li>
<li>Elixir - <a href="https://github.com/sorentwo/oban">oban</a>. It uses a Postgres DB as a job queue.</li>
</ul>
<p>The async programming (async/await) can be used for background processing but it has several major disadvantages if used directly:</p>
<ul>
<li>It doesn’t give control of the number of tasks that are being executed at any given time. So a lot of spawned tasks can overload a thread/threads that they’re started on.</li>
<li>It doesn’t provide any monitoring which can be useful to investigate your system and find bottlenecks</li>
<li>Tasks are not persistent. So all enqueued tasks are lost on every application restart</li>
</ul>
<p>To solve these shortcomings of the async programming we implemented the async processing in <a href="https://github.com/ayrat555/fang">the fang library</a>.</p>
<h2 id="threaded-fang">Threaded Fang</h2>
<p>Fang is a background processing library for rust. The first version of Fang was released exactly one year ago. Its key features were:</p>
<ul>
<li>Each worker is started in a separate thread</li>
<li>A Postgres table is used as the task queue</li>
</ul>
<p>This implementation was written for a specific use case - <a href="https://github.com/ayrat555/el_monitorro">el monitorro bot</a>. This specific implementation of background processing was proved by time. Each day it processes more and more feeds every minute (the current number is more than 3000). Some users host the bot on their infrastructure.</p>
<p>You can find out more about the threaded processing in fang in <a href="https://www.badykov.com/rust/fang/">this blog post</a>.</p>
<h2 id="async-fang">Async Fang</h2>
<blockquote>
<p>
Async provides significantly reduced CPU and memory overhead, especially for workloads with a large amount of IO-bound tasks, such as servers and databases. All else equal, you can have orders of magnitude more tasks than OS threads, because an async runtime uses a small amount of (expensive) threads to handle a large amount of (cheap) tasks
</p>
<footer><cite title="Async book">From the Rust's Async book</cite></footer>
</blockquote>
<p>For some lightweight background tasks, it’s cheaper to run them on the same thread using async instead of starting one thread per worker. That’s why we implemented this kind of processing in fang. Its key features:</p>
<ul>
<li>Each worker is started as a tokio task</li>
<li>If any worker fails during task execution, it’s restarted</li>
<li>Tasks are saved to a Postgres database. Instead of diesel, <a href="https://github.com/sfackler/rust-postgres">tokio-postgres</a> is used to interact with a db. The threaded processing uses the <a href="https://github.com/diesel-rs/diesel">diesel</a> ORM which blocks the thread.</li>
<li>The implementation is based on traits so it’s easy to implement additional backends (redis, in-memory) to store tasks.</li>
</ul>
<h2 id="usage">Usage</h2>
<p>The usage is straightforward:</p>
<ol>
<li>Define a serializable task by adding <code class="language-plaintext highlighter-rouge">serde</code> derives to a task struct.</li>
<li>Implement <code class="language-plaintext highlighter-rouge">AsyncRunnable</code> runnable trait for fang to be able to run it.</li>
<li>Start workers.</li>
<li>Enqueue tasks.</li>
</ol>
<p>Let’s go over each step.</p>
<h3 id="define-a-job">Define a job</h3>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">use</span> <span class="nn">fang</span><span class="p">::</span><span class="nn">serde</span><span class="p">::{</span><span class="n">Deserialize</span><span class="p">,</span> <span class="n">Serialize</span><span class="p">};</span>
<span class="nd">#[derive(Serialize,</span> <span class="nd">Deserialize)]</span>
<span class="nd">#[serde(crate</span> <span class="nd">=</span> <span class="s">"fang::serde"</span><span class="nd">)]</span>
<span class="k">pub</span> <span class="k">struct</span> <span class="n">MyTask</span> <span class="p">{</span>
<span class="k">pub</span> <span class="n">number</span><span class="p">:</span> <span class="nb">u16</span><span class="p">,</span>
<span class="p">}</span>
<span class="k">impl</span> <span class="n">MyTask</span> <span class="p">{</span>
<span class="k">pub</span> <span class="k">fn</span> <span class="nf">new</span><span class="p">(</span><span class="n">number</span><span class="p">:</span> <span class="nb">u16</span><span class="p">)</span> <span class="k">-></span> <span class="n">Self</span> <span class="p">{</span>
<span class="n">Self</span> <span class="p">{</span> <span class="n">number</span> <span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Fang re-exports <code class="language-plaintext highlighter-rouge">serde</code> so it’s not required to add it to the <code class="language-plaintext highlighter-rouge">Cargo.toml</code> file</p>
<h3 id="implement-the-asyncrunnable-trait">Implement the AsyncRunnable trait</h3>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">use</span> <span class="nn">fang</span><span class="p">::</span><span class="n">async_trait</span><span class="p">;</span>
<span class="k">use</span> <span class="nn">fang</span><span class="p">::</span><span class="n">typetag</span><span class="p">;</span>
<span class="k">use</span> <span class="nn">fang</span><span class="p">::</span><span class="n">AsyncRunnable</span><span class="p">;</span>
<span class="k">use</span> <span class="nn">std</span><span class="p">::</span><span class="nn">time</span><span class="p">::</span><span class="n">Duration</span><span class="p">;</span>
<span class="nd">#[async_trait]</span>
<span class="nd">#[typetag::serde]</span>
<span class="k">impl</span> <span class="n">AsyncRunnable</span> <span class="k">for</span> <span class="n">MyTask</span> <span class="p">{</span>
<span class="k">async</span> <span class="k">fn</span> <span class="nf">run</span><span class="p">(</span><span class="o">&</span><span class="k">self</span><span class="p">,</span> <span class="n">queue</span><span class="p">:</span> <span class="o">&</span><span class="k">mut</span> <span class="n">dyn</span> <span class="n">AsyncQueueable</span><span class="p">)</span> <span class="k">-></span> <span class="n">Result</span><span class="o"><</span><span class="p">(),</span> <span class="n">Error</span><span class="o">></span> <span class="p">{</span>
<span class="k">let</span> <span class="n">new_task</span> <span class="o">=</span> <span class="nn">MyTask</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="k">self</span><span class="py">.number</span> <span class="o">+</span> <span class="mi">1</span><span class="p">);</span>
<span class="n">queue</span>
<span class="nf">.insert_task</span><span class="p">(</span><span class="o">&</span><span class="n">new_task</span> <span class="k">as</span> <span class="o">&</span><span class="n">dyn</span> <span class="n">AsyncRunnable</span><span class="p">)</span>
<span class="k">.await</span>
<span class="nf">.unwrap</span><span class="p">();</span>
<span class="k">log</span><span class="p">::</span><span class="nd">info!</span><span class="p">(</span><span class="s">"the current number is {}"</span><span class="p">,</span> <span class="k">self</span><span class="py">.number</span><span class="p">);</span>
<span class="nn">tokio</span><span class="p">::</span><span class="nn">time</span><span class="p">::</span><span class="nf">sleep</span><span class="p">(</span><span class="nn">Duration</span><span class="p">::</span><span class="nf">from_secs</span><span class="p">(</span><span class="mi">3</span><span class="p">))</span><span class="k">.await</span><span class="p">;</span>
<span class="nf">Ok</span><span class="p">(())</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<ul>
<li>Fang uses the <a href="https://github.com/dtolnay/typetag">typetag library</a> to serialize trait objects and save them to the queue.</li>
<li>The <a href="https://github.com/dtolnay/async-trait">async-trait</a> is used for implementing async traits</li>
</ul>
<h3 id="init-queue">Init queue</h3>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">use</span> <span class="nn">fang</span><span class="p">::</span><span class="nn">asynk</span><span class="p">::</span><span class="nn">async_queue</span><span class="p">::</span><span class="n">AsyncQueue</span><span class="p">;</span>
<span class="k">let</span> <span class="n">max_pool_size</span><span class="p">:</span> <span class="nb">u32</span> <span class="o">=</span> <span class="mi">2</span><span class="p">;</span>
<span class="k">let</span> <span class="k">mut</span> <span class="n">queue</span> <span class="o">=</span> <span class="nn">AsyncQueue</span><span class="p">::</span><span class="nf">builder</span><span class="p">()</span>
<span class="nf">.uri</span><span class="p">(</span><span class="s">"postgres://postgres:postgres@localhost/fang"</span><span class="p">)</span>
<span class="nf">.max_pool_size</span><span class="p">(</span><span class="n">max_pool_size</span><span class="p">)</span>
<span class="nf">.duplicated_tasks</span><span class="p">(</span><span class="k">true</span><span class="p">)</span>
<span class="nf">.build</span><span class="p">();</span>
</code></pre></div></div>
<h3 id="start-workers">Start workers</h3>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">use</span> <span class="nn">fang</span><span class="p">::</span><span class="nn">asynk</span><span class="p">::</span><span class="nn">async_worker_pool</span><span class="p">::</span><span class="n">AsyncWorkerPool</span><span class="p">;</span>
<span class="k">use</span> <span class="nn">fang</span><span class="p">::</span><span class="n">NoTls</span><span class="p">;</span>
<span class="k">let</span> <span class="k">mut</span> <span class="n">pool</span><span class="p">:</span> <span class="n">AsyncWorkerPool</span><span class="o"><</span><span class="n">AsyncQueue</span><span class="o"><</span><span class="n">NoTls</span><span class="o">>></span> <span class="o">=</span> <span class="nn">AsyncWorkerPool</span><span class="p">::</span><span class="nf">builder</span><span class="p">()</span>
<span class="nf">.number_of_workers</span><span class="p">(</span><span class="mi">10_u32</span><span class="p">)</span>
<span class="nf">.queue</span><span class="p">(</span><span class="n">queue</span><span class="nf">.clone</span><span class="p">())</span>
<span class="nf">.build</span><span class="p">();</span>
<span class="n">pool</span><span class="nf">.start</span><span class="p">()</span><span class="k">.await</span><span class="p">;</span>
</code></pre></div></div>
<h3 id="insert-tasks">Insert tasks</h3>
<div class="language-rust highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">let</span> <span class="n">task</span> <span class="o">=</span> <span class="nn">MyTask</span><span class="p">::</span><span class="nf">new</span><span class="p">(</span><span class="mi">0</span><span class="p">);</span>
<span class="n">queue</span>
<span class="nf">.insert_task</span><span class="p">(</span><span class="o">&</span><span class="n">task1</span> <span class="k">as</span> <span class="o">&</span><span class="n">dyn</span> <span class="n">AsyncRunnable</span><span class="p">)</span>
<span class="k">.await</span>
<span class="nf">.unwrap</span><span class="p">();</span>
</code></pre></div></div>
<h2 id="pitfalls">Pitfalls</h2>
<p>The async processing is suitable for lightweight tasks. But for heavier tasks it’s advised to use one of the following approaches:</p>
<ul>
<li>start a separate tokio runtime to run fang workers</li>
<li>use the threaded processing feature implemented in fang instead of the async processing</li>
</ul>
<h2 id="future-directions">Future directions</h2>
<p>There are a couple of features planned for fang:</p>
<ul>
<li>Retries with different backoff modes</li>
<li>Additional backends (in-memory, redis)</li>
<li>Graceful shutdown for async workers (for the threaded processing this feature is implemented)</li>
<li>Cron jobs</li>
</ul>
<h2 id="conclusion">Conclusion</h2>
<p>The project is available on <a href="https://github.com/ayrat555/fang">GitHub</a></p>
<p>The async feature and this post is written in collaboration between <a href="https://www.badykov.com/">Ayrat Badykov</a> (<a href="https://github.com/ayrat555">github</a>) and <a href="https://pxp9.github.io/">Pepe Márquez Romero</a> (<a href="https://github.com/pxp9">github</a>)</p>Pepe Márquez RomeroEven though the first stable version of Rust was released in 2015, there are still some holes in its ecosystem for solving common tasks. One of which is background processing.