Patrick Clokehttps://patrick.cloke.us/2024-02-23T16:01:00-05:00Joining the Matrix Spec Core Team2024-02-23T16:01:00-05:002024-02-23T16:01:00-05:00Patrick Cloketag:patrick.cloke.us,2024-02-23:/posts/2024/02/23/joining-the-matrix-spec-core-team/<p>I was recently invited to join the Matrix “Spec Core Team”, the group who
steward the Matrix protocol, from their <a class="reference external" href="https://matrix.org/about/">own documentation</a>:</p>
<blockquote>
The contents and direction of the Matrix Spec is governed by the Spec Core Team;
a set of experts from across the whole Matrix community, representing all aspects …</blockquote><p>I was recently invited to join the Matrix “Spec Core Team”, the group who
steward the Matrix protocol, from their <a class="reference external" href="https://matrix.org/about/">own documentation</a>:</p>
<blockquote>
The contents and direction of the Matrix Spec is governed by the Spec Core Team;
a set of experts from across the whole Matrix community, representing all aspects
of the Matrix ecosystem. The Spec Core Team acts as a subcommittee of the Foundation.</blockquote>
<p>This was the <a class="reference external" href="https://matrix.org/blog/2024/02/09/this-week-in-matrix-2024-02-09/#dept-of-status-of-matrix-face-with-th">announced a couple of weeks ago</a> and I’m just starting to get my feet
wet! You can see an interview between myself, Tulir (another new member of the Spec
Core Team), and Matthew (the Spec Core Team lead) in today’s <a class="reference external" href="https://matrix.org/blog/2024/02/23/this-week-in-matrix-2024-02-23/">This Week in Matrix</a>.
We cover a range of topics including <a class="reference external" href="https://www.thunderbird.net/">Thunderbird</a> (and <a class="reference external" href="http://instantbird.com/">Instantbird</a>), some
improvements I hope to make and more.</p>
<div class="youtube youtube-16x9"><iframe src="https://www.youtube-nocookie.com/embed/8vjjwxx7k1w" allowfullscreen seamless frameBorder="0"></iframe></div>Synapse URL Previews2024-02-23T15:35:00-05:002024-02-23T15:35:00-05:00Patrick Cloketag:patrick.cloke.us,2024-02-23:/posts/2024/02/23/synapse-url-previews/<p>Matrix includes the ability for a client to request that the server
<a class="reference external" href="https://spec.matrix.org/v1.8/client-server-api/#get_matrixmediav3preview_url">generate a “preview” for a <span class="caps">URL</span></a>. The client provides a <span class="caps">URL</span> to the server which
returns <a class="reference external" href="https://ogp.me/">Open Graph</a> data as a <span class="caps">JSON</span> response. This leaks any URLs detected in
the message content to the server, but protects the …</p><p>Matrix includes the ability for a client to request that the server
<a class="reference external" href="https://spec.matrix.org/v1.8/client-server-api/#get_matrixmediav3preview_url">generate a “preview” for a <span class="caps">URL</span></a>. The client provides a <span class="caps">URL</span> to the server which
returns <a class="reference external" href="https://ogp.me/">Open Graph</a> data as a <span class="caps">JSON</span> response. This leaks any URLs detected in
the message content to the server, but protects the end user’s <span class="caps">IP</span> address, etc.
from the <span class="caps">URL</span> being previewed. <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a> (Note that clients generally disable <span class="caps">URL</span> previews
for encrypted rooms, but it can be enabled.)</p>
<div class="section" id="improvements">
<h2>Improvements</h2>
<p>Synapse implements the <span class="caps">URL</span> preview endpoint, but it was a bit neglected. I was
one of the few main developers running with <span class="caps">URL</span> previews enabled and sunk a bit of
time into improving <span class="caps">URL</span> previews for my on sake. Some highlights of the improvements
made include (in addition to lots and lots of refactoring):</p>
<ul class="simple">
<li>Support <a class="reference external" href="https://oembed.com/">oEmbed</a> for <span class="caps">URL</span> previews:
<a class="reference external" href="https://github.com/matrix-org/synapse/pull/7920">#7920</a>,
<a class="reference external" href="https://github.com/matrix-org/synapse/pull/10714">#10714</a>,
<a class="reference external" href="https://github.com/matrix-org/synapse/pull/10759">#10759</a>,
<a class="reference external" href="https://github.com/matrix-org/synapse/pull/10814">#10814</a>,
<a class="reference external" href="https://github.com/matrix-org/synapse/pull/10819">#10819</a>,
<a class="reference external" href="https://github.com/matrix-org/synapse/pull/10822">#10822</a>,
<a class="reference external" href="https://github.com/matrix-org/synapse/pull/11065">#11065</a>,
<a class="reference external" href="https://github.com/matrix-org/synapse/pull/11669">#11669</a> (combine with <span class="caps">HTML</span> results),
<a class="reference external" href="https://github.com/matrix-org/synapse/pull/14089">#14089</a>,
<a class="reference external" href="https://github.com/matrix-org/synapse/pull/14781">#14781</a>.</li>
<li>Reduction of 500 errors:
<a class="reference external" href="https://github.com/matrix-org/synapse/pull/8883">#8883</a> (empty media),
<a class="reference external" href="https://github.com/matrix-org/synapse/pull/9333">#9333</a> (unable to parse),
<a class="reference external" href="https://github.com/matrix-org/synapse/pull/11061">#11061</a> (oEmbed errors).</li>
<li>Improved support for document encodings:
<a class="reference external" href="https://github.com/matrix-org/synapse/pull/9164">#9164</a>,
<a class="reference external" href="https://github.com/matrix-org/synapse/pull/9333">#9333</a>,
<a class="reference external" href="https://github.com/matrix-org/synapse/pull/11077">#11077</a>,
<a class="reference external" href="https://github.com/matrix-org/synapse/pull/11089">#11089</a>.</li>
<li>Support previewing <span class="caps">XML</span> documents (<a class="reference external" href="https://github.com/matrix-org/synapse/pull/11196">#11196</a>)
and <tt class="docutils literal">data:</tt> URIs (<a class="reference external" href="https://github.com/matrix-org/synapse/pull/11767">#11767</a>).</li>
<li>Return partial information if images or oEmbed can’t be fetched:
<a class="reference external" href="https://github.com/matrix-org/synapse/pull/12950">#12950</a>,
<a class="reference external" href="https://github.com/matrix-org/synapse/pull/15092">#15092</a>.</li>
<li>Skipping empty Open Graph (<tt class="docutils literal">og</tt>) or <tt class="docutils literal">meta</tt> tags:
<a class="reference external" href="https://github.com/matrix-org/synapse/pull/12951">#12951</a>.</li>
<li>Support previewing from <a class="reference external" href="https://developer.twitter.com/en/docs/twitter-for-websites/cards/guides/getting-started">Twitter card information</a>:
<a class="reference external" href="https://github.com/matrix-org/synapse/pull/13056">#13056</a>.</li>
<li>Fallback to favicon if no images found:
<a class="reference external" href="https://github.com/matrix-org/synapse/pull/12951">#12951</a>.</li>
<li>Ignore navgiation tags: <a class="reference external" href="https://github.com/matrix-org/synapse/pull/12951">#12951</a>.</li>
<li>Document how Synapse <a class="reference external" href="https://github.com/matrix-org/synapse/blob/be65a8ec0195955c15fdb179c9158b187638e39a/synapse/media/url_previewer.py#L101-L154">generates <span class="caps">URL</span> previews</a>:
<a class="reference external" href="https://github.com/matrix-org/synapse/pull/10753">#10753</a>,
<a class="reference external" href="https://github.com/matrix-org/synapse/pull/13261">#13261</a>.</li>
</ul>
<p>I also helped review many changes by others:</p>
<ul class="simple">
<li>Improved support for encodings: <a class="reference external" href="https://github.com/matrix-org/synapse/pull/10410">#10410</a>.</li>
<li>Safer content-type support: <a class="reference external" href="https://github.com/matrix-org/synapse/pull/11936">#11936</a>.</li>
<li>Attempts to fix Twitter previews: <a class="reference external" href="https://github.com/matrix-org/synapse/pull/11985">#11985</a>.</li>
<li>Remove useless elements from previews: <a class="reference external" href="https://github.com/matrix-org/synapse/pull/12887">#12887</a>.</li>
<li>Avoid crashes due to unbounded recursion:
<a class="reference external" href="https://github.com/matrix-org/synapse/security/advisories/GHSA-22p3-qrh9-cx32"><span class="caps">GHSA</span>-22p3-qrh9-cx32</a>.</li>
</ul>
<p>And also fixed some security issues:</p>
<ul class="simple">
<li>Apply <tt class="docutils literal">url_preview_url_blacklist</tt> to oEmbed and pre-cached images:
<a class="reference external" href="https://github.com/matrix-org/synapse/pull/15601">#15601</a>.</li>
</ul>
</div>
<div class="section" id="results">
<h2>Results</h2>
<p>Overall, there was an improved result (from my point of view). A summary of some
of the improvements. I tested 26 URLs (based on ones that had previously been
reported or found to give issues). See the table below for testing at a few versions.
The error reason was also broken out into whether JavaScript was required or some
other error occurred. <a class="footnote-reference" href="#footnote-2" id="footnote-reference-2">[2]</a></p>
<table border="1" class="docutils">
<colgroup>
<col width="9%" />
<col width="14%" />
<col width="20%" />
<col width="28%" />
<col width="29%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">Version</th>
<th class="head">Release date</th>
<th class="head">Successful preview</th>
<th class="head">JavaScript required error</th>
<th class="head">Found image <span class="amp">&</span> description?</th>
</tr>
</thead>
<tbody valign="top">
<tr><td>1.0.0</td>
<td>2019-06-11</td>
<td>15</td>
<td>4</td>
<td>14</td>
</tr>
<tr><td>1.12.0</td>
<td>2020-03-23</td>
<td>18</td>
<td>4</td>
<td>17</td>
</tr>
<tr><td>1.24.0</td>
<td>2020-12-09</td>
<td>20</td>
<td>1</td>
<td>16</td>
</tr>
<tr><td>1.36.0</td>
<td>2021-06-15</td>
<td>20</td>
<td>1</td>
<td>16</td>
</tr>
<tr><td>1.48.0</td>
<td>2021-11-30</td>
<td>20</td>
<td>1</td>
<td>11</td>
</tr>
<tr><td>1.60.0</td>
<td>2022-05-31</td>
<td>21</td>
<td>0</td>
<td>21</td>
</tr>
<tr><td>1.72.0</td>
<td>2022-11-22</td>
<td>22</td>
<td>0</td>
<td>21</td>
</tr>
<tr><td>1.84.0</td>
<td>2023-05-23</td>
<td>22</td>
<td>0</td>
<td>21</td>
</tr>
</tbody>
</table>
</div>
<div class="section" id="future-improvements">
<h2>Future improvements</h2>
<p>I am no longer working on Synapse, but some of the ideas I had for additional improvements included:</p>
<ul class="simple">
<li>Use <a class="reference external" href="https://www.crummy.com/software/BeautifulSoup/">BeautifulSoup</a> instead of a custom parser to handle some edge cases in <span class="caps">HTML</span>
documents better (<span class="caps">WIP</span> @ <a class="reference external" href="https://github.com/matrix-org/synapse/tree/clokep/bs4"><tt class="docutils literal">clokep/bs4</tt></a>).</li>
<li>Always request both oEmbed and <span class="caps">HTML</span> (<span class="caps">WIP</span> @ <a class="reference external" href="https://github.com/matrix-org/synapse/tree/clokep/oembed-and-html"><tt class="docutils literal"><span class="pre">clokep/oembed-and-html</span></tt></a>).</li>
<li>Structured data support (<a class="reference external" href="https://json-ld.org/"><span class="caps">JSON</span>-<span class="caps">LD</span></a>, <a class="reference external" href="https://html.spec.whatwg.org/multipage/">Microdata</a>, <a class="reference external" href="https://rdfa.info/">RDFa</a>) (<a class="reference external" href="https://github.com/matrix-org/synapse/issues/11540">#11540</a>).</li>
<li>Some minimal JavaScript support (<a class="reference external" href="https://github.com/matrix-org/synapse/issues/14118">#14118</a>).</li>
<li>Fixing any of the other issues with particular URLs (see this <a class="reference external" href="https://github.com/matrix-org/synapse/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3AA-URL-Preview+">GitHub search</a>).</li>
<li>Thumbnailing of <span class="caps">SVG</span> images (which sites tend to use for favicons) (<a class="reference external" href="https://github.com/matrix-org/synapse/issues/1309">#1309</a>).</li>
</ul>
<p>There’s also a ton more that could be done here if you wanted, e.g. handling more
data types (text and <span class="caps">PDF</span> are the ones I have frequently come across that would be
helpful to preview). I’m sure there are also many other URLs that don’t work right
now for some reason. Hopefully the <span class="caps">URL</span> preview code continues to improve!</p>
<table class="docutils footnote" frame="void" id="footnote-1" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td>See some <a class="reference external" href="https://github.com/matrix-org/matrix-spec/blob/main/attic/drafts/url_previews.md">ancient documentation</a> on the tradeoffs and design of <span class="caps">URL</span> previews.
<a class="reference external" href="https://github.com/matrix-org/matrix-spec-proposals/pull/4095"><span class="caps">MSC4095</span></a> was recently written to bundle the <span class="caps">URL</span> preview information into
evens.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-2" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-2">[2]</a></td><td>This was done by instantiating different Synapse versions via Docker and
asking them to preview URLs. (See <a class="reference external" href="https://github.com/clokep/test-matrix-url-previews/tree/e0e20154ec348fc25d203546ddede0c881b9772a/docker">the code</a>.) This is not a super realistic
test since it assumes that URLs are static over time. In particular some
sites (e.g. Twitter) like to change what they allow you to access without
being authenticated.</td></tr>
</tbody>
</table>
</div>
Matrix Intentional Mentions explained2023-12-15T15:41:00-05:002023-12-15T15:41:00-05:00Patrick Cloketag:patrick.cloke.us,2023-12-15:/posts/2023/12/15/matrix-intentional-mentions-explained/<p>Previously I have written about how <a class="reference external" href="https://patrick.cloke.us/posts/2023/05/08/matrix-push-rules-notifications/">push rules generate notifications</a> and how
<a class="reference external" href="https://patrick.cloke.us/posts/2023/01/05/matrix-read-receipts-and-notifications/">read receipts mark notificiations as read</a> in the Matrix protocol. This article
is about a change that I instigated to improve <em>when</em> a “mention” (or “ping”)
notification is created. (This is a “highlight” notification in the Matrix specification …</p><p>Previously I have written about how <a class="reference external" href="https://patrick.cloke.us/posts/2023/05/08/matrix-push-rules-notifications/">push rules generate notifications</a> and how
<a class="reference external" href="https://patrick.cloke.us/posts/2023/01/05/matrix-read-receipts-and-notifications/">read receipts mark notificiations as read</a> in the Matrix protocol. This article
is about a change that I instigated to improve <em>when</em> a “mention” (or “ping”)
notification is created. (This is a “highlight” notification in the Matrix specification.)</p>
<p>This was part of the work I did at Element to reduce <a class="reference external" href="https://github.com/vector-im/element-meta/issues/886">unintentional pings</a>. I
preferred thinking of it in the positive — that we should only generate a mention
on purpose, hence “intentional” mentions. <a class="reference external" href="https://github.com/matrix-org/matrix-spec-proposals/pull/3952"><span class="caps">MSC3952</span></a> details the technical protocol
changes, but this serves as a bit of a higher-level overview (some of this content
is copied from the <span class="caps">MSC</span>).</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">This blog post assumes that default push rules are enabled, these can be heavily
modified, disabled, etc. but that is ignored in this post.</p>
</div>
<div class="section" id="legacy-mentions">
<h2>Legacy mentions</h2>
<p>The legacy mention system searches for the current user’s display name or the
localpart of the Matrix <span class="caps">ID</span> <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a> in the text content of an event. For example, an
event like the following would generate a mention for me:</p>
<div class="highlight"><pre><span></span><span class="p">{</span>
<span class="w"> </span><span class="c1">// Additional fields ignored.</span>
<span class="w"> </span><span class="s2">"content"</span><span class="o">:</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="s2">"body"</span><span class="o">:</span><span class="w"> </span><span class="s2">"Hello @clokep:matrix.org!"</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</pre></div>
<p>A <tt class="docutils literal">body</tt> content field <a class="footnote-reference" href="#footnote-2" id="footnote-reference-2">[2]</a> containing <tt class="docutils literal">clokep</tt> or <tt class="docutils literal">Patrick Cloke</tt>
would cause a “highlight” notification (displayed as red in Element). This isn’t uncommon
in chat protocols and is how <span class="caps">IRC</span> and <span class="caps">XMPP</span>.</p>
<p>Some of the issues with this are:</p>
<ul class="simple">
<li>Replying to a message will re-issue pings from the initial message due to
<a class="reference external" href="https://spec.matrix.org/v1.5/client-server-api/#fallbacks-for-rich-replies">fallback replies</a>.</li>
<li>Each time a message is edited the new version will be re-evaluated for mentions.</li>
<li>Mentions occurring <a class="reference external" href="https://github.com/matrix-org/matrix-spec/issues/16">in spoiler contents</a> or <a class="reference external" href="https://github.com/matrix-org/matrix-spec/issues/15">code blocks</a> are evaluated.</li>
<li>If the <a class="reference external" href="https://github.com/matrix-org/matrix-spec-proposals/issues/3011">localpart of your Matrix <span class="caps">ID</span> is a common word</a> then spurious notifications
might occur (e.g. Travis <span class="caps">CI</span> matching if your Matrix <span class="caps">ID</span> is <tt class="docutils literal">@travis:example.org</tt>).</li>
<li>If the <a class="reference external" href="https://github.com/matrix-org/matrix-spec-proposals/issues/2735">localpart or display name of your Matrix <span class="caps">ID</span> matches the hostname</a>
(e.g. <tt class="docutils literal">@example:example.org</tt> receives notifications whenever <tt class="docutils literal">@foo:example.org</tt>
is replied to).</li>
</ul>
<p>There were some <a class="reference external" href="https://github.com/matrix-org/matrix-spec-proposals/blob/main/proposals/3952-intentional-mentions.md#prior-proposals">prior attempts</a> to fix this, but I would summarize them as attempting
to reduce edge-cases instead of attempting to rethink how mentions are done.</p>
</div>
<div class="section" id="intentional-mentions">
<h2>Intentional mentions</h2>
<p>I chose to call this “intentional” mentions since the protocol now requires
explicitly referring to the Matrix IDs to mention in a dedicated field, instead
of implicit references in the text content.</p>
<p>The overall change is simple: include a list of mentioned users in a new
content field, e.g.:</p>
<div class="highlight"><pre><span></span><span class="p">{</span>
<span class="w"> </span><span class="c1">// Additional fields ignored.</span>
<span class="w"> </span><span class="s2">"content"</span><span class="o">:</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="s2">"body"</span><span class="o">:</span><span class="w"> </span><span class="s2">"Hello @clokep:matrix.org!"</span>
<span class="w"> </span><span class="s2">"m.mentions"</span><span class="o">:</span><span class="w"> </span><span class="p">{</span>
<span class="w"> </span><span class="s2">"user_ids"</span><span class="o">:</span><span class="w"> </span><span class="p">[</span><span class="s2">"@clokep:matrix.org"</span><span class="p">]</span>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="p">}</span>
<span class="p">}</span>
</pre></div>
<p>Only the <tt class="docutils literal">m.mentions</tt> field is used to generate mentions, the <tt class="docutils literal">body</tt> field is
no longer involved. Not only does this remove a whole class of potential bugs,
but also allows for “hidden” mentions and paves the way for mentions in extensible
events (see <a class="reference external" href="https://github.com/matrix-org/matrix-spec-proposals/pull/4053"><span class="caps">MSC4053</span></a>).</p>
<p>That’s the gist of the change, although the <span class="caps">MSC</span> goes deeper into backwards
compatibility, and interacting with replies or edits.</p>
</div>
<div class="section" id="comparison-to-other-protocols">
<h2>Comparison to other protocols</h2>
<p>The <tt class="docutils literal">m.mentions</tt> field is similar to how <a class="reference external" href="https://developer.twitter.com/en/docs/twitter-api/data-dictionary/object-model/tweet">Twitter</a>, <a class="reference external" href="https://docs.joinmastodon.org/entities/Status/#Mention">Mastodon</a>, <a class="reference external" href="https://discord.com/developers/docs/resources/channel#message-object">Discord</a>,
and <a class="reference external" href="https://learn.microsoft.com/en-us/graph/api/resources/chatmessagemention?view=graph-rest-1.0">Microsoft Teams</a> handle mentioning users. The main downside of this approach
is that it is not obvious <em>where</em> in the text the user’s mention is (and allows
for hidden mentions).</p>
<p>The other seriously considered approach was searching for “pills” in the <span class="caps">HTML</span>
content of the event. This is similar to how <a class="reference external" href="https://api.slack.com/reference/surfaces/formatting#mentioning-users">Slack</a> handles mentions, where the
user <span class="caps">ID</span> is encoded with some markup <a class="footnote-reference" href="#footnote-3" id="footnote-reference-3">[3]</a>. This has a major downside of requiring <span class="caps">HTML</span>
parsing on a hotpath of processing notifications (and it is unclear how this would
work for non-<span class="caps">HTML</span> clients).</p>
</div>
<div class="section" id="can-i-use-this">
<h2>Can I use this?</h2>
<p>You can! The <span class="caps">MSC</span> was approved and <a class="reference external" href="https://spec.matrix.org/v1.9/client-server-api/#user-and-room-mentions">included in Matrix 1.7</a>, Synapase has had
support since v1.86.0; it is pretty much up to clients to implement it!</p>
<p>Element Web has handled (and sent intentional mentions) since v1.11.37, although
I’m not aware of other clients which do (Element X might now). Hopefully it will
become used throughout the ecosystem since many of the above issues are still
common complaints I see with Matrix.</p>
<table class="docutils footnote" frame="void" id="footnote-1" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td>This post ignores room-mentions, but they’re handled very similarly.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-2" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-2">[2]</a></td><td>Note that the plaintext content of the event is searched <em>not</em> the “formatted”
content (which is <a class="reference external" href="https://spec.matrix.org/v1.9/client-server-api/#mroommessage-msgtypes">usually <span class="caps">HTML</span></a>).</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-3" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-3">[3]</a></td><td>This solution should also reduce the number of unintentional mentions, but
doesn’t allow for hidden mentions.</td></tr>
</tbody>
</table>
</div>
Matrix Presence2023-12-15T11:24:00-05:002023-12-15T11:24:00-05:00Patrick Cloketag:patrick.cloke.us,2023-12-15:/posts/2023/12/15/matrix-presence/<p>I put together some notes on presence when implementing <a class="reference external" href="https://github.com/matrix-org/synapse/pull/16066">multi-device support for presence</a>
in Synapse, maybe this is helpful to others! This is a combination of information
from the specification, as well as some information about how Synapse works.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">These notes are true as of the v1.9 of …</p></div><p>I put together some notes on presence when implementing <a class="reference external" href="https://github.com/matrix-org/synapse/pull/16066">multi-device support for presence</a>
in Synapse, maybe this is helpful to others! This is a combination of information
from the specification, as well as some information about how Synapse works.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">These notes are true as of the v1.9 of the Matrix spec and also cover some
Matrix spec changes which may or may not have been merged since.</p>
</div>
<div class="section" id="presence-in-matrix">
<h2>Presence in Matrix</h2>
<p>Matrix includes basic presence support, which is explained decently from <a class="reference external" href="https://spec.matrix.org/v1.7/client-server-api/#presence">the specification</a>:</p>
<blockquote>
<p>Each user has the concept of presence information. This encodes:</p>
<ul class="simple">
<li>Whether the user is currently online</li>
<li>How recently the user was last active (as seen by the server)</li>
<li>Whether a given client considers the user to be currently idle</li>
<li>Arbitrary information about the user’s current status (e.g. “in a meeting”).</li>
</ul>
<p>This information is collated from both per-device (<tt class="docutils literal">online</tt>, <tt class="docutils literal">idle</tt>, <tt class="docutils literal">last_active</tt>)
and per-user (status) data, aggregated by the user’s homeserver and transmitted
as an <tt class="docutils literal">m.presence</tt> event. Presence events are sent to interested parties where
users share a room membership.</p>
<p>User’s presence state is represented by the presence key, which is an enum of
one of the following:</p>
<ul class="simple">
<li><tt class="docutils literal">online</tt> : The default state when the user is connected to an event stream.</li>
<li><tt class="docutils literal">unavailable</tt> : The user is not reachable at this time e.g. they are idle. <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a></li>
<li><tt class="docutils literal">offline</tt> : The user is not connected to an event stream or is explicitly suppressing their profile information from being sent.</li>
</ul>
</blockquote>
<p><a class="reference external" href="https://github.com/matrix-org/matrix-spec-proposals/pull/3026"><span class="caps">MSC3026</span></a> defines a <tt class="docutils literal">busy</tt> presence state:</p>
<blockquote>
the user is online and active but is performing an activity that would prevent
them from giving their full attention to an external solicitation, i.e. the user
is online and active but not available.</blockquote>
<p>Presence information is returned to clients in the <tt class="docutils literal">presence</tt> key of the
<a class="reference external" href="https://spec.matrix.org/v1.7/client-server-api/#_matrixclientv3sync_presence">sync response</a> as a <tt class="docutils literal">m.presence</tt> <span class="caps">EDU</span> which contains:</p>
<ul class="simple">
<li><tt class="docutils literal">currently_active</tt>: Whether the user is currently active (boolean)</li>
<li><tt class="docutils literal">last_active_ago</tt>: The last time since this used performed some action, in milliseconds.</li>
<li><tt class="docutils literal">presence</tt>: <tt class="docutils literal">online</tt>, <tt class="docutils literal">unavailable</tt>, or <tt class="docutils literal">offline</tt> (or <tt class="docutils literal">busy</tt>)</li>
<li><tt class="docutils literal">status_msg</tt>: An optional description to accompany the presence.</li>
</ul>
<div class="section" id="updating-presence">
<h3>Updating presence</h3>
<p>Clients can call <a class="reference external" href="https://spec.matrix.org/v1.8/client-server-api/#put_matrixclientv3presenceuseridstatus"><tt class="docutils literal"><span class="caps">PUT</span> <span class="pre">/_matrix/client/v3/presence/{userId}/status</span></tt></a> to update the presence state <span class="amp">&</span> status message or
can set the presence state via <a class="reference external" href="https://spec.matrix.org/v1.8/client-server-api/#get_matrixclientv3sync">the <tt class="docutils literal">set_presence</tt> parameter on <tt class="docutils literal">/sync</tt> request</a>.</p>
<p>Note that when using the <tt class="docutils literal">set_presence</tt> parameter, <tt class="docutils literal">offline</tt> is equivalent to
“do not make a change”.</p>
</div>
<div class="section" id="user-activity">
<h3>User activity</h3>
<p>From the <a class="reference external" href="https://spec.matrix.org/v1.7/client-server-api/#last-active-ago">Matrix spec on last active ago</a>:</p>
<blockquote>
The server maintains a timestamp of the last time it saw a pro-active event from
the user. A pro-active event may be sending a message to a room or changing presence
state to <tt class="docutils literal">online</tt>. This timestamp is presented via a key called <tt class="docutils literal">last_active_ago</tt>
which gives the relative number of milliseconds since the pro-active event.</blockquote>
<p>If the presence is set to <tt class="docutils literal">online</tt> then <tt class="docutils literal">last_active_ago</tt> is not part of the
<tt class="docutils literal">/sync</tt> response and <tt class="docutils literal">currently_active</tt> is returned instead.</p>
</div>
<div class="section" id="idle-timeout">
<h3>Idle timeout</h3>
<p>From the <a class="reference external" href="https://spec.matrix.org/v1.7/client-server-api/#idle-timeout">Matrix spec on automatically idling users</a>:</p>
<blockquote>
The server will automatically set a user’s presence to <tt class="docutils literal">unavailable</tt> if their
last active time was over a threshold value (e.g. 5 minutes). Clients can manually
set a user’s presence to <tt class="docutils literal">unavailable</tt>. Any activity that bumps the last active
time on any of the user’s clients will cause the server to automatically set their
presence to <tt class="docutils literal">online</tt>.</blockquote>
<p><a class="reference external" href="https://github.com/matrix-org/matrix-spec-proposals/pull/3026"><span class="caps">MSC3026</span></a> also recommends:</p>
<blockquote>
If a user’s presence is set to <tt class="docutils literal">busy</tt>, it is strongly recommended for implementations
to not implement a timer that would trigger an update to the <tt class="docutils literal">unavailable</tt> state
(like most implementations do when the user is in the <tt class="docutils literal">online</tt> state).</blockquote>
</div>
</div>
<div class="section" id="presence-in-synapse">
<h2>Presence in Synapse</h2>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p>This describes Synapse’s behavior <em>after</em> v1.93.0. Before that version Synapse
did not account for multiple devices, essentially meaning that the latest device
update won.</p>
<p class="last">This also only applies to <em>local</em> users; per-device information for remote users
is not available, only the combined per-user state.</p>
</div>
<p>User’s devices can set a device’s presence state and a user’s status message.
A user’s device knows better than the server whether they’re online and should
send that state as part of <tt class="docutils literal">/sync</tt> calls (e.g. sending <tt class="docutils literal">online</tt> or <tt class="docutils literal">unavailable</tt>
or <tt class="docutils literal">offline</tt>).</p>
<p>Thus a device is only ever able to set the “minimum” presence state for the user.
Presence states are coalesced across devices as
<tt class="docutils literal">busy</tt> > <tt class="docutils literal">online</tt> > <tt class="docutils literal">unavailable</tt> > <tt class="docutils literal">offline</tt>. You can build simple
truth tables of how these combine with multiple devices:</p>
<table border="1" class="docutils">
<colgroup>
<col width="33%" />
<col width="33%" />
<col width="33%" />
</colgroup>
<thead valign="bottom">
<tr><th class="head">Device 1</th>
<th class="head">Device 2</th>
<th class="head">User state</th>
</tr>
</thead>
<tbody valign="top">
<tr><td><tt class="docutils literal">online</tt></td>
<td><tt class="docutils literal">unavailable</tt></td>
<td><tt class="docutils literal">online</tt></td>
</tr>
<tr><td><tt class="docutils literal">busy</tt></td>
<td><tt class="docutils literal">online</tt></td>
<td><tt class="docutils literal">busy</tt></td>
</tr>
<tr><td><tt class="docutils literal">unavailable</tt></td>
<td><tt class="docutils literal">offline</tt></td>
<td><tt class="docutils literal">unavailable</tt></td>
</tr>
</tbody>
</table>
<p>Additionally, users expect to see the latest activity time across all devices.
(And therefore if any device is online and the latest activity is recent then
the user is currently active).</p>
<p>The status message is global and setting it should always override any previous
state (and never be cleared automatically).</p>
<div class="section" id="automatic-state-transitions">
<h3>Automatic state transitions</h3>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">Note that the below only describes the logic for <em>local</em> users. Data received
over federation is handled differently.</p>
</div>
<p>If a device is <tt class="docutils literal">unavailable</tt> or <tt class="docutils literal">offline</tt> it should transition to <tt class="docutils literal">online</tt>
if a “pro-active event” occurs. This includes sending a receipt or event, or syncing
without <tt class="docutils literal">set_presence</tt> or <tt class="docutils literal">set_presence=online</tt>.</p>
<p>If a device is <tt class="docutils literal">offline</tt> it should transition to <tt class="docutils literal">unavailable</tt> if it is syncing
with <tt class="docutils literal">set_presence=unavailable</tt>.</p>
<p>If a device is <tt class="docutils literal">online</tt> (either directly or implicitly via user actions) it should
transition to <tt class="docutils literal">unavailable</tt> (idle) after a period of time <a class="footnote-reference" href="#footnote-2" id="footnote-reference-2">[2]</a> if the device is
continuing to sync. (Note that this implies the sync is occurring with
<tt class="docutils literal">set_presence=unavailable</tt> as otherwise the device is continuing to report as
<tt class="docutils literal">online</tt>). <a class="footnote-reference" href="#footnote-3" id="footnote-reference-3">[3]</a></p>
<p>If a device is <tt class="docutils literal">online</tt> or <tt class="docutils literal">unavailable</tt> it should transition to <tt class="docutils literal">offline</tt>
after a period of time if it is not syncing and not making other actions which
would transition the device to <cite>online</cite>. <a class="footnote-reference" href="#footnote-4" id="footnote-reference-4">[4]</a></p>
<p>Note if a device is <tt class="docutils literal">busy</tt> it should not transition to other states. <a class="footnote-reference" href="#footnote-5" id="footnote-reference-5">[5]</a></p>
<p>There’s a <a class="reference external" href="https://github.com/matrix-org/synapse/blob/be65a8ec0195955c15fdb179c9158b187638e39a/tests/handlers/test_presence.py#L971-L1106">huge testcase</a> which checks all these transitions.</p>
<div class="section" id="examples">
<h4>Examples</h4>
<ol class="arabic simple">
<li>Two devices continually syncing, one <tt class="docutils literal">online</tt> and one <tt class="docutils literal">unavailable</tt>. The
end result should be <cite>online</cite>. <a class="footnote-reference" href="#footnote-6" id="footnote-reference-6">[6]</a></li>
<li>One device syncing with <tt class="docutils literal">set_presence=unavailable</tt> but had a “pro-active”
action, after a period of time the user should be <tt class="docutils literal">unavailable</tt> if no additional
“pro-active” actions occurred.</li>
<li>One device that stops syncing (and no other “pro-active” actions” are occurring),
after a period of time the user should be <tt class="docutils literal">offline</tt>.</li>
<li>Two devices continually syncing, one <tt class="docutils literal">online</tt> and one <tt class="docutils literal">unavailable</tt>. The
<tt class="docutils literal">online</tt> device stops syncing, after a period of time the user should be
<tt class="docutils literal">unavailable</tt>.</li>
</ol>
<table class="docutils footnote" frame="void" id="footnote-1" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td>This should be called <tt class="docutils literal">idle</tt>.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-2" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-2">[2]</a></td><td>The period of time is implementation specific.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-3" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-3">[3]</a></td><td>Note that syncing with <tt class="docutils literal">set_presence=offline</tt> does not transition to offline,
it is equivalent to not syncing. (It is mostly for mobile applications to
process push notifications.)</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-4" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-4">[4]</a></td><td>The spec doesn’t seem to ever say that devices can transition to offline.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-5" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-5">[5]</a></td><td>See the <a class="reference external" href="https://github.com/matrix-org/matrix-spec-proposals/pull/3026/files#r1287453423">open thread on the <span class="caps">MSC3026</span></a>.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-6" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-6">[6]</a></td><td>This is essentially the <a class="reference external" href="https://github.com/matrix-org/synapse/issues/16057">bug illustrated by the change in Element Web’s behavior</a>.</td></tr>
</tbody>
</table>
</div>
</div>
</div>
Handling GitHub Notifications2023-10-06T07:55:00-04:002023-10-06T07:55:00-04:00Patrick Cloketag:patrick.cloke.us,2023-10-06:/posts/2023/10/06/handling-github-notifications/<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">This was originally written for some coworkers and assumes a mostly GitHub-based
workflow. It has been lightly edited to be more readable, but if your organization
doesn’t use GitHub like we do then it might not apply great.</p>
</div>
<p><a class="reference external" href="https://github.com">GitHub</a> can generate a lot of notifications which can be …</p><div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">This was originally written for some coworkers and assumes a mostly GitHub-based
workflow. It has been lightly edited to be more readable, but if your organization
doesn’t use GitHub like we do then it might not apply great.</p>
</div>
<p><a class="reference external" href="https://github.com">GitHub</a> can generate a lot of notifications which can be difficult to follow,
this documents some of my process for keeping up with it! For reference, I
subscribe to:</p>
<ol class="arabic simple">
<li>All notifications for the repositories I work in somewhat frequently;</li>
<li>Only releases and security alerts for repositories which might affect me (e.g.
upstream repositories);</li>
<li>Other issues that might be related to the project I’m working on (e.g. bugs
in upstream projects).</li>
</ol>
<p>I also watch a bunch of open source projects and have some of my own projects.
(These are mostly Twisted or Celery related.)</p>
<p>I generally enjoy having some idea of “everything” going on in my team (in enough
detail to know what people are generally working on).</p>
<p>To avoid being overwhelmed by notifications I only subscribe to specific issues
for repositories from other teams or projects. These are usually:</p>
<ul class="simple">
<li>Things that personally annoy me (and I want to see fixed);</li>
<li>Things that are directly related to or blocking my work;</li>
</ul>
<p>For reference, I currently <a class="reference external" href="https://github.com/watching">watch 321 repositories</a>, although most of my
notifications probably come from < 20 repositories. I also have 32 repositories
with custom notification rules — those are set to only releases <span class="amp">&</span> Security alerts.
(And I have 1 muted repository.) <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a></p>
<div class="section" id="when-how">
<h2>When / how</h2>
<p>I tend to do the following daily:</p>
<ul class="simple">
<li>Catch-up on notifications in the morning (takes ~15 - 45 minutes for GitHub,
chat, e-mail, etc.).</li>
<li>Check notifications a few times during the day (between meetings, after lunch,
while tests runs, etc.).</li>
</ul>
<p>Each time I check notifications I quickly triage each notification by skimming
the title to see if I’m interested (sometimes the title is enough info!). From
this I do one of several things:</p>
<ul class="simple">
<li>Open any issue in a separate tab to come back to if I need to read more (or
potentially take action). I usually skim the update, leaving it open if I need
to respond, closing the tab if I don’t.</li>
<li><span class="dquo">“</span>Mark as read” if I know it does not require anything from me:<ul>
<li>A review someone else is handling (unless it is a bit of code I’m keen to
understand, or know is tricky and feel some ownership over).</li>
<li>The title contains enough information I don’t need to read the issue (e.g.
a colleague following a follow-up issue).</li>
<li>Obvious support requests, unless I’m the maintainer. <a class="footnote-reference" href="#footnote-2" id="footnote-reference-2">[2]</a></li>
<li>Random MSCs / matrix-doc issues that I don’t care about.</li>
</ul>
</li>
<li><strong>Unsubscribing</strong> if I’m not interested in following the issue (e.g. an open
source project is re-doing their <span class="caps">CI</span>). This was key for me watching other
projects that I only somewhat care about.</li>
</ul>
<p>I use both Thunderbird and the GitHub website (specifically the
<a class="reference external" href="https://github.com/notifications?query=is%3Aunread">unread notifications view</a>) to go through notifications. Note that the website
has quick buttons on the right which I use frequently: “Done” and “Unsubscribe”
(there is also “Save” — which I do not use, I mark as unread if I need to come back).
It can also be useful to “Mark as done” an entire repository for projects I
follow out of vague interest, but don’t have time to read at the moment.</p>
<p><span class="dquo">“</span>Open unread” is useful to get everything into separate tabs for later processing
(and to avoid waiting for GitHub to load). I usually use it when I have < 10
notifications left for a particular repository.</p>
<p>I usually attempt to go through notifications that I know I won’t have to respond
to first, as they can be quickly processed and reduce the overwhelming number of notifications.</p>
</div>
<div class="section" id="setup">
<h2>Setup</h2>
<p>This workflow refers to using GitHub with <a class="reference external" href="https://www.thunderbird.net/">Mozilla Thunderbird</a> (via <a class="reference external" href="https://www.fastmail.com/">Fastmail</a>)
and <a class="reference external" href="https://getfirefox.net">Mozilla Firefox</a>, none of it is particular to those applications and can be
adapted to others.</p>
<div class="section" id="github">
<h3>GitHub</h3>
<p>If you use GitHub for both work and other personal / open source projects it can
be helpful to route your work notifications to a separate email address. (This is
a good idea regardless for security <span class="amp">&</span> intellectual property concerns.)</p>
<p>Your default email can be configured on the <a class="reference external" href="https://github.com/settings/notifications">Notifications page</a> and
separation by organization can be configured on the <a class="reference external" href="https://github.com/settings/notifications/custom_routing">Custom routing page</a>.
Under “Subscriptions” on the Notification page, I have both “Watching” and
“Participating, @mentions and custom” set to notify on both GitHub <span class="amp">&</span> email.</p>
<p>You may also want to tweak your “Customize email preferences”. I have the
following enabled:</p>
<ul class="simple">
<li><span class="dquo">“</span>Pull Request reviews”</li>
<li><span class="dquo">“</span>Comments on Issues and Pull Requests”</li>
<li><span class="dquo">“</span>Include your own updates” — this sounds weird, but you only need to lose a
massive comment on GitHub once to want a copy of it in your inbox. (I
automatically mark them as read, see below.)</li>
</ul>
<p>I disable “Pull Request pushes” because I don’t find it useful, although you will
still get these via the website.</p>
</div>
<div class="section" id="fastmail">
<h3>Fastmail</h3>
<p>I have two mail rules setup in Fastmail to move all GitHub email to a separate
folder and to mark my own emails as read: <a class="footnote-reference" href="#footnote-3" id="footnote-reference-3">[3]</a></p>
<ol class="arabic simple">
<li>From: <tt class="docutils literal">Patrick Cloke <notifications@github.com></tt>:
1. Mark as read
2. Move to “GitHub”</li>
<li>From email address: <tt class="docutils literal">notifications@github.com</tt>:
1. Move to “GitHub”</li>
</ol>
<p>Similar filters can be setup on other mail services, e.g. Google Mail:</p>
<ol class="arabic simple">
<li>Matches: <tt class="docutils literal"><span class="pre">from:(Patrick</span> Cloke <notifications@github.com>)</tt>
1. Skip Inbox
2. Mark as read
3. Apply label: “GitHub”</li>
<li>Matches: <tt class="docutils literal"><span class="pre">from:(notifications@github.com)</span></tt>
1. Skip Inbox
2. Apply label: “GitHub”</li>
</ol>
<p>You can also check for <a class="reference external" href="https://docs.github.com/en/account-and-profile/managing-subscriptions-and-notifications-on-github/setting-up-notifications/configuring-notifications#filtering-email-notifications">more ways to filter GitHub emails</a>.</p>
</div>
<div class="section" id="mozilla-thunderbird">
<h3>Mozilla Thunderbird</h3>
<p>For all of my folders I use threads (View > Sort By > Threaded) and only view
threads which have unread messages (View > Threads > Threads with Unread).</p>
<p>Other things that are useful:</p>
<ul class="simple">
<li>Enable “Automatically mark messages as read”, but with a short delay (I have
“After displaying for” set to 1 second). (This lets you move through messages
quickly using the keyboard or shortcuts without marking them all by mistake.)</li>
<li>Add GitHub to the exceptions list in under “Allow remote content in messages”
for either <cite>notifications@github.com</cite> or the <cite>https://github.com</cite>: this can
be added when viewing an email from GitHub. (This will
<a class="reference external" href="https://docs.github.com/en/account-and-profile/managing-subscriptions-and-notifications-on-github/setting-up-notifications/configuring-notifications#notification-delivery-options">mark the notification as read</a> the GitHub website automatically.)</li>
</ul>
<p>I sort my threads by date, oldest first so I can just click the “n” hotkey to
move through messages quickly. I also use the message pane to have some context
on remaining unread messages per thread, but it should work fine without that.
If you decide you don’t care about the rest of the thread “r” marks it as read.
Note that reading any messages in a thread will mark the entire issue or pull
request as done on the website. I find this extremely efficient for going through
a small number of notifications quickly.</p>
<p>I very much wish there was a way to sync back the read status of notifications from
GitHub back to Thunderbird. Lacking that I tend to mark the entire folder as read
(Shift+C) if I’ve caught up on the website. <a class="footnote-reference" href="#footnote-4" id="footnote-reference-4">[4]</a></p>
</div>
<div class="section" id="mozilla-firefox">
<h3>Mozilla Firefox</h3>
<p>I use a few GitHub related extensions which can help:</p>
<ul class="simple">
<li><a class="reference external" href="https://github.com/sindresorhus/refined-github/">Refined GitHub</a>: includes lots of small tweaks to make GitHub “better”.</li>
<li><a class="reference external" href="https://github.com/fregante/github-issue-link-status">GitHub Issue Link Status</a>: colors the issue / <span class="caps">PR</span> links with whether it is
open / closed / etc and marks it as an issue / <span class="caps">PR</span>.</li>
<li><a class="reference external" href="https://github.com/tanmayrajani/notifications-preview-github">Notifications Preview for GitHub</a>: makes the notification button a dropdown
for quick processing.</li>
<li><a class="reference external" href="https://github.com/homerchen19/github-file-icons">File Icons for GitHub and GitLab</a>: adds file icons per file type for GitHub</li>
<li><a class="reference external" href="https://github.com/freaktechnik/advanced-github-notifier">Advanced GitHub Notifier</a>: adds a Firefox toolbar button with easy access to
your notifications (including a count of unread notifications)</li>
</ul>
</div>
</div>
<div class="section" id="conclusion">
<h2>Conclusion</h2>
<p>Hopefully some of this is helpful, please let me know if you have any questions
or thoughts!</p>
<table class="docutils footnote" frame="void" id="footnote-1" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td>In August 2021 I was watching 263 repositories and had 18 repositories with
custom notification settings.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-2" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-2">[2]</a></td><td>My team rotates through who is the first-line of contacts for incoming
community requests, releases, etc.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-3" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-3">[3]</a></td><td>I have similar filters setup for <a class="reference external" href="https://gitlab.com">GitLab</a>, <a class="reference external" href="https://sentry.io">Sentry</a>, etc.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-4" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-4">[4]</a></td><td>You could probably do this with an Thunderbird extension, but I’ve failed to
find time to look into it.</td></tr>
</tbody>
</table>
</div>
Matrix Live demo on Linearized Matrix2023-10-04T09:55:00-04:002023-10-04T09:55:00-04:00Patrick Cloketag:patrick.cloke.us,2023-10-04:/posts/2023/10/04/matrix-live-demo-on-linearized-matrix/<p>I demoed some of my work at <a class="reference external" href="https://element.io">Element</a> on Matrix Live back on August 4th’s
<a class="reference external" href="https://matrix.org/blog/2023/08/04/this-week-in-matrix-2023-08-04/">This Week in Matrix</a> (and failed to mention it here). I talked a bit about
what <a class="reference external" href="https://turt2live.github.io/ietf-mimi-linearized-matrix/draft-ralston-mimi-linearized-matrix.html">Linearized Matrix</a>, Element’s effort for the <span class="caps">IETF</span>’s
<a class="reference external" href="https://datatracker.ietf.org/group/mimi/about/">“More Instant Messaging Interoperability” (<span class="caps">MIMI</span>)</a> working group.</p>
<p>I demoed …</p><p>I demoed some of my work at <a class="reference external" href="https://element.io">Element</a> on Matrix Live back on August 4th’s
<a class="reference external" href="https://matrix.org/blog/2023/08/04/this-week-in-matrix-2023-08-04/">This Week in Matrix</a> (and failed to mention it here). I talked a bit about
what <a class="reference external" href="https://turt2live.github.io/ietf-mimi-linearized-matrix/draft-ralston-mimi-linearized-matrix.html">Linearized Matrix</a>, Element’s effort for the <span class="caps">IETF</span>’s
<a class="reference external" href="https://datatracker.ietf.org/group/mimi/about/">“More Instant Messaging Interoperability” (<span class="caps">MIMI</span>)</a> working group.</p>
<p>I demoed two <a class="reference external" href="https://github.com/matrix-org/synapse">Synapse</a> instances as a dual-stack Matrix/Linearised Matrix
homeserver communicating over federation with two <a class="reference external" href="https://github.com/matrix-org/eigen-server">eigen-server</a> instances, an
example greenfield Linearized Matrix server. THis work server as a proof of
concept for interoperability between Matrix and Linearized Matrix.</p>
<p>You can find a bit more about the approach (and the code) on <a class="reference external" href="https://github.com/matrix-org/synapse/issues/15954">GitHub</a>.</p>
<p>Check out the video below (my bit starts around the 5:27 mark) and let me know
if you have any questions!</p>
<div class="youtube youtube-16x9"><iframe src="https://www.youtube-nocookie.com/embed/1LxkbTku0XQ?start=327" allowfullscreen seamless frameBorder="0"></iframe></div>Celery architecture breakdown2023-09-15T15:28:00-04:002023-09-15T15:28:00-04:00Patrick Cloketag:patrick.cloke.us,2023-09-15:/posts/2023/09/15/celery-architecture-breakdown/
<p>The <a class="reference external" href="https://docs.celeryq.dev/en/v5.2.7/">Celery project</a>, which is often used Python library to run “background tasks”
for synchronous web frameworks, describes itself as:</p>
<blockquote>
<p>Celery is a simple, flexible, and reliable distributed system to process vast
amounts of messages , while providing operations with the tools required to
maintain such a system.</p>
<p>It’s a …</p></blockquote>
<p>The <a class="reference external" href="https://docs.celeryq.dev/en/v5.2.7/">Celery project</a>, which is often used Python library to run “background tasks”
for synchronous web frameworks, describes itself as:</p>
<blockquote>
<p>Celery is a simple, flexible, and reliable distributed system to process vast
amounts of messages , while providing operations with the tools required to
maintain such a system.</p>
<p>It’s a task queue with focus on real-time processing, while also supporting
task scheduling.</p>
</blockquote>
<p>The documentation goes into great detail about how to configure Celery with
its plethora of options, but it does not focus much on the <a class="reference external" href="https://docs.celeryq.dev/en/v5.2.7/getting-started/introduction.html">high level architecture</a>
or how messages pass between the components. Celery is <em>extremely</em> flexible (almost
every component can be easily replaced!) but this can make it hard to understand.
I attempt to break it down to the best of my understanding below. <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a></p>
<div class="section" id="high-level-architecture">
<h2><a class="toc-backref" href="#toc-entry-1">High Level Architecture</a></h2>
<p>Celery has a few main components <a class="footnote-reference" href="#footnote-2" id="footnote-reference-2">[2]</a>:</p>
<ol class="arabic simple">
<li>Your application code, including any <a class="reference external" href="https://docs.celeryq.dev/en/v5.2.7/userguide/tasks.html"><tt class="docutils literal">Task</tt></a> objects you’ve defined. (Usually
called the “client” in Celery’s documentation.)</li>
<li>A <a class="reference external" href="https://docs.celeryq.dev/en/v5.2.7/getting-started/backends-and-brokers/index.html">broker</a> or message transport.</li>
<li>One or more Celery <a class="reference external" href="https://docs.celeryq.dev/en/v5.2.7/internals/worker.html">workers</a>.</li>
<li>A (results) <a class="reference external" href="https://docs.celeryq.dev/en/v5.2.7/getting-started/backends-and-brokers/index.html">backend</a>.</li>
</ol>
<blockquote class="text-center">
<a class="reference external image-reference" href="https://patrick.cloke.us/images/celery-architecture/celery-overview.png"><img alt="Celery overview" src="https://patrick.cloke.us/images/celery-architecture/celery-overview.png" style="width: 50%;"/></a>
<p>A simplified view of Celery components.</p>
</blockquote>
<p>In order to use Celery you need to:</p>
<ol class="arabic simple">
<li>Instantiate a Celery <tt class="docutils literal">application</tt> (which includes configuration, such as
which broker and backend to use and how to connect to them) and define one or
more <tt class="docutils literal">Task</tt> definitions.</li>
<li>Run a broker.</li>
<li>Run one or more Celery workers.</li>
<li>(Maybe) run a backend.</li>
</ol>
<p>If you’re unfamiliar with Celery, below is an example. It declares a simple
<tt class="docutils literal">add</tt> task using the <tt class="docutils literal">@task</tt> decorator and will request the task to be executed
in the background twice (<tt class="docutils literal"><span class="pre">add.delay(...)</span></tt>). <a class="footnote-reference" href="#footnote-3" id="footnote-reference-3">[3]</a> The results are then fetched
(<tt class="docutils literal">asyncresult_1.get()</tt>) and printed. Place this in a file named <tt class="docutils literal">my_app.py</tt>:</p>
<pre class="code python highlight literal-block">
<span class="kn">from</span> <span class="nn">celery</span> <span class="kn">import</span> <span class="n">Celery</span><span class="w">
</span><span class="n">app</span> <span class="o">=</span> <span class="n">Celery</span><span class="p">(</span><span class="w">
</span> <span class="s2">"my_app"</span><span class="p">,</span><span class="w">
</span> <span class="n">backend</span><span class="o">=</span><span class="s2">"rpc://"</span><span class="p">,</span><span class="w">
</span> <span class="n">broker</span><span class="o">=</span><span class="s2">"amqp://guest@localhost//"</span><span class="p">,</span><span class="w">
</span><span class="p">)</span><span class="w">
</span><span class="nd">@app</span><span class="o">.</span><span class="n">task</span><span class="p">()</span><span class="w">
</span><span class="k">def</span> <span class="nf">add</span><span class="p">(</span><span class="n">a</span><span class="p">:</span> <span class="nb">int</span><span class="p">,</span> <span class="n">b</span><span class="p">:</span> <span class="nb">int</span><span class="p">)</span> <span class="o">-></span> <span class="nb">int</span><span class="p">:</span><span class="w">
</span> <span class="k">return</span> <span class="n">a</span> <span class="o">+</span> <span class="n">b</span><span class="w">
</span><span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s2">"__main__"</span><span class="p">:</span><span class="w">
</span> <span class="c1"># Request that the tasks run and capture their async results.</span><span class="w">
</span> <span class="n">asyncresult_1</span> <span class="o">=</span> <span class="n">add</span><span class="o">.</span><span class="n">delay</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span><span class="w">
</span> <span class="n">asyncresult_2</span> <span class="o">=</span> <span class="n">add</span><span class="o">.</span><span class="n">delay</span><span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">)</span><span class="w">
</span> <span class="n">result_1</span> <span class="o">=</span> <span class="n">asyncresult_1</span><span class="o">.</span><span class="n">get</span><span class="p">()</span><span class="w">
</span> <span class="n">result_2</span> <span class="o">=</span> <span class="n">asyncresult_2</span><span class="o">.</span><span class="n">get</span><span class="p">()</span><span class="w">
</span> <span class="c1"># Should result in 3, 7.</span><span class="w">
</span> <span class="nb">print</span><span class="p">(</span><span class="sa">f</span><span class="s2">"Results: </span><span class="si">{</span><span class="n">result_1</span><span class="si">}</span><span class="s2">, </span><span class="si">{</span><span class="n">result_2</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
</pre>
<p>Usually you don’t care where (which worker) the task runs on it, or how it gets
there but sometimes you need! We can break down the components more to reveal more detail:</p>
<blockquote class="text-center">
<a class="reference external image-reference" href="https://patrick.cloke.us/images/celery-architecture/celery-components.png"><img alt="Celery components" src="https://patrick.cloke.us/images/celery-architecture/celery-components.png" style="width: 50%;"/></a>
<p>The Celery components broken into sub-components.</p>
</blockquote>
<div class="section" id="broker">
<h3><a class="toc-backref" href="#toc-entry-2">Broker</a></h3>
<p>The message broker is a piece of off-the-shelf software which takes task requests
and queues them until a worker is ready to process them. Common options include
<a class="reference external" href="https://www.rabbitmq.com/">RabbitMQ</a>, or <a class="reference external" href="https://redis.io">Redis</a>, although your cloud provider might have a custom one.</p>
<p>The broker may have some sub-components, including an exchange and one or more
queues. (Note that Celery tends to use <span class="caps">AMQP</span> terminology and sometimes emulates
features which do not exist on other brokers.)</p>
<p>Configuring your broker is beyond the scope of this article (and depends heavily
on workload). The Celery <a class="reference external" href="https://docs.celeryq.dev/en/v5.2.7/userguide/routing.html">routing documentation</a> has more information on how and
why you might route tasks to different queues.</p>
</div>
<div class="section" id="workers">
<h3><a class="toc-backref" href="#toc-entry-3">Workers</a></h3>
<p>Celery workers fetch queued tasks from the broker and then run the code defined in
your <tt class="docutils literal">task</tt>, they can optionally return a value via the results backend.</p>
<p>Celery workers have a “consumer” which fetches tasks from the broker: by default
it requests many tasks at once, equivalent to “<a class="reference external" href="https://docs.celeryq.dev/en/v5.2.7/userguide/configuration.html#std-setting-worker_prefetch_multiplier">prefetch multiplier</a> x <a class="reference external" href="https://docs.celeryq.dev/en/v5.2.7/userguide/configuration.html#std-setting-worker_concurrency">concurrency</a>“.
(If your prefetch multiplier is 5 and your concurrency is 4, it attempts to
fetch up to 20 queued tasks from the broker.) Once fetched it places them into
an in-memory buffer. The task pool then runs each task via its <tt class="docutils literal">Strategy</tt> —
for a normal Celery <tt class="docutils literal">Task</tt> the task pool essentially executes tasks from the
consumer’s buffer.</p>
<p>The worker also handles scheduling tasks to run in future (by queueing them
in-memory), but we will not go deeper into that here.</p>
<p>Using the “prefork” pool, the consumer and task pool are separate processes, while
the “gevent”/”eventlet” pool uses coroutines, and the “threads” pool uses threads.
There’s also a “solo” pool which can be useful for testing (everything is run in
the same process: a single task runs at a time and blocks the consumer from
fetching more tasks.)</p>
</div>
<div class="section" id="backend">
<h3><a class="toc-backref" href="#toc-entry-4">Backend</a></h3>
<p>The backend is another piece of off-the-shelf software which is used to store the
results of your task. It provides a key-value store and is commonly <a class="reference external" href="https://redis.io">Redis</a>,
although there are many options depending on how durable and large your results
are. The results backend can be queried by using the <tt class="docutils literal">AsyncResult</tt> object which
is returned to your application code. <a class="footnote-reference" href="#footnote-4" id="footnote-reference-4">[4]</a></p>
<p>Much like for brokers, how you configure results backends is beyond the scope of
this article.</p>
</div>
</div>
<div class="section" id="dataflow">
<h2><a class="toc-backref" href="#toc-entry-5">Dataflow</a></h2>
<p>You might have observed that the above components discussed at least several
different processes (client, broker, worker, worker pool, backend) which may also
exist on different computers. How does this all work to pass the task between
them? Usually this level of detail isn’t necessary to understand what it means
to “run a task in the background”, but it can be useful for diagnosing performance
or configuring brokers and backends.</p>
<p>The main thing to understand is that there’s lots of serialization happening across
each process boundary:</p>
<blockquote class="text-center">
<a class="reference external image-reference" href="https://patrick.cloke.us/images/celery-architecture/celery-dataflow.png"><img alt="Celery dataflow" src="https://patrick.cloke.us/images/celery-architecture/celery-dataflow.png" style="width: 50%;"/></a>
<p>A task message traversing from application code to the broker to a worker,
and a result traversing from a worker to a backend to application code.</p>
</blockquote>
<div class="section" id="request-serialization">
<h3><a class="toc-backref" href="#toc-entry-6">Request Serialization</a></h3>
<p>When a client requests for a task to be run the information needs to be passed to
the broker in a form it understands. The necessary data includes:</p>
<ul class="simple">
<li>The task identifier (e.g. <tt class="docutils literal">my_app.add</tt>).</li>
<li>Any arguments (e.g. <tt class="docutils literal">(1, 2)</tt>) and keyword arguments.</li>
<li>A request <span class="caps">ID</span>.</li>
<li>Routing information.</li>
<li>…and a bunch of other metadata.</li>
</ul>
<p>Exactly what is included is defined by the <a class="reference external" href="https://docs.celeryq.dev/en/v5.2.7/internals/protocol.html#message-protocol-task-v2">message protocol</a> (of which Celery
has two, although they’re fairly similar).</p>
<p>Most of the metadata gets placed in the headers while the task arguments, which
might be any Python class, need to be serialized into the body. Celery supports
<a class="reference external" href="https://docs.celeryq.dev/en/v5.2.7/userguide/calling.html#calling-serializers">many serializers</a> and uses <a class="reference external" href="https://docs.python.org/3/library/json.html"><span class="caps">JSON</span></a> by default (<a class="reference external" href="https://docs.python.org/dev/library/pickle.html#module-pickle">pickle</a>, <a class="reference external" href="http://yaml.org/"><span class="caps">YAML</span></a>, and <a class="reference external" href="http://msgpack.org/">msgpack</a>,
as well as custom schemes can be used as well).</p>
<p>After serialization, Celery also supports <a class="reference external" href="https://docs.celeryq.dev/en/v5.2.7/userguide/calling.html#compression">compressing the message</a> or
<a class="reference external" href="https://docs.celeryq.dev/en/v5.2.7/userguide/security.html#message-signing">signing the message</a> for additional security.</p>
<p>An example <span class="caps">AMQP</span> message containing the details of a task request (from RabbitMQ’s
<a class="reference external" href="https://www.rabbitmq.com/management.html">management interface</a>) is shown below:</p>
<blockquote class="text-center">
<a class="reference external image-reference" href="https://patrick.cloke.us/images/celery-architecture/rabbitmq-task-message.png"><img alt="Celery task wrapped in a RabbitMQ message" src="https://patrick.cloke.us/images/celery-architecture/rabbitmq-task-message.png" style="width: 50%;"/></a>
<p>The example Celery task wrapped in a RabbitMQ message</p>
</blockquote>
<p>When a worker fetches a task from the broker it deserializes it into a <a class="reference external" href="https://docs.celeryq.dev/en/v5.2.7/userguide/tasks.html#task-request">Request</a>
and executes it (as discussed above). In the case of a “prefork” worker pool the
<tt class="docutils literal">Request</tt> is serialized <em>again</em> using pickle when passed to task pool <a class="footnote-reference" href="#footnote-5" id="footnote-reference-5">[5]</a>.</p>
<p>The worker pool then unpickles the request, loads the task code, and executes
it with the requested arguments. Finally your task code is running! Note that the
task code itself is not contained in the serialized request, that is loaded
separately by the worker.</p>
</div>
<div class="section" id="result-serialization">
<h3><a class="toc-backref" href="#toc-entry-7">Result Serialization</a></h3>
<p>When a task returns a value it gets stored in the results backend with enough
information for the original client to find it:</p>
<ul class="simple">
<li>The result <span class="caps">ID</span>.</li>
<li>The result.</li>
<li>…and some other metadata.</li>
</ul>
<p>Similarly to tasks this information must be serialized before being placed in the
results backend (and gets split between the headers and body). Celery provides
configuration options to customize this serialization. <a class="footnote-reference" href="#footnote-6" id="footnote-reference-6">[6]</a></p>
<p>An example <span class="caps">AMQP</span> message containing the details of a result is shown below:</p>
<blockquote class="text-center">
<a class="reference external image-reference" href="https://patrick.cloke.us/images/celery-architecture/rabbitmq-result-message.png"><img alt="Celery result wrapped in a RabbitMQ message" src="https://patrick.cloke.us/images/celery-architecture/rabbitmq-result-message.png" style="width: 50%;"/></a>
<p>The example Celery result wrapped in a RabbitMQ message</p>
</blockquote>
<p>Once the result is fetched by the client it can deserialized the true (Python)
return value and provide it to the application code.</p>
</div>
<div class="section" id="final-thoughts">
<h3><a class="toc-backref" href="#toc-entry-8">Final thoughts</a></h3>
<p>Since the Celery protocol is a public, documented <span class="caps">API</span> it allows you to create
task requests externally to Celery! As long as you can interface to the Celery broker
(and have some shared configuration) you can use a different application (or programming
language) to publish and/or consume tasks. This is exactly what others have done:</p>
<ul class="simple">
<li>JavaScript / TypeScript<ul>
<li><a class="reference external" href="https://github.com/mher/node-celery">node-celery</a></li>
<li><a class="reference external" href="https://github.com/node-celery-ts/node-celery-ts">node-celery-ts</a></li>
</ul>
</li>
<li><span class="caps">PHP</span>:<ul>
<li><a class="reference external" href="https://github.com/gjedeer/celery-php">celery-php</a></li>
</ul>
</li>
<li>Rust<ul>
<li><a class="reference external" href="https://github.com/rusty-celery/rusty-celery">rusty-celery</a></li>
</ul>
</li>
<li>Go<ul>
<li><a class="reference external" href="https://github.com/gocelery/gocelery">gocelery</a></li>
<li><a class="reference external" href="https://github.com/marselester/gopher-celery">gopher-celery</a></li>
</ul>
</li>
</ul>
<p>Note that I haven’t used any of the above projects (and can’t vouch for them).</p>
<table class="docutils footnote" frame="void" id="footnote-1" rules="none">
<colgroup><col class="label"/><col/></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td>Part of this started out as an <a class="reference external" href="https://github.com/clokep/celery-batches/issues/69#issuecomment-1181855643">explanation of how celery-batches works</a>.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-2" rules="none">
<colgroup><col class="label"/><col/></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-2">[2]</a></td><td><a class="reference external" href="https://docs.celeryq.dev/en/v5.2.7/userguide/periodic-tasks.html">Celery beat</a> is another common component used to run scheduled or periodic
tasks. Architecture wise it takes the same place as your application code,
i.e. it runs forever and requests for tasks to be executed based on the time.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-3" rules="none">
<colgroup><col class="label"/><col/></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-3">[3]</a></td><td>There’s a <a class="reference external" href="https://docs.celeryq.dev/en/v5.2.7/userguide/calling.html">bunch of ways</a> to do this, <tt class="docutils literal">apply_async</tt> and <tt class="docutils literal">delay</tt> are the
most common, but don’t impact the contents of this article.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-4" rules="none">
<colgroup><col class="label"/><col/></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-4">[4]</a></td><td>As a quick aside — <tt class="docutils literal">AsyncResult</tt> does not refer to async/await in Python.
<tt class="docutils literal">AsyncResult.get()</tt> is <strong>synchronous</strong>. A <a class="reference external" href="https://patrick.cloke.us/posts/2018/10/23/calling-celery-from-twisted/">previous article</a> has some more
information on this.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-5" rules="none">
<colgroup><col class="label"/><col/></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-5">[5]</a></td><td>This is not configurable. The Celery <a class="reference external" href="https://docs.celeryq.dev/en/v5.2.7/userguide/security.html#serializers">security guide</a> recommends not using
pickle for serializers (and it is <a class="reference external" href="https://docs.python.org/dev/library/pickle.html">well known</a> that pickle can be a security
flaw), but it does not seem documented anywhere that pickle will be used with
the prefork pool. If you are using <span class="caps">JSON</span> to initially serialize to the broker
then your task should only be left with “simple” types (strings, integers,
floats, null, lists, and dictionaries) so this should not be an issue.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-6" rules="none">
<colgroup><col class="label"/><col/></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-6">[6]</a></td><td>Tasks and results can be configured to have different serializers (or different
compression settings) via the <tt class="docutils literal">task_</tt> vs. <tt class="docutils literal">result_</tt> configuration options.</td></tr>
</tbody>
</table>
</div>
</div>
Matrix Push Rules & Notifications2023-05-08T14:56:00-04:002023-05-08T14:56:00-04:00Patrick Cloketag:patrick.cloke.us,2023-05-08:/posts/2023/05/08/matrix-push-rules-notifications/<p>In a previous post about <a class="reference external" href="https://patrick.cloke.us/posts/2023/01/05/matrix-read-receipts-and-notifications/">read receipts <span class="amp">&</span> notifications in Matrix</a> I briefly
mentioned that push rules generate notifications, but with little detail. After
completing a rather large project to improve notifications in Matrix I want to
fill in some of those blanks. <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a></p>
<!-- comment:
Adapted from https://docs.google.com/presentation/d/1odrbD5wMwGz_qUtG5U1pFb7p3sFwLApDaYtyHpdI-Oo/edit -->
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">These notes are true as of …</p></div><p>In a previous post about <a class="reference external" href="https://patrick.cloke.us/posts/2023/01/05/matrix-read-receipts-and-notifications/">read receipts <span class="amp">&</span> notifications in Matrix</a> I briefly
mentioned that push rules generate notifications, but with little detail. After
completing a rather large project to improve notifications in Matrix I want to
fill in some of those blanks. <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a></p>
<!-- comment:
Adapted from https://docs.google.com/presentation/d/1odrbD5wMwGz_qUtG5U1pFb7p3sFwLApDaYtyHpdI-Oo/edit -->
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">These notes are true as of the v1.6 of the Matrix spec and also cover some
Matrix spec changes which may or may not have been merged since.</p>
</div>
<div class="section" id="push-notifications-in-matrix">
<h2><a class="toc-backref" href="#toc-entry-1">Push notifications in Matrix</a></h2>
<p>Matrix includes a <a class="reference external" href="https://spec.matrix.org/v1.6/client-server-api/#push-notifications">push notifications module</a> which defines when Matrix events
are considered an unread <strong>notification</strong> or <strong>highlight notification</strong> <a class="footnote-reference" href="#footnote-2" id="footnote-reference-2">[2]</a>
and <em>how</em> those events are <strong>sent to third-party push notification services</strong>.</p>
<p><strong>Push rules</strong> are a set of <em>ordered</em> rules which clients upload to the homeserver.
These are shared by all device and are evaluated per event by the homeserver (and
also by clients). <a class="reference external" href="https://spec.matrix.org/v1.6/client-server-api/#predefined-rules">Default push rules</a> are defined in the Matrix spec. Push rules
power the unread (and highlight) counts for each room, push notifications, and the
notifications <span class="caps">API</span>.</p>
<p>Each rule defines <strong>conditions</strong> which must be met for the rule to match and
<strong>actions</strong> to take if the rule matches.</p>
<p>Processing of push rules occur until a rule matches or all rules have been evaluated.</p>
<div class="section" id="getting-notifications">
<h3><a class="toc-backref" href="#toc-entry-2">Getting notifications</a></h3>
<p>As some background, clients receive notifications in one of two ways, via polling
<tt class="docutils literal">/sync</tt> and/or via push notifications.</p>
<p>Web-based clients often receive events via polling:</p>
<blockquote class="text-center">
<a class="reference external image-reference" href="https://patrick.cloke.us/images/matrix-push-rules-and-notifications/web-push-flow.png"><img alt="Notification flow for web applications." src="/thumbnails/matrix-push-rules-and-notifications/web-push-flow_medium.png"/></a>
</blockquote>
<p>The <a class="reference external" href="https://spec.matrix.org/v1.6/client-server-api/#_matrixclientv3sync_unread-notification-counts">sync response</a> (both initial and incremental) include the count of unread
notifications and unread highlight notifications per room.</p>
<p>Mobile applications often <a class="reference external" href="https://spec.matrix.org/v1.6/push-gateway-api/#overview">receive events via push</a> <a class="footnote-reference" href="#footnote-3" id="footnote-reference-3">[3]</a>:</p>
<blockquote class="text-center">
<a class="reference external image-reference" href="https://patrick.cloke.us/images/matrix-push-rules-and-notifications/mobile-push-flow.png"><img alt="Notification flow for mobile applications." src="/thumbnails/matrix-push-rules-and-notifications/mobile-push-flow_medium.png"/></a>
</blockquote>
<p>Push notifications include the event information (or just the event <span class="caps">ID</span>) and
whether the event was a highlight notification. (The event being pushed implies
it increased the notification count.)</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">The deployment of the push gateway must be paired with the application (the
push keys must be paired). I.e. if you make your own application (or even
your own build of Element iOS / Android) you cannot re-use the deployment at
matrix.org and must have your own deployment.</p>
</div>
</div>
<div class="section" id="getting-events-which-generated-notifications">
<h3><a class="toc-backref" href="#toc-entry-3">Getting events which generated notifications</a></h3>
<p>There’s <a class="reference external" href="https://spec.matrix.org/v1.6/client-server-api/#listing-notifications">an <span class="caps">API</span> to retrieve a list of events</a> which the user has been notified
about. This powers the “notification panel” on Element Web and is meant to help
users catch-up on missed notifications.</p>
<p>It is fairly underspecified and the Synapse implementation has limitations:</p>
<ul class="simple">
<li>Highlight notifications are only kept for 30 days</li>
<li>Non-highlight notifications are only kept for 72 hours</li>
</ul>
<p>Additionally it <a class="reference external" href="https://github.com/vector-im/element-web/issues/6874">works poorly for encrypted rooms</a>.</p>
</div>
</div>
<div class="section" id="push-rules-background">
<h2><a class="toc-backref" href="#toc-entry-4">Push rules background</a></h2>
<div class="section" id="getting-the-configured-push-rules">
<h3><a class="toc-backref" href="#toc-entry-5">Getting the configured push rules?</a></h3>
<p>There’s a <a class="reference external" href="https://spec.matrix.org/v1.6/client-server-api/#push-rules-api">set of APIs to fetch or modify push rules</a>, they let you:</p>
<ul class="simple">
<li>Fetch all push rules</li>
<li>Create or delete an individual push rule</li>
<li>Fetch or update an individual push rule’s actions</li>
<li>Fetch or enable/disable an individual push rule</li>
</ul>
<p>An initial sync includes all of a user’s push rules under the user’s account data.</p>
<p>Any changes to push rules are included in incremental syncs. <em>Except</em> for newly
added rules to the specification (this is likely a homeserver bug).</p>
<p>Note that you cannot use <a class="reference external" href="https://spec.matrix.org/v1.6/client-server-api/#client-config">the account data APIs</a> to configure push rules. <a class="footnote-reference" href="#footnote-4" id="footnote-reference-4">[4]</a></p>
</div>
<div class="section" id="what-makes-up-a-push-rule">
<h3><a class="toc-backref" href="#toc-entry-6">What makes up a push rule?</a></h3>
<p>A push rule is a <a class="reference external" href="https://spec.matrix.org/v1.6/client-server-api/#_matrixclientv3pushrules_pushrule"><span class="caps">JSON</span> object with the following fields</a>:</p>
<ul class="simple">
<li><tt class="docutils literal">rule_id</tt>: Unique (per-user) <span class="caps">ID</span> for the rule.<ul>
<li>The <tt class="docutils literal">rule_id</tt> for default rules have a special form (they start with a
dot: <tt class="docutils literal">.</tt>).</li>
</ul>
</li>
<li><tt class="docutils literal">default</tt>: Whether the rule is part of the predefined set of rules.</li>
<li><tt class="docutils literal">enabled</tt>: Whether the rule is enabled.</li>
<li><tt class="docutils literal">conditions</tt>: an array of 0 or more conditions to match.</li>
<li><tt class="docutils literal">actions</tt>: 0 or more actions to take if the rule matches.</li>
</ul>
<p>All conditions must match for a push rule to match. If there are no conditions,
then the push rule always matches. Possible conditions include:</p>
<ul class="simple">
<li>Check event properties against patterns or exact values<ul>
<li>Strings can be compared via globbing or exact values.</li>
<li>The globbing behavior changes if you’re checking the <tt class="docutils literal">body</tt> property or not.</li>
</ul>
</li>
<li>Check against the number of room members<ul>
<li>Used to (incorrectly) check if a room is a direct message.</li>
</ul>
</li>
<li>Check if a user can <a class="reference external" href="https://spec.matrix.org/v1.6/client-server-api/#mroompower_levels">perform an action</a> via power rules<ul>
<li>The only defined option is whether a user can send @room.</li>
</ul>
</li>
</ul>
<p>Push rule actions define <a class="reference external" href="https://spec.matrix.org/v1.6/client-server-api/#actions">what to do once a push rule</a> matches an event.</p>
<ul class="simple">
<li><tt class="docutils literal">notify</tt>: increment the notification count and send a push notification. Uses
“tweaks” to optionally:<ul>
<li>Play a sound.</li>
<li>Create a highlight notification, this causes the highlight count to be
incremented (in addition to the notification count).</li>
</ul>
</li>
<li>Can be an empty list to do nothing.</li>
</ul>
<p>There are other undefined or no-op actions (<tt class="docutils literal">dont_notify</tt>, <tt class="docutils literal">coalesce</tt>) which will be
removed in the next version of the spec. <a class="footnote-reference" href="#footnote-5" id="footnote-reference-5">[5]</a></p>
</div>
<div class="section" id="types-of-push-rules">
<h3><a class="toc-backref" href="#toc-entry-7">Types of push rules</a></h3>
<p>Push rules have a type associated with them, these are executed in order:</p>
<ul class="simple">
<li>Override: generic high priority rules</li>
<li>Content-specific: applies to messages which have a <tt class="docutils literal">body</tt> that matches a <tt class="docutils literal">pattern</tt></li>
<li>Room-specific: applies to messages of a room</li>
<li>Sender-specific: applies to messages from a sender</li>
<li>Underride: generic low priority rules</li>
</ul>
<p>The previously discussed shape of push rules is not the full story! There are
special cases which do not accept conditions, but can be mapped to them.</p>
<ul class="simple">
<li>Content-specific: has a <tt class="docutils literal">pattern</tt> field which maps to a pattern against the
<tt class="docutils literal">body</tt> property.</li>
<li>Room-specific: the <tt class="docutils literal">rule_id</tt> is re-used to match against the room <span class="caps">ID</span>.</li>
<li>Sender-specific: the <tt class="docutils literal">rule_id</tt> is re-used to match against the event <tt class="docutils literal">sender</tt>.</li>
</ul>
</div>
</div>
<div class="section" id="why-do-clients-care-doesnt-the-homeserver-do-this-all-for-me">
<h2><a class="toc-backref" href="#toc-entry-8">Why do clients care? Doesn’t the homeserver do this all for me?</a></h2>
<p>Encryption ruins everything! Some of the push rules require the decrypted event
content to be properly processed. The enable this, the default rules declare
<strong>all encrypted events as notifications</strong>. Clients are expected to
<strong>re-run push rules on the decrypted content</strong>. <a class="footnote-reference" href="#footnote-6" id="footnote-reference-6">[6]</a></p>
<p>This can result in one of the following: <a class="footnote-reference" href="#footnote-7" id="footnote-reference-7">[7]</a></p>
<ul class="simple">
<li>Increment the highlight count (the decrypted event results in a highlight)</li>
<li>No change (the decrypted event results in a notification)</li>
<li>Decrement notification counts (the decrypted event results in no notification)</li>
</ul>
<p>Due to gappy syncs clients frequently can only make a best estimate of the true
unread / highlight count of events in encrypted rooms.</p>
<div class="admonition warning">
<p class="first admonition-title">Warning</p>
<p class="last">Element iOS / Android get encrypted events pushed to them, but do not properly
implement mentions <span class="amp">&</span> keywords.</p>
</div>
</div>
<div class="section" id="what-happens-by-default">
<h2><a class="toc-backref" href="#toc-entry-9">What happens by default?</a></h2>
<p>The <a class="reference external" href="https://spec.matrix.org/v1.6/client-server-api/#predefined-rules">default rules are in the Matrix spec</a> and include:</p>
<ul class="simple">
<li>Highlight:<ul>
<li>Tombstones</li>
<li>Room <span class="amp">&</span> user mentions</li>
</ul>
</li>
<li>Do nothing:<ul>
<li>Notice messages</li>
<li>Other room member events</li>
<li>Server <span class="caps">ACL</span> updates</li>
</ul>
</li>
<li>Notification:<ul>
<li>Invites to me</li>
<li>Messages and encrypted events in non-DMs</li>
</ul>
</li>
<li>Notification with sound:<ul>
<li>Incoming calls</li>
<li>Messages and encrypted events in DMs</li>
</ul>
</li>
</ul>
<p>Default rules can be disabled or have their actions modified on a per-user basis.
Some of the above features are handled by multiple push rules.</p>
<div class="section" id="other-standard-rules">
<h3><a class="toc-backref" href="#toc-entry-10">Other “standard” rules</a></h3>
<p>Element creates custom push rules based on a known form. <a class="footnote-reference" href="#footnote-8" id="footnote-reference-8">[8]</a></p>
<ul class="simple">
<li>Keywords (implemented as a content-specific rule with a pattern)</li>
<li>Per-room overrides:<ul>
<li>All messages (implemented as a room-specific rule with a notify action)</li>
<li>Mentions <span class="amp">&</span> keywords (implemented as a room-specific rule with no actions)</li>
<li>Mute (implemented as an override rule to match the room <span class="caps">ID</span> with no actions)</li>
</ul>
</li>
</ul>
<p>Matrix also allows defining arbitrary rules (e.g. to change behavior for particular
rooms, senders, message types, etc.)</p>
</div>
<div class="section" id="what-about-unread-rooms">
<h3><a class="toc-backref" href="#toc-entry-11">What about unread rooms?</a></h3>
<p>The unread (“bold”) rooms logic in Element Web is completely custom and outside
of the Matrix specification.</p>
<ul class="simple">
<li>Will the <a class="reference external" href="https://github.com/matrix-org/matrix-react-sdk/blob/d33e416fc75369d3fec1c1f27ef9d5b2ea0b3703/src/shouldHideEvent.ts#L58-L82">event be shown</a> ?</li>
<li>Is it <a class="reference external" href="https://github.com/matrix-org/matrix-react-sdk/blob/d33e416fc75369d3fec1c1f27ef9d5b2ea0b3703/src/Unread.ts#L41-L48">not an ignored event type</a> ?</li>
<li>Is it <a class="reference external" href="https://github.com/matrix-org/matrix-react-sdk/blob/d33e416fc75369d3fec1c1f27ef9d5b2ea0b3703/src/Unread.ts#L52">not redacted</a> ?</li>
<li>Does a <a class="reference external" href="https://github.com/matrix-org/matrix-react-sdk/blob/d33e416fc75369d3fec1c1f27ef9d5b2ea0b3703/src/Unread.ts#L53">renderer exist for the event</a> ?</li>
</ul>
<p>Note that if you enable hidden events (or tweak other options to show events)
then the behavior changes!</p>
</div>
</div>
<div class="section" id="putting-it-altogether">
<h2><a class="toc-backref" href="#toc-entry-12">Putting it altogether…</a></h2>
<p>…it gets complicated trying to figure out whether a message will generate a
notification or not.</p>
<blockquote class="text-center">
<a class="reference external image-reference" href="https://patrick.cloke.us/images/matrix-push-rules-and-notifications/default-push-rules.png"><img alt="Flow chart of the default Matrix push rules when using Element." src="/thumbnails/matrix-push-rules-and-notifications/default-push-rules_medium.png"/></a>
<p>The default Matrix push rules (also showing the options available within Element).</p>
</blockquote>
<table class="docutils footnote" frame="void" id="footnote-1" rules="none">
<colgroup><col class="label"/><col/></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td>Improving unintentional mentions (<a class="reference external" href="https://github.com/matrix-org/matrix-spec-proposals/pull/3952"><span class="caps">MSC3952</span></a>) is the main feature we were
working on, but this was powered by <a class="reference external" href="https://github.com/matrix-org/matrix-spec-proposals/pull/3758"><span class="caps">MSC3758</span></a> (from <a class="reference external" href="https://www.beeper.com/">Beeper</a>),
<a class="reference external" href="https://github.com/matrix-org/matrix-spec-proposals/pull/3873"><span class="caps">MSC3873</span></a> (from a coworker), and <a class="reference external" href="https://github.com/matrix-org/matrix-spec-proposals/pull/3966"><span class="caps">MSC3966</span></a>. <a class="reference external" href="https://github.com/matrix-org/matrix-spec-proposals/pull/3980"><span class="caps">MSC3980</span></a> was also a
follow-up for consistency.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-2" rules="none">
<colgroup><col class="label"/><col/></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-2">[2]</a></td><td><p class="first">Notification count (the grey badge with count in Element Web) is the number
of unread messages in a room. Highlight count (the red badge with count in
Element Web) is the number of unread mentions in a room.</p>
<div class="admonition warning">
<p class="first admonition-title">Warning</p>
<p class="last">The unread (“bold”) rooms feature in Element Web, which represents a room
with unread messages (but no notification count) is not powered by push
rules (and is not specced).</p>
</div>
<p class="last">See the <a class="reference external" href="https://github.com/matrix-org/matrix-react-sdk/blob/develop/docs/room-list-store.md#list-ordering-algorithm-importance">Element Web docs on the room list</a>.</p>
</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-3" rules="none">
<colgroup><col class="label"/><col/></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-3">[3]</a></td><td>This post generally defines “push notifications” as a notification which
is sent via a push provider to an application. Push providers include Apple,
Google, Microsoft, or Mozilla.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-4" rules="none">
<colgroup><col class="label"/><col/></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-4">[4]</a></td><td><a class="reference external" href="https://github.com/matrix-org/matrix-spec-proposals/pull/4010"><span class="caps">MSC4010</span></a> aims to make this explicit.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-5" rules="none">
<colgroup><col class="label"/><col/></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-5">[5]</a></td><td>See <a class="reference external" href="https://github.com/matrix-org/matrix-spec-proposals/pull/3987"><span class="caps">MSC3987</span></a>.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-6" rules="none">
<colgroup><col class="label"/><col/></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-6">[6]</a></td><td>It was not clear how clients should handle encrypted events <a class="reference external" href="https://github.com/matrix-org/matrix-spec/pull/1461">until recently</a></td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-7" rules="none">
<colgroup><col class="label"/><col/></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-7">[7]</a></td><td>Adapted from a <a class="reference external" href="https://gist.github.com/Half-Shot/f9501916363894761a1659250aa25181">Gist from Half-Shot</a>.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-8" rules="none">
<colgroup><col class="label"/><col/></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-8">[8]</a></td><td>These don’t seem to be specced, I’m unsure if other clients create similar
rules or understand these rules.</td></tr>
</tbody>
</table>
</div>
Python str Collection Gotchas2023-02-24T19:42:00-05:002023-02-24T19:42:00-05:00Patrick Cloketag:patrick.cloke.us,2023-02-24:/posts/2023/02/24/python-str-collection-gotchas/<p>We have been slowly adding Python type hints <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a> to <a class="reference external" href="https://github.com/matrix-org/synapse">Synapse</a> and have made
great progress (see <a class="reference external" href="https://matrix.org/blog/2021/12/03/type-coverage-for-sydent-motivation">some of our motivation</a>). Through this process we have
learned a lot about Python and type hints. One bit that was unexpected is that
many of the <a class="reference external" href="https://docs.python.org/3/library/collections.abc.html#collections-abstract-base-classes">abstract base classes</a> representing groups of …</p><p>We have been slowly adding Python type hints <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a> to <a class="reference external" href="https://github.com/matrix-org/synapse">Synapse</a> and have made
great progress (see <a class="reference external" href="https://matrix.org/blog/2021/12/03/type-coverage-for-sydent-motivation">some of our motivation</a>). Through this process we have
learned a lot about Python and type hints. One bit that was unexpected is that
many of the <a class="reference external" href="https://docs.python.org/3/library/collections.abc.html#collections-abstract-base-classes">abstract base classes</a> representing groups of <tt class="docutils literal">str</tt> instances
also match an individual <tt class="docutils literal">str</tt> instance. This has resulted in more than one
<em>real</em> bug for us <a class="footnote-reference" href="#footnote-2" id="footnote-reference-2">[2]</a>: a function which has parameter of type <tt class="docutils literal">Collection[str]</tt>
was called with a <tt class="docutils literal">str</tt>, for example <a class="footnote-reference" href="#footnote-3" id="footnote-reference-3">[3]</a>:</p>
<div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">send</span><span class="p">(</span><span class="n">event</span><span class="p">:</span> <span class="n">Event</span><span class="p">,</span> <span class="n">destinations</span><span class="p">:</span> <span class="n">Collection</span><span class="p">[</span><span class="nb">str</span><span class="p">])</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
<span class="w"> </span><span class="sd">"""Send an event to a set of destinations."""</span>
<span class="k">for</span> <span class="n">destination</span> <span class="ow">in</span> <span class="n">destinations</span><span class="p">:</span>
<span class="c1"># Do some HTTP.</span>
<span class="o">...</span>
<span class="k">def</span> <span class="nf">create_event</span><span class="p">(</span><span class="n">sender</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">content</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">room</span><span class="p">:</span> <span class="n">Room</span><span class="p">)</span> <span class="o">-></span> <span class="kc">None</span><span class="p">:</span>
<span class="w"> </span><span class="sd">"""Create & send an event."""</span>
<span class="n">event</span> <span class="o">=</span> <span class="n">Event</span><span class="p">(</span><span class="n">sender</span><span class="p">,</span> <span class="n">content</span><span class="p">)</span>
<span class="n">send</span><span class="p">(</span><span class="n">event</span><span class="p">,</span> <span class="s2">"matrix.org"</span><span class="p">)</span>
</pre></div>
<p>The correct version should call <tt class="docutils literal">send</tt> with a <tt class="docutils literal">list</tt> of destinations instead
of a single one. The “s” at the end of “destinations” takes on quite a bit of
importance! See the fix:</p>
<div class="highlight"><pre><span></span><span class="gu">@@ -7,5 +7,5 @@</span>
<span class="w"> </span> def create_event(sender: str, content: str, room: Room) -> None:
<span class="w"> </span> """Create & send an event."""
<span class="w"> </span> event = Event(sender, content)
<span class="gd">- send(event, "matrix.org")</span>
<span class="gi">+ send(event, ["matrix.org"])</span>
</pre></div>
<p>A possible solution is redefine the <tt class="docutils literal">destinations</tt> parameter as a <tt class="docutils literal">List[str]</tt>,
but this forces the caller to convert a <tt class="docutils literal">set</tt> or <tt class="docutils literal">tuple</tt> to a <tt class="docutils literal">list</tt>
(meaning iterating, allocate memory, etc.) or maybe using a <tt class="docutils literal"><span class="pre">cast(...)</span></tt> (and
thus losing some of the protections from type hints). As a team we have a desire
to keep the type hints of function parameters as broad as possible.</p>
<p>Put another way, <tt class="docutils literal">str</tt> is an instance of <tt class="docutils literal">Collection[str]</tt>, <tt class="docutils literal">Container[str]</tt>,
<tt class="docutils literal">Iterable[str]</tt>, and <tt class="docutils literal">Sequence[str]</tt>. <a class="footnote-reference" href="#footnote-4" id="footnote-reference-4">[4]</a> <a class="footnote-reference" href="#footnote-5" id="footnote-reference-5">[5]</a></p>
<p>Since our type hints are only used internally we do not need to worry too much
about accepting exotic types and eventually came up with <a class="reference external" href="https://github.com/matrix-org/synapse/blob/335f52d595c2c32e4b512b97e2851bc98b819ca7/synapse/types/__init__.py#L84-L86"><tt class="docutils literal">StrCollection</tt></a>:</p>
<div class="highlight"><pre><span></span><span class="c1"># Collection[str] that does not include str itself; str being a Sequence[str]</span>
<span class="c1"># is very misleading and results in bugs.</span>
<span class="n">StrCollection</span> <span class="o">=</span> <span class="n">Union</span><span class="p">[</span><span class="n">Tuple</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="o">...</span><span class="p">],</span> <span class="n">List</span><span class="p">[</span><span class="nb">str</span><span class="p">],</span> <span class="n">AbstractSet</span><span class="p">[</span><span class="nb">str</span><span class="p">]]</span>
</pre></div>
<p>This covers lists, tuples, sets, and frozen sets of <tt class="docutils literal">str</tt>, the one case it does
not seem to cover well is if you are using a dictionary as an <tt class="docutils literal">Iterable[str]</tt>,
the easy workaround there is to call <tt class="docutils literal">keys()</tt> on it to explicitly convert to a
<a class="reference external" href="https://docs.python.org/3/library/collections.abc.html#collections.abc.KeysView"><tt class="docutils literal">KeysView</tt></a>, which inherits from <tt class="docutils literal">Set</tt>.</p>
<table class="docutils footnote" frame="void" id="footnote-1" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td>Looking at the <a class="reference external" href="https://github.com/matrix-org/synapse/commits/develop/mypy.ini">commits</a> to <tt class="docutils literal">mypy.ini</tt> is probably the best way to see progress.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-2" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-2">[2]</a></td><td><a class="reference external" href="https://github.com/matrix-org/synapse/issues/14809">matrix-org/synapse#14809</a> is our tracking issue for fixing this, although
<a class="reference external" href="https://github.com/matrix-org/synapse/pull/14880/files#diff-0b449f6f95672437cf04f0b5512572b4a6a729d2759c438b7c206ea249619885R1161">matrix-org/synapse#14880</a> shows a concrete bug fix which occurred.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-3" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-3">[3]</a></td><td>This is heavily simplified, but hopefully illustrates the bug!</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-4" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-4">[4]</a></td><td>And <tt class="docutils literal">Reversible[str]</tt>, but I don’t think I have ever seen that used and
I think it less likely to introduce a bug.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-5" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-5">[5]</a></td><td><tt class="docutils literal">bytes</tt> doesn’t have quite the same issue, but it might be surprising
that <tt class="docutils literal">bytes</tt> is an instance of <tt class="docutils literal">Collection[int]</tt>, <tt class="docutils literal">Container[int]</tt>,
<tt class="docutils literal">Iterable[int]</tt>, and <tt class="docutils literal">Sequence[int]</tt>. I find this less likely to
introduce a bug.</td></tr>
</tbody>
</table>
Researching for a Matrix Spec Change2023-01-12T15:24:00-05:002023-01-12T15:24:00-05:00Patrick Cloketag:patrick.cloke.us,2023-01-12:/posts/2023/01/12/researching-for-a-matrix-spec-change/<p>The <a class="reference external" href="https://spec.matrix.org/">Matrix protocol</a> is modified via <a class="reference external" href="https://spec.matrix.org/proposals/">Matrix Spec Changes</a> (frequently abbreviated
to “MSCs”). These are short documents describing any technical changes and why they
are worth making (see <a class="reference external" href="https://github.com/matrix-org/matrix-spec-proposals/pull/2457">an example</a>). I’ve <a class="reference external" href="https://github.com/matrix-org/matrix-spec-proposals/pulls?q=is%3Apr+author%3Aclokep+label%3Aproposal">written a bunch</a> and wanted to
document my research process. <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a></p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">I treat my research as a …</p></div><p>The <a class="reference external" href="https://spec.matrix.org/">Matrix protocol</a> is modified via <a class="reference external" href="https://spec.matrix.org/proposals/">Matrix Spec Changes</a> (frequently abbreviated
to “MSCs”). These are short documents describing any technical changes and why they
are worth making (see <a class="reference external" href="https://github.com/matrix-org/matrix-spec-proposals/pull/2457">an example</a>). I’ve <a class="reference external" href="https://github.com/matrix-org/matrix-spec-proposals/pulls?q=is%3Apr+author%3Aclokep+label%3Aproposal">written a bunch</a> and wanted to
document my research process. <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a></p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">I treat my research as a <em>living document</em>, not an <em>artifact</em>. Thus, I don’t worry
much about the format. The important part is to start writing everything down
to have a single source of truth that can be shared with others.</p>
</div>
<p>First, I write out a <strong>problem statement</strong>: what am I trying to solve? (This step
might seem obvious, but it is useful to frame the technical changes in why
they matter. Many proposals seem to skip this step.) Most of my work tends to be
from the point of view of an end-user, but some changes are purely technical. Regardless,
there is benefit from a shared written context of the issue to be solved.</p>
<p>From the above and prior knowledge, I list any <strong>open questions</strong> (which I update
through this process). I’ll augment the questions with answers I find in my research,
write new ones about things I don’t understand, or remove them as they become irrelevant.</p>
<p>Next, I begin collecting any previous work done in this area, including:</p>
<ul>
<li><p class="first">What is the <strong>current specification</strong> related to this? I usually pull out blurbs
(with links back to the source) from <a class="reference external" href="https://spec.matrix.org/v1.5/client-server-api/">the latest specification</a>.</p>
</li>
<li><p class="first">Are there any <a class="reference external" href="https://github.com/matrix-org/matrix-spec/issues"><strong>related known issues</strong></a>? It is also worth checking the issue
trackers of projects: I start with the <a class="reference external" href="https://github.com/matrix-org/synapse">Synapse</a>, <a class="reference external" href="https://github.com/vector-im/element-meta">Element Meta</a>, and
<a class="reference external" href="https://github.com/vector-im/element-web">Element Web</a> repositories.</p>
</li>
<li><p class="first">Are there <strong>related outstanding MSCs</strong> or <strong>previous research</strong>? I search the
<a class="reference external" href="https://github.com/matrix-org/matrix-spec-proposals/">matrix-spec-proposals</a> repository for keywords, open anything that looks
vaguely related and then crawl those for mentions of other MSCs. I’ll document
the related ones with links and a brief description of the proposed change.</p>
<p>I include both proposed and closed MSCs to check for previously rejected ideas.</p>
</li>
<li><p class="first">Are others interested in this? Have others had conversation about it? I roughly
follow the <a class="reference external" href="https://matrix.to/#/#matrix-spec:matrix.org">#matrix-spec</a> room or search for rooms that might be interested in
the topic. I would recommend joining the <a class="reference external" href="https://matrix.to/#/#matrix-spec:matrix.org">#matrix-spec</a> room for brainstorming
and searching.</p>
<p>This step can help uncover any missed known issues and MSCs. I will also ask
others with a longer history in the project if I am missing anything.</p>
</li>
<li><p class="first">A brief <strong>competitive analysis</strong> is performed. Information can be gleaned from
technical blog posts and <span class="caps">API</span> documentation. I consider not just competing
<em>products</em>, but also investigate if others have solved similar <em>technical</em>
problems. Other protocols are worth checking (e.g. <span class="caps">IRC</span>, <span class="caps">XMPP</span>, <span class="caps">IMAP</span>).</p>
</li>
</ul>
<p>You can see an example of my research on <a class="reference external" href="https://patrick.cloke.us/posts/2023/01/05/matrix-read-receipts-and-notifications/">Matrix read receipts <span class="amp">&</span> notifications</a>.</p>
<p>Once I have compiled the above information, I jump into the <strong>current implementation</strong>
to ensure it roughly matches the specification. <a class="footnote-reference" href="#footnote-2" id="footnote-reference-2">[2]</a> I start considering what
protocol changes would solve the problem and are reasonable to implement. I find
it useful to write down all of my ideas, not just the one I think is best. <a class="footnote-reference" href="#footnote-3" id="footnote-reference-3">[3]</a></p>
<p>At this point I have:</p>
<ul class="simple">
<li>A problem statement</li>
<li>A bunch of background about the current protocol, other proposed solutions, etc.</li>
<li>A list of open questions</li>
<li>Rough ideas for proposed solutions</li>
</ul>
<p>The next step is to iterate with my colleagues: answer any open questions, check
that our product goals will be met, and seek agreement that we are designing a
buildable solution. <a class="footnote-reference" href="#footnote-4" id="footnote-reference-4">[4]</a></p>
<p><em>Finally</em>, I take the above and formalize it in into one or more Matrix Spec Changes.
At this point I’ll think about error conditions / responses, backwards compatibility,
security concerns, and any other parts of the full <span class="caps">MSC</span>. Once it is documented, I
make a pull request with the proposal and self-review it for loose ends and clarity.
I leave comments for any parts I am unsure about or want to open discussion on.</p>
<p>Then I ask me colleagues to read through it and wait for feedback from both them and
any interested community members. It can be useful to be in the <a class="reference external" href="https://matrix.to/#/#matrix-spec:matrix.org">#matrix-spec</a> room
as folks might want to discuss the proposal.</p>
<table class="docutils footnote" frame="void" id="footnote-1" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td>There’s a useful <a class="reference external" href="https://github.com/matrix-org/matrix-spec-proposals/blob/9b3f01b0193caa1e1c563cfc861ab021bcddcb2c/proposals/0000-proposal-template.md">proposal template</a> that I eventually use, but I do much
of this process before constraining myself by that.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-2" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-2">[2]</a></td><td>This consists of looking through code as well as just trying it out by
manually making <span class="caps">API</span> calls or understanding how APIs power product features.</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-3" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-3">[3]</a></td><td>Part of the <span class="caps">MSC</span> proposal is documenting alternatives (and why you didn’t
choose one of those). It is useful to brainstorm early before you’re set
in a decision!</td></tr>
</tbody>
</table>
<table class="docutils footnote" frame="void" id="footnote-4" rules="none">
<colgroup><col class="label" /><col /></colgroup>
<tbody valign="top">
<tr><td class="label"><a class="fn-backref" href="#footnote-reference-4">[4]</a></td><td>I usually do work with Matrix homeservers and am not as experienced with
clients. It is useful to bounce ideas off a client developer to understand
their needs.</td></tr>
</tbody>
</table>