Thunderbird 60, RSS feeds, HTTP 429, and `close_connection`
So, Ubuntu finally pushed Thunderbird 60 to the repos.
Which was great, because it broke so much stuff for me.
Setting aside the new icon theme (unusable due to clashing colors with a lightweight theme), the most major problem was a behavior change in how the RSS feeds were updated.
I had two groups of feeds, one for news / security notices etc. which is configured to update every 30 minutes, and another for blogs / leisure reading, updating every 24 hours. This new version decided for some reason to change them all to 100 minutes…
But that’s not all. I had a website in my daily feeds that provides multiple feeds, i.e. /rss?user=alice, /rss?user=bob, /rss?search=keyword, etc. They have also configured rate limiting in their server, which means that every 5 or so request would get an HTTP 429 (Too Many Requests). Previously this wouldn’t be a problem – due to network latency or other factors, it would probably not be the same feed getting the 429 every time. So perhaps I’ll see it a few days late, but no big deal.
But Thunderbird 60 decided that when a feed receives a network error, it should “pause” a.k.a. disable the feed, meaning that it would not check for updates for that feed ever again. Like, what?
So I figured the easiest way to “fix” this, would be to just write an HTTP proxy running on localhost, which would throttle the requests (ensure at least 100ms between each) to the server. This was done quite easily using Python.
Half an hour of coding and reading the docs later, I changed the URLs
of the feeds to point to my proxy, and tried again. It didn’t work.
Specifically, only the first 4 feeds worked, and the later feeds all
timed out for some reason. It turned out that the BaseHTTPRequestHandler
class doesn’t automatically close connections, and I never realised it
because I’ve almost always run it multi-threaded.