Handling Infinite Loops with Backpressure
amasad (1224)

In 2017 we published a post on how we handle infinite loops at Repl.it. The post was very popular and went on to inspire similar work at React.js and other projects.

Our approach then was a mixture of code instrumentation and throttling. Throttling, however, turned out to be tricky. We used to throttle all output indiscriminately at a certain rate (20 messages / second), essentially taxing all output to reduce the chance of lock-up in the most underpowered of our clients. As our users' applications got more complex and ambitious it became apparent that this handicapped our development environment and restricted a potential for high output applications.

Consider a reimplementation of this very forum in the terminal by @mat1 that needs to respond to mouse events and needs to (re)paint large portions of the screen:

Another complication was the introduction of Multiplayer: Now every program had multiple clients receiving output and sending input. So how do you build a system that works well for fast clients and degrades gracefully for slow clients?

Enter Backpressure

To understand backpressure consider a scenario where you have a producer sending data to a consumer for processing. Often times the consumer is slower than the producer and might get flooded with messages before getting a chance at processing. At which point the consumer typically crashes. Backpressure is the mechanism in which the consumer tells the producer to slow the heck down!

Backpressure explained — the flow of data through software

In our case the producer is the your program and the consumer is your browser. If your program had an infinite loop with a print in it, it will be sending data at a rate which your browser will normally crash. However, with backpressure we made it so that your program will blocked until the write goes all the way from the container, through our servers, network, your router, browser and to your terminal! That way your program writes at the perfect speed for you.

But this presents a new problem -- what if you want to interact with your program by providing keyboard or mouse input? If it's blocked then it can't handle your input. Enter bursts:


Bursts are periods for which the program can write freely without being constrained by the consumer. Every program starts out with a burst so that the initial set of output gets sent out to the client. Additional bursts happen with input. Every time you interact with your application you put it into burst mode so that it can catch up and be able to handle your input.

That way interactive programs remains interactive.


Now imagine coding with a group friends and one of them has a wooden laptop that can barely handle much output. At first the program will slow down to accommodate said friend. But say your friend's computer crashes and never processes any output ever -- what should we do? Sadly, we have to sacrifice the one to save the many: Simply disconnect your friend with the wooden computer.

Your friend's computer.

Try it

while True:
  print("hello world")


You are viewing a single comment. View All