Note: this guide was written by Jim, author of OBS Studio.
The author’s style is saved in this document.
So you’re dropping frames, and you don’t know why. You think “Hmm, I don’t understand why this is happening, I see other people reporting it, so I suspect my streaming program is at fault!”
So, let’s examine what frame drops are and why they happen. TL;DR at the bottom, but please don’t skip to the TL;DR because I spent way too much time writing this when I probably should have been coding.
Your streaming program creates video/audio frames, compresses them, and sends them out right after they’re compressed, at whatever framerate you set your streaming program to. To send them, it calls a single function called “send” for each compressed audio/video packet as soon as they’re created. This “send” function is socket function implemented by the operating system itself (it’s a Microsoft/Apple/Linux function — not a function created by your streaming program). If your connection has no problems, that “send” function will finish right away, with almost no delay, and you will not get frame drops.
However, if that “send” function takes too long (due to network congestion of some sort), excess audio/video packets that you’re trying to send out will start to back up in a queue so to speak, sitting there waiting to be sent off. Meanwhile, you’re encoding data at however many frames per second, and they can’t just sit there and wait for that function to finish on end forever until your memory runs out, eventually something’s got to give. When they get to a specific point (about 700ish milliseconds worth of backed up data), it will trigger the phenomena you know as “dropping frames”, where it’s forced to dump video frames for a bit to clear up that backed up packet queue. This is why your stream will experience what appears to be video stuttering when that happens.
In the case of some random spontaneous network congestion, dropping frames after a specific amount of congestion is both necessary and important to ensure stream stability and minimize stream latency.
Why is it important for stability?
Why is it important for latency?
So to ensure network stability and minimize latency, after about 700ish milliseconds of backed up audio/video packets, it decides to start dropping frames. If it were lower, it would make it a bit too sensitive to congestion. If it were higher, it could cause stream delay or latency issues, so it tries to keep it at a reasonable number.
If it is happening consistently, it’s typically a sign that your bitrate is too high for you to be able to maintain with stability.
So frame drops do not happen because of some bug or error in the program, or some problem with the program — they literally happen only because the the program calls the “send” function, the send function takes too long, and queued audio/video packets start piling up waiting to be sent out, until they reach a critical mass in which they need to be chopped out to maintain stream latency and stability.
In other words, it only happens because of network congestion. Streaming programs do not drop frames for any other reason than network congestion.
What causes that network congestion?
Unstable connection to the server, poor/unstable ISP or connection to ISP are most common, in more rare cases it can be bad routers or modems, in very rare cases it can be other system issues or certain very uncommon network devices or drivers.
Note the key words above: “maintain with stability”. The rate in which you send out audio/video packets must be consistent and stable to ensure both latency and further stability, as stated above. This does not necessarily mean that your internet connection is unable to upload at a higher rate, it only means that it’s unable to upload at the set rate with stability. You could hypothetically have like 30mb/s up and only be able to maintain 3mb/s at a fully stable rate. Streaming programs need to be able to upload their data at a consistent, stable rate, otherwise there will be congestion, which can cause frame drops.
Again, there are many reasons why you can have network congestion, which is specifically why we created an entire section in our help for it:
So, if you’ve actually managed to read this far, I hope you’ve been enlightened as to what frame drops are, why they happen, why you hear other people report them, and why, despite that fact, they are very likely not the fault of your streaming program.
TL;DR? A single function of your operating system, “send” is called with your audio/video packets, and if it takes too long (due to network congestion and only network congestion), audio/video packets pile up (get queued), and after a certain point the program is forced to dump data to maintain stream latency and stability. It is quite literally not the streaming program’s fault. It means you have network congestion for whatever reason.
I wrote all this for you. Yep, you on twitter there. For some reason I guess I just sort of snapped and wrote this entire thing just now. I did it because I literally cannot control the send function, because network congestion is not my fault, because there are a million and one reasons why network congestion can occur, and of course because I love you.