aSoL moviepy system memory leak
I noticed that memory usage grows continuously while generating the subclips from the content descriptor. It maxes out at ~14G on a 16G system, and then stays there. Sometimes it’s killed by the kernel, but sometimes it’s fine. Which makes me think it’s some kind of “non-serious” memory leak where the system can reclaim memory if absolutely needed, but takes way too much by default. However:
I used Fil, a great memory profiler for python to track the memory usage. It looks like creating the subclips has a lot of overhead. Probably because it creates 1920x1080x3 images for each one, as the background. That makes 6MB of raw data for each image. So I can see how this would add up quickly.
I don’t understand why it’s able to keep running at maxed memory rather than crashing every time. But since the videos can generate just fine, I’ll leave this problem alone for now. If it becomes more of an issue I can refactor to render in stages, cleaning up memory after each, and concatenating the output at the end.
Right-click and “open image” to interact with the profile:
