Most of us probably ran into a bug in some application at one point. Sometimes it’s not very obvious, e.g. a random crash, while other times you may know that, if you for example perform certain five steps it will crash. But what do you do when you find an issue? Do you just avoid the crashing steps? Do you restart the app and keep working? Or do you put in some time and try and report the issue to the developer?
As a developer myself, detailed bug reports and active involvement in the bug report have been among the most valuable things in fixing bugs. As such, I can only highly recommend to send in bug reports whenever possible! Here is just some bug hunting story from last weeks.
Last week I opened the great video player VLC to playback some Opus audio file. Shortly after opening the application, it reported that there was a software update. I vaguely remembered that it already previously mentioned that, so I was a bit confused why I didn’t update then. Of course I accepted to download the new version and after 22% through the download process, VLC simply crashed. Aha! That’s why I didn’t update VLC last time. And indeed whenever you try to update VLC it would simply crash.
I remembered back when ShareX didn’t update when you clicked on the auto-update message and also remembered that they put up a notice on their website to manually update it. Certainly a broken update function in a quite popular media player would have been reported by now and thus I checked the website. To my surprise however there was no notice and searching a bit further I couldn’t find a report either. So I took things to the VideoLAN IRC channel and quickly found some active user.
After some digging we found the debug nightly builds. With that in hand, I thought it would be very easy to get a stack trace of the crash, but because this was a nightly build, the update feature points to a test server and while it actually triggered an update to be run, the build to update to wasn’t available anymore and thus nothing happened.
My next idea was to simply spoof the update server and change the update information. Having Python installed on my system I could just run python -m http.server 80 and it would launch a web server that served the files in the working directory. Then I added the update server to my hosts file, placed the update text file in the web server’s directory and … VLC’s update feature now simply threw an error. Luckily the debug build also output a lot of useful information in the console, so I saw that the update tests a signature file. This security feature prevents exactly this kind of “attack” where the DNS is spoofed and the fake server serves a malicious update. Having a file with a PGP signature ensures that only the ones with the private key can provide binaries with matching signatures.
So the new plan was to simply take the update file for the latest release version and its signature file. Finally, VLC crashed and I could get the stack trace. With an impressive list of 1637 entries, it became clear that the crash must be some sort of infinite recursion, which fills up the stack and finally leads to the crash. Since the problem seems to occur around lots of Qt functions, it may end up being less of an VLC issue and more one of Qt. A report was filed and now the waiting game beings.
Critique
In my books having a broken update function seems quite critical, as your users won’t get any security updates anymore, but need to go to the website and manually update VLC and many will probably just disable auto-update to not receive that message anymore. As such the reaction so far have been too low for my liking and I’m surprised that nobody else has experienced this issue. Then again VLC is an open source software maintained by developers in their free time, so it’s not a surprise that things take a bit longer.
The fact that the update feature doesn’t work due to outdated files in the nightly/development builds is a bit concerning. If it can’t be tested easily, then it simply won’t be tested at all and bugs like the one mentioned won’t be found during development.
VLC’s automatic reporting system doesn’t work (anymore). The FTP server the reports gets send to, doesn’t seem to exist anymore. In the consumer market direct reports to the team like mine are quite rare, so an automatic crash report system should work, so you actually see in case VLC suddenly keeps crashing all around the world.
Summary
As a software user, report bugs whenever you can and include as much information as possible!
As a software developer, make sure your features can be tested and that debug builds/symbols are available to get more detailed information.
I wanted to write some more on how to report bugs, but this post has already turned into a longer one, so I’ll reserve that for a dedicated entry.