When building an RTC-enabled video application, we just want it to work. But what does that really mean when user experience is so subjective? Let's look at open-source tools that can be used to automate objective testing of video quality and how encoders, networks, simulcast, and SVC affect it.
Our LiveSwitch and IceLink customers often ask us how to tune things for the best possible user experience. One of our primary objectives at Frozen Mountain is to eliminate the need to ask this question. We want our customers focused on building great applications, not worrying about bandwidth estimation, encoder configuration, or simulcast setup. As we develop new features and algorithms to assist with delivering a great end-user experience, we also, in parallel, develop new testing methodology that proves it works as expected. Since manual QA/testing is expensive and prone to subjectivity, we use open-source tools to automate objective video quality estimation. This allows us to test far more scenarios and with greater repeatability and consistency than we would be able to otherwise. We would like to share some of our learnings with the community, along with a live demonstration or two if time permits.