5G is the new paradigm of telecommunication and wireless networking. The greatest difference between 4G to 5G is a massive increase in speed (bandwidth) due to updated network infrastructure. We imagine remote-controlled surgery, distributed sporting events, and even stable Networked Music Performances (NMP).

Playing music over the internet is exceptionally challenging because it requires really stable, high-quality and fast low-latency communication in a synchronous (real-time) environment. In fact, research tells us that the ideal roundtrip audio latency (audio from you, to your distant partner, and back again) for realtime NMPs is between 25-50ms.

For perspective - here is what 30ms of audio delay sounds like from left to right ear:

In 2022, my colleague (Stefano Faciani) and I got in touch with Telenor Research to explore the feasibility of conducting NMPs over 5G. During spring, we were able to schedule and complete a series of audio experiments to test the audio latency on private and commercial 5G networks, using our specialized equipment. In this post, I present our experiments and results in detail.

Contents

1. Test sites and Equipment
2. 5G Experiments
   1. Network Coverage
   2. Signal Quality
   3. Roundtrip Latency
3. Results
4. Summary

---

Test sites and Equipment

To connect, Telenor provided us with a pair of Huawei H138-380 CPE Pro 3 5G Routers. These modems were pre-configured to access specific Access-Point-Names (APNs) based on our location to enable faster packet routing and better p2p connectivity. Becuase of this, we had to carefully plan our testing locations in advance.

Our first experiment was on a first-generation commercial 5G network at the Musicology Department (IMV), UiO. A little later, we got the opportunity to run the same tests on a private 5G Network-on-Wheels (5GNoW) at the Terningmoen Army Base in Elverum. The 5GNoW is apparently only used for specialized field experiments and was therefore a chance for us to experiment on near optimal conditions.

To transmit the audio over 5G, we used a pair of custom-built NMP hardware kits from IMV. These hardware bundles are designed specifically for ultra-low-latency AV transmission making them perfect for NMP-related education and research. The kits use high-end RME PCI audio interfaces connected to Behringer ADAT converters for XLR inn/out.

Finally, we needed lightweight transmission software that was agnostic about network conditions, could support uncompressed formats, and worked well cross-platform. Here, JackTrip from the Jack-audio connection kit seemed to fit the bill. For the record, we also did several experiments with the high-quality audio-video transmission software called Lola. Unfortunatley, it seems that Lola is+currently uncompatible with the 5G protocol due to its high packet transmission rate, sending fixed-size packets as small as 1kb.

5G Experiments

Having lightning-fast communication is always a high priority for any NMP. However, low latency is not synonymous with high-quality transmissions with minimal unwanted artifacts (glitches) and dropouts. Knowing this, we ran experiments to find the best possible tradeoff between stability, quality, and latency, instead of just benchmarking various metrics. We were more intersted in finding suitable conditions where musical activity actually could occur.

In total, we did three tests at each location that measured the network coverage, signal quality and audio latency.

Network Coverage

By using a combination of iPerf networking utilities, measurements from the Huawei routers' software, Telenor’s online coverage map, and Ookla's online speedtester, we were able to make network bandwidth and coverage estimates throughout the experiments.

These metrics made us confident that our transmission load did not exceed the capacity of the networks.

Signal Quality

To locate the quality and stability sweetspots for our audio transmission, we created a network loopback systems between our two NMP kits, as depicted in the diagram below. By listening and inspecting the audio we sent across these loopback systems (preferably a single mid-range sine tones), we could monitor the audio quality of our connections in real-time.

If we experienced any unwanted artifacts, glitches, or drops in the audio, we just adjusted the buffersize of our audio interfaces (and in jacktrip) to the lowest possible configuration where we found a stable transmission over time.

Roundtrip Latency

With the software and hardware parameters fine-tuned, we used the same loopback system to measure the digital and analog audio roundtrip time (RTT).

With digital RTT, we refer to the time it takes audio to travel from software to software (or PC to PC) and back again. With this method, we bypass and control for the latency induced by our external soundcards and mixer hardware. By including this additional hardware, we measure the analog RTT. This measurement is the most relevant metric for NMP musicians because it captures the entire audio chain.

To achieve higher accuracy and certainty in our analog RTT measurements, we used a secondary pc+soundcard setup, as shown in the diagram above. With this configuration, we bypass and control unwanted latency compensation mechanisms and other side effects caused by our hardware or software modules.

Results

Commercial 5G

Since the 5G reception inside IMV was poor, we decided to place the routers outside, hoping it would generate better coverage and boost overall performance. Just outside IMV's main auditorium (at this location), we measured a stable 75% 5G coverage with 60Mbps bandwidth.

When monitoring the audio quality on the commercial 5G, we experienced aweful results with buffersizes below 512 on our audio interfaces, sampling at 48khz. Although processing 512 samples at 48Khz only takes 10ms, our audio has to pass through these interfaces 4x times(!) during every analog RTT round. This quickly adds up.

We measured the average digital and analog RTT to be 110ms and 165ms (respectively) on the commercial 5G network, using a buffersize of 512, sampled at 48khz.

For perspective - here is 165ms audio delay from left to right ear:

Private 5G

During our second round of testing, time was of the essence. We only had about 3-4 hours to set up our equipment and run tests on the private 5GNoW in Elverum. From the very start, we ran into unexpected connectivity and bandwidth issues but we managed to get a stable 14-15Mbps link which is more than enough for transmitting stereo uncompressed audio.

On the private 5G, we were able to get better overall results overall, meaning we got better audio quality using a lower buffersize on our audio interfaces. We measured an average digital and analog RTT of 60ms and 74ms (respectively), using a buffersize of 256, sampled at 48khz.

For perspective - here is 74ms audio delay from left to right ear:

Although this was impressive, the configuration rendered borderline audio quality that would be unpleasant in the long run. When using a buffersize of 512, we got much more stable audio with 90-100ms digital RTT (similar to the commercial 5G, only slightly faster) and an analog RTT at about 140ms.

Summary

In early 2022, me and a UiO colleague investigated the feasibility of conducting NMPs over commercial and private 5G networks in collaboration with Telenor Research. We measured the stability, quality, and latency of transferring uncompressed audio over two 5G networks with high-end hardware and software utilities. When accepting borderline conditions on a private 5G network (5GNoW), we managed to push the audio RTT latency down to 75ms. However, in more realistic conditions on a first-generation commercial 5G, we achieved an analog RTT audio latency of 165ms.

Results in table form:

Compared with audio RTT benchmarks mentioned in the introduction (between 25-50ms), our 5G test results were not particularly promising. I believe we could conduct successful NMPs over 5G if we could get the audio analog RTT down between 50-70ms. Considering that we had unfortunate testing conditions in Elverum, there is reason to be optimistic about achieving better RTT audio scores at a later stage.

Also, lower latency should be possible when the Ultra-Reliable Low Latency Communications (URLLC) feature of 5G is implemented on future Telenor networks.