In late 2022, my good friend and artist Henrik Nordahl asked me to collaborate with him on a sound sculpture entitled "01101000 01100101 01101100 01101100 01101111 00100000 01110111 01101111 01110010 01101100 01100100 (hello world)", for his MFA thesis project at the Parsons, National Academy of Arts, NYC. In computer science, "Hello world" programs are often the first thing you learn to write in a given language.

My contributions were purely to the audio of the sculpture. My mission was to ensure that the structure sonically reacted to nearby wifi traffic using Bluetooth transducers in combination with MaxMSP (Max) and Python software.

Hello world was exhibited in April 2023 at The Sheila Johnson Design Center, Parsons, NYC.

Contents

1. Materials and Family
2. Hardware and Software
3. Sonification
4. Links and Resources

Materials and Family

Hello world is one of several sculptures in a series that interpret Nature and Architecture, such as Gjallarhorn and Frequencies. The sculptures are made from burned drywall paper tape, methylcellulose, aluminum flashing, concrete and steel rods where the drywall paper tape rolls have been pulled from the center to create cones that are repeated in iterations.

Overall, the works focus on the multiplicity of one form and are the results of Nordahl's research and process-led experimentation into the behavior of these materials.

Hardware and Software

To make Hello world physical vibrate and create unique sounds, four small Bluetooth transducers (speaker drivers) were installed on custom membranes at the connection points between the drypaper cones and the steel section in the middle of the structure. This turned the entire structure into speaker elements that we could control from a nearby laptop. With wireless audio transmission, we did not need to purchase cables, amps and other audio gear, nor find a way to hide these items inside the sculpture. And since we did not care too much about latency, the Bluetooth devices seemed like the obvious choice.

Sonifying network traffic first requires a method to extract packets from your network. With Python, the go-to method for network sniffing is to use the pyshark library, a wrapper for something called tshark, allowing network packet parsing using WireShark dissectors. Together with python-osc, a library for OSC transmission, I was able to use pyshark to capture wifi packets being sent from our local machine out to the world. Ideally, we would've liked to get all messages on the network, from any person to anywhere, but Wireshark does not allow this behavior (for obvious safety reasons). After the packets were captured, desired information was encapsulated into OSC messages and sent to Max internally for audio processing.

Below is a code example of how you can accomplish this WiFi capture and simple OSC messaging with Python.

from pythonosc import udp_client
import argparse
import pyshark

"""
Capture and process wifi packets from host_ip to the world.
Send source IP, desitination IP and packet length to localhost port 8888.
"""

host_ip = "192.168.86.39" # Local machine IP
max_port = 8888 # Internal IP (localhost) port that Max will receive OSC messages from python on.
max_ip = "127.0.0.1"
prev_pkt_len = "" # previous packet length

if __name__ == "__main__":
    client = udp_client.SimpleUDPClient(max_ip, max_port)  # define a simple UDP client for sending OSC messages to Max
    print(f"Sending OSC packets to {max_ip} on port {max_port}.")
    print(f"Press Ctrl+C to stop.")

    cap = pyshark.LiveCapture(interface="Wi-Fi")  # Start capturing WiFi network packets

    prev_pkt_len = ""
    for pkt in cap: # For every packet
        if (prev_pkt_len != pkt.length): # Prevent identical connections from clogging our program
            if "IP" in pkt: # If the packet has a property called IP
                protocol = pkt.highest_layer
                source = pkt.ip.src
                dest = pkt.ip.dst
                prev_pkt_len = pkt.length
                try:
                    if source == ip_host:  # If the source IP of the packet matches my ip_host
                        client.send_message("/packet_len", int(pkt.length)) # send stuff to Max
                        client.send_message('/src_ip', source)
                        client.send_message("/dst_ip", dest)
                        # etc ...
                        # etc ...
                except:
                    print("Error while sending OSC..")
                    break

Sonification

In Max, OSC messages from Python are received and unpacked before being used for sonification and effect processing. Central to the synthesis engine is a process that turns every IP address into a series of binary numbers. The binary numbers then controlled important aspects of the sound generation, such as oscillator onset, sustain, gain and other envelope properties.

But not all sound properties were controlled automatically by OSC packets. In fact, several parameters required manual adjustments. This enabled us to fine-tune central aspects of the sonification to cater to different physical spaces with varying acoustic qualities. The combination of manual fine-tuning and the sporadic uncontrollable nature of the web traffic made the sonification unique for every iteration.

Here is a short excerpt of the audio from the installation:

It was important for us that this wifi sonification concept was quickly grasped by audience members without us having to compromise too much on artistic qualities. We wanted the audio aesthetic and sonification to reflect the hidden and sporadic qualities of wireless networks. The result was a fairly basic synthesis engine that mostly focused on low-frequency sounds, with some added mysterious/ghostly qualities.

The wifi traffic rate, packet lengths, and IP addresses all controlled aspects of the audio. For instance, the traffic rate was directly related to the intensity and density of the sound (a technique often known as audification of data). There were also links between network packet lengths and audio amplitude, and between IP address and frequency content.

Links and Resources

To learn more about how to sonify and capture web traffic using Python, I recommend two great blog posts by my good friend Jackson Goode. Here, he explains the process in great detail:

• Soniweb: An Experiment in Web Traffic Sonification and Ambisonics
• Soniweb: Epilogue

Also, to learn more about Henrik Nordahl and his awesome artworks, check him out online at his:

• Official Website
• Instagram