VideoAnalysis a standalone beginners tool for analyzing basic music-related body motion in videos, one of the larger applications part of the Musical Gestures Toolbox for MaxMSP created by Alexander Refsum Jensenius for the RITMO FourMS lab at the University of Oslo in the early 2000s.

In 2019, I was hired to update and refurbish VideoAnalysis together with Balint Lackzo. This post takes a quick look at VideoAnalysis v2.1 and shares some its key features.

Contents

1. How To Use It
2. Key Features
   1. Batch Processing
   2. Crop Trim and Skip
   3. Motion Visualizations
   4. Export Options
3. Download Link and Resources

---

How To Use It

VideoAnalysis (VA) is an asynchronous analysis software, designed to locate movement in pre-recorded videos. After uploading video content, the goal is to isolate motion using various pre-processing and visualization techniques. To guide the user in this process, VA includes an adjustable display window where you can view the processed videos in real-time, and a range of export options for extracting motion features and images for further analysis.

For documentation and guides, visit the official VideAnalysis Wiki. You can also get a detailed overview of all software operations from the app itself, under "help".

Key Features

VA sports many advanced video and display options. Let's look closer at some of the unique import and export options, pre-processing features, and visualizations.

Batch Processing

VA supports batch video processing for those interested in working with multiple video files or creating larger datasets with motion data. In practice, batch processing enables users to import folders full of videos and process them all in one go.

Also, VA includes export options designed to create better batch-processing workflows. For example, you can use the same processing settings for all videos or choose to configure custom settings for each file. Also, to ensure that users can continue their work later and use the same settings for multiple batches, it's possible to save and recall presets when needed.

Crop Trim and Skip

The most useful pre-processing features in VA are cropping, skipping, and trimming. When doing motion video analysis, there is no need for high resolutions and frame rates to get high-quality data. Therefore, it's wise to reduce video dimensions, skip frames, and trim length to minimize CPU load and processing time.

Cropping, trimming, and skipping features are available from the Processing settings left of the Source Display. To crop videos, simply highlight a desired bounding box region in the Source Display. You can monitor the results in the detachable UI display in real-time.

Skipping values decide which frames are processed. With a skipping value of 1, every other frame is skipped, meaning frames 1, 3, 5, etc. are processed. Skipping like this is powerful because it cuts the video length in half while preserving most of its content, reducing processing time by 50%(!).

Finally, trimming can help isolate interesting sections for analysis, removing unnecessary content. Trimming can be adjusted numerically from the Processing UI section or more dynamically by adjusting the yellow timeline under the Source Display playbar.

Motion Visualizations

Despite being optimized for non-real-time use, VideoAnaysis can create many good real-time motion visualizations, such as motion images/videos, the centroid of motion, and motiongrams.

Motion images are generated by calculating the difference between current and previous frames in videos. The result is an image that shows only the difference between the previous and current frames. A motion video is simply a series of motion images.

The quality of the motion videos depends on the quality of the source video. Small changes in the lighting, camera motion, compression artifacts, etc. will influence the final image. However, visual "noise" can be removed by applying filters that remove pixel values below certain thresholds. In VideoAnalysis, these settings are found in the Filter UI section.

The centroid of motion (CoM), and area of motion (AoM), are features that can be extracted from motion videos. The CoM and AoM are used to illustrate where motion occurs in a video, as well as the spatial displacement of motion over time.

In the centroid video in the VA display window, the CoM is visualized by a series of red circles (snake). From the UI, It's possible to customize the length, radius, and threshold of the snake to better track and visualize the motion history.

Finally, with motion videos we can also create motiongrams; images that represent motion over time. These images are good for getting a general sense of the spatiotemporal distribution of videos. They can also be exported and viewed directly from the VA UI.

Motiongrams are composed of columns of pixels, each representing a unique motion video frame. With each motion video frame, the mean values of every row are gathered and used to produce single pixel columns (one-dimensional matrices). These columns are then printed across a canvas (left to right) to create the motiongram effect.

Export Options

VA can export lots of motion features and images for further analysis. By default, the software exports a pre-selected bundle of data. You can customize this selection with the export options in the navbar. Here is a complete list of the available data:

• Quantitative .csv analysis data, including Quantity of motion (QoM), Height of motion (HoM), Width of motion (WoM), Area of motion (AoM), Centroid of motion (CoM), and much more.
• General log about the analyzed file and the analysis in a .txt file.
• Basic statistics in a .txt file.
• Average image, based on taking the average of all images in the video stream.
• Motion average image, based on taking the average of all motion images in the video stream.
• Horizontal motiongram, based on averaging over each column in each motion image.
• Horizontal motiongram, based on averaging over each column in each motion image.
• Motion history image: displaying motion average image (mavg) on top of the first video frame.
• Motion history image: displaying motion average image (mavg) on top of the last video frame.

Download Link and Resources

VideoAnalysis is open source and available on both OSX and WIN. The software can be downloaded from the FourMS GitHub page. If you want to contribute, you are free to do so. Simply fork the VideoAnalysis repo, and make a pull request for us to review.

• Jensenius, Alexander Refsum (2005). Developing Tools for Studying Musical Gestures within the Max/MSP/Jitter Environment. Proceedings of the International Computer Music Conference, p. 282-285. Available at: https://www.duo.uio.no/handle/10852/26907

• VideoAnalysis has been acknowledged by Cycling74 (maxMSP creators) on their official homepage.