VS3: visual synthesizer rev. 3

vs3 touchscreen rig with Roland V4 mixerVS3: realtime visual synthesis - a visual instrument based on the architecture/approach of a music synthesizer: original images are created in a performance environment by selecting sources (as one would select waveforms on a synthesizer) and applying some sort of method (algorithm) to combine and/or modulate them. Unlike a typical post-production environment, the VS3 system is designed to allow creative decisions to be made on-the-fly in a live performance. This approach is also more advanced than the typical 'VJ' paradigm, as the system doesn't just mix existing sources but actually generates original footage in real-time.

The original VS3 prototype system is a Shuttle Athlon 64 (Win2K) with dual touchscreens, midi and audio interfaces, and scan converter. The prototype software was developed in Director/Lingo.

Visual sources for the system include video, 2d images (with optional alpha channels), Flash, 3D models (OpenGL accelerated Shockwave 3D), text/fonts, color palettes. The system is controlled in live performance via touchscreen input and MIDI controllers/triggers (e.g., keyboard and percussion pads).

The prototype application has been used in a number of performances, in venues ranging from galleries and festivals to rock clubs.

Source and algorithm selections can be saved in 24 preset locations.On the control panel (the lower touchscreen in the picture, above), sources are selected using the 20 buttons on the left of the touchscreen (source type) and the list controls in the center (source select); algorithms are selected using the 20 buttons at the bottom; algorithm-specific parameter controls are at the right.

Screen two (the upper screen in the picture, above) is the output, scan-converted to NTSC and fed to the video mixer.

VS3 supports a variety of media types and algorithms, as illustrated below (control panel on left, sample output on right):

the 2d blend algorithm generates realtime montages using the selected 2d elements, including bitmap, Flash, and typographical assets. Modulation is sync'ed to an adjustable beat/bar counter.
User-definable frame capture settings allow the output feed to be captured to or combined with a 60-frame buffer, using a variety of masks, transfer modes, and positioning options. Current buffer contents are displayed as thumbnails
Alternate 2d montage algorithm and sources.
3D models and environments can be manipulated in realtime via midi data; for example, percussion hits can be translated into motion of objects in the 3D world
Video playback algorithm with 'scratch ribbon' controller.