SAN: Audio
Table of Contents
1 Introduction
SAN is a construction system for architectural structures that interact with the environment visually and auditory. This document discusses the auditory part: How can an architectural structure be a musical instrument?
Figure 1: Dodecahedron built by two little girls at the end of Maker Faire Rome. Notice the visualization in the background: The structure understands how it is assembled.
2 Approaches
What follows is a discussion of vairous approaches to turning the structure into sound or even have it generate music.
2.1 Sound of molecular vibrations
2.1.1 Idea
Treat the structure as a molecule made up of carbon atoms. Turn molecular vibrations into a sound.
Like the spherical building elements of SAN, carbon atoms have four bonds arranged in tetrahedral symmetry. Molecules can be made to vibrate. These molecular vibrations have different frequencies which together make up a spectrum:
Figure 2: Molecule with six atoms in one of its 12 possible vibration modes
2.1.2 Examples
Scientists and musicians have explored turning molecular spectra into sound:
Molecular Sounds (ejectamenta), a keyboard with each key mapped to a different molecule and its sound:
Figure 3: Screenshot
Molecular composition based on NMR spectra by Falk Morawitz:
See also his 2016 talk Sonification of nuclear magnetic resonance in ½ spin systems: An overview at ZKM Karlsruhe.
2.1.3 Calculation of vibrations
There are many ways to calculate molecular vibrations. Simple estimates can be done on a piece of paper. Relatively accurate results are given by the harmonic approximation where the bonds between atoms are treated as harmonic oscillators (see the book Molecular Vibrations by Wilson, Decius, and Cross, 1955). Then there are quantum mechanical methods that are ever more accurate and complex.
To get an idea concerning how different spectra look, the software Gaussian 09 can be used:
2.2 Collaborative modular synthesizer
2.2.1 Idea
Each node is module in a big modular synthesizer:
- Using a web UI, users can tweak the properties of individual modules.
- Reconnecting nodes updates how the modular synthesizer is patched.
- Master output: There is one root node in the network, which is where the structure is connected to the computer. This root node serves as the master output.
2.2.2 A module
Each node has several child nodes and one parent node. By default each node acts as an oscillator possibly at a base frequency detuned depending on the node’s position. All oscillators are simply added. This makes the structure make sound by default without any user interaction.
- Inputs: child nodes
- Outputs: parent nodes, or root node (master)
Figure 4: Screenshot of a module connected to master, June 12 2019
2.2.3 Possible extensions
It could be interesting to give the user a selection of inputs:
- outputs from the neighboring child nodes
- oscillator
- position of the node in space: x, y, z
- color of node
There could be various predefined properties, such as pitch or the duration of a repeating loop, that the user can attach inputs to. Also the node could be made freely programmable, perhaps globally for all nodes at once.
In addition to the sliders, users could be given the option to tap on number and modify it using text entry.
2.2.4 Examples for collaborative synths
- Popular Synth by Carlo Cattano (host of Sonic Code Sessions in Berlin)
- Socket Synthesizer presented by Steve Kinney at JSConf US 2015
2.3 Sequencer
Idea: Walk through the molecule, atom by node. Trigger an action at each step, such as playing a note associated with node.
Open question: Is there a standard way to walk the atoms of a molecule?
2.4 Adjacency matrix
The adjaceny matrix is visible in the main interface. Create sound based on that.
2.5 Neurons
:CUSTOMID: neurons
Treat SAN as a neural network. Investigate how pulses are passed around in a biological neural network.
(idea of Brian / Sonic Code Sessions 02/2019)
2.6 Mood
Ivan Palmegiani pointed out that Science Hack Lab Berlin on June 8 2019 that there is research that certain sound waves affect mood.
This means that mood can be changed according to the structure built.
2.7 Cognimates extension
Stafania suggested on June 17 CEST at Betahaus in Berlin to create an extension for SAN for her Cognimates AI teaching platform. She outlined that it may be possible to train the AI to create music based on certain constellations.