{"id":2692,"date":"2016-10-19T03:27:41","date_gmt":"2016-10-19T03:27:41","guid":{"rendered":"https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/?page_id=2692"},"modified":"2016-12-05T17:57:23","modified_gmt":"2016-12-05T17:57:23","slug":"software-instruments","status":"publish","type":"page","link":"https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/software-instruments\/","title":{"rendered":"Software Instruments"},"content":{"rendered":"<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-large wp-image-2693\" src=\"https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-content\/uploads\/2016\/10\/Screen-Shot-2016-10-18-at-10.44.07-PM-1024x281.png\" alt=\"Screen Shot 2016-10-18 at 10.44.07 PM\" width=\"840\" height=\"231\" srcset=\"https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-content\/uploads\/2016\/10\/Screen-Shot-2016-10-18-at-10.44.07-PM-1024x281.png 1024w, https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-content\/uploads\/2016\/10\/Screen-Shot-2016-10-18-at-10.44.07-PM-300x82.png 300w, https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-content\/uploads\/2016\/10\/Screen-Shot-2016-10-18-at-10.44.07-PM-768x211.png 768w, https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-content\/uploads\/2016\/10\/Screen-Shot-2016-10-18-at-10.44.07-PM-1200x330.png 1200w, https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-content\/uploads\/2016\/10\/Screen-Shot-2016-10-18-at-10.44.07-PM.png 1340w\" sizes=\"auto, (max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px\" \/><\/p>\n<p>Synesthesia is a condition where one of the senses is coupled with another, where people report of hearing images, of words having colors or of tasting sounds. The translation between senses has been an ongoing source of inspiration for creators for centuries with responses being instruments such as the color organ where a note triggers a visual experience. Even prior to the invention of electricity, there were have been scores of attempts to create color organs.  Some had popups, opened curtains to show colored panels or later with electricity, turned on projected colored lights when a key was struck.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-large wp-image-2714\" src=\"https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-content\/uploads\/2016\/10\/Screen-Shot-2016-10-19-at-7.07.12-AM-1024x739.png\" alt=\"Screen Shot 2016-10-19 at 7.07.12 AM\" width=\"840\" height=\"606\" srcset=\"https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-content\/uploads\/2016\/10\/Screen-Shot-2016-10-19-at-7.07.12-AM-1024x739.png 1024w, https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-content\/uploads\/2016\/10\/Screen-Shot-2016-10-19-at-7.07.12-AM-300x216.png 300w, https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-content\/uploads\/2016\/10\/Screen-Shot-2016-10-19-at-7.07.12-AM-768x554.png 768w, https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-content\/uploads\/2016\/10\/Screen-Shot-2016-10-19-at-7.07.12-AM.png 1184w\" sizes=\"auto, (max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px\" \/><\/p>\n<p>Along with these elaborate inventions, various artists, scientists and thinkers have sought\u00a0correspondences between the elements of color (hue, saturation and value) and those of sound (pitch, amplitude, and tone color) developing color scales such as the one as shown here.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-2715\" src=\"https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-content\/uploads\/2016\/10\/colorScales.gif\" alt=\"colorScales\" width=\"601\" height=\"348\" \/><\/p>\n<p>Read more on the history of visual music machines here:<\/p>\n<ul>\n<li><a href=\"http:\/\/www.awn.com\/mag\/issue2.1\/articles\/moritz2.1.html\">The Dream of Color Music and the Machines that made it possible.<\/a><\/li>\n<li><a href=\"http:\/\/rhythmiclight.com\/archives\/timeline.html\">A timeline\u00a0of color organs<\/a><\/li>\n<\/ul>\n<p>Painters such as Kandinsky have also been captivated by the translation between the visual and the sonic domains. His paintings have been described as having visual &#8220;chords&#8221;. Working in the early 1900s he often described his works as compositions, symphonies and harmonies.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-2695\" src=\"https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-content\/uploads\/2016\/10\/kandinsky_wwi.jpg\" alt=\"kandinsky_wwi\" width=\"800\" height=\"534\" srcset=\"https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-content\/uploads\/2016\/10\/kandinsky_wwi.jpg 800w, https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-content\/uploads\/2016\/10\/kandinsky_wwi-300x200.jpg 300w, https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-content\/uploads\/2016\/10\/kandinsky_wwi-768x513.jpg 768w\" sizes=\"auto, (max-width: 800px) 85vw, 800px\" \/><\/p>\n<p>Today this exploration continues in the medium of software and sound visualizers have become ubiquitous, shipping with almost every music player and app.<\/p>\n<p><strong>Sound visualization\/synthesis works:<\/strong><\/p>\n<ul>\n<li><a href=\"http:\/\/www.thisispersona.com\/joshue-ott-kenneth-kirschner\/\">Joshue Ott + Kenneth Kirschner release composition systems as apps that combine visuals and sound.<\/a><br \/>\n<iframe loading=\"lazy\" src=\"https:\/\/www.youtube.com\/embed\/BR1Ernyv7-w\" width=\"640\" height=\"400\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/li>\n<li>Or the other way.. \u00a0<a href=\"http:\/\/www.zachdunham.com\/found-rhythm\">Zack Dunham<\/a> makes sound from visual patterns in Found Rhythm.<br \/>\n<iframe loading=\"lazy\" src=\"https:\/\/player.vimeo.com\/video\/152839098\" width=\"640\" height=\"400\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/li>\n<li>Amanda Ghassaei makes interactive sound sculpture that are synthesizers that respond to ambient light. <iframe loading=\"lazy\" src=\"https:\/\/player.vimeo.com\/video\/38626867\" width=\"640\" height=\"360\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/li>\n<\/ul>\n<p><strong>Software Instruments:<\/strong><\/p>\n<p>Software\u00a0instruments can also richly\u00a0tie visual sensation to the auditory or musical. An instrument however is a tool and demands interactivity. It must be able to be played and when manipulated, yield a predictable result.<\/p>\n<ul>\n<li><a href=\"http:\/\/flong.com\/projects\/miw\/\">The Manual Input Work Station<\/a>, 2004 by Golan Levin and Zach Lieberman is an instrument that interprets hand gestures and finger movements across the interface of an augmented overhead projector.<\/li>\n<p><iframe loading=\"lazy\" width=\"560\" height=\"315\" src=\"https:\/\/www.youtube.com\/embed\/3paLKLZbRY4\" frameborder=\"0\" allowfullscreen><\/iframe><\/p>\n<li><a href=\"http:\/\/sws.cc\/index.html\">Sonic Wire Sculpture<\/a> by Amit Pitaru creates sound via drawing in 3D space.<\/li>\n<\/ul>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-2690\" src=\"https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-content\/uploads\/2015\/01\/amit.gif\" alt=\"amit\" width=\"472\" height=\"316\" \/><\/p>\n<ul>\n<li>Keyboard instruments: A classic of this genre of work is the keyboard instrument\u00a0with\u00a0<a href=\"http:\/\/patatap.com\/\">Patatap<\/a>\u00a0being a particularly refined example. As you try it, concentrate of the transitions and interaction design.<\/li>\n<p><iframe loading=\"lazy\" src=\"https:\/\/player.vimeo.com\/video\/95057507?color=911201\" width=\"640\" height=\"400\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<li><a href=\"https:\/\/runhello.com\/p\/396\">Drum Circle<\/a> by <a href=\"https:\/\/runhello.com\/\">Andi Mcc <\/a>that is comprised of a visual system for composition.<\/li>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-content\/uploads\/2016\/10\/dc1.png\" alt=\"dc1\" width=\"320\" height=\"480\" class=\"alignnone size-full wp-image-2761\" srcset=\"https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-content\/uploads\/2016\/10\/dc1.png 320w, https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-content\/uploads\/2016\/10\/dc1-200x300.png 200w\" sizes=\"auto, (max-width: 320px) 85vw, 320px\" \/><\/p>\n<li>SUFI PLUG INS is an interdisciplinary project by Jace Clayton and is a free suite of Ableton add-ons that are based on nonwestern notions of sound &#038; a poetic interface. This brief explanation video shows them in action. <a href=\"http:\/\/www.beyond-digital.org\/sufiplugins\/\">Download them here.<\/a><br \/>\n<iframe loading=\"lazy\" src=\"https:\/\/player.vimeo.com\/video\/41943354\" width=\"640\" height=\"480\" frameborder=\"0\" webkitallowfullscreen mozallowfullscreen allowfullscreen><\/iframe><\/li>\n<li><a href=\"http:\/\/www.musicfromtext.com\/\">TransProse is a project by Hannah Davis<\/a>. It is the first iteration of a program that finds different emotions throughout text and programmatically creates music with a similar emotional tone. <\/li>\n<li>\n<a href=\"http:\/\/amy-alexander.com\/live-performance\/pigs.html\">PIGS is a performable, improvisable instrument<\/a> by Amy Alexander. It uses silent percussion to create expressive, improvised cinematic performance. It can be used with performance interfaces including iPads, Leap Motion controller, and silent MIDI drums. Real-time gestures are drawn on the iPads and Leap Motion, and can then replayed with variations by striking the MIDI drums.\n<\/li>\n<li> Cory Arcangel, video instruments. Here is Schonenberg composed algorithmically with with cat videos.\n<p><iframe loading=\"lazy\" width=\"560\" height=\"315\" src=\"https:\/\/www.youtube.com\/embed\/6ay0nOIWSo4?list=PLepGI0nLPsVMJmESaDVqv9DtaQU9ljoFn\" frameborder=\"0\" allowfullscreen><\/iframe><\/p>\n<p>For more of these <a href=\"https:\/\/www.youtube.com\/playlist?list=PLepGI0nLPsVMJmESaDVqv9DtaQU9ljoFn\">see this playlist.<\/a>\n<\/li>\n<\/ul>\n<p><strong>Other instruments of note:<\/strong><\/p>\n<ul>\n<li><a href=\"http:\/\/www.imogenheap.com\">Imogen Heap<\/a>&#8216;s software gloves are a new interface for performance and composition.<br \/>\n<iframe loading=\"lazy\" src=\"https:\/\/player.vimeo.com\/video\/90252137\" width=\"640\" height=\"360\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/li>\n<li>David Rokeby&#8217;s <a href=\"http:\/\/www.davidrokeby.com\/vns.html\">Nervous System<\/a> (1982-1991) is a classic and very early computer vision work where the body is the controller.<br \/>\n<iframe loading=\"lazy\" src=\"https:\/\/player.vimeo.com\/video\/8120954\" width=\"640\" height=\"424\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/li>\n<li>Alison Parish&#8217;s <a href=\"http:\/\/www.decontextualize.com\/projects\/nite\/\">writing instruments <\/a>(warning! these are not sound but take on the principles of musical instruments).<\/li>\n<li>Todd Anderson&#8217;s Chrome Extension, <a href=\"https:\/\/chrome.google.com\/webstore\/detail\/synthtyper\/ahmmmjockhcjpepcbiacehhlpbbgofdo\">Synth Typer.<\/a> Uses your interactions with the browser to synthesis beautiful synthy music.<\/li>\n<li>Rebecca Fiebrink&#8217;s <a href=\"http:\/\/www.wekinator.org\/\">Wekinator software tool <\/a>is a platform for the development of instruments using machine learning techniques. See <a href=\"http:\/\/www.bbc.com\/news\/technology-35868083\">a video on this here.<\/a> And here is a tutorial on using <a href=\"https:\/\/www.youtube.com\/watch?v=bE2EimjdUmM\">wekinator in Processing.<\/a><\/li>\n<li><a href=\"https:\/\/www.youtube.com\/watch?v=dpW0wRkijQY&amp;app=desktop\">Here is Phoenix Perry<\/a> on using Wekinator to make a gesture based instrument also using the kinect.<\/li>\n<li>Apps <a href=\"http:\/\/www.lullatone.com\/category\/games\/\">by Lullatone<\/a><\/li>\n<\/ul>\n<p><strong>Assignment brief: Software Instrument<\/strong><br \/>\nYour task is to design a software instrument. The interaction design of your instrument should either be using the mouse or keyboard. Your instrument could trigger sound from files using the <a href=\"https:\/\/processing.org\/reference\/libraries\/sound\/SoundFile.html\">Soundfile Player class<\/a>, or they could be synthesized using the <a href=\"https:\/\/processing.org\/reference\/libraries\/sound\/index.html\">sound oscillator functions<\/a>. These functions\u00a0produce different types of\u00a0sound waves such as sine, saw, square, triangle or pulse waves (note their differing aesthetic qualities). If you are triggering sound files, carefully consider where you source your suite of sounds come from. Will you record them from a group of things &#8211; eg. a collection of recordings of people saying\/singing\/shouting a particular word? Or sounds sourced from water\/hitting particular objects\/the body? Or will your sounds be cut from a particular film\/song\/source media? Or will you construct a particular scale such as the western chromatic scale? Or a different scale (there are many).<\/p>\n<p>What visuals will your instrument trigger? They could trigger photos, drawings, animation walk cycles or other visuals.<\/p>\n<p>If you use a suite of files for you sounds and\/or images, these must be stored and accessed from an array.<\/p>\n<p>Note the example below on how to deal with triggering events only once from a key stroke (rather than over and over again when the key is pressed. This same technique could also be applied to a mouseclick.<\/p>\n<p><strong>Due week 10, November 2nd in class.<\/strong><br \/>\nCome ready to play or have your software instrument played in class!<\/p>\n<p><strong>Dealing with triggering sound files just once.<\/strong><\/p>\n<p>The draw loop of Processing can make it challenging to trigger a sound just once, instead of over and over again. You will need to use a boolean variable to manage this as shown in the template below. <\/p>\n<p>You will need to add <a href=\"https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-content\/uploads\/2016\/10\/soundtrack.mp3\">this sound file<\/a> to the data folder of this saved sketch.<\/p>\n<pre code=\"java\">\r\n\/\/import processing sound library\r\nimport processing.sound.*;\r\n\r\n\/\/Make an instance of the soundFile class called soundtrack\r\nSoundFile soundtrack;\r\n\/\/Make an instance of the AudioIn and Amplitude classes too!\r\nAudioIn in;\r\nAmplitude amp;\r\n\r\nboolean isPlaying=false; \/\/boolean to check if sound is playing\r\n\r\nvoid setup() \r\n{\r\n  size(400, 400);\r\n  \/\/load in new file to the soundtrack object\r\n  soundtrack = new SoundFile(this, \"soundtrack.mp3\");\r\n  \/\/ loop sound\r\n  soundtrack.loop();\r\n  \/\/ stop sound to prevent it from playing automatically\r\n  soundtrack.stop();\r\n\r\n  \/\/ Create an Input stream which is routed into the Amplitude analyzer\r\n  in = new AudioIn(this, 0);\r\n  amp = new Amplitude(this);\r\n  \/\/ start the Audio Input\r\n  in.start();\r\n\r\n  \/\/ patch the AudioIn\r\n  amp.input(in);\r\n}\r\n\r\nvoid draw(){\r\n  background(255);\r\n\r\n  \/\/ draw an ellipse based on current volume level\r\n  float vol = amp.analyze();\r\n  println(vol);\r\n  noStroke();\r\n  \/\/visualizer\r\n  fill(0,0,255);\r\n  ellipse(width\/2, height\/2, map(vol, 0, 0.5, 0, width), map(vol, 0, 0.5, 0, height));\r\n\r\n  \/\/draw some play and stop buttons\r\n    \/\/ play button\r\n  fill(0,255,0);\r\n  ellipse(25, 25, 50, 50);\r\n    \/\/ stop button\r\n  fill(255, 0, 0);\r\n  ellipse(75, 25, 50, 50);\r\n\r\n  println(isPlaying);\r\n}\r\n\r\nvoid playsound() {\r\n  \/\/check our file is not already playing\r\n  if (isPlaying == false) {\r\n    \/\/play file\r\n    soundtrack.play();\r\n    \/\/set isPlaying to true as our file is playing\r\n    isPlaying=true;\r\n  }\r\n}\r\n\r\n\r\nvoid stopsound() { \r\n\/\/check file is actually playing\r\n  if (isPlaying == true) {\r\n   \/\/stop it playing\r\n    soundtrack.stop();\r\n    \/\/therefore isPlaying is false\r\n    isPlaying=false;\r\n  }\r\n}\r\n\r\nvoid mousePressed() { \/\/if the mouse is clicked\r\n  \/\/ and if the mouse is inside the play button\r\n  if (dist(mouseX, mouseY, 25, 25)<25) {\r\n    playsound();\r\n  }\r\n  \/\/and if the mouse is inside the stop button\r\n  if (dist(mouseX, mouseY, 75, 25)<25) {\r\n    stopsound();\r\n  }\r\n}\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Synesthesia is a condition where one of the senses is coupled with another, where people report of hearing images, of words having colors or of tasting sounds. The translation between senses has been an ongoing source of inspiration for creators for centuries with responses being instruments such as the color organ where a note triggers &hellip; <a href=\"https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/software-instruments\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Software Instruments&#8221;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-2692","page","type-page","status-publish","hentry"],"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-json\/wp\/v2\/pages\/2692","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-json\/wp\/v2\/comments?post=2692"}],"version-history":[{"count":10,"href":"https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-json\/wp\/v2\/pages\/2692\/revisions"}],"predecessor-version":[{"id":2942,"href":"https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-json\/wp\/v2\/pages\/2692\/revisions\/2942"}],"wp:attachment":[{"href":"https:\/\/www.courses.tegabrain.com\/2016\/PVA2\/wp-json\/wp\/v2\/media?parent=2692"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}