Hold on … plants playing music, come on, really? Yes! And by listening closely you may find a better understanding of plants and of yourself!
By measuring microcurrent fluctuations occurring across the surface of a plant’s leaf, this Biodata Sonification device generates MIDI notes when a change in conductivity is detected, allowing a user to listen into the invisible biological processes occurring within plants.
Here you can see two electrodes are connected to the leaves of a Snake Plant using ‘Tens’ style sticky pads, which are reusable, washable, and ‘plant-safe’.
How it works….
Tiny changes in electrical conductivity are measured between the electrodes and fed into a programmable microcontroller. Changes are detected using averages and standard deviation calculations which turn on LEDs and produce MIDI Notes and Control Changes. The circuit used to detect biological galvanic conductance is based on a 555 timer IC set as an astable multivibrator, similar to a simple lie-detector.
The exploration and practice of Biodata Sonification can allow a student, musician, scientist, or florist to listen in on the secret life of plants, and understand how your tools work!
Development and Future of Biodata Sonification
This device began as an interactive human touch music project with my friends Jon Shapiro and Gus Depenbrock, developing into a plant-focused system with Data Garden and growing as I designed the MIDI Sprout, the project evolved further with my work at Haverford College using a wifi daughter board for the Adafruit Circuit Playground, and now continues as a part of my educational Biodata Sonification project using the Adafruit Feather ecosystem providing serial MIDI, Bluetooth MIDI, WiFi MIDI, or even direct translation of biodata into musical sounds!
One key aspect of this biodata system is the electrical isolation of the plant sensor, similar to a medical device, we need to ensure that the plant is not impacted by the sensor in order to detect how it reacts to stimulus ‘in nature’. One way this is done is by using MIDI which is opto-isolated, an additional level of separation can be provided by bluetooth and wifi wireless connections. USB connections are usually not isolated, and special circuitry is required in order to have charging, USB/MIDI, or other ‘wired’ connectivity. This also means that using battery power is much preferred over having the device plugged into a power supply.
Bluetooth MIDI is very well implemented on most Android and iPhones, as well as modern computers, making it a great way to connect one (or perhaps two) biodata systems to a computer/phone.
While working with Biodata Sonification I have found that connecting multiple plants to produce music can be very challenging, and the desire to collaborate with other plants over long distances is strong. In order for plants from multiple locations to be monitored/listened to simultaneously, we can use RTP-MIDI. This service allows MIDI to be sent over a local (wifi/ethernet) or external (internet) networks. By using services like McLaren Labs MIDI Cloud who provide MIDI Chat Rooms, we can connect multiple MIDI streams and listen to plants from around the world!
After a series of iterations, I always have errors and mistakes on my first run of circuit boards, I have a successful Bill of Materials listing (BOM) and designs for a Biodata Feather Wing circuit board. This daughter board should work with many of Adafruit’s Feather main boards, while this has been specifically built for the ESP32 Feather, Huzzah (ESP8266), or 32u4 Feather. I’m hopeful it will additionally work with some of the Bluetooth boards, will just need to ensure the pins in usage for the IO and serial port for MIDI are not conflicting.
For those who want to build their own or review the details of of the system works, all of the files and so much more are available on my Electricity for Progress GitHub repository
I also have designed a neat (simple) 3D printed enclosure which allows a LiPo battery to be stacked along with the Feather and Biodata boards. Check it out on Tinkercad.
What is MIDI?
MIDI is a language for computers to talk to synthesizers, drum machines, and other electronic devices in order to play music. Classic synthesizers in the 80’s began to adopt the MIDI 5-pin DIN standard connector allowing interconnectivity for synchronization of timing/beats and allowing elaborate and repeatable programming of multiple devices. This revolutionized recording studios and live stage production, bringing a new level of digital control of instrumentation. In the modern day using big DIN cables does feel clunky and cumbersome, now BLE bluetooth and WIFI RTP-MIDI can allow wireless connections between music making devices, phones, computers, and even the internet.
Standard MIDI specifications allow connected devices to be ‘electrically isolated’ using optical transistors on the Input of any MIDI synthesizer. By ensuring electrical isolation, MIDI prevents old synths which may be in poor repair from destroying any other equipment it is connected to. This also limits any noise which would be present in the audio output of an interconnected synth or drum machine.
Review Sparkfun’s MIDI information page for excellent details about MIDI circuits, pinouts, and how MIDI is used!
For these reasons I have chosen MIDI as the musical data transport for this system. The plant and electrodes are provided with some level of electrical isolation, and MIDI can be easily used with keyboards, computers, and many other musical devices.
What does it all mean?
Ask a botanist, biologist, philosopher, or florist and try it for yourself! While there are always flourishes and strange/exciting events heard when interacting with a plant, as an engineer I personally am interested in the longer sets of raw data. I want to better understand how the plants change over many days, cycles, seasons, and how using this type of recorded data may be able to help measure and track plants in nature, in the office, on the farm, and in the lab under study.
What Biodata Sonification data can show…
The data which is produced by the Biodata Sonification system is not inherently ‘musical’ in itself. The biodata readings do not occur ‘on beat’, rather a note is generated when a change is detected within the plant with pitch corresponding amount of change. This is sometimes a criticism of the device, as users hope that plant music will sound magical ‘out of the box plant’. It can be challenging to use the relatively thick data stream of MIDI notes, while the threshold knob on the Biodata system can help to reduce the number of notes. As many users can attest, the rich MIDI data can be very useful for creating lush multi-instrumental soundscapes. Filters added to the MIDI data can allow ‘scaling’ and limiting of notes. Scaling of the notes can quickly take the raw biodata and make it sound musical or like a lullabye. Personally, I prefer working with the Raw data in hopes of better understanding the environmental conditions which influence changes and patterns.
When it comes to Biodata Sonification experimentation is key, and you may never hear the ‘same’ sounds again! The notes coming out of my system sound and ‘feel’ fairly random (a common criticism), but over long periods of time there are very real patterns which emerge and these are the daily cycles of activity happening within each plant. The algorithm detects changes in electrical conductivity, and the ‘amount of change’ is represented by a number/musical note.
Below is an image of MIDI data over a 16 hour session with a Monstera on top and a trusty Snake Plant at the bottom. You can see the ‘waves’ and changes in the overall pitch, density, and distribution of the notes. This type of process is really hard to ‘hear’ during a brief performance, and if the performance is at the wrong time of day (or in the wrong environment) there might not be any notes at all! Plants play a long-game and the patient are rewarded.
Below you will find an image showing raw MIDI Biodata which was recorded over multiple days from the Snake Plant sitting on my office desk. Laid out in a row the colored Clips show examples of MIDI data from different times of day. You can easily see by the distribution of notes in each Clip that the overall pitch or root note varies throughout the day, while the structure of the biodata signal is similar in each segment.
You can see in the second ‘brown’ clip that the MIDI notes appear to be slowly moving downward in pitch. You can also see the final ‘magenta’ clip from the ‘early morning’ shows notes very tightly distributed compared against the ‘orange’ clip from earlier in the evening.
In the MIDI piano roll images below, you can see a zoomed in segment of each recording.
- We can see two ‘parallel’ rows of notes with flourish and outliers near the high and low band of MIDI.
- In the Evening segment the low notes are well below C2
- We can see that by the Morning the lowest notes have crept up around C2
- In the third image the data from the Afternoon shows the lowest notes well above C2
This upward flow is a very common type of progression seen in the Biodata throughout a normal day. Had I recorded data for longer, the signals would have slowly (and sometimes abruptly) reduced back down to lower octaves. The ebbs and flows of the plant’s biological cycle.
I have included audio recordings of the ‘sonification’ of this MIDI data and links to the MIDI tracks themselves which are considerably longer. For this rendering I am using the ‘Chimes and Bow’ Instrument in Live 9 which splits the low notes to a low bowed bass string, spooky mid tones, and high notes in crispy chimes.
✿¸♫♪*.•¨•.¸♪¸.•¨•.¸✿¸.•♫♪*•.¸¸.•♫♪*•.¸♪¸.•¨•.¸✿
MIDI File – Biodata Evening 01
✿¸♫♪*.•¨•.¸♪¸.•¨•.¸✿¸.•♫♪*•.¸¸.•♫♪*•.¸♪¸.•¨•.¸✿
MIDI File – Biodata Morning 02
✿¸♫♪*.•¨•.¸♪¸.•¨•.¸✿¸.•♫♪*•.¸¸.•♫♪*•.¸♪¸.•¨•.¸✿
MIDI file – Biodata Afternoon 02
✿¸♫♪*.•¨•.¸♪¸.•¨•.¸✿¸.•♫♪*•.¸¸.•♫♪*•.¸♪¸.•¨•.¸✿
MIDI file – Early Morning 03
✿¸♫♪*.•¨•.¸♪¸.•¨•.¸✿¸.•♫♪*•.¸¸.•♫♪*•.¸♪¸.•¨•.¸✿
Who is using Biodata Sonification? An overview of the landscape..
Data Garden, leetronics, Damanhur, artists
Open source hardware and software
- Electricity for Progress Github: https://github.com/electricityforprogress
- Biodata Feather Bluetooth, WIFI, HardwareSerial: https://github.com/electricityforprogress/BiodataFeather
- Biodata Sonification MIDIsprout
- BreadBoard Kit Instructions: https://github.com/electricityforprogress/BiodataSonificationBreadboardKit
- Original MIDIsprout Design: https://github.com/electricityforprogress/MIDIsprout
- Assembly Videos on Youtube: https://www.youtube.com/watch?v=lbbWzdTiL8w
- Biodata Wifi with Adafruit Circuit Playground (old): https://github.com/electricityforprogress/BiodataWifi
Is a kit of parts or the preprogrammed chip available?
Many thanks in advance and kind regards
Stu
Hello Stu sorry for my late reply. Yes I offer kits on my Store page, you can get a full kit of parts with a programmed chip or just the circuit board. I also am working on a new Bluetooth/Wifi version of the kit. let me know how i can help get you setup for some biodata sonification adventuring!
-sam
The concept of translating the raw signals produced by evapotranspiration into tones is a fascinating concept. I see you have found differences in produced sounds within the day/night cycle. Have you noticed differences in the lunar cycles (mini tides) as well? Has the biodata units been tested in the field to determine an area of stressed vegetation prior to the proliferation, for example invasive plants. (for instance sonic output beyond what was expected for the time of day, plant species and such.) I think you and your colleagues may be onto something.
Hello Linda, and thank you for your order(s)! There are definitely differences with the day/night cycle, but each plant variety has its own clock which (along with Temperature) drives its activity cycle.
While I believe there is a scientific and data based study which is possible using this device, most users are just having fun 😉 I am working with a variety of students from around the world who are interested in Biodata, and I am hoping through their studies this device can move from a novelty noise sequencer into a usable detector for environmental states and changes.
I’m an Australian, retired, Electrical Engineer (BE)
At first glance, your recordings look suspiciously like harmonics from 60Hz (US) AC mains power?
Have you done any measurements in an electro-magnetic screened enclosure (to eliminate AC mains pick-up) ?
Regards.
ps. We (and the UK and others) have 50HZ AC Mains. It would be interesting to discover what results are obtained in other (50Hz) countries with the same device set-up.
Hello John, thank you very much for question. I am VERY interested in noise and its impact to the sensors. One thing you need to keep in mind, the diagrams which you see representing the data are MIDI notes displayed over long periods of time. Different plants under study display different characteristic patterns.
It is very possible to have noise, interference, and ground loop issues when using a Biodata (or any medical) Device. The two ways which I combat this issue are 1. DC/DC power Isolation and optical isolation for the 555 timer (biodata sensor) 2. MIDI and wireless connection eliminate the ground loops which can occur with a computer/synth/amplifier set.
Plants themselves are large bags of salty fluid extended in space, which will absorb (and potentially resonate with) electromagnetic waves. There is no doubt that plants can act as antenna, different plant varieties act as physical filters by mass and structure.
I’m sure you’ve got some triple-5’s, a 4700 puff cap,Swa and a 100k resistor in the junk box … try spinning up the circuit (it’s really just the old lie detector from Forrest Mims III)
Hi Dear, I love your project
what kind of electrode do you use?
thanks a lot
Fabio
Hello Fabio! I use three different types of electrodes. Using standard ‘Tens’ gel pads and snap electrodes it is easy to attach to the leaves of a plant. Using small flat copper clips can allow switching between different plants for testing or during expeditions. The third type of electrode uses Silver Wire which can be inserted into the Mycelium base of Fungi and can sonify mushrooms! It is easy to try and build your own electrodes, connecting to the Biodata Device using a mono 3.5mm jack.
Hello Sam,
today I stumpled over this amazing project with the plant made music. As a DIY guy who is also in modular Synthesizers, currently I´m about to build my own system, this project seems perfect.
After reading all the stuff regarding the Biodata devices, and watching a review from the “Plantwave”, I realized there is not much of a difference between the two versions.
I know, you wanna go commercialy – but is there any chance you would share the code?? 😀
Would be great for beeing in nature without a synth for enjoying these lovely tunes…
220 Bucks – it is definitly worth your efforts – is quite a lot for me, especially if all the parts are allready laying around in my toy box.
Would be really nice.
greetings from germany,
Ralf
Hello Ralf. Yes all of my code is opensource on my github.com/electricityforprogress. The Plantwave is 95% the same code as my device, the only difference being the Plantwave app is locked to their hardware.
The main reason that you can find Biodata Sonification devices commercially today (from a variety of sellers) can be traced back to my publication of my code back in 2014. There are really ‘two methods’ for Plant music making out there, one uses my open source methods with a 555 timer as a sensor and the ‘other’ is from Music of the Plants and their wheatstone bridge detector.
You can find instructables, a PDF for breadboard built on my github, or I suggest if you are really interested in Biodata you should try my new Bluetooth version which uses an Adafruit ESP32 Feather brain-board and an easy to build biodata sensor.
Hey, whats up? I’ve been looking for this open source for a long time and I was very exciting to find it on your Github and more information here on your website. I’m a multi-artist in Brazil and I know almost nothing about programming, more recently I’ve been learning a little more thanks to some projects that I’ve been trying to build with Arduino, but it’s all very new and I’m walking in baby steps. I’m working using your open source, I followed the steps with the informative pdf and used your code I found here ( https://github.com/electricityforprogress/BiodataSonificationBreadboardKit/blob/master/BiodataSonification_026_kit.ino ) but I didn’t get good results. I believe it has somting to do with these two other codes (https://github.com/electricityforprogress/BiodataSonificationBreadboardKit/blob/ac8b28a86406faa1d3ad3d88cac917c3296f4fc5/Serial_MIDI_Converter_V2D.pde) and (https://github.com/electricityforprogress/BiodataSonificationBreadboard05Git/blovbShield/masterKit/ .brd) that I don’t know what to do with them. I really appreciate it if you could explain me better how i use the codes programming in Arduino. thanks a lot.
Hello Luly and thank you for your interest in Biodata sonification.
Programming with Arduino is always a bit challenging, and if you are jumping right into a project like this I recommend learning a bit about how uploading code works, getting a Blink sketch running, and learning about libraries
All of the code is in the 026_kit.ino file, and when you compile you must select your specific arduino board (probably an Uno). You need to install libraries for the items in the #include statements at the top of the code, links are provided in the comments: //store and read variables to nonvolitle memory //https://github.com/thomasfredericks/Bounce-Arduino-Wiring //manage LEDs without delay() jgillick/arduino-LEDFader https://github.com/jgillick/arduino-LEDFader.git
#include
#include
#include
You will need to describe a bit more about the specific Arduino which you have, what you use for making sounds, and describe the error/issue. Are you having upload problems or are you having MIDI issues? Does the LED light show appear when powering on? Do you have a synthesizer or specific MIDI computer connection that you are using? Are you stuck in Windows and trying to use the old hairless Serial->MIDI bridge?
But first, make sure you understand enough about Arduino to make the ‘Blink’ sketch work. Then get the Biodata code to compile. Then rock out with plants.
Hi Sam,
Firstly, thank you so much for allowing us all to create this device ourselves!
I am a postgraduate art student in London and have been working on your device for a few months- I finally managed to plug it into Ableton with a block of mycelium on the other end today and the results were just beautiful!
I can see that you’ve suggested that it’s easy to make your own silver wire electrodes using a 3.5mm mono jack plug but I’m struggling to find clear instructions online and I’m a bit scared of making them wrong and then damaging the device in the process. Would I simply buy a 3.5mm jack with bare ended cables and solder silver wire onto the bare ends? Thank you!!
Hello Matilde, sorry for the slow response, i don’t often check these comments. You are always welcome to email me directly 😉
Yes it is very easy to make additional electrodes and you won’t break anything.
The ‘best’ way to build the electrodes would be to purchase a mono 3.5mm solder jack and use some two conductor wire (speaker wire, or anything really). When connecting to the jack make sure that the two stripped wires do not touch, i recommend stripping off some of the wire insulator, then ‘tin’ the wire buy soaking some solder to the end of the wire with your iron, then cut the tinned end of the wire nice and short, then ‘tin’ the connectors on the 3.5mm jack, then carefully solder each of the wires into place 😉 This is a youtube video for soldering ‘headphones’, which gives you the general idea. https://www.youtube.com/watch?v=wYH1KZ_14uE
You could even salvage an old set of headphones for the wiring, but the wires are usually really tiny and fiddly in headphones beware!
Make sure that the ‘electrode wire’ you use is really silver, sterling jewelry wire works best, it is easy to get ‘silver colored’ wire by mistake. Other metals like copper will oxidize and loose sensitivity. I use 3cm of silver wire for the ‘trodes that i build, you wanna stick them pretty deep into the mycelium and i think that 3cm of exposed wire will make great conductive contact.
Good luck with your build, and make sure to put your device in an enclosure 😉
Hi Sam,
Thank you so much for your very helpful response!
I’ve ordered the bits now so I’ll give it a go.
Ooh yes, I definitely need to look into buying/building an enclosure!
Thanks!