1

Topic: Audio and Python

Hi All

I would like to store small (say, 5s) chunks of audio in the NION and send them out (randomly) to different outputs. This is of course possible to do by using the wave file players, but I wonder if there is a more efficient way to do it. Is it e.g. possible to store small sequences of audio in the NION's ram by using a Python script and retreiving these in subsequent calls to the script? Or is there a better way?

Is there any documentation on what subset of python modules that are implemented in the NION?

Thanks!
Per

2

Re: Audio and Python

Why not use a wave player, or players, and use Python to control the playback and routing?

3

Re: Audio and Python

Yepp, that is what I'm doing right now, but there is a limit to how many wave players you can run simultaneously (this is for a sound design/art installation). According to a previous discussion a couple of weeks ago, the limit is around seven. I'm not quite sure if the limit is set by disc access or by the control resources. But the machine is supposed to do some other tasks as well...
We have 40 outputs.
smile
And it's a NION Nx.
Thanks anyway.
Per

4

Re: Audio and Python

Hi Per-

How many signals do you need to play back at the same time?

How many different audio clips are there to be played back?

If you want to play more than 6 audio clips at the same time, I would encourage you to look into some of the dedicated playback devices. Something like the Audio Science ASI 6416 PCI CobraNet card will allow you to play back 8 stereo audio clips at once and put them on CobraNet. Your NION can take this CobraNet and route it to the system outputs you desire. Or, of course there are many other devices from the likes of Alcorn McBride and others who are into the higher numbers of playback channels.

Thanks!

-Josh

Josh Millward
Burnt Orange Studios

5

Re: Audio and Python

I think the ASI card is pretty cool, however I would tend towards the Alcorn box if you need a good amount of simultaneous playback channels. Take a look at their Digital Binloop. Reasons off the top of my head why it may be a good fit:

1. Cobranet option, you can get the audio into the Nion w/o the hassle of messing with analog wiring. The balanced audio outputs on the Binloop are on DB-25 connectors, which compared to a RJ45s are somewhere between a moderate and major PITA. Plus, additional LAN connections contribute to reason #5.

2. Solid state storage. The failures I've seen in playback systems are by far rotary hard drives. The Binloop is all compact flash based.

3. LAN controllable. It should be pretty easy to cook up a Python script in the Nion project to control the Binloop via TCP/IP.

4. Not PC based. I've also seen plenty of systems that have been taken down because someone got to the PC and loaded up games, spyware, etc. The Binloop is a standalone box that would only need 2 LAN connections(assuming Cobranet in use) and a power cord.

5. Front panel flashy lights.

I've used them before and they work quite well, although I've heard evil things about their 8-Traxx.

http://www.alcorn.com/products/binloop/index.html

6

Re: Audio and Python

Hi and thanks for many good suggestions.

We're actually using the Alcorn McBride 8TraXX (please - I'm dying to hear about all evil rumours, per@sound.se). We've used it before and it seems to work fine - so far (but you're making me nervous now) smile.

Haven't looked into the Binloop, but it seems like a good idea if you can tie them together using CobraNet.

The tricky part is the audio processing bit: In this installation, people can speak (sound) and add chunks of audio to the complex. I've made a Python script that randomly, and at random time intervals, selects one wave player out of four. The audio is processed using the existing processors in MM. But I would also like to add all incoming audio to an accumulator (with an exponential decay) to create a gravy of all audio ever added. This is art smile !

Maybe I can achieve this by using a delay and a feedback loop with a gain slightly less than unity...
Merry X-mas to you all.
Per

7

Re: Audio and Python

o.O

So you are recording sound bytes with the Nion? At first glance I'd work this with a front and back end. The front end of course would be the Nion. The back end would be a Linux host running a database along with some other little toys. Depending on the storage requirements, the filesystem may serve as an adequate 'database'. It might work something like this...

Record audio with Nion .wav recorder.

Use a python script to push that file over to the server.

The server can store that clip in the database as a binary blob with a timestamp of when it was recorded.

Cook up some scripts in the server(Python would probably be good for this also, or maybe just bash) to grab the latest X number of clips from the database and utilize one of the CLI sound editors to string them together and possibly modify their amplitudes based on the timestamp.

Push the result back to the Nion.

Play w/ Nion wave player.

Now I *KNOW* that some lunatic has asked for NFS support in the Nion. I don't know who or when, but I'm *PRETTY* sure the request has been made. With NFS the Nion could record/play directly from a NFS share directly on the server, which would make things simpler. OTOH, if shuffling around files for the Nion wav players/recorders is too much trouble, you could just outfit the server with a Cobranet sound card and handle the recording and playback directly from the server and leave the routing in the Nion. Check out AudioScience, they make Cobranet  PCI sound cards with Linux ALSA drivers. There are CLI utilities for playing and recording audio directly from the command line, which means you can automate them pretty easily.

8

Re: Audio and Python

Footprint wrote:

Hi and thanks for many good suggestions.

We're actually using the Alcorn McBride 8TraXX (please - I'm dying to hear about all evil rumours, per@sound.se). We've used it before and it seems to work fine - so far (but you're making me nervous now) smile.

Haven't looked into the Binloop, but it seems like a good idea if you can tie them together using CobraNet.

The tricky part is the audio processing bit: In this installation, people can speak (sound) and add chunks of audio to the complex. I've made a Python script that randomly, and at random time intervals, selects one wave player out of four. The audio is processed using the existing processors in MM. But I would also like to add all incoming audio to an accumulator (with an exponential decay) to create a gravy of all audio ever added. This is art smile !

Maybe I can achieve this by using a delay and a feedback loop with a gain slightly less than unity...
Merry X-mas to you all.
Per

Oh. My.

Per- What an interesting way to use the NIONs ability to record and play audio! Now you have me really interested! How do the people speak into the system? Do they have to push a button or anything like that? Do you have a defined limit for how long their entry can be? Do you have a method of purging tracks after some predetermined amount of time, or do they just accumulate over time? What will happen when you overload the storage capacity of the NION? Would you be willing to share any of this with us? I know I would be very interested in looking at your project design even if you would prefer that we not use it as an example of what can be done.

Thank you for sharing what you have so far! If you would rather not share on the board, my email works... there is a link right under my name.

-Josh

Josh Millward
Burnt Orange Studios

9

Re: Audio and Python

jvalenzuela wrote:

o.O

indeed!

jvalenzuela wrote:

So you are recording sound bytes with the Nion? At first glance I'd work this with a front and back end. The front end of course would be the Nion. The back end would be a Linux host running a database along with some other little toys. Depending on the storage requirements, the filesystem may serve as an adequate 'database'. It might work something like this...

Record audio with Nion .wav recorder.

Use a python script to push that file over to the server.

The server can store that clip in the database as a binary blob with a timestamp of when it was recorded.

Cook up some scripts in the server(Python would probably be good for this also, or maybe just bash) to grab the latest X number of clips from the database and utilize one of the CLI sound editors to string them together and possibly modify their amplitudes based on the timestamp.

Push the result back to the Nion.

Play w/ Nion wave player.

I like this concept Jason! Very cool! If you were interested in making this work, you might be interested to know (or maybe you already do know) that there is an FTP client in the Linux operating system already. I have not had an opportunity to work with it at all, and I should be honest and tell you that Ivor informed me that it exists. So, if you just wanted to run an FTP server on the back end, I'm sure there is a method to transfer the files back and forth.

(insert obligatory comment here about how MediaMatrix Technical Services DOES NOT recommend anyone EVER go digging around in the operating system files in the NION... if you brick your box, you are on your own, but we might be able to help give you some ideas of how to bring it back, maybe... we can not promise that. If you know what you are doing, good for you. If you do NOT know what you are doing, PLEASE do not break it. Linux can be downloaded for free and installed on your own computer, this is a good way to gain some experience so you don't wreck your NION. Thank you!)

jvalenzuela wrote:

Now I *KNOW* that some lunatic has asked for NFS support in the Nion. I don't know who or when, but I'm *PRETTY* sure the request has been made. With NFS the Nion could record/play directly from a NFS share directly on the server, which would make things simpler. OTOH, if shuffling around files for the Nion wav players/recorders is too much trouble, you could just outfit the server with a Cobranet sound card and handle the recording and playback directly from the server and leave the routing in the Nion. Check out AudioScience, they make Cobranet  PCI sound cards with Linux ALSA drivers. There are CLI utilities for playing and recording audio directly from the command line, which means you can automate them pretty easily.

Yes, NFS support for the NION is an oft asked for feature. Thank you for bringing it up again as it has been a while since anyone said anything about it.

Thanks for reminding me, Jason!

-Josh

Josh Millward
Burnt Orange Studios

10

Re: Audio and Python

JoshM wrote:

If you were interested in making this work, you might be interested to know (or maybe you already do know) that there is an FTP client in the Linux operating system already. I have not had an opportunity to work with it at all, and I should be honest and tell you that Ivor informed me that it exists. So, if you just wanted to run an FTP server on the back end, I'm sure there is a method to transfer the files back and forth.

Ya, I've used that to add some Python modules that don't come w/ the Nion. AFAIK the MontaVista version you guys use seems to have most of what one would expect on a typical *nix host. So many ways to void one's warranty I don't even know where to begin....

With regards to the parent thread, the more I think about it, the more I would prefer to handle all the recording/playback in the storage host rather than the Nion. Use the Nion to route and apply what ever processing(high pass, comp/limit) you like to the record stream. The PCI Cobranet card eliminates the issues of transferring media between the Nion and storage server, not to mention the 'uncharted waters' nature of doing so w/o using Nware. No messing with FTP, whining about NFS, etc. Use the Nion for what it is good for, and use the server for the rest.

11

Re: Audio and Python

jvalenzuela wrote:

AFAIK the MontaVista version you guys use seems to have most of what one would expect on a typical *nix host. So many ways to void one's warranty I don't even know where to begin....

Yeah, pretty much. As you know, we will help however we can. Unfortunately, if someone manages to seriously corrupt the system, it is going to cost them. This is why I was pleased to see:

jvalenzuela wrote:

With regards to the parent thread, the more I think about it, the more I would prefer to handle all the recording/playback in the storage host rather than the Nion. Use the Nion to route and apply what ever processing(high pass, comp/limit) you like to the record stream. The PCI Cobranet card eliminates the issues of transferring media between the Nion and storage server, not to mention the 'uncharted waters' nature of doing so w/o using Nware. No messing with FTP, whining about NFS, etc. Use the Nion for what it is good for, and use the server for the rest.

I agree completely. This is the much better, totally supported way to get the job done without having to mess up your NION. Nothing freaky or tricky about getting this one to work.

Josh Millward
Burnt Orange Studios

12

Re: Audio and Python

Hi guys.

The solutions you have discussed so far are very interesting, although I don't fully understand how this should be implemented. And with the time schedule (and budget) for this project in mind, I really hope that the capabilities of the NION Nx will do. I haven't seen a NION until three weeks ago, but I'm trying hard to learn as much as possible about this thing and it doesn't look too bad right now. There are still some open ends, though, so I'll make a short presentation here.

I work with this group, Urban Sound Institute (www.usit.nu), comsisting of two musical composers, two architects and me, who is a sound designer/producer/acoustician. I used to work with research in adaptive filters for active noise and vibration control.

We're doing this installation in a museum in Sweden. In a big hall we've built a labyrinth (approx 8 by 13 meters) that is ment to be pedagogical as well as entertaining. It has has four different sounding parts: The name, the story, the music and the room (with long reverberation). In all theese we're running pre-recorded sounds (with the 8TraXX = 16 channels of sound). In three of theese, (the name, the story, the room) people can add sounds.

The name: A push button, connected to one pin of the GPIO interface, starts a recording. The length of this recording is preset to something between 3-5 seconds. I've got four recorders, recording to files name1...name4. A python script decides which recorder to use. This is usually the next one in line, but if the corresponding file is currently used by a player, the second next recorder is picked. Four players are designated to play theese files. A pyhton script selects players randomly (avoiding the player that corresponds to a file currently beeing recorded) and at random times. This means that a maximum of four players/recorders will be running simultaneously. The names are played clean as well as processed and mixed with a pre-recorded marinade.

The story: In this case, lifting a telephone receiver starts the recording (another GPIO pin). The recording continues as long as the receiver is off the hook, or until a preset time limit is reached. As before, there are four recorders recording to story1...story4, respectively. A python script decides which writer to use and as before, I'm not recording to a file that is currently beeing played back. The stories are played back one after the other, again controlled by a script. This was quite tricky to write, since the playback has to work even if e.g. only one file has been recorded, so I maintain status flags (in the state variable) on files written.

You raise a good question here: What happens when I run out of disk space? The answer is: I don't know! (Yet!). In fact, I don't even know how much space there is to store data in. How can I find that out? My unit was shipped with the 256 Mb disc and the problem would definitely be smaller (I imagine) if I had the 2 Gb disc. I'll talk with TTS (your associate in Sweden) about this.
I don't know how clever the file system is, but I imagine it might become clogged after a (short) while. Maybe a solution would be to clean the disc once a day. How do I do that? Automatically?

The third place is just a room with long reverberation: A microphone in the room is fed back through loudspeakers with delay and reverberation, mixed with pre recorded material. The Feedback Ferret comes in handy smile Nothing is recorded or played back on the NION.

To save DSP time and disc space, I'm running the system at 24 kHz sampling rate, but somehow it seems that the CABs are still running at 48 kHz (I need to interpolate). How's that? I'm also getting compilation notes saying "Device has sample rate override† that makes me a bit confused.

This was the short vesrion. smile
I'm wide open to suggestions.
Per

13

Re: Audio and Python

I just want to say that the idea with saving recorded audio to a database is brilliant, thinking about the (unlimited?) amount of data that could be stored and played back. The artistic level would definitely gain from this implementation. Maybe next time...
Thanks!
Per

14

Re: Audio and Python

Hmm - read the manuals (again) and saw that the CAB 16i/o only run at 48 kHz.
Maybe it's still a good idea to run the NION (and its IO cards) at 24 kHz and do the decimation/interpolation to interface the CABs (one CAB 16i and two CAB 16o).
Cheers
Per

15

Re: Audio and Python

Footprint wrote:

To save DSP time and disc space, I'm running the system at 24 kHz sampling rate, but somehow it seems that the CABs are still running at 48 kHz (I need to interpolate). How's that? I'm also getting compilation notes saying "Device has sample rate override�  that makes me a bit confused.

Cobranet only supports sampling rates of 48,000Hz and 96,000Hz. The Nion of course can operate at other rates, but must be converted before being transferred via Cobranet.

16

Re: Audio and Python

Footprint wrote:

I just want to say that the idea with saving recorded audio to a database is brilliant, thinking about the (unlimited?) amount of data that could be stored and played back. The artistic level would definitely gain from this implementation. Maybe next time...

A general purpose *nix machine running a database does indeed address the tasks that the Nion may not do so well. If you do find that you need(or want) to add this level of functionality at a later time, the Nion need not be replaced. It is a rather simple to recode your existing Python scripts to instead control the server(likely via a socket) as opposed to the wav players/recorders currently residing in the Nion. All the GPIO interfacing is probably fine. By using Cobranet to interface between the Nion and server, the physical wiring changes that would be required are pretty much nil.

Speaking of sockets, it seems that Nions do have the standard Python socket module available.

Good luck...

17

Re: Audio and Python

You are taking the CPU right to the limits.  Watch the 5m load on the LCD screen (press next on the Status until you see CPU on the right).  If this exceeds 5, and keeps climbing, the NioNode will reboot.  If the 5m climbs very slowly, I have seen it exceed 8, but not for long.

Josh's suggestion from 12-15 of using an AudioScience CobraNet to off load some of the recording/playback (which would give you the long record times that you seek).  Stepping down to 24k will save the DSP, and I THINK (no guarantee here) that what you are wanting to do would work with the DSP, but everything you are doing (including the Feedback Ferret) is using CPU, and this is where you will need to watch.

Good luck, and keep us informed of how things are going.

Make it intuitive, never leave them guessing.

18

Re: Audio and Python

Hi

It seems that the CPU load is around 10%. 5M is around 1.3 (if I remember correctly). What is this indicator anyway? There's another one too, right? 15M? (I'm not by the machine right now).

The DSP load is on average around 25%, but the DSP memory blocks 0 and 1 are used up to 99%. I've had to specify the DSP (A, B or C) for some DSP functions for the compiler to go through. The shared memory is used to 2%. Is there any way to make better use of this memory capacity?

The big question is still how much disk storage space there is available for storing wave files. Is there any way for me to find out? I think I read somewhere that the Python OS modules are not included in the MM implementation.

Per

PS. We're going live in two weeks...

19

Re: Audio and Python

I would have guessed that the 5M would be just under 5, so recorders are not using near as much CPU as I would have thought.

As to the memory, it sounds like you are doing the right things.  You can click on the "Details" box, and then look at which algorithms are using the most memory, and the decide if they are really needed, or if they can be modified to use less.  For example, 8 single channel gain controls will use FAR more memory then 1 eight channel gain.  The Blocks 0 and 1 are used by the algorithms, the shared is used for delay times.

The 256 meg Compact Flash has 128 reserved for the OS, and 128 megs for user files.  The 2 gig card, about 180 is reserved, leaving the rest for user files...Big change.  If you need the 2 gig, let us know soon.  I'm not sure we can push the replacement flash through the many hands between us and you in time (going live in two weeks means you would want the flash in one).

Make it intuitive, never leave them guessing.

20

Re: Audio and Python

Yes, I would very much like to have the 2G Compact Flash smile
Per