Interesting dual-core ARM Cortex M4+M0 from NXP

LeafLabs Garden » Community » The Lounge

Interesting dual-core ARM Cortex M4+M0 from NXP

(19 posts) (5 voices)

Started 4 years ago by cbrunschen
Latest reply from gbulmer

12 Next »

cbrunschen
Member

Hi all,

The one thing I've been missing from the STM32 chips is a way to add lots of RAM, in order to be able to process large amounts of data, such as audio processing at high sampling rates with long delay lines: Basically, looking at the openmusiclabs site, their 'MICrODEC' effects unit uses an Atmel processor with a fair amount of memory to allow up to 6s of delay, but obviously has less processing power; whereas if you use their AudioCodenShield on top of a Maple, that gives the processing power but not the memory for delays etc.

Further, for audio use, a Cortex M4 with its floating-point unit and DPS instructions would of course be excellent. In the process of searching I came across the NXP LPC4300 series, which combine a Cortex M4 with an auxiliary Cortex M0, I2C, SPI and I2S (which is great for audio of course), 1 or 2 USB, and options of ethernet and LCD support - and with an external memory controller for oodles of external memory:

http://www.nxp.com/#/pip/pip=pfp=71562|pp=[t=pfp,i=71562]

Not as much in the way of ADC/DAC, and in general seems to be not quite aimed at the same kind of market as the STM32s, which makes sense. But this seems - in particular the dual-core aspect, with a Cortex-M4 processor for the main aspects, and a Cortex-M0 for I/O and other auxiliary processing – quite intriguing.

Hmmmm ...

Best wishes,

// Christian Brunschen

Posted 4 years ago #
mbolivar
LeafLabs

whereas if you use their AudioCodenShield on top of a Maple, that gives the processing power but not the memory for delays etc.

Funny you should bring this up! We put their AudioCodecShield on the Maple Native just this week, and with its 1 MB external RAM, were able to get 45 seconds of delay. A blog post (with video!) on this will be coming out soon.

Posted 4 years ago #
cbrunschen
Member

How do you get 45 seconds of delay in 1MB? At CD audio fidelity (16 bits per sample, 44100 samples per second), 1MB = 512k samples = (524288 / 44100 = ) 11.89 seconds in mono, or half of that, 5.94 seconds, in stereo.

For comparison, the MICrODEC already has 512kB (4Mb) of memory, so the Maple Native just doubles that. With the higher performance of the CPU in the Maple, it would be nice to be able to connect a bit more RAM as well in order to be able to use that power, and not be so rather limited on the storage space.

And something like the LPC4300, which runs at over twice the clock frequency of the STM32 on the Maple, and with its Cortex M4 with DSP & floating-point operations should be able to give a further fair bit of oomph over the Maple, could possibly process audio at higher sample resolution and/or sample frequency - but also requiring the associated extra memory ... and indeed offering a way to put that memory there and use it.

Best wishes,

// Christian Brunschen

Posted 4 years ago #
gbulmer
Moderator

Further, for audio use, a Cortex M4 with its floating-point unit and DPS instructions would of course be excellent.

I agree completely.

There are some very interesting video's from the NXP announcement earlier this year about LPC4300.

AFAIK, LPC4300 might not much further long than ST Micro's STM32F4 (may actually be behind STM32F4, but I really do not know)

I looked at:
http://ics.nxp.com/products/lpc4000/lpc43xx/
and they are designated as samples, and not in production.

Please have a look at the thread about the ST Micro Cortex-M4 http://forums.leaflabs.com/topic.php?id=1056

I should say that the criteria about what I want may be very different from yours.

I would like the balance between power and simplicity to favour simplicity because I would like school children to be able to use the computer power 'from the get go'. So though *I* like the idea of a dual processor, and I can think of a bunch of nifty things *I* might like to do with it, I fear that it may be a bit too much for a new programmer with an hour or two of programming experience.

IMHO, it may be more interesting for audio than the NXP because STM32F4 has three reasonably fast (2.4Msps) 12-bit ADC's (which, I believe, with oversampling + e.g. a none-correlated triangle wave, can achieve better than 44.1Ksps 16bit without any external ADC's). It also has stereo DACs.

How do you get 45 seconds of delay in 1MB? At CD audio fidelity (16 bits per sample, 44100 samples per second), 1MB = 512k samples = (524288 / 44100 = ) 11.89 seconds in mono, or half of that, 5.94 seconds, in stereo.

I'd like to understand that too.

One possible path, which hasn't been attempted AFAIK, is using the STM32F's SDIO interface.
In theory this is capable of 12.5MBytes/second from 2GB SD memory cards.
I think that might be a way to extend the memory in a very useful way.
I estimate that is over 6 hours of uncompressed CD-quality mono.

(full disclosure: I am not a member of LeafLabs staff, or ST Micro's !-)

Posted 4 years ago #
cbrunschen
Member

Oh, I'm not saying that this chip would suit the Maple-or-similar target market, or that Leaflabs should switch. My thought is more something along the lines of the open music labs MICrODEC or the Mutable Instruments Shruthi-1 - a programmable / hackable sound generation and processing platform, mostly self-contained; but with enough persistent memory (Flash) to be able to store a fair chunk of sampled waveforms for generating audio, enough RAM for delay lines and other storage for audio processing, enough processing power to be able to be able to both generate sound and process it while being programmable without having to resort to squeeze-performance-out-of-every-cycle tricks, and a nice set of connectivity (USB, MIDI, possibly even ethernet) to boot.

I am of course making noises on the open music labs forum as well. I just thought I'd mention this chip here because of its rather interesting asymmetric dual-core approach (and put it into a context that sprang to my mind).

Best wishes,

// Christian Brunschen

Posted 4 years ago #
poslathian
LeafLabs

Sorry everyone! Marti merely propagated a lie/honest error I was floating around the office. I got confused for a moment and back of the enveloped the 8Mb Ram as allowing for 90Sec of 16bit 44KHz audio. Obviously I was off by a factor of 8. The actual value is 11.3 seconds.

However, with the SD card, you could stash arbitrary stuff. The real question is whether or not you can read/write from the SD card and produce/capture audio at the same time. Using SPI and SDFat, the answer is "no" but just barely. Using SDIO (which doesnt have lib support) I bet it would just work. Use the DMA you are probably golden. You may have to clear flash pages in advance though, depending on what the wait is there. I suppose you could asynchronously clear flash pages, while writing to already cleared ones.

I have been really wanting a more capable loop pedal, for storing several tracks simultaneously and providing a better interface to do so. A sampler would also be cool.

Posted 4 years ago #
gbulmer
Moderator

Chrsitian - I've been working on a sound card which may or may not get to production.

My ultimate specification sounds similar to yours. I am happy to use STM32F1 as an initial step. I agree hardware floating point may make it much easier for people to program it without having to resort to fixed point, or more complicated tricks. But an STM32F1 is quite a lot of processing power.

As I wrote I am most interested in providing a platform that both children and adults could program to create their own 'sound machinery'.

Having watched some of the NXP videos, I am genuinely worried that it will be significantly harder to use than a STM32F4. I believe some of the work can be offloaded from the processor to DMA controllers.

What sort of things do you think would benefit from the Cortex-M0, user interface?

Are there any groups or people you'd recommend I should read or contact?

Posted 4 years ago #
cbrunschen
Member

My thoughts are really rather loosely plucked out of the air from looking at the MICrODEC (http://www.openmusiclabs.com/projects/microdec/) and the Shruthi-1 (http://mutable-instruments.net/shruthi1); both use microcontrollers, the Shruthi-1 to generate sound, the MICrODEC to process and filter it. With something based on a more powerful chip that also allows a fair chunk of memory, both persistent for things like waveforms to generate and transient for things like delay lins while processing the audio, you could have a single box that does both jobs - and with enough power that you can use a simpler programming language & programming model than having to squeeze performance out of every cycle.

The LCP43xx simply seem like they offer both a nice amount of power, including suitable DSP-like instructions for generating & processing audio, as well as allowing access to nice amounts of memory.

As an added bonus, they offer connectivity (USART, USB, Ethernet even), a possibility for a bit more of a UI (an LCD interface is included on some of the chips), and the secondary processor (the Cortex-M0 one) could be used to offload I/O, user interaction, etc, while letting the main processor focus on the audio generation & processing.

I'd think the Open Music Labs people and the Mutable Instruments people would be good to talk to about anything microcontroller-audio-related.

Best wishes,

// Christian Brunschen

Posted 4 years ago #
gbulmer
Moderator

cbrunschen -

With something based on a more powerful chip ..., you could have a single box that does both jobs - and with enough power that you can use a simpler programming language & programming model than having to squeeze performance out of every cycle.

Yes, that is exactly my approach. It is Soooo much easier to get something working when there is plenty of spare 'headroom'.

The LCP43xx simply seem like they offer both a nice amount of power, including suitable DSP-like instructions for generating & processing audio, as well as allowing access to nice amounts of memory.

Yes, that Cortex-M4 CPU is very similar to the Cortex-M4 in STM32F4xx (I haven't looked at the details of how LPC43xx caches flash memory so I can't say they are identical).

The STM32F4xx ADC & DAC peripherals appear to be significantly faster, and STM32F4 has stereo DACs when compared to LPC43xx. So in that area, the STM32F4 part appears to be a better choice for audio applications.

As an added bonus, they offer connectivity (USART, USB, Ethernet even), a possibility for a bit more of a UI (an LCD interface is included on some of the chips), ...

The STM32F4 and LPC43xx seem to be comparable there.

... and the secondary processor (the Cortex-M0 one) could be used to offload I/O, user interaction, etc, while letting the main processor focus on the audio generation & processing.

So this is the key benefit for you?
It sounds like a great idea, but I am very concerned that it will be hard to use, especially for new programmers. Have you any plans on how you'd make it easy enough for folks too use?

Thank you for the pointers to Open Music Labs and Mutable. I'll have a dig around and get in touch.

Posted 4 years ago #
cbrunschen
Member

I'd actually completely ignore the built-in ADC & DACs on the devices, because they're not really aimed at audio applications on either the STM23F4 or on the LPC4300; I'd use a dedicated stereo Codec, like the Wolfson WM8731 as is used on the OpenMusicLabs projects (see http://www.wolfsonmicro.com/products/codecs/WM8731/ ), which also has a headphone driver. The LPC4300 and the STM32F4 both have I2S interfaces, which are specifically for digital audio I/O and which can be used with the WM8731 and its friends, so that makes things nice and straightforward - and equally so in both cases.

The main thing that makes me look at the LPC4300 over the STM32F4 is the ease of connecting lots of memory, actually - both for storing pre-generated-or-recorded waveforms (in Flash) and for offering lots of space for delay lines or similar.

But having a separate I/O processor would make it possible to have an environment where there was simply no question of I/O interfering with the audio generation and/or processing, especially if that I/O involved user interaction; even without using an LCD or similar, the I/O processor could handle any and all encoders, potentiometers, etc - interrupts, ADC, and all.

In fact, imagine a simplified exploration environment, similar to the Arduino / Maple environment, where instead of a 'loop' function, you could simply write a 'generateNextSample' and a 'processSample' function perhaps, which would execute on the main processor, and would be called from the 'framework' code at suitable audio resolution – something similar to how the OpenMusicLabs AudioCodecShield does it when connected to an Arduino or Maple, perhaps.

At the same time, the software on the I/O processor would be fixed and always remain the same, and simply write the values it read/processed from encoders, potentiometers, buttons and whatnot, into predefined well-known memory locations that could be exposed through well-documented variables or functions, since on the LPC4300, the main (M4) and I/O (M0) processors share the same memory and peripherals.

This gives us the advantage that the main audio processor would never have to wait for an ADC conversion to finish, or have to handle interrupts when the decoder changes or a button is pressed, or similar, and instead focus entirely on the generation and processing of audio data in the presence of whatever values have already been processed bu the I/O processor.

Posted 4 years ago #
gbulmer
Moderator

cbrunschen - my current STM32F103 design is simpler than the MICrODE.
I am aiming for something with a 'softer' user interface because part of the goal is to enable children to create that too.
I like Wolfson on I2S; as well as the WM8731, I've been looking at some slightly higher capability parts.

I am aiming for something more flexible than the MICrODE.
So, while the quality for the high-quality audio channels will be comparable, I am not aiming at a specialised use. Hence I think having upto 16 more input channels, and stereo DAC output as well as PWM output, to supplement the discrete audio CODEC seems much more fun to me.

I agree with much of what you say about the second processor.
But my target audience and some use cases are likely different.
I prefer to minimise the amount of fixed user interface, so the Cortex-M0 would be less valuable.
After all, human interaction is soft real-time, and rarely happens in under 10ms, and the audio I/O can use DMA.
Plus, it is aimed at education, so having understandable problems is "a good thing".

The main thing that makes me look at the LPC4300 over the STM32F4

They both have SDIO, both 16-bit external memory, so I are you referring to the "Quad SPI Flash Interface"?

Posted 4 years ago #
cbrunschen
Member

If I read the datasheet correctly, the STM32F4 has an FSMC (Flexible Static Memory Controller) that allows connection of 4 banks of 128 MB (26 address lines, 16 bit wide memory bus => 128 MB) each of static memory (SRAM, Flash, or similar). The LPC4300 has an EMC (External Memory Controller) that allows connection of 4 banks of memory including SDRAM, 2 of which can hold 256 MB each, giving a much less expensive way to connect a lot more memory.

The SPI Flash Interface also allows connection of supposedly-less-expensive serial Flash chips (instead of parallel ones).

If you want something simple, then I'm not sure you need something that has more than 2 inputs. Either way, both the STM32F4 and the LPC4300 have 2 I2S interfaces so you can easily go up to 4 inputs and outputs on each.

What do you mean by 'minimise fixed user interface'?

If you expect there to be some set of inputs such as rotary encoders and/or potentiometers, I actually think that it's an easier model to handle for experimentation if the values from any available user interactions 'automagically' appear in variables that the program can simply use without having to factor in any particular delay in acquisition; so having the reading of values from encoders / potentiometers handled by a secondary processor - which the user never actually has to programs themselves! - actually would make things conceptually easier.

You mention that your current design is simpler than the MICrODEC one (by the way, you're missing a 'C' at the end of 'MICrODEC') – would you care to elucidate?

Best wishes,

// Christian Brunschen

Posted 4 years ago #
gbulmer
Moderator

If I read the datasheet correctly, the STM32F4 has an FSMC (Flexible Static Memory Controller) that allows connection of 4 banks of 128 MB (26 address lines, 16 bit wide memory bus => 128 MB) each of static memory (SRAM, Flash, or similar). The LPC4300 has an EMC (External Memory Controller) that allows connection of 4 banks of memory including SDRAM, 2 of which can hold 256 MB each, giving a much less expensive way to connect a lot more memory.

I agree with your analysis. As I said, my use cases are different.

Ideally I'd like to have the simplest board possible. That might mean no external memory, use a 64-pin part, and enable the kids to use the same board to make a robot or a music gizmo, or blink LEDs like KITT in Knight Rider. Minimise extras.

If you want something simple, then I'm not sure you need something that has more than 2 inputs. Either way, both the STM32F4 and the LPC4300 have 2 I2S interfaces so you can easily go up to 4 inputs and outputs on each.

I mean simplicity of hardware, not simplicity of concept or purpose.
I don't need to do much to the ADC inputs other than make them accessible. I can make a board which carries a microphone, or small amplifier, or socket, and filter which the users can plug in. Similarly with output.
If a kid (or adult) wants to try and sing a duet with a bat, and has a go at building it, I will be utterly thrilled.

What do you mean by 'minimise fixed user interface'?
If you expect there to be some set of inputs such as rotary encoders and/or potentiometers, ...

I don't want to have anything more than gain-control on the analogue side. Ideally everything else is for the user to choose.

A key goal is to provide an educational experience for people who have never programmed, or have done very little programming.
A goal is to give them a something which provides the fewest constraints, and allows them to explore and discover.
It is not a goal to give them a perfect audio solution out of the box.

... (by the way, you're missing a 'C' at the end of 'MICrODEC ...

Thank you. Sorry.

You mention that your current design is simpler than the MICrODEC ... would you care to elucidate?

May I delay that? We need to agree stuff, which may later change.

I will say, take a *near minimal* usable stereo analogue audio chain (buffer, low-pass filters, gain controls), with normal audio-quality ADC/DAC, with an STM32F103, and accessible pins, and you are mostly there.

An issue is external RAM. I am not currently intending to have more RAM than Native. Even that seems like too much of a luxury :-)

Another issue is how ready-to-go is it. I like the "ready to go" argument to give a great 'out-of-the-box-experience'.
I also like the minimalist "let them choose as much as possible" argument.
My 'druthers' is as little as possible that will 'do audio' out of the box.

I think that is plenty for a person to learn with, and has lots of potential.

I am only one voice. We have a proposed process which generates evidence, so I am much happier getting some evidence than using 'gut'.

If we discover it is too hard for our users to build a physical UI, then we can change. If we discover that building the physical UI is very interesting to some learners, and maybe they would like even more options, then that needs to be taken forward.

Okay?

(full disclosure: I am not a member of LeafLabs staff.)

Posted 4 years ago #
cbrunschen
Member

If I understand correctly, your use case is more aimed towards offering audio in addition to other aspects - blinking lights, robot control, etc - as a general teaching & experimentation platform?

That's quite different indeed from what I had in mind, so it's not surprising you'd prefer a different set of tradeoffs: Since my thoughts were more in the direction of a platform specifically for _audio_ generation & processing, and experimentation and playing with _those_, having a more predefined set of user controls (pots & encoders) would make more sense, even though for your use that would be too restrictive and at the same time adding needless complexity.

Best wishes,

// Christian Brunschen

Posted 4 years ago #
gbulmer
Moderator

If I understand correctly, your use case is more aimed towards offering audio in addition to other aspects - blinking lights, robot control, etc - as a general teaching & experimentation platform?

I apologise, I guess I am not expressing myself clearly.
It is intended to be part of a general teaching & experimentation platform.
It does not need all the facilities for all the experiments.
A goal is to have high enough audio quality on this module so that users can do many of the obvious fun and interesting audio experiments, and more, but it is not an 'application specific' solution, so it won't have all of the controls that, for example MICrODEC has.
It is intended to be extensible, so it should be able to blink LEDs, maybe control motors etc. for a robot.

Yes, we are balancing different sets of requirements and tradeoffs.

Posted 4 years ago #

RSS feed for this topic

12 Next »

Reply »

You must log in to post.