LeafLabs Garden » Topic: High Speed Data Acquisition

LeafLabs Garden » Topic: High Speed Data Acquisition - What's the best to expect? http://forums.leaflabs.com/topic.php?id=1292 A place to share, learn, and grow... en-US Fri, 22 Jan 2016 00:20:37 +0000 http://bbpress.org/?v=1.0.2 <![CDATA[Search]]> q http://forums.leaflabs.com/search.php djk on "High Speed Data Acquisition - What's the best to expect?" http://forums.leaflabs.com/topic.php?id=1292#post-7927 Sat, 28 Jan 2012 09:20:14 +0000 djk 7927@http://forums.leaflabs.com/ tend to agree with gbulmer's comments. working in the data acq industry for many years now i think trying to do a high-speed project this on open source hw/sw you are going to find out why professional systems do cost $. don't forget overall signal accuracy (not the same as resolution), Anti-alias filtering, isolation, handling common-mode noise etc...by the time you hire someone to do all of this work you'll have paid for a proper system. not to mention if you are going to do published research, i'd think you'd want some form of calibrated traceability (NIST). All this being said, the very reason the thread caught my eye is that i believe there is a market for an nicely packaged (rugged), physically small, open-source data acq/toolkit with properly developed hw for applications such as this....where one could easily port their algorithms (such as for HUMS) to a platform. gbulmer on "High Speed Data Acquisition - What's the best to expect?" http://forums.leaflabs.com/topic.php?id=1292#post-7819 Sat, 21 Jan 2012 07:08:18 +0000 gbulmer 7819@http://forums.leaflabs.com/ bubulindo - IMHO, you do NOT need DMA to get close to 1Msamples/second. People have spent a long time trying to understand and use the DMA. It might be more productive to get other parts of your application working before attempting to apply DMA. Your code will spend a lot of time servicing the ADC, but code will likely be enough to make progress. Another thing folks should be aware of is the DMA on the STM32F4 is different in architecture from the STM32F1, so code would need to be ported. IMHO, a big problem is the quality of the input signal. The ADC functions in libmaple are configured to give plenty of time to charge up the sample and hold circuitry on the input of the ADC. This makes them relatively robust for beginners to use. The setup can (theoretically) cope with sampling a signal with an output impedance of 50Kohm. Have a look in the STM32F103xb/xB datasheet, section 5.3.18 "12-bit ADC characteristics", at Equation 1, and Table 47. To get to a 1.5cycle sample time, the output impedance of the signal being sampled needs to be about 0.4Kohms. A lot of source signals are no where near that good. For example, lots of people use a 10Kohm potentiometer as an input, which is way to high. It is also worth noting, that 1Msamples/second is an absolute maximum, and can't be achieved at the noromal 72MHz main processor clock. Even at 1.5 cycles sample, the ADC conversion cycle will take about 1.17microseconds (1.17us) when running the main processor clock at 72MHz. To get the maximum speed of conversion (1us), the main clock must run at an integer multiple of 14MHz; 14MHz is the maximum ADC clock, and is derived by dividing down the main processor clock. Simultaneously, the main processor clock is constrained to be an integer multiple of 8MHz, the crystal frequency; that 8MHz crystal is multiplied up to generate the main processor clock. The lowest common denominator, which would give 1Msamples/second is 56MHz. It is probably easier to leave this alone, run at 72MHz, and accept 1.17us ADC conversion time :-) bubulindo on "High Speed Data Acquisition - What's the best to expect?" http://forums.leaflabs.com/topic.php?id=1292#post-7817 Sat, 21 Jan 2012 00:51:38 +0000 bubulindo 7817@http://forums.leaflabs.com/ Hello, Thanks for the reply. My question wasn't based on the standard analogRead() since I know that it takes up too much to do anything and doesn't let you configure it either. I was actually thinking about writing software to configure the ADCs in many ways but without verification code and the thing that made me stop a bit was the DMA since that apparently can make wuite a big difference in the speed of the ADC. I'll read up on the DMA part of the manual to start messing with the ADC as soon as I have the chance. :) gbulmer on "High Speed Data Acquisition - What's the best to expect?" http://forums.leaflabs.com/topic.php?id=1292#post-7816 Fri, 20 Jan 2012 20:43:03 +0000 gbulmer 7816@http://forums.leaflabs.com/ AeroProf - Yes, Maple Native or RET6 are options. They are the same part family (STM32F103) as Maple, so they are 72MHz parts, and the ADCs are 1Msamples/second, but unlike Maple, they both have dual DAC's, so they may be enough to drive pulses. Also they have 3 ADCs (which might be enough to triangulate signals with a small sensor array). The extra MByte of external RAM on Native might be handy for buffering samples. By the summer (when you can get your tame student), I'd expect STM32F4 boards and software to be stable. (Full disclosure: I am not a member of LeaLabs staff, and I have no financial relationship with LeafLabs) (Bed time now) AeroProf on "High Speed Data Acquisition - What's the best to expect?" http://forums.leaflabs.com/topic.php?id=1292#post-7815 Fri, 20 Jan 2012 20:16:28 +0000 AeroProf 7815@http://forums.leaflabs.com/ All - thank you for the comments and advice. We'll order a couple of the STM32F4 development boards from Mouser to add to our lab's collection - we have a Maple on the way at the moment. For that price even if we don't use them for this project, I'm sure we can find some other project to put them into. @gbulmer - Would the Maple Native or RET6 be options - I wasn't sure from the comments above? As for the programming/signal processing background...in all honesty mine is quite weak, just enough to be dangerous :) I'll be hiring a summer student to do all the heavy lifting and this gives me a better idea of the kind of student to recruit! Thanks again! gbulmer on "High Speed Data Acquisition - What's the best to expect?" http://forums.leaflabs.com/topic.php?id=1292#post-7810 Fri, 20 Jan 2012 10:50:33 +0000 gbulmer 7810@http://forums.leaflabs.com/ bubulindo <blockquote>@gbulmer, are those speeds with DMA activated? </blockquote> Each ADC can sample at 1Msamples/second irrespective of DMA. There has been several threads on this topic in the forum. Summary Maple will not give close to 1Msamples/second data rate using libmaple for three reasons: 1. libmape configures the ADC sample rate to be about 5microseconds/sample to make it more tolerant of low quality input signals. So even with DMA, the sample rate is relatively low (about 20x Arduino). To speed up the sample rate up, use a buffer amplifier (e.g. op amp) to drive the input signal to the ADC, and go straight to the ADC hardware to set the ADC sample rate to maximum speed. 2. libmaple does some validation and conversion of pin numbers before getting to the ADC hardware and starting it to take a sample, so there is a bit of lag between calling analogRead and starting the hardware ADC 3. libmaple ADC read (analoRead) 'blocks' waiting for the sample, and then it has function overhead returning the sample, so there is a bit of a lag after each sample is ready IIRC, libmaple analogRead gets samples at roughly 160ksps in software. It could give 1msamples/second in code, without DMA, if it were programmed carefully, but the code would spend a lot of its time grabbing samples. It is more efficient to grab blocks of ADC samples with DMA, and concurrently process them. gbulmer on "High Speed Data Acquisition - What's the best to expect?" http://forums.leaflabs.com/topic.php?id=1292#post-7809 Fri, 20 Jan 2012 10:33:17 +0000 gbulmer 7809@http://forums.leaflabs.com/ AeroProf -sorry for my slow follow up, I am on GMT, and had other things to do, like sleep :-) <blockquote>The lamb wave frequency I'd want to measure would be on the order of 25-250 kHz. I wouldn't monitor the structure continuously, only when I need to interrogate the health of it - so storing a second or two of data for further processing is fine if real time transmission isn't possible. </blockquote> These frequencies are completely beyond an Arduino. You'd need external hardware, and IMHO would really struggle to do anything with the data. I believe a reasonable assumption is capture at least 10 samples of a wave cycle to have a decent chance of doing anything with it (Nyquist–Shannon sampling theorem <a href="http://en.wikipedia.org/wiki/Nyquist–Shannon_sampling_theorem" rel="nofollow">http://en.wikipedia.org/wiki/Nyquist–Shannon_sampling_theorem</a> is a theoretical limit) So if you want to handle 250KHz, you should aim to sample at 2.5Msamples/second or more. A pair of ADC's on a Maple can run in an 'interleaved' mode, each alternately sampling the same input signal, to give double the sample rate, but even then, it is a bit slow. But you could certainly get started with Maple. For your 'production system' I'd recommend you consider the significantly faster STM32F4 MCU. It has 3 ADC's, each runs at 2.4Msamples/second, and all three can sample the same signal in an interleaved mode, giving a theoretical 7.2Msamples/second, so you'd have a bit of headroom. The processor runs at 168MHz, and has hardware floating point, making it easier to write processing and analysis code. It also has 192KBytes of RAM, which would be handy for buffering data. It has dual Digital to Analogue Converters (DACs), so you might be able to do some experiments using them to generate your pulses. MCU's in the STM32F4 families come in versions which can support High-Speed USB or Ethernet which might be just about (theoretically) quick enough to get the data off in real time (though IMHO hard to do in practice). There is a very low-cost development board using a member of that STM32F4 processor family available at the moment: <a href="http://www2.mouser.com/ProductDetail/STMicroelectronics/STM32F4DISCOVERY/?qs=J2qbEwLrpCGdWLY96ibNeQ%3d%3d" rel="nofollow">http://www2.mouser.com/ProductDetail/STMicroelectronics/STM32F4DISCOVERY/?qs=J2qbEwLrpCGdWLY96ibNeQ%3d%3d</a> It is about $18 because it is an ST development board to promote the MCU. Don't expect this to be available at this price forever, but it is a cheap, extremely powerful piece of hardware to get off the ground. To reach these sorts of performance levels, you will need to program straight to the hardware for some functions. For those things, the libraries will slow things down too much. You'll still be programming in C/C++, but will need to read the manuals for the hardware. Having set your expectation, I think you could do initial experiments, and start to really understand how to tackle the problems with higher level libraries. I have recently being doing some research on ultrasonic transmission & reception (though nothing implemented yet). In my reading, it is useful to have quit a lot of flexibility over the transmitted wave. So I'd likely try to use the on-board DAC's. IMHO the MCP4725 external DAC is likely too slow to generate a good quality 'excitation' wave. <a href="http://www.sparkfun.com/products/8736" rel="nofollow">http://www.sparkfun.com/products/8736</a> It is 12 bit, and only 3.4Mbps interface, so it would appear to be a little under 300ksamples/second which even at the Nyquist–Shannon sampling theorem limit is only a 150KHz signal. IMHO, you might as well use a Maple RET6, Maple Native or STM32F4 DAC. The AD9835 signal generator appears to be more useful, though I have barely glanced at the datasheet. A lot of this is quite sophisticated software. Have you a strong programming and signal processing background? bubulindo on "High Speed Data Acquisition - What's the best to expect?" http://forums.leaflabs.com/topic.php?id=1292#post-7806 Fri, 20 Jan 2012 06:46:15 +0000 bubulindo 7806@http://forums.leaflabs.com/ The latest Maple has a DAC onboard. :) AeroProf on "High Speed Data Acquisition - What's the best to expect?" http://forums.leaflabs.com/topic.php?id=1292#post-7805 Fri, 20 Jan 2012 06:18:46 +0000 AeroProf 7805@http://forums.leaflabs.com/ @bublindo, yes, I'm sending a short sine wave pulse to the piezo actuator on the structure. Typically this might be only a few cycles to create the initial lamb wave. For that I was looking at a DAC like: <a href="http://www.sparkfun.com/products/9169" rel="nofollow">http://www.sparkfun.com/products/9169</a> or <a href="http://www.sparkfun.com/products/8736" rel="nofollow">http://www.sparkfun.com/products/8736</a> Thanks! bubulindo on "High Speed Data Acquisition - What's the best to expect?" http://forums.leaflabs.com/topic.php?id=1292#post-7802 Fri, 20 Jan 2012 02:51:10 +0000 bubulindo 7802@http://forums.leaflabs.com/ When you say pulse the input signal, what do you mean? Some sort of actuator on the structure? @gbulmer, are those speeds with DMA activated? AeroProf on "High Speed Data Acquisition - What's the best to expect?" http://forums.leaflabs.com/topic.php?id=1292#post-7801 Thu, 19 Jan 2012 22:21:58 +0000 AeroProf 7801@http://forums.leaflabs.com/ Cool, thanks for the quick reply! Here's some additional info - I'm looking at detecting lamb waves propagating in structures - basically guided stress waves. The lamb waves would be generated with a piezo bonded to the part and then another piezo, or an array of piezos, would listen for the dispersed lamb waves. Commercial systems are very expensive, power hungry and heavy - we're using one now and its a beast. I think with open source hardware it can be done much less expensively with comparable performance - that's what I'd like to demonstrate. The lamb wave frequency I'd want to measure would be on the order of 25-250 kHz. I wouldn't monitor the structure continuously, only when I need to interrogate the health of it - so storing a second or two of data for further processing is fine if real time transmission isn't possible. Basically I would pulse the input signal, listen, record the propagating waves, run some analysis (compare the signal to a stored baseline) and then pulse again. It'd be nice to have an array of piezos, pulse repeatedly and listen to each receiver in turn. Thanks! gbulmer on "High Speed Data Acquisition - What's the best to expect?" http://forums.leaflabs.com/topic.php?id=1292#post-7800 Thu, 19 Jan 2012 20:52:31 +0000 gbulmer 7800@http://forums.leaflabs.com/ Would you tell us something about the kind of data rates and volumes you are aiming for so we can help a bit more? IMHO for Maple, the bottleneck is the speed that you can save data to storage, or get it off the board. Maple is theoretically about 200x faster than Arduino for analogue data sampling. An Arduino has one 10Ksamples/second (actually slightly slower) 10bit Analogue to Digital Converter (ADC). It can only sample one of 8 ADC input channels at a time (8 on a nano, 6 on an UNO). So you pack a channel number and value into 2bytes/sample. That is less than 20kBytes/second. A 2GByte SD memory would take about 100,000 seconds to fill, or about a day and a quarter (a bit less). The bottleneck is how fast it can sample data. Compare that with an ordinary Maple. Maple has two ADC, each one is 12 bits, and each can run at a maximum of 1Msamples/second. A Maple has 16 ADC channels it can sample, two at a time, so when packing the channel number into the results, the two bytes would be fully used. A Maple could capture at about 4mBytes of sample data/second. That is faster than full-speed USB. A 2GB SD memory would take about 500 seconds to fill, or a bit over 8 minutes. To be honest, it is hard to move that data rate off the board. The Maples sampling system isn't the bottleneck as much as output or storage. That sort of data rate would warm up my Mac laptop, and use a lot of the bandwidth writing to the hard drive. Arduino and Maple are very different beasts. If you used a Maple Native or RET6, they have three ADC's, giving 50% more throughput. It is actually hard to deal with the sort of data volumes a Maple could capture. Edit: The processor on the Maple is a 32bit processor, it can do an arithmetic operation on a 32bit quantity in a single cycle (and runs at 72MHz). The processor in the Arduino is an 8bit processor, so it takes several cycles (of its 16MHz) to operate on a 10bit quantity. So you could do more processing of the captured signals on an Maple than an Arduino. The difference is not as dramatic as the ADC performance difference, but it is still significant. (Full disclosure: I am not a member of LeafLabs staff) AeroProf on "High Speed Data Acquisition - What's the best to expect?" http://forums.leaflabs.com/topic.php?id=1292#post-7799 Thu, 19 Jan 2012 20:16:27 +0000 AeroProf 7799@http://forums.leaflabs.com/ So here's my challenge - whether or not Maple would be suitable for use as part of a portable, low cost, high speed data acquisition system. First, I'm a structures person and a total microcontroller newbie - although I'm learning fast - so pardon my question if it seems basic... Background - I'd like to explore the limits of high speed data logging with the Maple or other low cost microcontrollers for some structural health monitoring projects I'm working on. Here's what I've found on one person's experience with the Maple: <a href="http://thesolarpowerexpert.com/an-open-source-high-speed-data-logger/" rel="nofollow">http://thesolarpowerexpert.com/an-open-source-high-speed-data-logger/</a> Seems kind of low to me...especially in light of this Arduino project: <a href="https://sites.google.com/site/measuringstuff/the-arduino" rel="nofollow">https://sites.google.com/site/measuringstuff/the-arduino</a> Has anyone else explored this? Any advice or other devices/directions for me to consider? Thanks!