LeafLabs Garden » Topic: Sending data over USB

LeafLabs Garden » Topic: Sending data over USB http://forums.leaflabs.com/topic.php?id=639 A place to share, learn, and grow... en-US Fri, 22 Jan 2016 00:18:33 +0000 http://bbpress.org/?v=1.0.2 <![CDATA[Search]]> q http://forums.leaflabs.com/search.php gbulmer on "Sending data over USB" http://forums.leaflabs.com/topic.php?id=639&page=2#post-7595 Thu, 12 Jan 2012 15:06:08 +0000 gbulmer 7595@http://forums.leaflabs.com/ gagol - It might be helpful to know that Maple comes with the Open Source Newlib implementation of most of the Standard C library: <a href="http://sourceware.org/newlib/" rel="nofollow">http://sourceware.org/newlib/</a> The standard <code>atoi</code> has already been implemented. Use it as: <pre><code>#include <stdlib.h> //... char buffer[] = "00535"; int value = atoi(buffer[0]);</code></pre> All strings, e.g. <code>char buffer[] = "00535";</code> are terminated with a '\O' (end of string) character, so there is no need to pass the length of the string to <code>atoi()</code>. The atoi function converts until it reaches the end of string, or a non digit character. Newlib includes a further set of conversions functions including strtol, strtoul, etc. which handle number bases other than 10. It also includes <code>sscanf()</code> which can extract pieces of string and convert them into any basic C types. While is is only a convention, functions which convert to unsigned numbers, like that atoint, might be better named atouint. The comments might also mention the restriction that it only works for unsigned integer strings, so that people don't misunderstand. It might also be worth mentioning in the comment that it has 'unpredictable' results if the string contains characters which are not ASCII digits. The usual way of doing the conversion from a string of ASCII digits to integers starts with something like this: <pre><code>#include <ctype.h> uint32 atouint(char* t) { uint32 val = 0; while (*t != '\O' && isdigit(*t)) { val = val*10 + (*t - '0'); ++t; } return val; }</code></pre> This checks that the characters in the input string are the ASCII digits. It dispenses with the need to track the number of characters in the string, by relying on the end of string, and doesn't separately track powers of 10 because it converts left to right. A small additional point if you are aiming to produce code for other people to read. Through hundreds of years of research and experience, we believe that in general, mixed case text is easier to read than UPPER CASE text. We also believe that lower case text is more readable than upper case text, hence the reason that C is a lower case programming language. Mixed upper and lower case text, or only lower case, has a lot more 'shape variation' than upper case only, and so we read it more quickly and more reliably. HTH (WARNING: I haven't compiled and tested the code, it is just an illustration) (PS - because of the abysmal forum software, which strips '\' followed by '0', I have used the letter O to create '\O' to stand for the character with the value 0. Yes, ugly) gagol on "Sending data over USB" http://forums.leaflabs.com/topic.php?id=639&page=2#post-7593 Thu, 12 Jan 2012 13:16:17 +0000 gagol 7593@http://forums.leaflabs.com/ I'm a little late, but if you still needs to convert ascii to int, here is a little function that works for me : char buffer[] = "00535"; int value = atoint(buffer[0], 5); The code requires leading 0. uint32 atoint(uint8 *t, uint8 _digits) // CONVERT ASCII NUMBERS TO INT, PASS POINTER TO *CHAR LOCATION OF THE BEGINNING OF { uint32 _val = 0; xb = 1; // FACTOR FOR EACH DIGIT PROCESSING while(_digits--) // FOR EACH DIGIT STARTING FROM RIGHT END GOING LEFTWARD { _val += (t[_digits] - '0') * xb; // ADD CURRENT DIGIT * FACTOR POWER TO VALUE xb *= 10; // AUGMENT FACTOR POWER BY 10 } return _val; // RETURN INT } gbulmer on "Sending data over USB" http://forums.leaflabs.com/topic.php?id=639&page=2#post-7456 Wed, 28 Dec 2011 11:05:32 +0000 gbulmer 7456@http://forums.leaflabs.com/ I have no idea if there is bit shifting in vb2010, but it isn't needed. A left shift of 8 (which reverses the right shift of 8 in writebuff) is exactly he same as multiplying by 2^8, which is 256. So, if you read the values as 0 to 255, then to rebuild a 16 bit value an expression would be something like <code>wholeValue = read0 + (read1 * 256)</code> where read0 is the first byte and read1 is the second byte read. gcam on "Sending data over USB" http://forums.leaflabs.com/topic.php?id=639&page=2#post-7455 Wed, 28 Dec 2011 05:02:06 +0000 gcam 7455@http://forums.leaflabs.com/ hi (google traslator), I send the data on the USB with the following instructions void writebuff(uint16 value) { if (buffindex >= sizeof(buffer)-1) { SerialUSB.write(&buffer[0], sizeof(buffer)); buffindex = 0; } buffer[buffindex++] = value & 0x00ff; buffer[buffindex++] = (value>>8) & 0x00ff; } void loop() { writebuff(analogRead(0)); // x axis (value +- 32000) writebuff(analogRead(1)); // y axis (value +- 32000) writebuff(analogRead(2)); // z axis (value +- 32000) ... } On the PC (visual basic 2010) I read the values are between 0 and 255. How do I go back to the values obtained from the accelerometer. mbolivar on "Sending data over USB" http://forums.leaflabs.com/topic.php?id=639&page=2#post-3928 Thu, 17 Mar 2011 07:34:15 +0000 mbolivar 3928@http://forums.leaflabs.com/ StephenFromNYC, "ajme..." is poslathian on the LeafLabs forum. I asked how he obtained his figure, and (as far as I recall) his method was (on the host PC side) to use the Unix "cat" utility to pipe the output of the virtual serial port's device to a file in Linux (e.g. $ cat /dev/ttyACM0 > some-file). Meanwhile, the Maple was sending data in a tight loop using the low-level usbSendBytes() function. He then timed how long it took for the file to reach a certain size. I'd like to emphasize that this is only my recollection of his methods. Poslathian, perhaps you'd care to weigh in? I personally have been unable to achieve transfer rates greater than ~30KB/s using the current code. StephenFromNYC on "Sending data over USB" http://forums.leaflabs.com/topic.php?id=639&page=2#post-3912 Wed, 16 Mar 2011 10:50:19 +0000 StephenFromNYC 3912@http://forums.leaflabs.com/ March 16, 2011 Hello mbolivar- Any feedback from the author ("ajme...") who claimed a rate of 150 kilobytes/sec using <code>SerialUSB()</code> and the February 2011 snapshot? I appear to be getting a fast data transfer rate, but the rate does not appear to be consistent (depending on how the rate is tested the transfer rate bounces up and down a lot). It would help me to see the code and test method used by "ajme..." Thanks! Stephen from NYC mbolivar on "Sending data over USB" http://forums.leaflabs.com/topic.php?id=639&page=2#post-3721 Wed, 23 Feb 2011 23:18:38 +0000 mbolivar 3721@http://forums.leaflabs.com/ Stephen, <blockquote> mbolivar, in your comment from: <a href="http://code.google.com/p/leaflabs/issues/detail?id=10" rel="nofollow">http://code.google.com/p/leaflabs/issues/detail?id=10</a> Can you please share the code you used to determine the current SerialUSB() rate to be about 150KB/second (using the February 2011 snapshot)? </blockquote> It seems like you're referring to this comment, which was not left by me: <a href="http://code.google.com/p/leaflabs/issues/detail?id=10#c4" rel="nofollow">http://code.google.com/p/leaflabs/issues/detail?id=10#c4</a> I am personally unable to reproduce those results, but I will ask the comment's author to do so :). gbulmer on "Sending data over USB" http://forums.leaflabs.com/topic.php?id=639&page=2#post-3703 Tue, 22 Feb 2011 20:23:34 +0000 gbulmer 3703@http://forums.leaflabs.com/ StephenFromNYC <blockquote>In other words are the next two lines equivalent (in terms of machine code created by the compiler)? <code>SerialUSB.write(&buffer[0], sizeof(buffer));</code> <code>SerialUSB.write(buffer, sizeof(buffer));</code> </blockquote> Yes, in this case the machine code generated by the compiler for <code>&buffer[0]</code> and <code>buffer</code> should be identical, and should be identical in general. I did the <code>&buffer[0]</code> deliberately for a couple of reasons; partly to ensure the difference between a pointer (unsigned char *) and an array is clear (the number of bugs I've seen involving unallocated space!-), and partly it makes it more obvious (to me) that there may be a bigger buffer (than 64 bytes). For example, if the ADC values were being saved into a buffer using DMA, then the DMA 'end event' might trigger after 64 bytes worth of results. Then the DMA event handler might reset the DMA to another buffer, or the other half of the same buffer, and trigger sending the USB data to the host. That sort of approach might give a more uniform sampling rate for the set of ADC samples (I don't know how long the SerialUSB.write(...) might block for, so this is an unknown concern). (I could have defined something like MAX_USB_PACKET, and used that instead of sizeof(), but it started to clutter the code.) (Technically it was the case that <code>buffer</code> is a <code>const</code> pointer, so is only a right-hand-side expression because it can't be assigned to, but I haven't worried about that level of C language precision for years, and I'd need to check Harbison & Steele <a href="http://www.amazon.com/Reference-Manual-Samuel-P-Harbison/dp/013089592X/ref=dp_ob_title_bk" rel="nofollow">http://www.amazon.com/Reference-Manual-Samuel-P-Harbison/dp/013089592X/ref=dp_ob_title_bk</a> but I can't find my copy of the 5th edition :-( StephenFromNYC on "Sending data over USB" http://forums.leaflabs.com/topic.php?id=639&page=2#post-3701 Tue, 22 Feb 2011 17:48:10 +0000 StephenFromNYC 3701@http://forums.leaflabs.com/ Hello, Yes, I understand that <code>buffer[buffindex++]</code> increments the value of buffindex AFTER the array assignment is made. I understand that the lines <code>buffindex = buffindex + 1;</code> can both be replaced by either <code>buffindex++</code> or <code>++buffindex</code>. I have a question about expressions and compiler optimizations used by the Maple. In the line <code>SerialUSB.write(&buffer[0], sizeof(buffer));</code> is there any difference between <code>&buffer[0]</code> (the address of the zeroth element of the array named buffer; in other words the beginning of the array) versus just using just "<code>buffer</code>". In other words are the next two lines equivalent (in terms of machine code created by the compiler)? <code>SerialUSB.write(&buffer[0], sizeof(buffer));</code> <code>SerialUSB.write(buffer, sizeof(buffer));</code> Thanks! Stephen from NYC gbulmer on "Sending data over USB" http://forums.leaflabs.com/topic.php?id=639&page=2#post-3694 Sun, 20 Feb 2011 19:38:25 +0000 gbulmer 3694@http://forums.leaflabs.com/ StephenFromNYC - yes my code may be a bit confusing. I was focusing on sending the data so that it could be accurately reconstituted. I try to ensure the sequence of bytes is defined by my code rather than assuming both sides are little endian or big endian. <blockquote>The loop is executed eight times for a total of 64 bytes transferred. </blockquote> The loop transfers 64 bits, 8 bytes, not 64 bytes. <blockquote>Why is it not possible to send the entire variable res using a single SerialUSB.print() command? For example, SerialUSB.print(ser, LONG LONG). </blockquote> This doesn't work because: a. there is no such function as SerialUSB.print(ser, LONG LONG), (and I'm not sure what LONG LONG signifies, a parameter to print, or the data type <code>long long</code> with capitalisation for emphasis?) and b. all of the numerical print functions convert binary values into strings of ASCII digits. For example a 4 byte uint32 could be anything from 1 ASCII digit to 10 ASCII digits long. So even if it did exist, it is likely that it would convert binary to ASCII. The way to transfer a 64 byte block in a single USB packet is to do a <code>SerialUSB.write(void *, uint32);</code> with a 64 byte buffer. If you only need 20,000 12-bit data points/second, then I think the Maple can do that. The inefficiency is sending near-empty packets, not the peak throughput. I'd likely just send 32 values/64 byte packet, and avoid any concerns about packing for now. You have several choices about how to pack stuff into a 64 byte buffer. If there is no bit-level packing, I'd do something like this: <pre><code>unsigned char buffer[64]; int buffindex = 0; void writebuff(uint16 value) { if (buffindex >= sizeof(buffer)-1) { SerialUSB.write(&buffer[0], sizeof(buffer)); buffindex = 0; } buffer[buffindex++] = value & 0x00ff; buffer[buffindex++] = (value>>8) & 0x00ff; } void loop() { writebuff(analogRead(0)); writebuff(analogRead(1)); writebuff(analogRead(2)); writebuff(analogRead(3)); writebuff(analogRead(4)); // other stuff }</code></pre> (WARNING code not checked) This handles the write when the buffer gets full, without any packing. Take things from there. If packing becomes important, handle it inside writebuff. PS - the <pre><code>buffer[buffindex++] = value & 0x00ff; buffer[buffindex++] = (value>>8) & 0x00ff;</code></pre> maybe easier to read as <pre><code>buffer[buffindex] = value & 0x00ff; buffindex = buffindex + 1; buffer[buffindex] = (value>>8) & 0x00ff buffindex = buffindex + 1;;</code></pre> (sorry that the indentation is broken, I can't get leading spaces into the first line using backtick) StephenFromNYC on "Sending data over USB" http://forums.leaflabs.com/topic.php?id=639&page=2#post-3693 Sun, 20 Feb 2011 18:02:26 +0000 StephenFromNYC 3693@http://forums.leaflabs.com/ Hello, I think I have confused other users and myself, so let me step back and try to summarize what I understand from recent posts about how to optimize <code>SerialUSB()</code> data transfers. Then at the end of this post I will try to refocus my questions. In my post: <a href="http://forums.leaflabs.com/topic.php?id=639#post-3629">http://forums.leaflabs.com/topic.php?id=639#post-3629</a> I was wrong to focus on how to send just five 12-bit analog reads, because I incorrectly thought that sending five 12-bit values was the most one could fit into a single USB 64-byte packet. In my hurried and faulty logic I got my bits and bytes confused. Obviously, 60-bits is a lot smaller than 64-bytes. Forty (40) 12-bit values acquired using <code>analogRead()</code> should fit into a single USB 64-bit packet. For the moment, let's just stick with the goal to efficiently send five 12-bit values using <code>SerialUSB()</code>. I feel like I am missing something really fundamental in my reading of gbulmers Maple code (<a href="http://forums.leaflabs.com/topic.php?id=639#post-3660">http://forums.leaflabs.com/topic.php?id=639#post-3660</a>) which he said was compiled but not tested. I believe the five 12-bit values are first combined into a single variable (<code>res</code>). <code>res</code> is then broken down into pieces and each char is sent to the host using <code>SerialUSB()</code>. I understand the line from gbulmer's code: <code>long long res = (a[4]<<48) | (a[3]<<36) | (a[2]<<24) | (a[1]<<12) | a[0];</code> This uses bitshifting and ORs to combine the five 12-bit values into a single variable of type <code>long long</code>. However, I am confused about gbulmer's for loop: <blockquote> for (int i=0; i<8; i=i+1) { unsigned char c = res & 0xff; res = res >> 8; SerialUSB.print(c, BYTE); } </blockquote> This code uses a mask to strip the higher bits. Then to prepare the variable <code>res</code> for the next cycle of the loop bitshifting (<code>res = res >> 8</code>) is used before the value of c is sent via <code>SerialUSB.print(c, BYTE)</code>. The loop is executed eight times for a total of 64 bytes transferred. Why are multiple <code>SerialUSB.print()</code> statements used if the goal is to send the five 12-bit values in a single USB 64-byte packet? My assumption is that each time (for a total of eight) <code>SerialUSB.print()</code> is called the data is transferred over USB even if the 64 byte packet is mostly empty (not filled with data). If this assumption is wrong, does the USB system wait until the eighth <code>SerialUSB.print(c, BYTE)</code> command before a single, completely filled, 64-byte packet is sent? Why is it not possible to send the entire variable <code>res</code> using a single <code>SerialUSB.print()</code> command? For example, <code>SerialUSB.print(ser, LONG LONG). In the host routine, the process is reversed. Eight </code><code>SerialUSB.read()</code>s are used to fetch each 8-bit value, the variable res is shifted by eight bits to prepare for the use of OR. This combines eight 8-bit reads into a single <code>long long</code>. In the host code, should the line: <code>a[j] = res & 0x0fff;</code> instead be: <code>a[j] = res & 0xff;</code> Here are my refocused questions: Goal (present): I want to use my Maple to acquire 12-bit data via <code>analogRead()</code> as quickly as possible and to send the data to the host via USB as quickly as possible (without any data being dropped). A higher data transfer rate will let me apply software filters to the data set (for example, to remove 60 Hz noise from the data). I do not know if I will use the Maple to do the filtering or use the host. I will be happy to shift the filtering to the host if this maximizes the data transfer rate. A transfer rate of 20,000 data points a second (where each data point is a 12-bit <code>analogRead()</code> value) should be sufficient for my long term goals. However, a rate which is 10X slower will make me happy in the short term and will keep me plugging away. Goal (future): DMA will speed up the data transfer between the Maple and a host system, but I think for my purposes DMA is not needed. I may be wrong to minimize this task, but at the moment, the exact rate of analog reads is not that important to me. That can be a problem for another day. I appreciate that two bytes are wasted using <code>SerialUSB().println()</code><code>, but I was using </code><code>SerialUSB.println()</code>, to see the boundaries between analogRead() values when I debugging. mbolivar, in your comment from: <a href="http://code.google.com/p/leaflabs/issues/detail?id=10">http://code.google.com/p/leaflabs/issues/detail?id=10</a> Can you please share the code you used to determine the current <code>SerialUSB()</code> rate to be about 150KB/second (using the February 2011 snapshot)? I understand that at most five 12-bit values fit into one <code>long long</code> (using the line from gbulmer's bitshifting code). A <code>long long</code> is eight bytes, so at most forty (40) 12-bit numbers fit into 64 bytes. Therefore, in the absence of data compression/expansion a single USB 64-byte packet can transfer forty (40) 12-bit values (acquired using <code>analogRead()</code>). My next question is: "How do I combine eight <code>long long</code> values into a buffer which I can efficiently send in a single 64-byte USB packet using <code>SerialUSB.write(buffer, 64)</code>"? I assume it will be necessary to flush before calling <code>SerialUSB.write(buffer, 64)</code>: <a href="http://forums.leaflabs.com/topic.php?id=575">http://forums.leaflabs.com/topic.php?id=575</a> Thanks and sorry for any confusion caused! Stephen from NYC gbulmer on "Sending data over USB" http://forums.leaflabs.com/topic.php?id=639#post-3687 Sun, 20 Feb 2011 11:09:52 +0000 gbulmer 3687@http://forums.leaflabs.com/ StephenFromNYC - You could send the 12-bit values as simple two byte values. That would be very easy to do. But, you need to pack them into a 64-byte packet anyway, so it isn't hugely simpler. The code above packs 5 12-bit numbers (60 bits) into 1 <code>long long</code>. A <code>long long</code> is 8 bytes, so they are easy to pack into 64 bytes. 5 12-bit number/<code>long long</code> gives 5 12-bit number x 8 <code>long long</code> number / 64 bytes = 40 12-bit numbers use similar code (to above) to transfer the data. Don't use println, it is a waste of two bytes ("\r\n"). Don't try to write the <code>long long</code> as a single value because that will transform the number to a sequence of ASCII characters, which will drop the transfer efficiency from 8 bits to approximately 3.3 bits/byte, defeating the entire effort. Use SerialUSB.write(buffer, 64) to send the whole packet in one unit. StephenFromNYC on "Sending data over USB" http://forums.leaflabs.com/topic.php?id=639#post-3686 Sun, 20 Feb 2011 10:39:16 +0000 StephenFromNYC 3686@http://forums.leaflabs.com/ Hello, gbulmer, yes, I got my variable types switched. Thanks for the careful read. <code>long long</code> is what is needed. Here is a "hopefully someone can quickly answer this question, because I am too lazy to figure this one out" question. What is the largest number of 12-bit numbers which can be combined into a single long long which is then efficiently transferred in a single 64 byte block using <code>SerialUSB.println()</code>? Thirty two? If I want to use <code>println()</code> instead of <code>print()</code> is thirty one the largest number which should be combined? I understand that avoiding cross byte boundary errors (pointed out by mbolivar) would be easiest at the maple end and also the host ends by bit-shifting in increments of 12 (assuming the goal is to combine 12-bit analog read data). I believe this wastes 4-bits per number, but it makes decoding on the host end easier. Thanks! Stephen from NYC gbulmer on "Sending data over USB" http://forums.leaflabs.com/topic.php?id=639#post-3681 Sat, 19 Feb 2011 11:20:44 +0000 gbulmer 3681@http://forums.leaflabs.com/ StephenFromNYC <blockquote>code snippets needed to merge five 12-bit analog reads into a single variable of type uint16</blockquote> This is probably just a slip of the pen/finger but the five 12 bit values are merged into a <code>long long</code>, not a <code>uint16</code> The unpacked values are returned in an array of <code>uint16</code> values. mbolivar on "Sending data over USB" http://forums.leaflabs.com/topic.php?id=639#post-3674 Fri, 18 Feb 2011 17:34:48 +0000 mbolivar 3674@http://forums.leaflabs.com/ Stephen, Yes, you could pack the data that way, although you'd have to be careful to get the details right, since the readings would cross byte boundaries in an uneven way.