LeafLabs Garden » Topic: using sizeof with a struct, strange behavior?

LeafLabs Garden » Topic: using sizeof with a struct, strange behavior? http://forums.leaflabs.com/topic.php?id=1319 A place to share, learn, and grow... en-US Fri, 22 Jan 2016 00:24:12 +0000 http://bbpress.org/?v=1.0.2 <![CDATA[Search]]> q http://forums.leaflabs.com/search.php gbulmer on "using sizeof with a struct, strange behavior?" http://forums.leaflabs.com/topic.php?id=1319#post-8121 Wed, 08 Feb 2012 05:45:40 +0000 gbulmer 8121@http://forums.leaflabs.com/ <>... is it really elegant and simple?</> My favourite Einstein quote is <a href="http://www.alberteinsteinsite.com/quotes/einsteinquotes.html">"Everything should be as simple as possible, but not simpler."</a> IMHO the problem with packing the struct and memcpy (or the for loop) at both ends of a communication is it relies on fragile underlying assumptions about two compilers, two runtime systems, two processors, and how the two programs might evolve. If this is likely to last for some time, then it might be worth taking steps to check that both programs match. Maybe exchange a 'version number' as part of the communication so something can check that the protocol is understood and common. I wrote code to read & write binary files created using this "packed structure" technique in the '80s. It was converting between a well defined CAD file format and a proprietary binary format used by a Pascal program. Unpleasant. With judicious use of macros and include, and avoiding unique constructs, I think it *might* be possible to use the same code on the Maple and C#. If there will very likely be a real problem keeping the code on both sides of the communication compatible, I'd be tempted to write two sets of macro definitions, and generate the code from a common source. djk on "using sizeof with a struct, strange behavior?" http://forums.leaflabs.com/topic.php?id=1319#post-8116 Tue, 07 Feb 2012 21:42:51 +0000 djk 8116@http://forums.leaflabs.com/ All noted thanks! FYI - DIY packing it as you suggested seems to work well as "struct __attribute__ ((__packed__))" seems to test OK. Is it the most efficient (size, speed) for this target? is it most portable for the future? is it really elegant and simple?....well all are separate questions to be asked and answered based upon what the goal and i need to re-visit this. BTW - in my case the host at the moment is C# so is it really a mirror? well sort of, in the way a circus mirror is technically a mirror. Very much appreciated. gbulmer on "using sizeof with a struct, strange behavior?" http://forums.leaflabs.com/topic.php?id=1319#post-8103 Mon, 06 Feb 2012 19:58:29 +0000 gbulmer 8103@http://forums.leaflabs.com/ It is probably worth adding for anyone reading this thread, that the overhead of doing the calls to the pack functions would be minimal if they were <code>static inline</code>, e.g. before they are used: <pre><code>static inline uint8_t* pack_uint8_t(uint8_t* dest, const uint8_t value) { dest[0] = value; return dest + sizeof(value); } static inline uint8_t* pack_uint16_t(uint8_t* dest, const uint16_t value) { uint8_t* valueptr = (uint8_t*)&value; if (isSameEdian) { dest[0] = valueptr[0]; // Warning: preserves endian dest[1] = valueptr[1]; // Warning: preserves endian } else { dest[0] = valueptr[1]; // Warning: CHANGES endian dest[1] = valueptr[0]; // Warning: CHANGES endian } return dest+ sizeof(value); }</code></pre> That would remove function call overhead. As the value of <code>isSameEdian</code> is constant, a c++ compiler should remove the if test. So it likely compiles to byte copies. It might actually generate smarter code. So it should be comparable to the <code>for</code> loop. It also has the benefit of its effect being a bit more explicit and predictable. Finally, the code on the host could be a mirror image, so not too hard to maintain. Even if extra fields are inserted into the structs on the host or Maple; the host and Maple structs wouldn't need to be in the same order or have all of the same fields, the communication order is in the code. gbulmer on "using sizeof with a struct, strange behavior?" http://forums.leaflabs.com/topic.php?id=1319#post-8100 Mon, 06 Feb 2012 19:18:34 +0000 gbulmer 8100@http://forums.leaflabs.com/ djk - you could pack the data if you really want to. You could try putting the UINT8 after the UINT16's. I have not checked this, so I am guessing, but <code>command</code>, the second UINT16, would cross a word, 4 byte, boundary. So it take two cycles to access memory, instead of one, so I'd hope the compiler tries to ensure things are quick (and predictable) with the compiler flags that Maple IDE applies. Also, I believe "bit band" addressing only works for aligned addresses. So it would be a handy if the compiler tried to ensure things are aligned by default. If things were exactly laid out the way they are needed, consider using <code>memcpy</code> instead of the <code>for</code> loop: <code>memcpy(dest, &tester, sizeof(tester);</code> I think gcc might 'inline' it. For 5 bytes it might not matter, but for larger chunks of memory, I vaguley remember gcc does some quite sneaky things. djk on "using sizeof with a struct, strange behavior?" http://forums.leaflabs.com/topic.php?id=1319#post-8094 Mon, 06 Feb 2012 14:19:58 +0000 djk 8094@http://forums.leaflabs.com/ Thank you both very very much! something elegant and simple about: for(i=0; i<sizeof(tester) i++) { *src++ = *dest++; } and being done with it! ah wishful thinking i guess. I'm aware of the endianess issues on the host side and was going to sort that bit and individual parsing there (it's headed for windows) i guess for some reason i didnt expect the packing if the M3 can handle bytewise alignment (or maybe i've been using 8 bit micro's a bit much lately). Also i'm building a multi-layer protocol and what i left out of my struct above was a payload byte array and a checksum byte. So i've got another protocol layer to set on top of the payload array that will have to deal with struct packing issues as well. At least now i know what to expect and how to respond. Again greatly appreciated! gbulmer on "using sizeof with a struct, strange behavior?" http://forums.leaflabs.com/topic.php?id=1319#post-8072 Sun, 05 Feb 2012 07:42:20 +0000 gbulmer 8072@http://forums.leaflabs.com/ djk - In general different processors can (and do) represent the same data types in different ways. Then different compilers, or different compiler settings, can pack data in different ways. Network protocols, like TCP/IP have to solve this problem. The first variation is ARM processors can store multi-byte numbers in a different byte order to the way x86 and x86-64 store multi-byte numbers. Another variation is a compiler for the same architecture might pack data with some compiler options, and not pack the data in the same way with different compiler options. As jcsalomon wrote don't try to send a struct in one chunk. So, to make a binary data stream to be predictable, copy the data to a byte buffer. That will ensure that code, that you control, lays the data out and you can remove dependencies on the compiler and processor. For example, the host processor might store a multi-byte number (e.g. UINT16) with the 'high' part of the number in the byte with a high address, while the Maple might store it in the byte with the low address. So, you might code it as (written on Mac, haven't got Maple next to me): <pre><code>#include <stdio.h> #include <stdint.h> /** packing functions */ const int isSameEdian = 1; // Preserve endian of numbers uint8_t* pack_uint8_t(uint8_t* dest, const uint8_t value); uint8_t* pack_uint16_t(uint8_t* dest, const uint16_t value); struct Tester { uint8_t header; uint16_t packetType; uint16_t command; } tester = { 'c', 0x00ff, 1234 }; int main (int argc, const char * argv[]) { uint8_t buff[sizeof(tester)]; uint8_t* buffptr = &buff[0]; buffptr = pack_uint8_t(buffptr, tester.header); buffptr = pack_uint16_t(buffptr, tester.packetType); buffptr = pack_uint16_t(buffptr, tester.command); printf("Size=%d\n", buffptr-buff); for (int i=0; i<buffptr-buff; ++i) { printf("%#4x ", buff[i]); } printf("\n"); return 0; } uint8_t* pack_uint8_t(uint8_t* dest, const uint8_t value) { dest[0] = value; return dest + sizeof(value); } uint8_t* pack_uint16_t(uint8_t* dest, const uint16_t value) { uint8_t* valueptr = (uint8_t*)&value; if (isSameEdian) { dest[0] = valueptr[0]; // Warning: preserves endian dest[1] = valueptr[1]; // Warning: preserves endian } else { dest[0] = valueptr[1]; // Warning: CHANGES endian dest[1] = valueptr[0]; // Warning: CHANGES endian } return dest+ sizeof(value); }</code></pre> This example gives a mechanism to guarantee the packing and byte order independent of the underlying processor or compiler. Change <code>isSameEdian</code> to <code>isSameEdian = 0</code> and you'll get the other byte ordering for multi-byte numbers. It happens that ARM Cortex-M3 can handle misaligned numbers, but other members of the ARM family, and some other processors can not handle misaligned numbers. Hence IMHO it is better to avoid relying on a particular processor+compiler+compiler-settings properties. (Code tends to get reused.) jcsalomon on "using sizeof with a struct, strange behavior?" http://forums.leaflabs.com/topic.php?id=1319#post-8069 Sun, 05 Feb 2012 03:56:42 +0000 jcsalomon 8069@http://forums.leaflabs.com/ You might try <blockquote> <code>struct __attribute__ ((__packed__)) tester {…};</code> </blockquote> but if the padding is there for a good reason, access to the mis-aligned members might be slow or cause a trap. A better solution is not to try sending the whole struct over the wire at once, but field-by-field — and if the machine at the other end isn't guaranteed to be the same model, make sure to send things in a well-defined byte order. djk on "using sizeof with a struct, strange behavior?" http://forums.leaflabs.com/topic.php?id=1319#post-8065 Sat, 04 Feb 2012 22:50:26 +0000 djk 8065@http://forums.leaflabs.com/ i'm having unexpected results using sizeof when trying to get the size of a struct in bytes when that struct contains multiple data types. Example: struct tester { UINT8 header; UINT16 packetType; UINT16 command; }; doing a println(sizeof(tester),DEC); to the serial monitor returns 6! I expect this to be 5? It appears there is a byte getting stuffed between UINT8 and UINT16. Does this have to do with storage boundaries on the STM32? because we are dealing with 32bit words? This causing a problem doing a pointer and bytewise copy (binary serial payload) on a serial stream of binary data i'm working with. suggestions and feedback greatly appreciated!