Hi,
I have a number defined as type uint64 (or unsigned long long).
The number is built out of a byte multiplied by 0x1000000.
Yet, whenever the byte > 0x80 which makes the resulting msb equals 1, I get a negative value.
uint64 is not supposed to go negative. It is a simple 64 bits positive integer.
Did anyone encounter similar situation, or have a reasonable explanation to this?
Thanks
samtal
Thanks juzza_s,
I have built a minimum code that shows the problem.
code at: http://pastebin.com/NF4jCuU8
What I am doing:
I have built a maple real-time clock.
To set the clock, I use a windows NT program that sends out via RS232 a reduced time stamp of 5 bytes (preceded by 02, length byte, ending with 03).
Example of sent data : 02 05 1E 87 0F AF 22 03
My program reads the bytes, fills in a byte array and then builds a time stamp of one uint64 value.
The attached code, when run, shows that if, for example, I will make the second data byte > 0x80, the total value will be reduced because the byte conversion will be negative.
As this has occupies too much of my time, I think if there is no solution I will find an easier way to set the RTC time, but I still want to understand what is wrong, thus any help will be appreciated, especially if there is a solution.
samtal.
gbulmer
Moderator
samtal - the value in the pastebin code is 0x100000000 which is too big to fit into an unsigned int.
By default, unsigned values are unsigned int, so the literal value should be stored as 0.
The top 1 is trimmed off. (The compiler should give a warning, but I haven't checked).
Try writing it as 0x100000000ULL
I am not yet convinced that is enough to explain everything, but it is a start.
The example program is significantly bigger than a minimal example.
It should initialise everything, so that there is no need to read input data; it is more robust to initialise than type input.
There is no example of what is printed, so it is not possible to confirm the same thing is output when run on my machine.
Here is something closer to a minimal example. It fully specifies the expected values by throwing away all of the input reading. You should initialise in_command with the actual values used, and republish. Then we can be confident we are testing the same thing.
#include "stdlib.h"
byte op_code;
byte in_command[20] = {
0x14, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09,
0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F, 0x10, 0x11, 0x12, 0x13
};
byte in_command_length;
uint64 datetime;
void setup()
{
Serial1.begin(38400);
}
void loop()
{
sort_inputs();
delay(1000);
}
void sort_inputs()
{
//Reconstruct the datetime variable by combining the 5 bytes as uint64
Serial1.println();
datetime = in_command[0] * 0x100000000; /* ERROR should be 0x100000000ULL */
Serial1.print("datetime="); Serial1.println(datetime);
datetime += in_command[1] * 0x1000000;
Serial1.print("datetime="); Serial1.println(datetime);
datetime += in_command[2] * 0x10000;
Serial1.print("datetime="); Serial1.println(datetime);
datetime += in_command[3] * 0x100;
Serial1.print("datetime="); Serial1.println(datetime);
datetime += in_command[4];
Serial1.print("datetime="); Serial1.println(datetime);
}I've run this on my Mac, which also defaults to 32-bit ints, and the results are not negative.
It will not compile because 0x100000000 is too big to fit into an int. When that is changed to 0x100000000ULL
The output is:
datetime=85899345920
datetime=85916123136
datetime=85916254208
datetime=85916254976
datetime=85916254980Which looks plausible.
I changed it to output in hex, and it is (still on my Mac):
datetime=1400000000
datetime=1401000000
datetime=1401020000
datetime=1401020300
datetime=1401020304Which looks correct to me. There are only 10 hex digits, which is 40-bits, so nothing is negative.
Unless the number is printed as 64bits long (which it is not) and the MSB is 1 (which would make the number hex 8 to F), then it just means that bit is set.
Please paste your output so that we can inspect it. (Preferably print hex as that is easier to check.)
One other possibility: Serial1.println(unsigned long long) has a bug.
Thanks gbulmer for a very very thorough and full reply.
The problem was solved by adding the ULL.
I simply overlooked the fact that the compiler relates to a constant as a signed integer, unless casted.
This also explains why I did not get the same problem with the small multipliers.
It still does not explain why this depended on the array byte's sign, which is supposed to be unsigned.
I also changed the array byte type to uint8 which is 100% unsigned, but still, the outcome depended on the multiplier, unless casted by ULL.
I run all my test on the actual Maple5 board.
I can now continue my work (hopefully) until the next time...
Thanks again.
samtal
gbulmer,
I run your code on IDE12, Maple-r5. Changed the print to HEX.
I did not add the ULL, and got good result as follows:
datetime=1400000000
datetime=1401000000
datetime=1401020000
datetime=1401020300
datetime=1401020304
When I changed the array inputs to 0x80, 0x81, 0x82, 0x83, 0x84 (msb=1),
I got the same error I had: (The second element is smaller than the first due to negative second element)
datetime=8000000000
datetime=7F81000000
datetime=7F81820000
datetime=7F81828300
datetime=7F81828384
I then added the ULL to the first two elements, then it was OK:
datetime=8000000000
datetime=8081000000
datetime=8081820000
datetime=8081828300
datetime=8081828384
As I wrote, although is seems to work, I do not understand why the array byte makes the difference rather than only the multiplier sign.
thanks and let me know what your conclusions are.
samtal
gbulmer
Moderator
C/C++ integer operations are always carried out as if the two types were the same.
According to these, conversions preserve the bits, but not signed-ness (since C99):
https://www.securecoding.cert.org/confluence/display/seccode/INT02-C.+Understand+integer+conversion+rules
http://msdn.microsoft.com/en-us/library/fc9te331%28v=vs.80%29.aspx
The first one at securecoding.cert.org has an example "Noncompliant Code Example" involving 'port' which has some of similarity with the problem code.
when 0x1000000 is an int then operation in_command[1] * 0x1000000 will give int.
Then the int is converted to uint64 for the datetime + (in_command[1] * 0x1000000);
So writing 0x1000000 as 0x1000000u will cause the expression in_command[1] * 0x1000000 to be unsigned int, so there will be no sign propagation.
Similarly explicity casting will also ensure nothing becomes signed (unsigned long long)in_command[1] * 0x1000000.
The most explicit (unsigned long long)in_command[1] * 0x1000000u is better.
If you added this code to your program:
void show_inputs()
{
Serial1.println();
datetime = (uint64)in_command[0] << 32;
Serial1.print("datetime="); Serial1.println(datetime, HEX);
datetime = ((in_command[1]) << 24);
Serial1.print("datetime=(uint64)"); Serial1.println(datetime, HEX);
datetime = (unsigned int)((in_command[1]) << 24);
Serial1.print("datetime=(unsigned int)"); Serial1.println(datetime, HEX);
datetime = (uint64)((in_command[1]) << 24);
Serial1.print("datetime=(no cast)"); Serial1.println(datetime, HEX);
datetime = in_command[2] << 16;
Serial1.print("datetime="); Serial1.println(datetime, HEX);
datetime = in_command[3] << 8;
Serial1.print("datetime="); Serial1.println(datetime, HEX);
datetime = in_command[4];
Serial1.print("datetime="); Serial1.println(datetime, HEX);
}and ran it, hopefully the problem would be clearer.
(There should be a warning from datetime = in_command[0] << 32;)
EDIT: This is the output (on my Mac):
datetime=8000000000
datetime=(uint64)FFFFFFFF81000000
datetime=(unsigned int)81000000
datetime=(no cast)FFFFFFFF81000000
datetime=820000
datetime=8300
datetime=84Summary: integer literals will be interpreted as signed unless explicitly written as unsigned. Since C99, integer conversions will not lose bits, but might loose signed-ness.
Thanks gbulmer,
You had to dig quite deep to get all that info. Chapeau.
No doubt that left shift is better than multiplication (and probably significantly faster).
Originally, this is what I had, but then when I started to get problems I changes to multiplication, and was left there. I will change it back and see if it will become cleaner.
Regarding your comments:
My understanding is that because I did not specify the number as unsigned, the compiler defaulted to signed int (or long-long).
Then, because the high number was assumed to be signed and negative (msb=1), the byte became also signed, and negative (also, msb=1).
Question:
1. How come two negative numbers gave a negative product?
2. What happened to the byte? did it lose its msb to the sign bit?
Well, it can become quite complex.
I'll juts make sure it works for me, and be more careful next time.
I'll keep the securecoding link. It is important.
I hope others will gain some knowledge of our work (mainly yours...)
samtal
gbulmer
Moderator
samtal - I would have written the code I posted, and been puzzled too, so I was pleased to learn.
Thanks for finding such a good question :-)
My understanding is that because I did not specify the number as unsigned, the compiler defaulted to signed int (or long-long).
Yes, int which is signed. I would bet money that that wasn't the case in K&R UNIX BSD C.
Then, because the high number was assumed to be signed and negative (msb=1), the byte became also signed, and negative (also, msb=1).
It is the fact that constant is a signed int that causes the first half of the problem. That is a problem waiting to happen.
The int constant is wider than the unsigned char, so the unsigned char needs to be converted (widened), and that is to an int. AFAIK C99 widens to int. I assume it wasn't adequately specified before C99 (or they needed to change C to keep lots of consultants in work - sorry being a bit cynical).
The unsigned char is multiplied or shifted (both fail) to an (signed) int. Then the (signed) int (which has its msb=1) is 'widened' to unsigned long long.
1. How come two negative numbers gave a negative product?
What are the two negative numbers? I think the only one is in_command[[].
2. What happened to the byte? did it lose its msb to the sign bit?
when it was multiplied, yes its top bit became the sign bit of an int.
You must log in to post.