LeafLabs Garden » Topic: Loading data to / from flash

LeafLabs Garden » Topic: Loading data to / from flash http://forums.leaflabs.com/topic.php?id=1009 A place to share, learn, and grow... en-US Fri, 22 Jan 2016 00:13:06 +0000 http://bbpress.org/?v=1.0.2 <![CDATA[Search]]> q http://forums.leaflabs.com/search.php samtal on "Loading data to / from flash" http://forums.leaflabs.com/topic.php?id=1009&page=3#post-6812 Mon, 10 Oct 2011 11:47:48 +0000 samtal 6812@http://forums.leaflabs.com/ As I wrote (and you agreed): <blockquote> Lets close this issue for now. Thanks for your support that was overwhelming. I am sure I will need more of it. </blockquote> samtal gbulmer on "Loading data to / from flash" http://forums.leaflabs.com/topic.php?id=1009&page=3#post-6810 Mon, 10 Oct 2011 08:57:14 +0000 gbulmer 6810@http://forums.leaflabs.com/ samtal - <blockquote>The issue was with ram to flash, which you gave initially the impression that can be solved (was it my impression only?), while now you insist it is impossible. </blockquote> Yes it is only your impression. I did not mean to suggest that <code>memcpy(flash_array, ram_array, sizeof(flash_array));</code> could work. I had thought I was very clear and explicit, but I must believe that was not the case for you. Please post links, and ideally quotes, to my posts which caused that misunderstanding. I will endeavour to fix them so that no-one else will misunderstand. I did find sentences which might have been unclear, so I have edited them to try minimise the chances of anyone else misunderstanding. I believe that memcpy(flash, ram, ...) is not practical, and has never been practical (I think there is a special case, where it will work, but that has only caused confusion.) So I am going to be unambiguous and say memcpy(flash, ram, ...) WILL NEVER EVER WORK. Even though I believe that is slightly inaccurate. Maybe this metaphor will help. NEVER EVER jump (or fall) from an aeroplane, while it is in more than 1000 feet up the air, without a working parachute, you will die. Clearly, that isn't *always* true. People have fall from an aeroplane without a parachute, and survived. Maybe badly injured, but not dead. <a href="http://www.topyaps.com/top-10-people-who-survived-against-all-odds/" rel="nofollow">http://www.topyaps.com/top-10-people-who-survived-against-all-odds/</a> Of course you can go try the experiment. Maybe you have an approach which I don't understand, and we will gain from your discovery. Nevertheless, IMHO both my statements are extremely good rules of thumb for most people. I did suggest writing a function <code>int write_flash(byte flash_buffer[], byte ram_buffer[], uint num_bytes) {...}</code> to develop something like memcpy. Specifically write_flash would contain all of the flash-controller code as well as the uint16 memory copy. I am not suggesting you do not spend time on trying to make <code>memcpy(flash_array, ram_array, sizeof(flash));</code> work. You are of course free to do whatever you wish. I think it might be possible to get memcpy to work for the special case of two bytes, unit16 aligned. IMHO, that is merely an interesting engineering curiosity, with no practical value to the use case you described. I am not going to spend another moment of time on it. I am unlikely to try to answer any more questions on it because I seem unable to be clear enough. (full disclosure: I am not a member of LeafLabs staff.) samtal on "Loading data to / from flash" http://forums.leaflabs.com/topic.php?id=1009&page=3#post-6809 Mon, 10 Oct 2011 00:23:05 +0000 samtal 6809@http://forums.leaflabs.com/ gbulmer, Thanks for your detailed reply, but I believe we are wasting unnecessary words and our precious time. There was no big issue with flash to ram. The issue was with ram to flash, which you gave initially the impression that can be solved (was it my impression only?), while now you insist it is impossible. I fully agree with the first statement, while with the other I would say 'never say never'. We can not see a solution now, and probably it is not worth the bother, but <blockquote>memcpy(flash_array, ram_array, sizeof(flash)); will never work. It will never work no matter what you do with the flash memory controller. It can not work. </blockquote> seems too far to declare. Lets close this issue for now. Thanks for your support that was overwhelming. I am sure I will need more of it. And, back to an older non-related question: When compiling the IDE, I can see that the compiler builds a .cpp file, but I can not find that file. Is this a virtual or ram file, or can it be accessed and edited? gbulmer on "Loading data to / from flash" http://forums.leaflabs.com/topic.php?id=1009&page=3#post-6801 Sun, 09 Oct 2011 18:17:57 +0000 gbulmer 6801@http://forums.leaflabs.com/ samtal - How can I make it any clearer? -------- case 1 ------ FLASH TO RAM This case is extremely similar to RAM to RAM memcpy(demo_array,flash_array,sizeof(demo_array)) always works for valid addresses. <code>SerialUSB.println((int)memcpy(demo_array,flash_array,sizeof(demo_array)),HEX);</code> works for me. It works every time for valid addresses. I can only think of one way to stop it working. It just works. You do not have to do anything at all to make it work. You do not have to unlock flash, or use the flash keys. DO NOTHING to the flash controller. memcpy(ram, flash, sizeof(ram)) works. In fact, do NOT even TRY to do anything to the flash controller because that is the only thing that can make it fail (with valid addresses) memcpy FROM flash TO ram WILL ALWAYS WORK. It never ever fails for valid addresses, unless the flash memory controller is busy. So for all uses of memcpy() for case 1, where the code does not attempt to use the Flash memory controller, and the addresses are valid, memcpy() always work. ------ case 2 ------- RAM TO FLASH this is NOT like RAM to RAM, or FLASH to RAM memcpy(flash_array,demo_array,sizeof(flash_array)) does not work <code>SerialUSB.println((int)memcpy(flash_array,demo_array,sizeof(demo_array)),HEX);</code> never works for me. It does not matter if you use the Flash keys. That will not make it work. It does not matter if you unlock Flash. That will not make it work memcpy() will never ever work, no matter what sequence of Flash controller commands are used. memcpy FROM ram TO flash will NEVER work. (Note 1) IT WILL NEVER EVER WORK (Note 1) So for all uses of memcpy() for case 2, no matter what you do to the Flash memory controller, memcpy() will not work If you do nothing with the Flash memory controller, memcpy() for case 2 will not work. memcpy() is not useful for copying memory for case 2. --------------------------------------- You have read the flash memory programming manual. So I assume you understand how it works. You have written the code to copy from ram to flash in your original program, and you got it to work. It tests every single write to flash memory. That is necessary for it to work. memcpy does not do that test. None of the memxxx functions, builtins or macros are likely to do that test because they are generic memory-copying code designed to work on every CPU that C has ever run on. I think this is a sufficient explanation of why memcpy(flash_array,demo_array,sizeof(flash_array)) will not work. The memxxx functions and builtins assume RAM to RAM. (and implicitly FLASH to RAM, because the processor can just read Flash without any special machine code instructions) Those memxxx functions do not know about writing to Flash. You read in the Flash memory programming manual, you have to do an extra bunch of steps to program Flash. YOU HAVE TO WRITE THE CODE TO COPY FROM RAM TO FLASH You have already written the code in your earlier program. There is no work to do. It is done. ---------------- Summary: <code>memcpy(ram_array, flash_array, sizeof(ram));</code> will always work, if you leave the flash memory controller alone. <code>memcpy(flash_array, ram_array, sizeof(flash));</code> will never work. It will never work no matter what you do with the flash memory controller. It can not work. (Note 1) Is this clear? (Note 1: I think there is a way to copy exactly and only two bytes, but IMHO, there is no point wasting time on it. I'm not going to, it only confuses the issue) samtal on "Loading data to / from flash" http://forums.leaflabs.com/topic.php?id=1009&page=3#post-6800 Sun, 09 Oct 2011 17:21:12 +0000 samtal 6800@http://forums.leaflabs.com/ gbulmer, We seem to be beating around the bush and never getting to the point or solution. At the end of all, you typed the same code I have sent that does not work, and you do not show the reason why. Did you try make it work? did it work for you? If it does, please let me know what am I doing wrong? Your comment <blockquote>NEVER needs the Flash to be in program mode. That memcpy() will work correctly without the FLASH_BASE->CR |= 1<<0; and FLASH_BASE->CR = 0x80;</blockquote> makes no sense to me (unless it works for you!?). The memcpy is a generic C/C++ macro that does not relate to flash, just to memory. It does not know if the addressed memory location is flash, SRAM, DRAM or EPROM or whatever, or if it is STM32 or Microchip or Intel. On the other hand, the STM32 flash can not be programmed without unlocking the double key, erasing the target (if not clear) and setting the program bit. My logical conclusion is that one MUST unlock and set the program bit for a copy to flash to succeed. (It must also be 16 bits half word, no more no less). Actually, as I found no clean (or any) solution to the use of the memcpy for copying from / to STM32 Maple flash, I gave up and wrote my byte-by-byte copy function. If you want to play with the macros (there are more, like memmove, memset, etc. that may need similar treatment) and can make the memcpy work with flash, you'll be blessed. I have good use for it, but little time to struggle with a non-critical function in my project. Thanks, samtal gbulmer on "Loading data to / from flash" http://forums.leaflabs.com/topic.php?id=1009&page=3#post-6773 Fri, 07 Oct 2011 14:26:39 +0000 gbulmer 6773@http://forums.leaflabs.com/ samtal <blockquote>As you have probably seen before, I know how to copy from flash to ram and back, as well as dumping the flash (see <a href="http://pastebin.com/h0CxGRQ7)" rel="nofollow">http://pastebin.com/h0CxGRQ7)</a>. </blockquote> Yes, and very well done for slogging through the manuals and making that work. Kudos to you. <blockquote>The question here was how to use the memcpy macro to copy a block of data in both directions without the need for loop iterations. </blockquote> Okay. I think I understand. EDIT: It won't work in both directions. memcpy(flash_array,demo_array,sizeof(demo_array)),HEX); will not work <blockquote>I have built a small IDE test program that I can not make to work. It is at <a href="http://pastebin.com/CDKv0WTd" rel="nofollow">http://pastebin.com/CDKv0WTd</a>. </blockquote> Thank you very much for taking the time to post that code. I find it is much easier to work from specific code, rather than a forum post. There is less chance of making a mistake. I apologise that my previous explanation was not clear. Please let me try again. Edit: I have read your program, and I have tried to run it. It does not work in the form you posted. There are several errors. I think I understand what line 37 is trying to do, but it can not work: <code>Serial1.println((int)memcpy(flash_array,demo_array,sizeof(demo_array)),HEX);</code> Edit: for all practical purposes <code>memcpy(flash_array,demo_array,sizeof(demo_array))</code> does not work. AFAIK, your other program - <a href="http://pastebin.com/h0CxGRQ7" rel="nofollow">http://pastebin.com/h0CxGRQ7</a> works correctly. In that program, every pair of bytes it tries to write to flash, it checks the FLASH_BASE->SR bit (lines 356 & 357): <pre><code>while (FLASH_BASE->SR & (1<<0)) { }</code></pre> I am very sure memcpy() does not do that check. My suggestion in <a href="http://forums.leaflabs.com/topic.php?id=1009&page=2#post-6507" rel="nofollow">http://forums.leaflabs.com/topic.php?id=1009&page=2#post-6507</a> was to wrap that up in something like: <pre><code>int write_flash(byte flash_buffer[], byte ram_buffer[], uint num_bytes) {...} // returns the number of bytes written from ram_buffer to flash_buffer // or <0 if an error. This looks like memcpy() BUT // this requires flash_buffer to be a valid flash memory address</code></pre> I can see where I may have caused confusion, and failed to clarify in my subsequent posts. I apologise. Conversely, the memcpy() between lines 50-53: <pre><code>FLASH_BASE->CR |= 1<<0; //Set flash program bit Serial1.println((int)memcpy(demo_array,flash_array,sizeof(demo_array)),HEX); //Lock the flash FLASH_BASE->CR = 0x80;</code></pre> NEVER needs the Flash to be in program mode. That memcpy() will work correctly without the <code>FLASH_BASE->CR |= 1<<0;</code> and <code>FLASH_BASE->CR = 0x80;</code> so <code>Serial1.println((int)memcpy(demo_array,flash_array,sizeof(demo_array)),HEX);</code> is all that is needed. Edit: this memcpy() works. (full disclosure: I am not a member of LeafLabs staff.) samtal on "Loading data to / from flash" http://forums.leaflabs.com/topic.php?id=1009&page=3#post-6771 Fri, 07 Oct 2011 09:34:26 +0000 samtal 6771@http://forums.leaflabs.com/ gbulmer, As you have probably seen before, I know how to copy from flash to ram and back, as well as dumping the flash (see <a href="http://pastebin.com/h0CxGRQ7)" rel="nofollow">http://pastebin.com/h0CxGRQ7)</a>. The question here was how to use the memcpy macro to copy a block of data in both directions without the need for loop iterations. I have built a small IDE test program that I can not make to work. It is at <a href="http://pastebin.com/CDKv0WTd" rel="nofollow">http://pastebin.com/CDKv0WTd</a>. I am asking if someone can take a look at that test program and either make it work or tell me what needs to be done. The problem may be in a casting or addressing, or in the fact that the RAM is read in bytes, while writing to flash must be done in 16 bits half-words (See STM PM0075 Flash manual). samtal gbulmer on "Loading data to / from flash" http://forums.leaflabs.com/topic.php?id=1009&page=3#post-6767 Thu, 06 Oct 2011 16:54:38 +0000 gbulmer 6767@http://forums.leaflabs.com/ samtal - I don't understand your comments, <blockquote>I literally meant to copy from RAM to flash (and vise verse), thus the order of parameters was what I meant (and should work both directions). </blockquote> It "should work both directions" If, and only if, (IFF) flash is "programmable". <code>memcpy()</code> should always work from Flash to RAM, even if the flash is not programmable, because that is how the processor must work. So you could write something like: <pre><code>setup() { delay(5000); // give 5 seconds to start the serial console SerialUSB.println("Dump flash: "); for (byte* p=(byte*)0x08000000; p<=(byte*)0x0801FFFF; ++p) { SerialUSB.print(*p,HEX); SerialUSB.print(" "); } }</code></pre> Where 0x08000000 is the start address of flash memory, and 0x0801FFFF is the end address of flash memory. That should work in any case (on STM32F103xB), it doesn't need the Flash memory to be in "program mode". So I don't think there is much to be deduced from memcpy(demo_data, flash_array, sizeof(demo_data)); EDIT: But memcpy() from RAM to Flash will never work. It is as simple as that. End of story. Done. Nothing else to know. Finished. (WARNING: I have not compiled and tested that code, it's only an illustration) (full disclosure: I am not a member of LeafLabs staff) samtal on "Loading data to / from flash" http://forums.leaflabs.com/topic.php?id=1009&page=3#post-6759 Wed, 05 Oct 2011 14:01:08 +0000 samtal 6759@http://forums.leaflabs.com/ gbulmer, Thanks for your reply. I literally meant to copy from RAM to flash (and vise verse), thus the order of parameters was what I meant (and should work both directions). From ST PM0075 Flash manual: "The embedded Flash module can be addressed directly, as a common memory space". I tried your proposed corrections (and many other options), but, as before it does not work. One direction copies RAM to RAM (not to flash), while when switching direction the Maple crashes. memcpy(flash_array,demo_data,sizeof(demo_data) will output: Copy demo data from:200012F8 to (crash) memcpy(demo_data,flash_array,sizeof(demo_data) will output: Copy demo data from:200012F8 to 200012F8 (RAM to RAM, same address!). Can you or someone else please try to run the program and find what is wrong?? samtal gbulmer on "Loading data to / from flash" http://forums.leaflabs.com/topic.php?id=1009&page=3#post-6756 Tue, 04 Oct 2011 20:21:42 +0000 gbulmer 6756@http://forums.leaflabs.com/ samtal - The line: <code>SerialUSB.println((uint32)memcpy(flash_array,&demo_data,num_cells),HEX);</code> is incorrect. The way to remember what memcpy does is to think of it as <code>first-array = second-array</code> so the parameters in your code are the wrong way round, and your code is trying to copy RAM to Flash, which will not work, and may explain the error. Secondly. I wouldn't take the address of an array <code>&demo_data</code>, because an array name is already an address. I think it actually gives the correct value when the variable name is a global array, but I believe it would be wrong if the variable was a parameter (2am here), so I wouldn't write that expression. I would write the address this way: <code>SerialUSB.println((uint32)memcpy(demo_data, flash_array, num_cells),HEX);</code> but this is also okay <code>SerialUSB.println((uint32)memcpy(&demo_data[0], flash_array, num_cells),HEX);</code> But I wouldn't write the line that way anyway because it isn't clear that num_cells means the number of bytes you want to copy (which is what memcpy needs), or the number of cells of the array you want to copy (which are uint16). I'd write something like: <code>SerialUSB.println((uint32)memcpy(demo_data, flash_array, num_cells*sizeof(uint16)),HEX);</code> because it seems clearer. In reality, I wouldn't bother trying to keep track of the amount of array used. I'd just copy the entire array back and forth, and hence write: <code>SerialUSB.println((uint32)memcpy(demo_data, flash_array, sizeof(demo_data)),HEX);</code> which should be a bit more fool proof. samtal on "Loading data to / from flash" http://forums.leaflabs.com/topic.php?id=1009&page=2#post-6747 Tue, 04 Oct 2011 12:49:49 +0000 samtal 6747@http://forums.leaflabs.com/ Thanks mbolivar, As you can see in my code, (in a second commented line, starting with 'or'), I have tried first gbulmers version, but when it failed I tried another option (Need to correct [sizeof(ram_array)] to [PAGE_SIZE]). Unfortunately, neither works (I tried even more options). Did you try running the program? Did it work for you? The interesting thing is that if I set the copy length (num bytes) to 1 or 2, it seems to work, and I get back an address in the flash, but any other number of bytes >2 crash. For the full program, I get 'Copy demo data from:200012F8 to', and it crashes. For bytes to copy 2, I get: 'Copy demo data from:200012F8 to 8008800' which seems right. I have tried everything I know (which apparently isn't enough), but for no avail. (I have the feeling it is a variable type or casting problem?) And 2 more, non-related: 1. Regarding the backticks; I did not see your edit in my code. (I naturally always ready to learn..) 2. A general question: On each compile (IDE), I see that the compiler builds a C++ file. Is that file available? can I see or edit it? (I couldn't find it, and it looks like a RAM or virtual file??). samtal mbolivar on "Loading data to / from flash" http://forums.leaflabs.com/topic.php?id=1009&page=2#post-6738 Mon, 03 Oct 2011 16:26:18 +0000 mbolivar 6738@http://forums.leaflabs.com/ samtal, this appears to be a simple copy-paste error. you wrote: <code>uint8 flash_array[PAGE_SIZE] _FLASH_H_ __PAGE_ALIGNED__;</code> but gbulmer wrote: <code>uint8 flash_array[sizeof(ram_array)] __FLASH__ __PAGE_ALIGNED__;</code> note that you wrote <code>_FLASH_H_</code> instead of <code>__FLASH__</code>. <code>_FLASH_H_</code> is the #define used by flash.h to prevent recursive inclusion. <code>__FLASH__</code> is a macro that directs the compiler to store a variable in flash. see: <a href="http://leaflabs.com/docs/arm-gcc.html#arm-gcc-attribute-flash" rel="nofollow">http://leaflabs.com/docs/arm-gcc.html#arm-gcc-attribute-flash</a> edit: By the way, you can put code within backticks (the ` character underneath ~ on most US keyboards) in order to preserve formatting. i have edited your post to use this feature; i hope you don't mind. samtal on "Loading data to / from flash" http://forums.leaflabs.com/topic.php?id=1009&page=2#post-6736 Mon, 03 Oct 2011 04:50:11 +0000 samtal 6736@http://forums.leaflabs.com/ Hi, I have tried hard the above advices with many variations, but still can not copy an array from RAM to Flash. Here is a small IDE program that is supposed to copy a small array from the RAM to the Flash (I added flash unlock and lock. I guess this is required). As can be seen by running the program, it copies from the original RAM location to another RAM location rather than to the flash. I don't understand where is the flash address specified in the attribute. What is wrong with this program? samtal /************Program begins here*********************/ <pre><code>//Maple IDE memcpy test from RAM to flash by samtal Oct 3,2011 #include "stdlib.h" //Included explicitly for proper data printing. #include "flash.h" #include "string.h" #define PAGE_SIZE 1024 #define __PAGE_ALIGNED__ __attribute__((aligned (PAGE_SIZE))) uint8 flash_array[PAGE_SIZE] _FLASH_H_ __PAGE_ALIGNED__; //or uint8 flash_array[sizeof(ram_array)] __FLASH__ __PAGE_ALIGNED__; uint16 demo_data[1024]; //Build the demo array in the setup() uint16 num_cells=100; //Set up the number of demo cells to copy. void setup() { //Build the demo array for (uint16 d=0;d<num_cells;d++) { demo_data[d]=0xFFFF-d; } } void loop() { delay(1000); //Unlock flash: FLASH_BASE->KEYR = 0x45670123; //The STM Key1 FLASH_BASE->KEYR = 0xCDEF89AB; //The STM Key2 SerialUSB.println(); SerialUSB.print("Copy demo data from:"); SerialUSB.print((uint32)&demo_data,HEX); SerialUSB.print(" to "); SerialUSB.println((uint32)memcpy(flash_array,&demo_data,num_cells),HEX); //Lock the flash FLASH_BASE->CR = 0x80; }</code></pre> /******************End of program**************************/ gbulmer on "Loading data to / from flash" http://forums.leaflabs.com/topic.php?id=1009&page=2#post-6719 Sat, 01 Oct 2011 08:41:13 +0000 gbulmer 6719@http://forums.leaflabs.com/ samtal - memcpy() does work mbolivar explains how here: <a href="http://forums.leaflabs.com/topic.php?id=1076#post-6620" rel="nofollow">http://forums.leaflabs.com/topic.php?id=1076#post-6620</a> The 'trick' is to <code>#include <string.h></code> near the top of your program. That declares memcpy (and a bunch of other functions), so the error message should go away. As mbolivar explains on that thread, the IDE links against newlib, which provides many of the standard C functions. The documentation for newlib is at <a href="http://sourceware.org/newlib/" rel="nofollow">http://sourceware.org/newlib/</a> samtal on "Loading data to / from flash" http://forums.leaflabs.com/topic.php?id=1009&page=2#post-6718 Sat, 01 Oct 2011 00:41:13 +0000 samtal 6718@http://forums.leaflabs.com/ Kind members, Thank you all for your support. To the best of my knowledge, 'memcpy' and other C/C++ commands you use in your examples are not recognized by the Maple IDE and thus generate errors. As I use the IDE, such commands are of no use to me (have I missed something?). For the IDE we need lower level commands, or better drivers. When, in the past, I suggested that someone knowledgeable will write a driver for the dual adc, I was opposed by several members claiming that Maple has more important tasks. I therefore made my own unprofessional solution that, if used by others, may lead to errors and bugs. The same happened with the flash program. I made my own flash handling program, but it is an IDE program rather than C/C++, which I am forced to include as code in my main IDE, making it too long and difficult to handle. I wish to use the flash program as an included external c file, but I do not have the knowledge for doing it right. Can anyone lift that glove and convert my IDE (<a href="http://pastebin.com/h0CxGRQ7" rel="nofollow">http://pastebin.com/h0CxGRQ7</a>) into a viable C/C++ included file? This has to be done anyhow to make the Maple useful for a wider range of potential users. samtal