Loading data to / from flash « LeafLabs Garden

LeafLabs Garden » Support » Maple IDE Support

Loading data to / from flash

(40 posts) (6 voices)

Started 4 years ago by samtal
Latest reply from samtal

12 3 Next »

samtal
Member

Hi,
I have a calibration table (two-dim array) that I wish to upload to a non volatile memory area (flash).
Can anyone help with step-by-step info on how to load / read from the flash and to make sure it will be in a protected area ?
The table size can go up to 400 bytes.
By default, data from the table should be read to the ram at setup.
Thanks
samtal

Posted 4 years ago #
gbulmer
Moderator

samtal - I am sorry, but I am not sure what you are asking, I can read your post in a couple of different ways.

1. This may not be what you are asking, but better complete and simple ... :-)
If the values in the calibration table are known at compile time, then the data can be used to initialise a 2D table, as in
int table[WIDTH][HEIGHT] = { {1, 2, 3, 4 ...}, { 2, 3, 4, ...}, {...}, {...}};
In this case the upload will happen when the program is loaded into Flash.

2. If what you are asking is "how can a program running on the STM32F103 reprogram part of Flash while it is running?" (which I think you are asking) ...

Then PM0075 Programming manual, "STM32F10xxx Flash memory microcontrollers"
http://www.st.com/internet/com/TECHNICAL_RESOURCES/TECHNICAL_LITERATURE/PROGRAMMING_MANUAL/CD00283419.pdf
explains how to program Flash memory.

and the ST Micro Application note AN2594 "EEPROM emulation in STM32F10x microcontrollers" http://www.st.com/internet/com/TECHNICAL_RESOURCES/TECHNICAL_LITERATURE/APPLICATION_NOTE/CD00165693.pdf
has details of how to use Flash as EEPROM

The technique in the Application note might be a bit more intricate than you might need because the application note explains how to emulate EEPROM which is reprogrammable at the byte level, whereas Flash is only reprogrammable at a page level (1kByte for 128K Flash STM32F103RB). I imagine you can copy the table to RAM then reprogram the whole table.

Reprogramming is relatively slow, i.e. 10's of milliseconds. Will it be okay for your program to take that much time?

All variables with initial values are stored in Flash. The actual non-const variables used by the program, which exists in RAM, are initialised before main (or loop) is entered when the program first starts.

A slick approach might be to rewrite the Flash holding the initialisation values of the table to the new calibration values.

An easier approach might be to leave the table as un-initialised, but set aside another, second, table of the same size, explicitly declared as living in flash. Then initialise the RAM version 'by hand' (using for example a memcpy). The small advantage of this approach is you have control over when it happens, and the names of the two tables are easy to find (you choose them both).

Off the top of my hand, I don't know what the initialised data will be called for the first approach, but, I imagine if you can get a listing of the programs assembler, then you will be able to find it (at worst by making the intialised data easy to find while experimenting).

I'm not sure what you mean about putting the table into a "protected area".

Posted 4 years ago #
samtal
Member

Thanks gbulmer for the prompt and detailed reply.

Obviously, as you indicated yourself,I did not mean the first item on your reply.
The calibration is carried out at run time, and must be saved in a NVM for future use that may be permanent for the device lifetime, or may be updated at long intervals.

Due to my implementation, it is obvious that write and read times are of no concern whatsoever.
Furthermore, I could live with full page(s) write and read. I can download the full page, modify the table and upload the page.
I also need to use a backup page in case the page will be corrupted while editing. (Mainly because the flash page must be fully erased before writing to).

The documents you recommended seems to hold all the information I need.
While (as you indicated) the first document CD00283419.pdf is more than needed, the other one (CD00165693.pdf) is perfect and seems easy (I'll know for sure in a few days...).
Writing directly to a predefined area in the flash by a simple pointer, and use the many ST options registers to handle the data (config., read, write, protect, etc.).
It is naturally much more complex than 123, but I hope I can cope.

The native Maple core files flash.h and flash.c seem incomplete, but will also help.

The CD00283419.pdf document lists a sample program named eeprom.c (and h) that is supposed to be available, but unfortunately I could not find it anywhere on the net.
Do you have an idea where that sample can be found?
(It can help in implementation)

When writing 'protected area' I meant a memory area that is protected from random or accidental overwriting, which is naturally a standard ST (or all MC) feature.
samtal

Posted 4 years ago #
gbulmer
Moderator

samtal - The only example code I could find is at:
http://www.st.com/internet/com/SOFTWARE_RESOURCES/SW_COMPONENT/FIRMWARE/an2594.zip

It was listed at:
http://www.st.com/stonline/stappl/resourceSelector/app?page=resourceSelector&doctype=FIRMWARE&ClassID=1734

HTH

Posted 4 years ago #
samtal
Member

Thanks.
I managed to learn and implement flash reading and writing using the ST Flash manual CD00165693.pdf with direct access to the various flash registers (partially using the libMaple flash.h).
The problem I am having now is the question WHERE in the flash can I write my data without interfering with the program (mine and possibly the bootloader).
My understanding is that the medium density devices (the Maple I assume is medium density) has 128 flash pages of 1 KB each, from 0x0800 0000 up to 0x0801 FFFF.
My program uses part of it, (about 30K so far), but I have no idea which part.
I thought the program will start at the beginning of the memory, so I tested the upper 4 pages, but found them to contain some data as well.
Can anyone tell which memory area can be dedicated for my data storage?
samtal

Posted 4 years ago #
samtal
Member

Thanks gbulmer.
With your kind support I managed to learn and implement flash reading and writing using the ST Flash manual CD00165693.pdf with direct access to the various flash registers (quite complex, partially using the libMaple flash.h).
The problem I am having now is the question WHERE in the flash can I write my data without interfering with the program (mine and possibly the bootloader).
My understanding is that the medium density devices (the Maple I assume is medium density) has 128 flash pages of 1 KB each, from 0x0800 0000 up to 0x0801 FFFF.
My program uses part of it, (about 30K so far), but I have no idea which part.
I thought the program will start at the beginning of the memory, so I tested the upper 4 pages, but found them to contain some data as well.
Can anyone tell which memory area can be dedicated for my data storage?
samtal

Posted 4 years ago #
perry
Administrator

Hi samtal,

You can place it in the "USER_FLASH" section using a gcc attribute:
http://gcc.gnu.org/onlinedocs/gcc-3.4.0/gcc/Variable-Attributes.html

Then you can read/write to it using the normal flash read/write routines.

-perry

Posted 4 years ago #
robodude666
Member

If you need to read/write calibration data to a non-volatile memory, I highly recommend getting an external SPI EEPROM/FLASH IC. They're only a few dollars each, and work pretty well.

I've "ported" an arduino library for the Atmel AT45DB161D. It's not complete (missing DMA support), but it works very well otherwise and is very fast.

-robodude666

Posted 4 years ago #
poslathian
LeafLabs

for constant arrays, you can permanently located them via the method perry pointed out. However, we know have that macro'd to " __FLASH__ "

so you can do:
static unsigned char font[] __FLASH__ = { ... };

I dont recommend writing to FLASH at runtime, although as you learned this can be done. If you want to see how we do it in the bootloader, check out:
https://github.com/leaflabs/maple-bootloader/blob/master/hardware.h
and
https://github.com/leaflabs/maple-bootloader/blob/master/hardware.c

Posted 4 years ago #
poslathian
LeafLabs

also, to answer your question about how to make sure you dont write over anything important in flash - the USER_FLASH section is everything not used by your program. The easiest thing to do (without breaking into the linker scripts) would be to define a big constant array like I showed before using the __FLASH__ macro. Then take the pointer to your new array (in the above example:
char* myPtr = &font[0]; // you dont need the [0], but its more clear

that will give you an address that lives somewhere in USER_FLASH, and so everything after that should be fair game for writing. there is one big caveat though, you can only erase flash pages one flash page at a time ( 1KB on maple/mini and 2KB on RET6/Native). So if the start of USER_FLASH does not live on a flash boundary then clearing the page that contains font[0] might clear stuff behind font[0] as well. The linker scripts could be modified to force USER_FLASH to start on a page boundary, but currently they dont do that.

Posted 4 years ago #
mbolivar
LeafLabs
that will give you an address that lives somewhere in USER_FLASH, and so everything after that should be fair game for writing. there is one big caveat though, you can only erase flash pages one flash page at a time ( 1KB on maple/mini and 2KB on RET6/Native). So if the start of USER_FLASH does not live on a flash boundary then clearing the page that contains font[0] might clear stuff behind font[0] as well. The linker scripts could be modified to force USER_FLASH to start on a page boundary, but currently they dont do that.

to ensure page alignment, use GCC's aligned attribute.

http://gcc.gnu.org/onlinedocs/gcc-3.2.3/gcc/Type-Attributes.html

use e.g. 1024 as your alignment on Maple, where flash pages are 1KB each.

if you do that, it makes sense to allocate the entire page. example:
```
#define PAGE_SIZE 1024
#define __PAGE_ALIGNED__ __attribute__((aligned (PAGE_SIZE)))
uint8 flash_page[PAGE_SIZE] __FLASH__ __PAGE_ALIGNED__;
```
this defines flash_page as a page-aligned byte buffer occupying 1 page. here's the arm-none-eabi-nm output confirming that the resulting variable is indeed page aligned:
```
$ arm-none-eabi-nm maple.elf | grep flash_page
08004800 D flash_page
```
edit: more portably, you can use the following:
```
#ifdef STM32_MEDIUM_DENSITY
#define PAGE_SIZE 1024
#elif defined(STM32_HIGH_DENSITY)
#define PAGE_SIZE 2048
#endif
```
this will properly define PAGE_SIZE across all existing LeafLabs boards.
Posted 4 years ago #
samtal
Member

With everyone's help and my humble understanding of the STM32, and due to the fact that I do not know C/C++ sufficiently and because necessity is the mother of invention, I made a basic flash Read / Write / Erase IDE application that directly accesses the registers and works well (for me).
It is built as an example / demo / template and can be downloaded from:
http://pastebin.com/h0CxGRQ7

This was tested on my computer only, and needs verification by others.
Feedback is needed, please do.

I encourage anyone knowledgeable to convert that app into a C/C++ app, which is the right way to go, but, unfortunately is out of MY way...

Posted 4 years ago #
gbulmer
Moderator
samtel - that is a very splendid piece of work. Well done.

One thing you might want to consider, is doing everything in consistent data types.
I read PM0075 as saying that pages are writable as 1/2 words, or two bytes, or uint16. So maybe try to do everything in those units.
In this case, pages (on a medium density STM32F103) are 512 uint16's

For example:
```
uint16* flash_base=0x08000000;
#define PAGE (1024/sizeof(uint16))
```
is the base address of a 16-bit integers, and a page is 1024 bytes / the size of an unsigned 16 bit int.

In C, pointers uint16* foop; and arrays uint16 foo[...]; can use interchangeable notations, so you can make the code look simpler by using array notation.

So the whole for loop of read_flash might become:
```
uint16* data_base = flash_base + PAGE * offset; // get start address of 'offset' page

for (int i=0; i<demo_cells/2; i=i+1) {
    while ((FLASH_BASE->SR & (1<<0))==true) {  }   //Check not in use

    flash_data[i] = data_base[i];    // copy from flash into RAM resident array 'flash_data'
}
```
which seems simpler and clearer to me.
(I'd be tempted to change the name of flash_data to ram_data because it isn't in flash, but that is a minor point)

simlarly in write_flash, the for (uint16 k=... loop might become:
```
uint16* lower_address_limit = 0x801E000 ;   //Lower safety page limit for the demo Page 120
uint16*  upper_address_limit = 0x0801FFFF ;

uint16* write_address=flash_base+PAGE*offset;
if ((write_address < lower_address_limit || (write_address+demo_cells/2 > upper_address_limit) {
      Serial1.println("Address out of allowed limits");
      lock_flash();          //Wrong address emergency brake...
      return;
}

for (int k=0; k<demo_cells/2; k=k+1) {
    while ((FLASH_BASE->SR & (1<<0))==true) { }  // wait
    write_address[k] = demo_data[k];
}
```
Similarly in void erase_flash_page(char page) the code checks to see cells are erased.
```
uint16 read_address=flash_base+PAGE*page;  // maybe use 'offset' instead of 'page' to match other code
for (int i=0;i<PAGE;i++) {
    while ((FLASH_BASE->SR & (1<<0))) {   }

    if  ((read_address[i] != 0xFFFF) {
      Serial1.println("Page erase error");
      lock_flash();
      return;
    }
  }
```
The code while ((FLASH_BASE->SR & (1<<0))==true) isn't a very robust test.
If the bit being tested were any other bit than bit 0, it would fail because true is a real integer value, 1.
For example while ((FLASH_BASE->SR & (1<<1))==true) would always fail.
IMHO, better to do the simpler test for non-zero:
while ( (FLASH_BASE->SR & (1<<0)) )

Other thoughts:
demo_cells is a global variable, but is sometimes passed to a function, e.g.
```
case 'w':
      write_flash(demo_data, page, 0, demo_cells);   //write_flash(int page, int offset, int num_cells)
      break;
```
and sometimes it is used as a global, e.g. in void print_flash(uint16 data[],int data_address[])

In uint16 read_flash(uint8 page, uint16 offset, uint16 num_cells) the global variable flash_data is used to copy data to from flash.
In void write_flash(uint16 demo_data[],char page, uint16 offset, uint16 demo_cells) the data is passed in as a parameter.

I'd recommend being consistent, and doing the same thing in all cases. I would almost always pass stuff as a parameter. Consistency will help avoid confusion or incorrect assumptions.

In uint16 read_flash(uint8 page, uint16 offset, uint16 num_cells) the address of the global variable flash_data is returned as an int. If you want to return addresses, try to keep them as addresses, then the C compiler can help by checking for a few more kinds of errors.
So declare the function as uint16* read_flash(uint8 page, uint16 offset, uint16 num_cells) and return the address.

Summary: if you can make the datatypes match what you want to happen, C will do a lot of the work, and remove some opportunities to make mistakes.

Overall, well done for trudging through the documentation and getting it all to work.
Posted 4 years ago #
samtal
Member

gbulmer,
Thanks for your unexpectedly prompt and detailed response.
A few comments:
The PM0075 does not mention the option to change page size, and I was not aware of such possibility that can make it easier on certain occasions.
As you can see in my code, I read bytes and combine them into half-words, while writing half words directly (which is a ST32 dictate).
(In my application, using bytes is an advantage, as I read real-time hex commands from the serial port, and they are bytes).
Regarding your comment "uint16* data_base = flash_base + PAGE * offset; // get start address of 'offset' page" I may have misunderstood you: The 'offset' is offset from page start address (defaults to 0), so why do you multiply by page number?
Addresses: I calculated addresses as numbers, and then used them as address. Is this wrong?

Regarding the other comments - I will learn them one by one and try to implement and correct my posting so as not to mislead others who may wish to use it.
(fixed and posted the (FLASH_BASE->SR & (1<<0))==true) to (FLASH_BASE->SR & (1<<0))

By the way, I did something similar with the Dual Simultaneous ADC that I had the need for and posted it as well (almost a month ago), asking for comments, but no one bothered to react. see http://pastebin.com/p7P1RPFu.
There may be more comments there, as I was 'greener'.
Thank you again. I truly appreciate your support.
samtal

Posted 4 years ago #
gbulmer
Moderator
samtal

The PM0075 does not mention the option to change page size, and I was not aware of such possibility that can make it easier on certain occasions.

I didn't change the page size, did I?
I believe it is 1024 bytes on a medium density device, or (1024/sizeof(uint16))

As you can see in my code, I read bytes and combine them into half-words, while writing half words directly (which is a ST32 dictate).

I believe flash can be read as bytes, half words (two bytes) or words (four bytes), AFAIK there is no ST32F1xx dictate about this.
A uint16 is a half word, so my example obeys the STM32F1xx dictate on writing to flash.

If you want to explicitly build a uint16 from two bytes (uint8), maybe something like:
```
uint16 flash_data[PAGESIZE];
...
byte* read_address = flash_base+page*PAGEBYTESIZE;
...
flash_data[i] =  read_address[2*i+1]<<8 | read_address[2*i];
```
is clear?

(In my application, using bytes is an advantage, as I read real-time hex commands from the serial port, and they are bytes).

Okay. I'd probably cast the start address of the buffer to a uint16* before writing it.

Regarding your comment "uint16* data_base = flash_base + PAGE * offset; // get start address of 'offset' page" I may have misunderstood you: The 'offset' is offset from page start address (defaults to 0), so why do you multiply by page number?

I made a mistake.
I think it should be uint16* data_base = flash_base + PAGE * page;

Addresses: I calculated addresses as numbers, and then used them as address. Is this wrong?

I would never do that. I would always try to keep addresses as addresses.
For one thing, with uint16* p; p[i]; p+i; always works correctly. The compiler calculates the address correctly based on the size of data element. Whereas casting things to the appropriate type at the appropriate time is a bit more error prone.
If the addresses stay as addresses, and integers as integers, the compiler can do slightly better checking for errors too.
It might also generate better code.
Finally, it should be clearer to someone reading code what is happening.

I would probably create a type to represent the unit of flash access and keep everything consistent:
```
typedef uint16 FlashUnit;
#define PAGEBYTESIZE (1024)
#define PAGE (PAGEBYTESIZE/sizeof(FlashUnit))
FlashUnit* flash_base;
FlashUnit* data_base = flash_base + PAGE * page;
```
Then either
```
void write_flash(byte demo_data[], uint page, uint offset, uint demo_cells) {
    FlashUnit* flash_aligned_data = (FlashUnit*)demo_data;   // or = (FlashUnit*)&demo_data[0];
```
or
```
case 'w':
      write_flash((FlashUnit*)demo_data, page, 0, demo_cells);   //write_flash(int page, int offset, int num_cells)
      break;
```
I like the second one because I'd like to keep the read_flash and write_flash APIs similar.

EDIT: I hate the second one, I must have been "tired and emotional", or just lacking food, when I wrote that.

By the way, I did something similar with the Dual Simultaneous ADC that I had the need for and posted it as well (almost a month ago), asking for comments, but no one bothered to react.

Sorry about that, but I don't always have much time.

(full disclosure: I am not a member of LeafLabs staff)
Posted 4 years ago #

RSS feed for this topic

12 3 Next »

Reply »

You must log in to post.