Loading data to / from flash « LeafLabs Garden

LeafLabs Garden » Support » Maple IDE Support

Loading data to / from flash

(40 posts) (6 voices)

Started 4 years ago by samtal
Latest reply from samtal

« Previous 123 Next »

gbulmer
Moderator

samtal - after something to eat, my brain got into gear, so I think my suggestion that the second one is nicer:

Then either

void write_flash(byte* demo_data[], uint page, uint offset, uint demo_cells) {
    FlashUnit* flash_aligned_data = (FlashUnit*)demo_data;   // or = (FlashUnit*)&demo_data[0];

case 'w':
      write_flash((FlashUnit*)demo_data, page, 0, demo_cells);   //write_flash(int page, int offset, int num_cells)
      break;

I like the second one because I'd like to keep the read_flash and write_flash APIs similar.

Is horrible.

I think I might make the APIs:

uint16 read_flash(uint8 page, uint16 offset, uint16 num_cells)  {...}
void write_flash(uint16 demo_data[],char page, uint16 offset, uint16 demo_cells)  {...}

into

int read_flash(byte buffer[], uint num_bytes, uint page_num, uint page_offset) {...}
// returns the number of bytes read

int write_flash(byte buffer[], uint num_bytes, uint page_num, uint page_offset) {...}
// returns the number of bytes written or -1 if an error
...
case 'w':
      write_flash(demo_data, sizeof(demo_data), page_numb, 0);   //write_flash(int page, int offset, int num_cells)
      break;

If I understand correctly, what you want is something to copy flash to an array variable, or an array variable to flash. Is that correct?

In that case the API might be:

int read_flash(byte ram_buffer[], byte flash_buffer[], uint num_bytes) {...}
// returns the number of bytes copied from flash_buffer into ram_buffer
// this is identical to memcpy()

int write_flash(byte flash_buffer[], byte ram_buffer[], uint num_bytes) {...}
// returns the number of bytes written from ram_buffer to flash_buffer
// or <0 if an error. This looks like memcpy() BUT
// this requires flash_buffer to be a valid flash memory address

and let the code in the functions sort out the details.

If you have the names of the two variables (one in flash, and one in RAM), they should do the 'right thing'.
Assuming the variable stored in flash (that is being written to) is on its own page, it should be straightforward.
It is a bit more tricky if the variable stored in flash shares a page with other stuff, but I assume you are avoiding that.

(full disclosure: I am not a member of LeafLabs staff.)

Posted 4 years ago #

samtal
Member

gbulmer, wow....
I am overwhelmed by your detailed, timely and time consuming replies.
As I have a lot to learn, it's going to take me hours to comprehend your comments and implement them, knowing I still lack the necessary deep understanding of the pointers issue.
To tell you the truth, some of it is based on 'trial and error' until I get it to work as it should (not necessarily written as should).

In any case, I assume after eating and drinking you currently are sleeping.
Thanks again and good morning (once you wake up to a new day...)
samtal

Posted 4 years ago #
gbulmer
Moderator
smtal - good morning!-)

Arrays vs Pointers.

In C arrays and pointers are very close in meaning.
The differences are (practically) invisible when they are passed as parameters.

Let's have a function with two parameters:
void fun(int a[], int* p) { ... }

In C, when arrays or pointers are passed as parameters to a function, they are passed as address values.
In the case of an array variable, the address value is the address of the start of the array; either a, or &a[0].
In the case of a pointer variable, it is the value contained in the variable, p.

C makes pointer arithmetic 'just work':
```
*p = ...;
a[0] = ...;
p[0] = ...;
*a = ...;
```
All have the same meaning in the function fun.

More importantly
```
a[1]
*(a+1)
p[1]
*(p+1)
```
are the same address value.
The integer value of
```
(int)&a[1]
(int)(a+1)
(int)&p[1]
(int)(p+1)
```
are all the same.

The integer value is any of:
```
((int)&a[0])+sizeof(int)
((int)(a))+sizeof(int)
((int)&p[0])+sizeof(int)
((int)(p))+sizeof(int)
```
This generalises, so
*(p+1024*page+i) = ...;
p += 1024*page; *(p+i) = ...;
p += 1024*page; p[i] = ...;
are all the same.

To get the same effect as *(p+1024*page+i) = ...; using an integer, would be:
int i = (int)p; *((int*)(i+1024*page*sizeof(int)+i*sizeof(int))) = ...;

The key point is, when using pointer arithmetic, C knows the size of the type being pointed at, so it sorts out adding multiples of sizeof(type), in this case sizeof(int). So it also works for uint16, long, float, struct {...}, etc.
So code using pointer arithmetic is actually portable to machines which have different sizes of int, and the compiler will work it all out.

The difference between an array and pointer is the actual definition of array variables are 'constants' and pointer variables are variables. Once passed as parameters, that difference 'melts away' :-)

A handy on-line, free, C book is at:
http://publications.gbdirect.co.uk/c_book/
It can be used on-line, or downloaded as a pdf.

This section may be useful:
http://publications.gbdirect.co.uk/c_book/chapter5/pointers.html
Posted 4 years ago #
samtal
Member

Thanks.
It became a real tutorial....
It would be enough to send the links.
Truth is that we all have access to endless material, but we lack time and patience, so that forms are the ideal way out (or 'trial and error' is the preferred method).

I will learn and follow your ideas.
Thanks again.

Posted 4 years ago #
samtal
Member

To make a long story short, one more question as follows:

I have a 2-dim array of variable elements size and number.
I wish to save the array in the flash, but due to its unregular shape, saving it to specific location, and even more, reading it will require quite a work.
The question:
Is there an easy way to define one start address (pointer) and let the OS take care of all the rest?

After all, when we save an array of any size, we do not tell the system where and how to save it, thus the system knows how to break it down into cells and retrieve the data by its cell numbers in the correct manner.

I read the ST AN2594 Application note "EEPROM emulation in STM32F10x microcontrollers", but it uses a ST program which is not easy to implement with the Maple IDE.

samtal

Posted 4 years ago #
gbulmer
Moderator

samtal -

I have a 2-dim array of variable elements size and number.

How are you declaring or creating that array?
I assume not
int a[X][Y];
because that is rectangular.

Is there an easy way to define one start address (pointer) and let the OS take care of all the rest?

Which OS are you talking about? There is no OS on a Maple, so is this a host program?
If it is on Maple, the brief answer is their is no OS, and hence probably 'No'.

After all, when we save an array of any size, we do not tell the system where and how to save it,

I am confused because I don't understand what array you are taking about or where it exists.
Brief answer: Yes I think we do have to tell the system, even if it seems to be implicit and calculated by the compiler and linker. Very little of that information is available on Maple, it was consumed by the compiler and linker. So you probably have to provide it to the run-time code somehow.

Posted 4 years ago #
samtal
Member

Well, sorry, I was inaccurate and not clear. I did not mean the operating system as such, just 'the system', meaning the compiler in use together with the linker and the machine core.

In that case, all 'systems' seem to do the same job. You define the array by name and size, and it is transparently allocated to 'somewhere' in the RAM.
I wish to copy the array from that 'somewhere' by a runtime command to a flash page and back. (the type or shape or size of the array are irrelevant).

Is there a simple way to specify that location by ONE pointer that will keep track of all the array elements?

I need to be able to READ IN RUN TIME FROM THE FLASH AND UPDATE any single or bulk part AND SAVE IT BACK TO THE FLASH. I will probably also want to keep a backup in the flash.

I do know how to address the flash page, so the problem I have is in knowing or determining the location of the array in the RAM.

I can use 2 full flash pages, and replace the whole page for each change (this is normally done at initial start-up and then very seldom, if ever).

Thanks
samtal

Posted 4 years ago #
gbulmer
Moderator
samtal -

In that case, all 'systems' seem to do the same job. You define the array by name and size, and it is transparently allocated to 'somewhere' in the RAM.

Yes, that allocation is identified by the compiler, and the location of the label calculated by the linker. The linker simply leaves enough space between that label and the next one, and 'voilà' it's done. There is no need for the linker to retain the variable size and associate that value with the array in the linked binary, and even if it does, C/C++ hasn't a simple mechanism to get at it at run-time.

A running C/C++ program does not implicitly contain any information about an arrays size. You have to do that yourself.

It is very easy to do that at compile-time, for example, call the compile-time operation sizeof() on an array, and that will yield the size as calculated by the compiler. Hence:
memcpy(ramarray, flasharray, sizeof(flasharray));
would 'just work' (TM:-) when copying from flash to RAM.

Is there a simple way to specify that location by ONE pointer that will keep track of all the array elements?

No. There is no implicit size information available at run-time. It was stripped out at compile and link time.

I do know how to address the flash page, so the problem I have is in knowing or determining the location of the array in the RAM.

If you declare the RAM array, then the compiler knows the size, and will give you it through sizeof, and will generate code to give the position of the start of the array, e.g. 'ramarray' above, or by getting the address of the first element '&ramarray[0]`.

If you are dynamically allocating RAM for yourself at run-time, then you will have to solve the problem of tracking the location and size of chunks memory anyway.

The usual technique is to create a struct containing a pointer to the array, and its size:
```
struct _MyBlock_ {
    uint16 arr_size;
    byte* arr_p;
};
```
and copy the whole thing. But that uses two blocks of storage, making it a bit more complex.

You can actually do a slightly ugly thing, but that might be okay as it is only a few instances, where there is only one chunck of storage:
```
struct _MyBlock_ {
    uint16 arr_size;
    byte arr[SIZE];
} ram_array = { SIZE };
```
If you dig around the web, you'll likely find a C++ class which does that sort of thing, and overloads '[]' to make it easy to use. I.e. ram_array[i] would still work.
Most C++ classes that I have seen would check the array bounds, which would have a tiny performance impact, and may raise an exception, which might not be what you want.
Posted 4 years ago #
samtal
Member

Thanks gbulmer for your detailed reply.
I'll learn and try your ideas. Eventually there will be a way, or I'll have to write a function or class that can do it, which to me is not an easy task, although not impossible...
samtal

Posted 4 years ago #
gbulmer
Moderator
samtal - try to do the easiest possible approach. Keep your arrays a known-at-compile-time size.
Try to avoid doing anything more complex.

define the maximum size of the array that needs to be stored in ram and flash
#define RAMARRAYSIZE (100)

use that as the array size:
uint8 ram_array[RAMARRAYSIZE];

Earlier in this thread mobilvar showed how to set up an array in flash:
```
#define PAGE_SIZE 1024
#define __PAGE_ALIGNED__ __attribute__((aligned (PAGE_SIZE)))
uint8 flash_array[sizeof(ram_array)] __FLASH__ __PAGE_ALIGNED__;
```
you can initialise the the ram array from flash using memcpy:
memcpy(ram_array, flash_array, sizeof(ram_array));

You have successfully written the code to 'erase' flash, and 'program' flash from ram.

So I think you have all the pieces.
Posted 4 years ago #
samtal
Member

Kind members,

Thank you all for your support.
To the best of my knowledge, 'memcpy' and other C/C++ commands you use in your examples are not recognized by the Maple IDE and thus generate errors.
As I use the IDE, such commands are of no use to me (have I missed something?).
For the IDE we need lower level commands, or better drivers.

When, in the past, I suggested that someone knowledgeable will write a driver for the dual adc, I was opposed by several members claiming that Maple has more important tasks.
I therefore made my own unprofessional solution that, if used by others, may lead to errors and bugs.
The same happened with the flash program.
I made my own flash handling program, but it is an IDE program rather than C/C++, which I am forced to include as code in my main IDE, making it too long and difficult to handle.
I wish to use the flash program as an included external c file, but I do not have the knowledge for doing it right.
Can anyone lift that glove and convert my IDE (http://pastebin.com/h0CxGRQ7) into a viable C/C++ included file?
This has to be done anyhow to make the Maple useful for a wider range of potential users.

samtal

Posted 4 years ago #
gbulmer
Moderator

samtal - memcpy() does work
mbolivar explains how here:
http://forums.leaflabs.com/topic.php?id=1076#post-6620

The 'trick' is to
#include <string.h>
near the top of your program. That declares memcpy (and a bunch of other functions), so the error message should go away.

As mbolivar explains on that thread, the IDE links against newlib, which provides many of the standard C functions.
The documentation for newlib is at http://sourceware.org/newlib/

Posted 4 years ago #

samtal
Member

Hi,
I have tried hard the above advices with many variations, but still can not copy an array from RAM to Flash.
Here is a small IDE program that is supposed to copy a small array from the RAM to the Flash (I added flash unlock and lock. I guess this is required).

As can be seen by running the program, it copies from the original RAM location to another RAM location rather than to the flash.
I don't understand where is the flash address specified in the attribute.
What is wrong with this program?
samtal

/************Program begins here*********************/

//Maple IDE memcpy test from RAM to flash by samtal Oct 3,2011

#include "stdlib.h"      //Included explicitly for proper data printing.
#include "flash.h"
#include "string.h"

#define PAGE_SIZE 1024
#define __PAGE_ALIGNED__ __attribute__((aligned (PAGE_SIZE)))
uint8 flash_array[PAGE_SIZE] _FLASH_H_ __PAGE_ALIGNED__;
//or uint8 flash_array[sizeof(ram_array)] __FLASH__ __PAGE_ALIGNED__;

uint16 demo_data[1024]; //Build the demo array in the setup()
uint16 num_cells=100;  //Set up the number of demo cells to copy.

void setup()
{
//Build the demo array
  for (uint16 d=0;d<num_cells;d++)
  {
    demo_data[d]=0xFFFF-d;
  }
}

void loop()  

{
delay(1000);

  //Unlock flash:
  FLASH_BASE->KEYR = 0x45670123;   //The STM Key1
  FLASH_BASE->KEYR = 0xCDEF89AB;   //The STM Key2

SerialUSB.println();
SerialUSB.print("Copy demo data from:");
SerialUSB.print((uint32)&demo_data,HEX);
SerialUSB.print(" to ");
SerialUSB.println((uint32)memcpy(flash_array,&demo_data,num_cells),HEX);

//Lock the flash
FLASH_BASE->CR = 0x80;
}

/******************End of program**************************/

Posted 4 years ago #

mbolivar
LeafLabs

samtal,

this appears to be a simple copy-paste error. you wrote:

uint8 flash_array[PAGE_SIZE] _FLASH_H_ __PAGE_ALIGNED__;

but gbulmer wrote:

uint8 flash_array[sizeof(ram_array)] __FLASH__ __PAGE_ALIGNED__;

note that you wrote _FLASH_H_ instead of __FLASH__.

_FLASH_H_ is the #define used by flash.h to prevent recursive inclusion. __FLASH__ is a macro that directs the compiler to store a variable in flash. see:

http://leaflabs.com/docs/arm-gcc.html#arm-gcc-attribute-flash

edit: By the way, you can put code within backticks (the ` character underneath ~ on most US keyboards) in order to preserve formatting. i have edited your post to use this feature; i hope you don't mind.

Posted 4 years ago #
samtal
Member

Thanks mbolivar,

As you can see in my code, (in a second commented line, starting with 'or'), I have tried first gbulmers version, but when it failed I tried another option (Need to correct [sizeof(ram_array)] to [PAGE_SIZE]).
Unfortunately, neither works (I tried even more options).

Did you try running the program? Did it work for you?

The interesting thing is that if I set the copy length (num bytes) to 1 or 2, it seems to work, and I get back an address in the flash, but any other number of bytes >2 crash.

For the full program, I get 'Copy demo data from:200012F8 to', and it crashes.
For bytes to copy 2, I get: 'Copy demo data from:200012F8 to 8008800' which seems right.

I have tried everything I know (which apparently isn't enough), but for no avail.
(I have the feeling it is a variable type or casting problem?)

And 2 more, non-related:
1. Regarding the backticks; I did not see your edit in my code. (I naturally always ready to learn..)

2. A general question: On each compile (IDE), I see that the compiler builds a C++ file. Is that file available? can I see or edit it? (I couldn't find it, and it looks like a RAM or virtual file??).

samtal

Posted 4 years ago #

RSS feed for this topic

« Previous 123 Next »

Reply »

You must log in to post.