Potentially running programs off the SD?

LeafLabs Garden » Support » Maple Hardware Support

[closed]

Potentially running programs off the SD?

(24 posts) (5 voices)

Started 5 years ago by crenn
Latest reply from crenn

« Previous 12

gbulmer
Moderator

Poslathian

As Bill Gates fictionally said - no one would ever need more than 640K of memory anyway

Maybe I am cynical, but I can't imagine Gates owning up to that one after, about 1984. Further, if the reporter was dead before 1997, and it was an old audio piece, it was probably a pretty safe position to take. Once he stepped down from MS, he probably got a bit more relaxed, but it's a bit late to change history again.

AFAIK, it was only recently that Gates himself admitted he'd underestimated the impact and expansion rate of the Web, and was forced to rush out and buy Spry to get a web browser (became IE) because the need became so urgent.

I watched a TV 'documentary' (in 2009?), where he seemed quite happy to dissemble; though, to be fair, the TV producers can selectively edit interviews, and even Gates might ignore inaccuracies which make him look smarter.

I am more suspicious of its accuracy, because it is so like the quote by IBM's Thomas Watson Jnr about the number of computers needed by the world.

Posted 5 years ago #
gbulmer
Moderator

The paper which explains how Dynamically Loaded Libraries work (for Linux) is called:
"How To Write Shared Libraries"
by Ulrich Drepper
and can be retrieved from http://www.akkadia.org/drepper/dsohowto.pdf

The edition there is dated December 16, 2010
The one I read was 2006, so I don't know what else has been added, but I thought it covered most of the mechanisms. It has a lot about optimisation for Linux, and the security problems caused by shared libraries/DLL's, which aren't needed but are interesting. I think the 'meat' of the paper, for this purpose, is in the first 8 pages.

The mechanism I am suggesting is to userp the indirect jump table to trigger code load and eviction, by substituting your own 'fix up' 'thunk'.

Posted 5 years ago #
x893
Member

I've been laughing reading this thread. Around 1983, I happened to run into dynamic library in RT-11 for LSI-11/20. Nothing has changed since then except for memory size and processor speed. If someone does not have enough 1MB flash memory - let him spend a week of their lives for the implementation of the manager and no brains hovers around the globe.

Posted 5 years ago #
gbulmer
Moderator

x893 - I think if you look back in the history of computing, you will find techniques for dynamic loading of code at every stage of computing.

Prior to virtual memory, it was usually visible in the programming language. For example FORTRAN COMMON allows for overlaying data, and I vaguely remember mechanisms in a proprietary version of FORTRAN (1980's) to load a code module. A 'mini-computer' system I programmed in the late 70's (bought as Singer Computers by ICL) allowed programs to chain programs and keep data resident. Most Basic's did, but they were interpreted, so it's significantly easier to do the fixup. I wrote a Bill-Of-Material system in the early 80's on a Commodore Business Computer (a PET in a nice case, with floppy disks), which had 32K RAM 'fixed', and 32K window for overlays (shared with the peripherals, so file reads and writes were a bit weird). I could write statements to pull in 'overlays'.

I used RSX-11 on a normal PDP-11, and I honestly can't remember how dynamic loading worked. Most things were set up for assembler though, so I can imagine it would be pretty easy with some assembler macro's.

My proposition is to make it 'invisible' to the running program by hijacking the normal shared library mechanism and use it in a slightly bizarre. AFAIK, the DEC PDP-11-era software (including RT-11) didn't try to do that.

The OP's question is to load into RAM, and run in their. A multi-user, multi-tasking Edition 7 UNIX ran fine in 256K (supported 5+ users), so that is my mental model :-)

Posted 5 years ago #
x893
Member

Yes - i agree. RT-11 with more sophisticated software run on 56K memory. And RSX-11 with 256K was a high top.Dynamic manager was a very simple and this method can be implemented easy. But in a real i can't imagine apllication on STM32 which need more than 512K flash. Also can use CircleOS with full support SD, file operation and software slot mechanism.

Posted 5 years ago #
MessiahAndrw
Member

Well you could use a byte code interpreter, and divide your RAM up into say 10x 2KB pages that can store bytecode and variables. When your bytecode interpreter jumps to or references an address not in memory you can dump the least use page (to disk if it's marked as dirty) and replace it with the page you're referencing.

Routines that need to run fast (or are used in many programs) could be loaded into flash memory as a runtime extension, and each extension would register a custom opcode into the interpreter.

Posted 4 years ago #
gbulmer
Moderator

MessiahAndrw - Yes, one could write a byte code interpreter. Yes, it could divide RAM into fixed size 2K pages.
That would be a slow simulation of virtual addressing. Access to a 2KB segment of SD is likely to be, approximately, 300,000x slower than access to RAM, I guess, if it were via SPI. SPI would take about 4ms to read or write 2K at 4Mbit/second

I'd recommend avoiding swapping variables to SD.
If a page contains any variable that has been changed, the page must be written out to SD if the interpreter wants to replace the variables page with some other page.
Code never changes, so it can be discarded and read in again when it is needed.
So replacing a code page is 2x faster than replacing a variables page.
With a 2K page size, it would only do 250 2K 'page replacements' /second.
If variables had to be paged out, that rate would be halved to 128 'page replacements'/second.

I'd strongly recommend a much smaller page size.
I have not looked hard at SD, but normal flash memories support 128 byte pages, which would give 16x more page replacements/second.

The mechanism I've suggested avoids the need to write a byte code interpreter.

Just use the existing C/C++ compiler (or any other language compiler which supports shared libraries/dynamic libraries).
By having the compiler generate position independent code (a command line flag), it can be loaded into RAM or Flash (on the Maples ARM Cortex-M3). It isn't constrained to page boundaries, just instruction boundaries (4 bytes boundaries are okay)

The OP could put a chunk of code, preferably stuff that is needed across the entire run of the program, into flash and call it as ordinary functions, because that's all it needs to be.

Posted 4 years ago #
poslathian
LeafLabs

x893 im with you here...

incidentally forums user sinneb recently got chibiOS working on the Maple hardware, so if you need it....

Posted 4 years ago #
crenn
Moderator

I guess this isn't something that's needed.

Posted 4 years ago #

RSS feed for this topic

« Previous 12

Topic Closed

This topic has been closed to new replies.