DMA memcpy mem2mem « LeafLabs Garden

LeafLabs Garden » Support » Maple Hardware Support

DMA memcpy mem2mem

(20 posts) (6 voices)

Started 3 years ago by manitou
Latest reply from Vasudev

manitou
Member

Here's a snippet of code that uses maple DMA to do memcpy() and memset() for 32-bit int's. All of the DMA channels support MEM2MEM. Testing copying 1000 32-bit words took 107us (299mbs) with DMA and 348us (92 mbs) with a for-loop. Setting 1000 32-bit words with a 32-bit value took 94us (340mbs) with DMA and 223us (144mbs) with a for-loop. A C++ implementation could use overloaded methods.

#define DMAn DMA1
#define DMA_CHn DMA_CH1

volatile static bool DMADONE;

void dma_isr(){
	dma_detach_interrupt(DMAn, DMA_CHn);
	DMADONE = true;
}

void memcpy32(uint32_t *dst, uint32_t *src, uint32_t n) {
	DMADONE = false;
      dma_init(DMAn);
      dma_setup_transfer( DMAn, DMA_CHn,
                          dst, DMA_SIZE_32BITS,
                          src, DMA_SIZE_32BITS,
                          ( DMA_FROM_MEM | DMA_MEM_2_MEM
                           | DMA_MINC_MODE | DMA_PINC_MODE
                           | DMA_TRNS_CMPLT //tell me at the end
                           )
                          );

      dma_attach_interrupt(DMAn, DMA_CHn, dma_isr);
	  dma_set_priority(DMAn, DMA_CHn, DMA_PRIORITY_VERY_HIGH);
      dma_set_num_transfers(DMAn, DMA_CHn, n);
      dma_enable(DMAn, DMA_CHn);                    //enable it..
	while(!DMADONE);
      dma_disable(DMAn, DMA_CHn);
}

void memset32(uint32_t *dst, uint32_t word, uint32_t n) {
	DMADONE = false;
      dma_init(DMAn);
      dma_setup_transfer( DMAn, DMA_CHn,
                          dst, DMA_SIZE_32BITS,
                          &word, DMA_SIZE_32BITS,
                          ( DMA_FROM_MEM | DMA_MEM_2_MEM
                           | DMA_PINC_MODE
                           | DMA_TRNS_CMPLT //tell me at the end
                           )
                          );

      dma_attach_interrupt(DMAn, DMA_CHn, dma_isr);
	  dma_set_priority(DMAn, DMA_CHn, DMA_PRIORITY_VERY_HIGH);
      dma_set_num_transfers(DMAn, DMA_CHn, n);
      dma_enable(DMAn, DMA_CHn);                    //enable it..
	while(!DMADONE);
      dma_disable(DMAn, DMA_CHn);
}

Posted 3 years ago #

higwoshy
Member

Excellent! Excuse my ignorance, but is the DMA independent of the CPU? Can you setup the DMA to send data to a port? I'm thinking of some kind of background bitmap->TV composite-out refresh.

I'm dreaming aren't I? ;)

Posted 3 years ago #
manitou
Member

Yes, the DMA is independent of the CPU, but would compete for memory access. See the wiki to learn more about maple DMA. There is DMA for serial ports, I2C, SPI, ADC, timers ...

Posted 3 years ago #
manitou
Member

Well, I timed the C lib's memcpy() and measured 56us (571mbs) and memset() was 34us (941mbs)! Wow, who knew.

Posted 3 years ago #
gbulmer
Moderator

manitou - interesting results.

I think there are errors in the timings.

AFAIK, SRAM can be accessed as bytes, double bytes, or quad bytes (word).
Assuming 72MHz, and 4 bytes per clock cycle, would give a maximum of 288MBytes/second of bandwidth.

So it looks like there is something odd going on, because some of those results seem impossible.
Maybe use a hardware counter as a timer?

Also while(!DMADONE); might have consumed significant memory bandwidth, and caused DMA to be slower than it should be.

It might be worth trying again, but avoid using RAM to communicate completion, to reduce contention for memory bandwidth between the processor and DMA. Maybe test the DMA controller register DMA_CNDTRx for completion (it should be 0 when completed)?

The Maple 'standard C library' is newlib, which is at http://sourceware.org/newlib/

It contains optimised versions for many processor architectures; some are assembler, and some C with inline assembler.
There doesn't seem to be a specialised version of memcpy or memset for arm (though there is an arm strcmp, strcpy and strlen).

The default memcpy() unrolls the loop by copying 4 long values at a time (as much as practical), and hence has 1/4 the loop overhead compared to copying one long at a time. memset uses the same technique.

Gcc can also insert an 'built-in' version of functions like memcpy, strcpy, etc. so it might be worth objdump-ing the program to see what is in there.

newlib code has been polished for years, so it is worth looking at the source to see how it works (though I dislike some of the macros).

Posted 3 years ago #
manitou
Member

>Assuming 72MHz, and 4 bytes per clock cycle, would give a maximum of 288MBytes/second of bandwidth.

my little "b" in mbs is megabits/second, so 288MBs = 2304mbs

>while(!DMADONE); might have consumed significant memory bandwidth, and caused DMA to be slower than it should be.

good thought. i'll check it out

> Maple 'standard C library' is newlib

Linux kernel's often have good hard-coded memcpy/memset. i think fastest use LDM/STM. Here is a discussion of memcpy on Cortex A8

http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.faqs/ka13544.html

thanks for insights

Posted 3 years ago #
manitou
Member

>while(!DMADONE); might have consumed significant memory bandwidth, and caused DMA to be slower

dma.c doesn't have a function for querying DMA_CNDTRx, so i hacked
volatile uint32_t *dmacnt = (uint32_t *) 0x4002000C;
... while(*dmacnt);

No change, DMA's of 1000 4-byte words still 93us (348 megabits/sec)

Posted 3 years ago #
manitou
Member
Re: newlib

> There doesn't seem to be a specialised version of memcpy or memset for arm ...

the newlib-1.20 at redhat does have arm assembler versions (memcpy.S), which uses LDRD/STRD assuming suitable sizes/alignment.
```
1:
        .irp    offset, #0, #8, #16, #24, #32, #40, #48, #56
        ldrd    r4, r5, [r1, \offset]
        strd    r4, r5, [r0, \offset]
        .endr

        add     r0, r0, #64
        add     r1, r1, #64
        subs    r2, r2, #64
        bge     1b
```
that loop runs close to 940 megabits/sec at 72MHz, if I counted cycles correctly...
Posted 3 years ago #
gbulmer
Moderator

'my little "b" in mbs is megabits/second' - Mb is mega-bits, MB is mega-bytes. 'mbs'? I guessed wrong :-)

'No change' - that is surprising.

348 megabits/sec = 43.5MBytes/sec seems a bit of a weird number, and I'm surprised it isn't faster; assuming 4 byte transfers, that is under 11M reads, and 11M writes.

I wonder if the processor and DMA are contending for the bus matrix?

The bus-architecture diagrams show the two DMA controllers as having separate connections to the bus matrix.

Have you tried setting up both DMA controllers, and using them simultaneously?

The other thought is to 'wait for event' to get the processor off the bus, but I've never tried that.

Posted 3 years ago #
mbolivar
LeafLabs

dma.c doesn't have a function for querying DMA_CNDTRx, so i hacked
volatile uint32_t *dmacnt = (uint32_t *) 0x4002000C;
... while(*dmacnt);

that's what register maps are for. use DMAx_BASE->CNDTRy, e.g. DMA1_BASE->CNDTR1.

Posted 3 years ago #
manitou
Member

I took a look at disassembled code for memcpy() for IDE 0.12 which uses newlib 1.17, i think. If it can, it uses an unrolled loop of ldr.w/str.w, 16 of these word ld/st, to move 64 bytes per loop iteration. That memcpy() takes about 59 microseconds to move 1024 32-bit words. I then built a version using the ARM memcpy.S from newlib 1.20 and that took 53us, where it uses an unrolled loop of
ldrd/strd to move 64 bytes per loop iteration. so newlib in IDE 0.12 is doing just fine.

Posted 3 years ago #
gbulmer
Moderator

manitou - interesting. Have you tried running the DMA memcpy at the same time as ordinary memcpy? Do they slow each other down?

Posted 3 years ago #
manitou
Member

> running the DMA memcpy at the same time as ordinary memcpy

baseline: 1024 32-bit word DMA memcpy: 96us (microsceconds) lib memcpy: 58 us (total 154us)

dueling memcpy:
gather micros() timestamp in DMA isr. Start DMA memcpy followed by lib memcpy
... DMA memcpy: 119us lib memcpy: 64us (total 183us)

Though DMA memcpy started first, it completed after the lib memcpy.
I used 4 1024 32-bit word vectors.

Posted 3 years ago #
manitou
Member

update

>DMA memcpy: 119us lib memcpy: 64us (total 183us)

Actually, the elapsed time for the concurrent copies was 119us (551 megabits/sec) which is faster than the serial baseline time of 154us. Though two serial lib mcmcpy's would take 116us.

Posted 3 years ago #
blackswords
Member

manitou > When do you start counting for the DMA memcpy? If it's before memcpy32() you lose some time to initialize the DMA so you don't only measure the copy time but also the initialization time. It would be better to use a DMA init function then a start function with just this inside of it

dma_enable(DMAn, DMA_CHn); //enable it..
while(!DMADONE);
dma_disable(DMAn, DMA_CHn);

Posted 3 years ago #

RSS feed for this topic

12 Next »

Reply »

You must log in to post.