Bootloader-less Designs « LeafLabs Garden

LeafLabs Garden » Community » The Lounge

Bootloader-less Designs

(39 posts) (4 voices)

Started 3 years ago by gbulmer
Latest reply from gbulmer

12 3 Next »

gbulmer
Moderator

This is a new thread and repost, as suggested by a post here: http://forums.leaflabs.com/topic.php?id=73&page=3#post-20829
and follows: http://forums.leaflabs.com/topic.php?id=73&page=3#post-20823

...

The STM32F3 and STM32F4 both have manufactured-in USB bootloaders. I haven't tried to use that, but maybe that is an adequate solution in the future? AFAIK ST Micro provide a windows program which works with this.

I have used a USB to serial cable and the Python program on Mac OS X and Windows with STM32F103, as well as texanes st-link software with on-board ST-LINK/V2 on Mac OS X with STM32F4, and they work. A STM32F0Discovery is only $7.99, and it includes ST-LINK/V2 upload and a hardware debugger.

Maybe a way to go is to build *very* simple STM32F3/STM32F4 boards which only use the manufactured-in bootloader (and try to appeal to ST Micro to make them for $10 like their existing Discovery boards?-)

This might have the useful property that a new board can be made functional without any extra programming or upload hardware.

Posted 3 years ago #
feurig
Member

I was actually thinking about going in the opposite direction given the promised support of the low end non usb chips (like the F0) where the chips themselves are dirt cheap but share the same footprint as the f1s. I agree that it would be nice if there we an option to script or to use a programmer much like the current arduino allows you to bypass their boot loader within the ide.

Posted 3 years ago #
tormod
Member

gbulmer, what would be the advantage of using the manufactured-in USB bootloaders compared to using the maple bootloader? They are pretty much equivalent, only differ in DFU versions.

What are the requirements of a new "bootloader-less design" ? Being able to run a "sketch", reprogram it, rerun the new sketch - without any hardware intervention like pushing a button or plugging in an adapter?

Using the built-in serial bootloader through an on-board CP2102 would be one solution, but then you would have one USB connector for programming and one for the application (usually virtual serial port) so the user would have to replug the USB cable.

Posted 3 years ago #
gbulmer
Moderator

tormod - I have posted most of this before, but I'll post in this thread for ease of readers.

what would be the advantage of using the manufactured-in USB bootloaders compared to using the maple bootloader?

In my experience, the Maple bootloader works fine for Mac OS X, and is pretty easy to get working on Linux (if you know enough to use Linux), but the Maple bootloader seems to be very unreliable for Windows users. If you look through the last few years of forum posts, I think it is the dominant problem for users. LeafLabs have described plans for better bootloaders for more than two years, but there has been no improvement for Windows.

I run workshops for folks who are interested in microcontrollers, electronics and programming. I still use Arduino's (after a couple of years using Maple personally) because it is much easier to get Arduino's working on peoples Windows machines, which is still the predominant OS, than Maple. Sometime I would prefer to use an ARM processor over AVR; I find projects where the extra headroom from STM32F1/4 over AVR might enough to enable relatively inexperienced people to succeed. But IMHO Maple is just too hard for less technical Windows users.

So one advantage might be the manufactured-in bootloader works for Windows user!! I think there is enough documentation of the protocol that we can use the same manufactured-in bootloader for Linux and Mac OS X. So it might be easy enough for new users.

Further, I would like to run workshops where folks make their own microcontroller. In some UK schools, children are taught to do surface mount soldering because it is faster and more reliable than through-hole soldering. I use surface mount for the same reasons. I think I can help people be sucessful making a simple board with a surface mount parts. If there is no need to put a bootloader on, I need less equipment, or a school needs less equipment, and more people can DIY boards with minimal facilities.

I'd like to help folks make their own ARM based electronics. Again, having a slick way to load programs, on a brand-new chip, with no extra hardware is attractive.

Using the built-in serial bootloader through an on-board CP2102 would be one solution, ...

Agreed, but I am talking about using the manufactured-in USB bootloader on ST32F3/4 etc. So it doesn't require a CP102, or any electronics other than the USB interface (socket and either a couple of resistors and capacitors, or a USB 'protection' chip).

... but then you would have one USB connector for programming and one for the application (usually virtual serial port) so the user would have to replug the USB cable.

This is not what I am suggesting, but in that sort of case why not use two USB cables, one for programming and one for USB?

(Full disclosure: I am not a member of LeafLabs staff.)

Posted 3 years ago #
gbulmer
Moderator

feurig

I was actually thinking about going in the opposite direction given the promised support of the low end non usb chips (like the F0) where the chips themselves are dirt cheap but share the same footprint as the f1s

Sorry, I misunderstood. I was thinking "more computer power than an Arduino Due, easy to DIY, no extra hardware, under $10" :-)

I like the STM32F051R8T6 which are under $2, Seriously more fun than an Arduino UNO, and could be put on a board for under $6 (two Starbucks coffees?). Stripping off the USB support from my little experimental F1 board would make it smaller and cheaper. Siy's board, which is already good, would be even more compact.

My thought their is to either use the ST-LINK/V2 on a low-cost (STM32F0Discovery), which would make the board small and low-cost, or maybe use the same ATmegaU8/16 USB to serial adapter that the Arduino UNO uses. That design already exists, the electronics is available built and tested, and much of the firmware would be the same.

Posted 3 years ago #
tormod
Member

In my experience, the Maple bootloader works fine for Mac OS X, and is pretty easy to get working on Linux (if you know enough to use Linux), but the Maple bootloader seems to be very unreliable for Windows users. If you look through the last few years of forum posts, I think it is the dominant problem for users. LeafLabs have described plans for better bootloaders for more than two years, but there has been no improvement for Windows.

I believe the bootloader itself is reliable and is not the problem. It is the switching from run-time into bootloader and the associated timing that is troublesome.

So one advantage might be the manufactured-in bootloader works for Windows user!! I think there is enough documentation of the protocol that we can use the same manufactured-in bootloader for Linux and Mac OS X. So it might be easy enough for new users.

The manufactured-in USB bootloader works with dfu-util on Windows, MacOSX and Linux, just like the Maple bootloader. On Windows you will have the alternative of using ST's own program, but if you look around at STM32 (devices) forums you'll see no end of problems installing the drivers for it (Vista, Windows 7, 64-bit...) - and you would rely on ST to fix it, not a way to go for an open, do-it-yourself project like Maple IMO.

You would still have the same issue of how to switch the Maple board into the bootloader mode.

Using the built-in serial bootloader through an on-board CP2102 would be one solution, ...

Agreed, but I am talking about using the manufactured-in USB bootloader on ST32F3/4 etc. So it doesn't require a CP102, or any electronics other than the USB interface (socket and either a couple of resistors and capacitors, or a USB 'protection' chip).

I meant this as an alternative. One could let the hardware reset and boot into bootloader mode as soon as the programming USB socket is plugged.

I asked about the bootloader system requirement, because what I see as fragile in the current system is this run-time switching into bootloader mode. When people use the button (perpetual boot mode) there is less trouble, isn't it? If that button would simply reset the board as well (before going into bootloader mode) it would be straight-forward: To program the board, press the button before you click "Flash" in the GUI. But then the button can not be used by user programs, of course.

Posted 3 years ago #
tormod
Member

mbolivar, are you listening also? :) I have a suggestion for an incremental modification to the existing system which preserves backwards compatibility with older bootloader versions. Instead of always starting the bootloader for 2 seconds at reset (and confuse Windows and many Windows users), either go straight to the user application (sketch) or start the bootloader in perpetual mode. A magic value set in RAM before the reset will tell the bootloader start-up code what to do. The CDC code in the sketch (libmaple) will set the magic before resetting the system (after detection of the DTR negative edge and "1EAF" sequence sent by the IDE).

Untested patches at https://github.com/tormodvolden/libmaple/commits/master and https://github.com/tormodvolden/maple-bootloader/commits/master

(I have previously done similar magic-in-RAM signalling for other bootloaders like https://gitorious.org/dsonano/dso-bootl)

Posted 3 years ago #
gbulmer
Moderator

tormond -

I believe the bootloader itself is reliable and is not the problem. It is the switching from run-time into bootloader and the associated timing that is troublesome.

I think that is exactly a property of the bootloader design and implementation? Don't you? The way that the Maple bootloader interacts with a Windows system creates a problem.

If not, what causes the problems for Windows users that is not part of the design and implementation of the bootloader?
How do you think that should be fixed?

The manufactured-in USB bootloader works with dfu-util on Windows, MacOSX and Linux, just like the Maple bootloader.

I was assuming dfu-util would have to be modified to understand ST's manufacturede-in DFuSE bootloader because LeafLabs had written that Open Moko's dfu-util does not work for the manufactured-in bootloader. I haven't checked. Have you?

If that is not the case, and dfu-util does work on all platforms with the manufactured-in bootloader then there is already "no work" approach to programming, which does require the user to press the reset button. Folks can "ignore" the Maples bootloader, and use the built-in one. Given the amount of button pressing and uncertainty which Windows users report, it might be no worse, and maybe better.

On Windows you will have the alternative of using ST's own program, but if you look around at STM32 (devices) forums you'll see no end of problems installing the drivers for it (Vista, Windows 7, 64-bit...)

I am not prejudging how I would make the ST Micro manufactured-in bootloader work reliably.

- and you would rely on ST to fix it, not a way to go for an open, do-it-yourself project like Maple IMO.

Are you saying you'd rather write code to fix a problem created by ST, that will only work for ST chips, so that the solution is Open Source, rather than encourage ST to fix the problem they created, and write no code?

I asked about the bootloader system requirement, because what I see as fragile in the current system is this run-time switching into bootloader mode.

That is part of the problem I was writing about when I stared this thread. So have you now come around to my view; that the problem cause is a property of the Maple bootloader design and implementation. The code in the bootloader was designed and written to work this way.

When people use the button (perpetual boot mode) there is less trouble, isn't it?

It is not a single button push, the procedure is described here: http://leaflabs.com/docs/troubleshooting.html#perpetual-bootloader-mode

Have you used it?

Some people have significant problems getting the Maple to go into that mode, but AFAIK, once they get it into that mode, they can usually (always?) program it. So the manufactured-in bootloader may be simpler to invoke than Maple's perpetual bootloader mode :-)

If that button would simply reset the board as well (before going into bootloader mode) it would be straight-forward ...

Have you used Maple's perpetual bootloader mechanism? Part of the procedure to get the Maple bootloader into perpetual bootloader mode is to press Reset, so the MCU has been reset, and then the next button-push-dance tells the Maple bootloader to stay in bootloader mode.

Seems like you are agreeing that the Maple bootloader is causing problems. Yes?

Posted 3 years ago #
tormod
Member

gbulmer, I think you may have missed my follow-up post, which should have clarified many things :)

I think that is exactly a property of the bootloader design and implementation? Don't you? The way that the Maple bootloader interacts with a Windows system creates a problem.

Yes, I was by bootloader referring to the USB bootloader once it is running, so I agree, the whole implementation of how it is started and so on is the problem. My point was that if you replace the maple bootloader DFU code with the built-in USB bootloader, you replace the "running bootloader part", but not the implementation of how to get into bootloader mode. You would still have to provide a user-friendly solution for how to enter this mode. So no problem solved IMO.

If not, what causes the problems for Windows users that is not part of the design and implementation of the bootloader?
How do you think that should be fixed?

Well, I have posted patches in my previous post.

I was assuming dfu-util would have to be modified to understand ST's manufacturede-in DFuSE bootloader because LeafLabs had written that Open Moko's dfu-util does not work for the manufactured-in bootloader. I haven't checked. Have you?

I wrote that support 2 years ago. BTW, I am maintaining the official dfu-util (http://dfu-util.gnumonks.org/).

If that is not the case, and dfu-util does work on all platforms with the manufactured-in bootloader then there is already "no work" approach to programming, which does require the user to press the reset button. Folks can "ignore" the Maples bootloader, and use the built-in one. Given the amount of button pressing and uncertainty which Windows users report, it might be no worse, and maybe better.

Yes, if pressing a button is acceptable, there is no need for 2 second magic window etc. But it is convenient to control the whole process from the computer without physically accessing the board. Anyway, the patches in my previous post get rid of the magic time window AND maintain the full software control of the run-time/bootloader switching.

Are you saying you'd rather write code to fix a problem created by ST, that will only work for ST chips, so that the solution is Open Source, rather than encourage ST to fix the problem they created, and write no code?

Yes, that's what I am saying and that's what I did. Like the dfu-util DfuSe support. Hang around at ST's STM32 forum and you'll get an idea to which length ST is supporting the customers. ST makes great, cheap hardware, but they write awful software and their community support is miserable.

I asked about the bootloader system requirement, because what I see as fragile in the current system is this run-time switching into bootloader mode.

That is part of the problem I was writing about when I stared this thread. So have you now come around to my view; that the problem cause is a property of the Maple bootloader design and implementation. The code in the bootloader was designed and written to work this way.

You probably share my view, I was just not expressing it accurately enough. I was referring to "bootloader" as the code running while in bootloader mode, but that is strictly too limited. I used the term "bootloader system" where you are saying bootloader. No big deal.

It is not a single button push, the procedure is described here: http://leaflabs.com/docs/troubleshooting.html#perpetual-bootloader-mode

Have you used it?

Yes, I have no Maple but I ported it to a development board. I had no problem using the reset and user button to enter perpetual boot mode. But I agree this two finger dance it not obvious for everybody. One button instead of two would help a lot.

Have you used Maple's perpetual bootloader mechanism? Part of the procedure to get the Maple bootloader into perpetual bootloader mode is to press Reset, so the MCU has been reset, and then the next button-push-dance tells the Maple bootloader to stay in bootloader mode.

Seems like you are agreeing that the Maple bootloader is causing problems. Yes?

Not any longer! :) Please try out my patches.

Posted 3 years ago #
gbulmer
Moderator

tormod - Thanks for the clarification. I think I understand what you're describing.

To reduce ambiguity, let's call ST Micro's manufactured-in USB bootloader the "ST-bootloader", and Leafleab's Maple bootloader the "Maple-bootloader"

so I agree, the whole implementation of how it is started and so on is the problem. My point was that if you replace the maple bootloader DFU code with the built-in USB bootloader, you replace the "running bootloader part", but not the implementation of how to get into bootloader mode. You would still have to provide a user-friendly solution for how to enter this mode.

The way the that the ST-bootlader works can't be changed. AFAIK, it runs when power is applied or a system RESET is triggered.
So the user can unplug power, press reset, or a board can have a power switch.
Not as nice as a host-controlled "automatic" reboot, but consistent, easy to understand, and simpler than the Maple-bootloader 'perpetual bootloader button-dance'.

I wrote that support 2 years ago. BTW, I am maintaining the official dfu-util

Brill!
So there is already a low-effort way of making a more robust boot method that would work for 'raw' STM32F chips.

Folks can "ignore" the Maples bootloader, and use the built-in one. Given the amount of button pressing and uncertainty which Windows users report, it might be no worse, and maybe better.

Yes, if pressing a button is acceptable, there is no need for 2 second magic window etc.

The "Button-dance" is necessary right now to get into a state where a program uploads, so net/net it may be better to just hit reset.

But it is convenient to control the whole process from the computer without physically accessing the board.

Agree. That is better than having to press a button.

But the bootloader flipping between USB DFU and USB CDC causes more problems than just making it hard to upload a new program; for example a program can't reliably do I/O in setup, it has to wait until USB is ready, which has appeared as a problem on this forum repeatedly. So if we are discussing "better", we should factor that in too. IMHO, to ignore those problems is a bit like "rearranging deck chairs on the deck of the Titanic".

We have had lots of discussions, e.g. http://forums.leaflabs.com/topic.php?id=758

So my original proposal was to avoid requiring a bootloader, to make it extremely simple to make boards.
An alternative, if the manufactured-in bootloader is to be replaced, replace it with a bootloader which removes all major problems.

IMHO, a bootloader should only present itself as a single USB device all the time, unless the user has programmed it to be something else for their application (and so far no one has reported such an application to this forum).

That USB device should use one of the few cross-platform, pre-installed USB drivers. It needs to support both booloading, and be be capable of serial comms to support print/read. The USB DFU mechanism creates extra problems which Windows makes worse.

So, no DFU. CDC serial, HID or Flash drive (I think all OS's support those three 'out of the box').

I agree a custom bootloader would be better if it could be put into bootloader mode 'automatically' from the host.

I would like to make the boards as simple as practical, with minimal extra hardware, so that they could be incorporated into peoples own designs.

I would like it to work with Windows "Thin Clients" because that is the technology used by many British schools.

Feurig has reminded us that some of the new STM32F0 are supper-cheap, and don't have USB hardware built in. A good bootloader scheme would enable someone to use them easily.

--------------

Hang around at ST's STM32 forum and you'll get an idea to which length ST is supporting the customers.

I did hang around. I didn't find ST on the forum very helpful. But Clive1 is fantastically helpful.
Also I have found the ST UK tech support helpful.

ST makes great, cheap hardware, but they write awful software and their community support is miserable.

I don't use Windows, so my only experience of their software is the ST manufactured-in serial bootloader, and the STM32 libraries.
The serial bootloader worked okay to put bootloaders onto STM32F103's.
IMHO the STM32 libraries are very ugly, but some of that is constrained by ARMs CMSIS API definition.

Posted 3 years ago #
gbulmer
Moderator

Back to Feurig's point, about making very low-cost boards using e.g. STM32F051.
They are under $2 each, for an MCU more capable than an Arduino UNO's ATmega328.

As tormod points out, a cheap CP2102 USB-to-serial adapter could be used. CP2102 are about $2.50 USA ebay (£1.78 or less from UK ebay)

Their are several Open Source versions of upload software that talk directly to the ST manufactured-in serial bootloader.
Most CP2102 have 6 pins, and it looks like the extra pin could be used by a program running on the host to reset the STM32Fxxx into bootloader mode. So upload would be 'automatically' controlled, similar to the Arduino.

Has anyone tried to DIY (toaster oven) a UFQFPN-32 package?
That package is only 5mm x 5mm, and hence would support a very small board.

Posted 3 years ago #
ala42
Member

You can use the F4 built in boot loader after installing the libusb driver with the dfu-util when the CPU is in boot mode.
You can enter boot mode by pulling BOOT0 high and applying a reset or in case there is a program running, you can jump into the internal boot loader.
The solution I have choosen is using a boot code that is started after reset and decides depending on a flag set by the main program if the main program is started directly or if the built in boot loader is started. In this case I apply a USB disconnect first, as the USB DFU device was not detected without the disconnect.
The flag is set in the USB code of the main program which resets the CPU after reception of a magic sequence, which is about the same way it is done in libmaple.
This mechanism works as long as the main program still accepts USB data.

Posted 3 years ago #
tormod
Member

The way the that the ST-bootlader works can't be changed. AFAIK, it runs when power is applied or a system RESET is triggered.
So the user can unplug power, press reset, or a board can have a power switch.
Not as nice as a host-controlled "automatic" reboot, but consistent, easy to understand, and simpler than the Maple-bootloader 'perpetual bootloader button-dance'.

The ST-bootloader only runs at reset/power-up if two boot input pins are in the right state. Well, the way it works can be changed also, because you can jump from user code into the ROM bootloader with some precautions. So you would need some buttons/circuitry on board to control those pins - or have a bootloader/startup stub in the normal flash that jumps to the bootloader when the user wants, pretty much like the old perpetual bootloader mode.

But it is convenient to control the whole process from the computer without physically accessing the board.

Agree. That is better than having to press a button.

But the bootloader flipping between USB DFU and USB CDC causes more problems than just making it hard to upload a new program; for example a program can't reliably do I/O in setup, it has to wait until USB is ready, which has appeared as a problem on this forum repeatedly. So if we are discussing "better", we should factor that in too. IMHO, to ignore those problems is a bit like "rearranging deck chairs on the deck of the Titanic".

Again, the patches I suggested some posts ago, avoid these problems. If the device boots up in application mode, it does not go via any bootloader mode and leaves the USB to be set up or not as the application wants.

IMHO, a bootloader should only present itself as a single USB device all the time, unless the user has programmed it to be something else for their application (and so far no one has reported such an application to this forum).

That USB device should use one of the few cross-platform, pre-installed USB drivers. It needs to support both booloading, and be be capable of serial comms to support print/read. The USB DFU mechanism creates extra problems which Windows makes worse.

So, no DFU. CDC serial, HID or Flash drive (I think all OS's support those three 'out of the box').

Are you listing alternatives or do you want all of the above? "A single USB device all the time" and using e.g. Flash drive (USB storage) for programming the flash while keeping the CDC serial would call for a USB composite device, which is not simple to deal with consistently on all OS's. Maple already gave up on that. It is in particular the CDC which is difficult to combine with other USB class interfaces.

Feurig has reminded us that some of the new STM32F0 are supper-cheap, and don't have USB hardware built in. A good bootloader scheme would enable someone to use them easily.

USB functionality is IMO a great plus, I don't think it is worth saving a dollar or two (?) sacrificing it. Especially since you would need to add a serial/USB adapter anyway to program it. Makes sense for mass production of a product (without USB - boring! :) ), but not for something you gonna reprogram all the time.

Posted 3 years ago #
tormod
Member

ala42, exactly. This is also how it is done in my patches, with a "magic" flag in RAM. I did not add an explicit USB disconnect and you are probably right this is needed because the device reset is so fast.

Posted 3 years ago #
gbulmer
Moderator

tormond

That USB device should use one of the few cross-platform, pre-installed USB drivers. It needs to support both booloading, and be be capable of serial comms to support print/read. The USB DFU mechanism creates extra problems which Windows makes worse.

So, no DFU. CDC serial, HID or Flash drive (I think all OS's support those three 'out of the box').

Are you listing alternatives or do you want all of the above?

Alternatives.
Hopefully to be clearer.
The board should present itself as the same USB interface all of the time for both program upload, and print/read communication. It should never change it's USB Device type ever, unless the user writes that as part of their application.

The only USB devices which appear to be available across the major platforms of Windows, Linux and Mac OS X, with reasonably consistent (but not identical :-( behaviour are: CDC serial, HID or Flash drive. Implementers are free to choose any one of those. Implementers can choose more than one, and provide some mechanism for the board user to choose, but that is optional.

A good implementation will support Windows "Thin Clients".

"A single USB device all the time" and using e.g. Flash drive (USB storage) for programming the flash while keeping the CDC serial would call for a USB composite device ...

What you describe is not "A single USB device all the time". What you describe is two different USB devices; a Flash drive and a CDC serial.

USB functionality is IMO a great plus

Agreed.

I don't think it is worth saving a dollar or two (?) sacrificing it.

Okay. I have a lot of sympathy with that view. It isn't purely cost, though that is a significant criteria.

On a cost basis, it looks like the difference in cost between the lowest-cost STM32F051 ($1.34 no USB) and lowest-cost STM32F103 ($2.96) is more than 2x, or about $1.60 (at future electronics). That difference is just about enough to buy a small PCB (20-off quantity), and maybe enough of the passive components necessary for the board to function.

One of my interests are things as low-cost as:
http://shrimping.it/blog/
where the whole kit costs under 2 GBP. IMHO this is something anyone might make for the fun of it, for example to use in a DIY TV-be-gone. The point is, low-cost appeals to more than mass-producers. Low-cost appeals to someone who has very little spare money, or doesn't see value in spending more than essential. I do a public Arduino workshop almost every month in Birmingham UK. People are impressed by the affordability of an Arduino or Freeduino, then amazed and excited that they could make something for 3 GBP.

Other reasons for not needing USB:
- If a school is making boards with their pupils, they might prefer to have something between the PC's USB socket and the board and have no desire for onboard USB interface.
- Another example might be an Ethernet, or wireless network board where USB is not available.
- Some people building weight sensitive or constrained systems might prefer not to use USB; for example, very few (no competition robots I can think of) Micromouse robots have USB sockets onboard, but might have a micro-JTAG, SWD or serial header instead.

Considering different types of board, including USB-less, is part of what triggered this thread.
Thank you very much for your extremely constructive, informative and helpful comments.

Posted 3 years ago #

RSS feed for this topic

12 3 Next »

Reply »

You must log in to post.