Discussion on the Wiring API and the new style Includes for libmaple

LeafLabs Garden » Community » Help Us Help You

Discussion on the Wiring API and the new style Includes for libmaple

(36 posts) (6 voices)

Started 3 years ago by crenn
Latest reply from modkit

« Previous 123 Next »

siy
Member

Yes, yes, yes! I'll never forget the first time I saw a compile error related to misuse of std::string. Horrible.

Compilers significantly improved this part since then.

As for other things: I see no point for discussion. I've explained an idea, if it not present an interest - no problem. But before it will be completely discarded, I'd like to show an example of how it might look like (note that it is based on existing libmaple/wiring API, so it does not completely implements ideas described above):

class Pin {
        gpio_dev* dev;
        uint8 pin;
        gpio_reg_map *regs;
        uint32 bit;
public:
        Pin(gpio_dev* dev_, uint8 pin_):dev(dev_),pin(pin_), regs(dev->regs) { bit = BIT(pin);}
        Pin(gpio_dev* dev_, uint8 pin_, gpio_pin_mode mode_):dev(dev_),pin(pin_), regs(dev->regs) { bit = BIT(pin); mod

        inline void mode(gpio_pin_mode mode) {gpio_set_mode(dev, pin, mode);}
    inline volatile uint32 __attribute__((always_inline)) read()    {return (regs->IDR & bit) ? 1:0;}
    inline void __attribute__((always_inline)) set()       {regs->BSRR = bit;}
    inline void __attribute__((always_inline)) reset()     {regs->BRR = bit;}
    inline void __attribute__((always_inline)) set(uint8 val)  { if (val) regs->BSRR  = bit; else regs->BRR = bit;}
    inline void __attribute__((always_inline)) toggle()    {regs->ODR ^= bit;}
};

Most methods are inline and translated in very short and effective code. Also, set of methods includes .set(<value>) along with simple .set().
Following example taken from one of my projects:

...
Pin Enable(GPIOB, ENABLE_PIN, GPIO_OUTPUT_PP);
...
    if (button) {
        Enable.reset();
        setWorkingMode(MODE_MENU);
        return;
    }
    Enable.set();
...

Posted 3 years ago #

siy
Member

@gbulmer, I have a great respect to your experience of teaching beginners. And I definitely rely on your opinion in regard to what is simpler and more convenient for beginners. But community of little bit more experienced developers is definitely not smaller than community of beginners. And once beginners make their first blink sketch working and try to write something more complex, they quickly realize that simplicity of the existing Wiring API works against them - most errors can be detected only at run time, syntax is verbose and API is very limited. Without debugger built into IDE the only way to debug application is Serial.print(). Limitations of the API lead to appearing low level code in the application. Resulting application looks like ugly mix of the Wiring calls, libmaple calls and direct manipulation of the hardware registers. Needless to say that such a mix leads to even more complex errors, especially when some kind or RT OS is involved and beside other problems, Wiring performance and debug output start causing race conditions and other multitasking-specific issues to appear.
All of the above does not mean that simplicity of the API for the beginners is not important. I just want to note that while thinking about beginners we should not forget about more experienced developers too. After all most beginners sooner or later will get more experience and if new API will not be satisfying, developers will abandon it. The libmaple might be a good alternative in this case, but this is completely new API and there is no seamless way to switch from one API to another. Approach described above might provide API convenient for both, beginners and more experienced developers and with some efforts open seamless way to implement things not directly provided by API (by providing simple access to hardware registers). In this case there will be no need to switch to other API, just better study an existing one.
But again, if this not present an interest and accent is made on beginners only - no problem, lets be it.

Posted 3 years ago #
mbolivar
LeafLabs
I've explained an idea, if it not present an interest - no problem

The idea is interesting, but I share gbulmer's reservations about how the error messages will look. If you have the time to try out some simple mistakes and report back the compiler output and how to explain it to beginners, that'd be useful.

I have some other concerns as well, namely memory consumption (struct stm32_pin_info is relatively compact; how well does your pin class compare?) and the fact that some pins have multiple peripherals of the same type available as alternate functions (e.g., on STM32F4, PA4 can be NSS for either SPI1 or SPI3). How do you propose to handle that situation?

this is completely new API and there is no seamless way to switch from one API to another.

I've tried to address this in the current libmaple master by providing c_dev methods as "escape hatches" into the libmaple proper device pointers for each of HardwareTimer, HardwareSPI, and HardwareSerial. These are somewhat analagous to std::string::c_str, and work like this:
```
HardwareSPI hw_spi(1); // Makes a HardwareSPI for SPI1
spi_dev *spi_d = hw_spi.c_dev(); // spi_d points to the same thing as SPI1 from <libmaple/spi.h>
```
The same thing for timers and serial ports -- their c_dev methods respectively return the underlying struct timer_dev* and struct usart_dev*. My hope is that they allow a simple-enough way to transition to the expert interface; what do you think?

Approach described above might provide API convenient for both, beginners and more experienced developers and with some efforts open seamless way to implement things not directly provided by API (by providing simple access to hardware registers).

Simple access to hardware registers follows from the c_dev routines like this:
```
HardwareSPI spi(1);
uint32 spi_sr = spi.c_dev()->regs->SR; // reads SPI1_SR
```
That is, given a HardwareFOO foo, we get that foo.c_dev()->regs is a pointer to the underlying struct foo_reg_map.

Seem reasonable?

But again, if this not present an interest and accent is made on beginners only - no problem, lets be it.

That's not my vision for Maple. I think it's possible to design APIs that allow for both beginner and expert use. The Python programming language is an inspiration to me in this regard. I really want libmaple proper to be a "real C library" that behaves roughly how an experienced developer might expect, with the Wiring layer providing convenience and beginner-friendliness.
Posted 3 years ago #

siy
Member

I'll split message into few parts for convenience.

About error messages. First of all I think it worth to note, that ugly error reports usually generated by STL and similar libraries mostly caused by complex internal structure of the templates, such a traits, support for allocators and other such things. So, when compiler meets an error while instantiating resulting template, it ought to show up all these internals. For our purposes there is no need for such a complex approaches because we're not going to support "programming in general", we just solving our narrow tasks. The code of the Pin template I did show above, so I'll not be repeating it here, just examples of errors and resulting compiler output.
Bug:

void errorMessageTest()
{
    Pin badPin(GPIOA);
}

(there is no constructor with single argument)
Relevant compiler output:

../main.cpp: In function 'void errorMessageTest()':
../main.cpp:56: error: no matching function for call to 'Pin::Pin(gpio_dev* const&)'
/home/siy/arm-eclipse/projects/workspace1/timer2/lib/Pin.h:39: note: candidates are: Pin::Pin(gpio_dev*, uint8, gpio_pin_mode)
/home/siy/arm-eclipse/projects/workspace1/timer2/lib/Pin.h:38: note:                 Pin::Pin(gpio_dev*, uint8)
/home/siy/arm-eclipse/projects/workspace1/timer2/lib/Pin.h:32: note:                 Pin::Pin(const Pin&)

Bug:

void errorMessageTest()
{
    Pin badPin(GPIOA, UNKNOWN_PIN);
}

(the UNKNOWN_PIN is undefined)
Compiler output:

../main.cpp: In function 'void errorMessageTest()':
../main.cpp:56: error: 'UNKNOWN_PIN' was not declared in this scope

Bug:

#define UNKNOWN_PIN "pin"
void errorMessageTest()
{
    Pin badPin(GPIOA, UNKNOWN_PIN);
}

(incorrect argument type)
Compiler output:

../main.cpp: In function 'void errorMessageTest()':
../main.cpp:57: error: invalid conversion from 'const char*' to 'uint8'

Bug:

void errorMessageTest()
{
    Pin badPin(GPIOA;
}

(missing closing parenthesis)
Compiler output:

../main.cpp: In function 'void errorMessageTest()':
../main.cpp:57: error: expected ')' before ';' token
../main.cpp:57: warning: unused variable 'badPin'

Bug:

void errorMessageTest()
{
    Pin badPin(GPIOA, 8);
    badPin.set("");
}

(invalid parameter type)

Compiler output:

../main.cpp: In function 'void errorMessageTest()':
../main.cpp:58: error: invalid conversion from 'const char*' to 'uint8'
../main.cpp:58: error:   initializing argument 1 of 'void Pin::set(uint8)'

As one can see, messages are simple and informative, nothing that might significantly differ from error messages produced by compiler for errors in regular (non-template) classes and believe support for parsing such a messaged is already implemented.

Posted 3 years ago #

siy
Member
I have some other concerns as well, namely memory consumption (struct stm32_pin_info is relatively compact; how well does your pin class compare?)

Template-based approach will be at least as efficient in this regard as current implementation of Wiring or even more efficient. The Pin template stores additional pointer to hardware registers for performance reasons, but this is not strictly necessary. Otherwise it contains the same set of variables as pin-related part of stm32_pin_info. And while instantiating pins only static data for actually used pins actually reaches final firmware. Current implementation of Wiring places entire array of stm32_pin_info data for all pins even if just one call to appropriate Wiring function is present in the code. Also, every stm32_pin_info contains placeholders for ADC and timer reference even if pin actually has no such functionality. With template-based approach pin definition will contain only data necessary to access hardware actually assigned to the pin.

and the fact that some pins have multiple peripherals of the same type available as alternate functions (e.g., on STM32F4, PA4 can be NSS for either SPI1 or SPI3). How do you propose to handle that situation?

I don't know exact answer yet, but I think it might be possible to use enums to instantiate slightly different implementations depending on the particular application. Something like following (not a real code):
```
enum SPI_CONFIG {
 SPI1_USE_NSS,
 SPI3_USE_NSS
};
...
template HardwareSPI<..., typename spi_config> ....
...
(application)
HardwareSPI<..., SPI3_USE_NSS> tftBus;
```
this is completely new API and there is no seamless way to switch from one API to another.

I've tried to address this in the current libmaple master by providing c_dev methods as "escape hatches" into the libmaple proper device pointers for each of HardwareTimer, HardwareSPI, and HardwareSerial. These are somewhat analagous to std::string::c_str, and work like this:

HardwareSPI hw_spi(1); // Makes a HardwareSPI for SPI1
spi_dev *spi_d = hw_spi.c_dev(); // spi_d points to the same thing as SPI1 from <libmaple/spi.h>
The same thing for timers and serial ports -- their c_dev methods respectively return the underlying struct timer_dev* and struct usart_dev*. My hope is that they allow a simple-enough way to transition to the expert interface; what do you think?

I like this idea a lot, it definitely should cover most (if not all) cases not covered by API.
Posted 3 years ago #
gbulmer
Moderator
mbolivar

I have some other concerns as well, namely memory consumption (struct stm32_pin_info is relatively compact; how well does your pin class compare?)

There are two things which consume memory, the stm32_pin_info vs template class, and the code generated when using them.

To use stm32_pin_info, for digital I/O the processor needs to load:
1. the base address of the PIN_MAP array,
2. the pin number, and
3. an offset to the member from the stm32_pin_info struct for the register address
4. the register address
5. the bit offset within the register (which must be used to generate a mask, or bit-band address)
6. the I/O bit

So the code might be quite big. This discourages me from using lots of inline (until I have a handle on the solution).

I did not do thorough enough measurements, but in one template-based variant I experimented with, the digital I/O pin class used Bit Band addressing. Everything else was a constant, or constant expression applied to the bit band address. So for simple, but common digital pin I/O, there was only the bit-band address, and it was static. So it might even be in the instruction stream (avoiding the need to load a constant pool base address).

So I think an inline digitalRead/digitalWrite was maybe smaller than a call (to digitalRead or digitalWrite). If that is the case, the template-based-approach is likely to always win on both space, and time.

Small, efficient, template-based mechanisms might have the further advantage, as siy explains, of only expanding the stuff that is used.

I stopped work on that partly because I felt the error messages from the template expressions would be too much for even relatively experienced C programmers to dig through. But I didn't do enough to see if I could solve that.
```
HardwareSPI hw_spi(1); // Makes a HardwareSPI for SPI1
spi_dev *spi_d = hw_spi.c_dev(); // spi_d points to ...
```
I like the idea of yielding a pointer to the block of registers that underpin a peripheral. I think a clever template-based approach might make this extremely efficient. If that were the case, deriving a class from, e.g. HardwareSPI, or wrapping it in a new class (which adds more functionality) might become more attractive.

A small comment; I think device, peripheral or registers are more memorable than c_dev. Further, I don't really care if it is C or C++, I would expect it to work from C++ as well as C, so that name doesn't appear to add any value. Unless you are going to tell me that the code relies on an incompatibility between C and C++, then I think the distinction is an artefact of some (old) design decisions which might feel important to LeafLabs, but IMHO don't seem to be important to the developer consuming the functionality.

(If you do tell me that the code relies on an incompatibility between C and C++, then I'd suggest you think about removing that incompatibility. :-)

siy - interesting work on the error messages. I had expected more of the template flavour to be included. IMHO templates can inject a lot of complexity into error messages, so I am impressed.
Maybe we need to try it out on people?
Posted 3 years ago #
feurig
Member

One of the joys of libmaple/wiring is that the details are kept out of the way, but you are allowed to just bypass them when when it makes sense. Most of what is proposed here (outside of pinMode(42, INPUT_PULLUP) which is just common sense) is way too complicated. As mbolivar and gbulmer already said. Keep it simple, keep it approachable. These things are not just for beginners. As an experienced programmer I don't want to pay attention to more detail than I have to; which is why I have started with wiring for most of my professional projects and why I am working with libmaple instead of the multitude of other stm code bases.

I think you are WAY over-thinking this. . Classing pins is just freaking out there. Feel free to write me off as an stdio loving, c programming geezer . Problem is that even if it isn't elegant or technically interesting it ain't broken. .

tl;dr
omg @_@ NOOOOOOOO!!!!!!!!!

Posted 3 years ago #
gbulmer
Moderator

feurig - I did my experiments (using template classes) for a couple of reasons:
1. When I have dipped under the Arduino digitalRead/digitalWrite API (to get reasonable performance on a couple of projects), changes to the Arduino library broke my code,
2. the performance of some library calls, e.g. digitalWrite & digitalRead, on Maple are so poor, compared to the underlying hardware, that it is frustrating; problems that I feel I should be able to solve easily require special solutions. I've ended up coding ARM-specific solutions, which I later felt should have been portable.

Edit: so I have wanted a 'standard' "escape hatch", to get at the hardware, and have NOT wanted much else wrapped around it.

So, I don't completely agree that it isn't broken. It might not be broken for a lot of uses, but we do get quite a regular stream of people moving to Maple, and posting on the forum because they want a lot more performance than Arduino. So I feel more folks than me would appreciate performance improvements.

A solution I was experimenting with was intended to be almost invisible. Things like digitalWrite(13, HIGH), worked the same as always, but digitalWrite(D13, HIGH) had much better performace. The easiest way to get that effect (that I can think of) is using classes for pins. Some folks assume the 'right way' is to drag in all the syntax of classes just because they have encapsulated something in a class. I don't feel that is essential, or even desirable in some cases.

I do agree with the sentiment there is no benefit in moving to a new API which is either so complex it is unusable by the majority of folks (and I include error messages as part of this complexity), or users feel it is so opaque that it becomes magic, because that has defeated the purpose of improving it for me. (My base-reference for improvements is Wiring and not libmaple or Wirish.)

I think it is extremely hard to get things right, and the Wiring/Arduino API is remarkably good. I do think the best way to get something better is to try it with real problems, with real people, and not expect to get it right first time, no matter how 'smart' we think we are.

I think (but maybe wrong) that an evolutionary approach implies the improved interface needs to appear to be close to the old API.

Posted 3 years ago #
feurig
Member

Well said.

Posted 3 years ago #
mbolivar
LeafLabs

Wow, this discussion is really going somewhere. Great.

Regarding the pros and cons of a template-based pin approach, I'm not qualified to comment, as I generally avoid using C++ features not available in C as much as I can. I remain very receptive to specific proposals with the kind of concrete analysis that gbulmer and siy have begun performing. Because of my inexperience, I will say right away that proposals which don't come with patches are not likely to be adopted, as I don't feel capable of coming up with a good implementation myself.

I really don't like doing API design in isolation, and I totally agree that anything LeafLabs comes up with unilaterally is all but certain to have serious shortcomings that user feedback can easily correct. Case in point:

A small comment; I think device, peripheral or registers are more memorable than c_dev

Yes, I'm uncomfortable with the name as well, which is why I've been keeping this feature a little quiet. I chose c_dev to try to evoke intuition from c_str, but it does have all the flaws you've outlined. Thankfully, these have never been part of a release, so there's still time to come up with something better.

I like "device" best of the names you've suggested; it makes it easier to point to documentation about libmaple's device model.

We could additionally add a "registers" method, that did nothing more than return this->device()->regs. That would provide the standard escape hatch without any additional stuff wrapped around it which you desire.

Thoughts?

Posted 3 years ago #
siy
Member

@mbolivar, I think that many peoples trying to avoid C++ just because it really a "sharp tool" and any mistake may lead to not just "shot itself in the foot" but also "amputate legs, hands and head in the same time". To use it effectively one need to have very good understanding how every line of code will be handled by compiler. And C++ in many cases unnecessarily makes things complex even in cases which are very simple by nature but do not fit into "academic style" proposed by Stroustrup. Many years ago I had discussion with him in regard to some aspects of the C++ which were related to "real world" low level and system programming, but during that discussion I quickly realized that the main goal is "purity" rather than practical usability.
At present choice of languages usable for embedded programming is quite limited, so we ought to use tools we have. With some reasonable restrictions and careful use C++ is way better than pure C (although still far from perfect). Since embedded programming by its nature (limited resources, lack of full-featured underlying OS and so forth) requires careful use of programming language and establishes some restrictions, difference between C and C++ in this regard is not so big. But, as I've mentioned above, C++ is way better and more powerful language. So, I see no reasons to limit itself and avoid C++.

Posted 3 years ago #
mbolivar
LeafLabs

siy:

I think you're probably right that many people avoid C++ for the "sharp tool" reason. I don't think I'm one of them, but I'm also not interested in having a fight about C versus C++.

These two things are true: libmaple proper is in C, and the Wiring layer is in C++. Neither of those facts is going to change any time in the foreseeable future.

I'm a bad C++ programmer, so there's good reason for me to avoid using C++ features in the code I write. However, as I've said, if you can make a good argument for using a particular C++ feature in the Wiring layer that's not currently in use, I'm happy to listen. It looks like you and gbulmer are off to a good start; I look forward to seeing what you come up with.

Posted 3 years ago #
gbulmer
Moderator

mbolivar - The experiment I'm describing was very simple.

IIRC, I only added new versions of digitalRead, digitalWrite and pinMode, and a few constructors. It used a class to differentiate existing calls (which take a pin number) from the calls of the new functions (which take a Pin). All the pins were pre-declared, e.g. D13.

My main goal was to make digitalRead/digitalWrite fast with minimal change to the API.

BTW - I have also spoken to Bjarne Stroustrup. A chum, Mark, who taught friends and myself C++, probably had the first C++ compiler in Europe, and Bjarne cut him the tape. He was the UK representative to ISO for C++ for several years. I tried to help him by reading and discussing some of the proposed changes to C++. After reading them, I would have been obsessed with trying to keep the language coherent and 'pure' too. People complain about C++ complexity, but IMHO it could have been much, much more complicated if some of the proposals were accepted (NB: complex != complicated).

At that time, people were trying to put every 'pet' idea they had into a programming language. I am looking back to before Java, SmallTalk was one of the important Object Orientated (OO) programming languages, and OO was the 'next big thing'.

C++ clearly had support, industry attention and momentum, so it was likely to be used. Hence C++ standardisation became a 'magnet' for those people with 'pet' ideas for OO, and a target vehicle for implementing everything, including 'the kitchen sink':-)

Stroustrup was probably bombarded with proposals and criticisms of C++ every waking hour for years. He likely heard most ideas, in some variation, several times each week. Further, he was one of the implementers, so the proposals weren't just intellectually interesting; he would likely end up having to implement them. I know that changes the way I listen to ideas.

When designing any large, complex, software system, the hard part is not gathering 'wants', that is easy, everyone contributes those. The hard part is weeding out from the enormous volume of 'wants' the very few critical requirements, and focusing on the right ones well enough to build them adequately. Every new independent idea has a combinatorial impact on complexity, so we want to keep the core of systems simple and coherent (to keep it buildable, testable and flexible enough to evolve). As a more recent example, have a look at the volume of Java ideas, or look at VPRI's fonc mailing list.

IMHO, Stroustrup did well to focus on a few key areas, move forward, and keep moving forward.

I think it may be helpful for people to view conversations with him from this perspective.

Posted 3 years ago #
siy
Member

@gbulmer, I completely agree, that at the time when C++ started gaining popularity, everyone tried to put every 'pet' idea into language. And definitely not every idea was simple enough to implement. But things like 'finally' clause were very easy to implement and would save a lot of time and efforts for users. Yes, I know about RAI and destructors, but while working with low level OS resources such as handles or OS-allocated memory regions 'pure' approach results to countless tiny classes which do nothing (beside making code harder to read). In contract 'pure' dynamic_cast<> was included into standard, despite significant efforts necessary to implement RTTI and its doubtful usability in real life.
Must admit that I'm assessing C++ as a system programming language while standard is targeted toward application programming, hence different 'weights' for different features and their usefulness in real life.

Posted 3 years ago #
mbolivar
LeafLabs

IIRC, I only added new versions of digitalRead, digitalWrite and pinMode, and a few constructors. It used a class to differentiate existing calls (which take a pin number) from the calls of the new functions (which take a Pin). All the pins were pre-declared, e.g. D13.

My main goal was to make digitalRead/digitalWrite fast with minimal change to the API.

Out of curiosity, did you also try separate inline and "standard" versions plus use of __builtin_constant_p() to determine which to use (as tried by Arduino)? If so, what turned you off about it?

Posted 3 years ago #

RSS feed for this topic

« Previous 123 Next »

Reply »

You must log in to post.