Stephen,
Your post brings up some important issues in both the IDE's current implementation and the way we develop it. I've been considering these issues for a while, and I'd like to respond to your concerns in detail. I hope everyone will forgive me the length of this post.
These paragraphs from your previous post are particularly relevant:
Knowing that the "Examples >> Maple >> InteractiveTest" is used to test each board is informative, but also puzzling. On a Windows XP SP3 system I am still not able to compile the InteractiveTest on Maple IDE 0.0.9 (see below). I reported this problem when 0.0.9 was released (in December, 2010). My interpretation is that when boards are evaluated the InteractiveTest QA is not performed using a Windows system, which is fine. However, in the future please be sure that the InteractiveTest sketch works on all supported platforms before major software releases.
Question: I have not looked at how the Timers/PWM code changed between IDE release 0.0.7 and 0.0.9, but if Maple Rev 3/5 board runs the InteractiveTest sketch from 0.0.9 (which should evaluate PWM) onn a linux system why did crenn's code fail on a Windows system? Does crenn's code work on a linux system? Another possible way to ask the question is why does InteractiveTest work on linux, but not Windows (using IDE 0.0.9)?
In maple-test-procedures.pdf, under the section labeled "Upload test program", we specify that we use the C++ version of the interactive test program which is included in the libmaple repository. The section goes on to say that it should be possible to use the InteractiveTest example as well, but there's a parenthetical comment "(TODO: double check that this is true)". Unfortunately, when I was putting together the 0.0.9 release, I failed to make sure that the InteractiveTest included with the IDE reflected the changes that had been made to libmaple, so, as you noticed, it failed to compile.
This mistake thus does not reflect a difference between the behavior of a program when compiled on Linux vs. Windows, but rather reflects the fact that I carelessly did not ensure that all of the IDE examples were kept up to date when I released 0.0.9. These examples in general are a place where we could improve our quality significantly. It is obviously an unsatisfactory situation when the same code has to be maintained by hand in two separate places, specifically, as C++ versions in libmaple, and as .pde versions in the IDE.
The "best" solution (in the sense of "most correct", "leading to least duplication of effort") is to build a standalone mechanism which enables the automatic translation of sketches written in the Maple variant of the Wiring language into C++. Presently, this is accomplished from within the Sketch#preprocess(String, PdePreprocessor) method in the IDE's Sketch.java:
https://github.com/leaflabs/maple-ide/blob/master/app/src/processing/app/Sketch.java#L1261
Instead, we could write a standalone program which performs the PDE preprocessing step, which the IDE and libmaple could both make use of. The LeafLabs developers could then write all of our examples as .pde files, and still compile them using the Unix toolchain.
(It may seem as though my preference (and that of other LeafLabs developers) to continue using the Unix toolchain reflects a lack of confidence in our IDE. To respond preemptively to that: the IDE is meant to be a simple, easy-to-use tool, with minimal frills. While it definitely has its problems, I don't think it fails utterly in this regard. However, the fact remains that my tools of choice (namely, Emacs and the command line), while decidedly more difficult to use, are also significantly more powerful, and I am simply not productive enough without the benefits this extra power brings.)
Unfortunately, producing such a PDE preprocessor properly turns out to be a rather subtle task, so at least for the time being, I think it's best if we continue to maintain the two separate versions of each example. We at LeafLabs will simply have to be vigilant in maintaining the two parallel versions of the test code.
This is an unlovable situation to me, but I think it's the most workable solution. The remainder of this post explains my rationale.
To begin, let's establish the PDE preprocessor's jobs. Glossing over some details, these are:
(1) Concatenate all of the sketch's files together
(2) Parse the program to find the user's imports (so we know which extra libraries to link against) and declared functions (so we can produce prototypes for them)
(3) Output valid C++ which #includes wirish/ and defines prototypes for the user's functions, so they don't have to, along with a main() function and code that calls wirish's init() function.
The current implementation really fails hard in step (2); I've been wanting to get rid of it and replace it with something more robust and stand-alone since I first read its implementation. For example, try compiling this sketch, which should be perfectly legal code:
#define BB }}
void setup() {
if (true) {
BB
void loop() {
}
You'll get compile errors. This behavior isn't unique to the Maple IDE; the Arduino environment (from which we inherited the PDE preprocessor implementation) has the same problem.
While the above code is certainly a gross abuse of the C preprocessor, the fact remains that the IDE's PDE preprocessor doesn't take the C preprocessor into account, which is just Bad and Wrong.
For comparison, here is the equivalent C++ program:
#include "wirish.h"
#define BB }}
void setup() {
if (true) {
BB
void loop() {
}
__attribute__((constructor)) void premain() {
init();
}
int main(void) {
setup();
while (1) {
loop();
}
return 0;
}
This compiles just fine against the current libmaple master (cdd367bdd264c9e19180 at time of writing).
Now, with regards to producing function prototypes, the current implementation attempts to use regular expressions to parse function declarations:
https://github.com/leaflabs/maple-ide/blob/master/app/src/processing/app/preproc/PdePreprocessor.java
For reasons that have been perfectly well established in parsing theory for decades, regular expressions are simply inadequate to parse a context dependent grammar such as that of C++. So it's not just that the current implementation has flaws, it's that important pieces of its strategy are fundamentally invalid.
For example, consider the problem of parsing out a prototype from a function declaration which takes as an argument a function pointer, like so:
void func(void (*arg)(void)) {
SerialUSB.println("calling argument function");
arg();
SerialUSB.println("done!");
}
Again, perfectly valid Wiring and C++. However, note that the argument function pointer could also take arguments which are function pointers. We could do this an arbitrarily large number of times:
void func1(void (*func2)(void (*func3)(...))) {
...
}
Thus, in order to detect if such a declaration is proper, we need among other things to make sure that all the nested parentheses match properly. It's an elementary example in most texts on parsing that regular expressions are unable to determine whether or not an expression with parentheses is properly balanced.
Rather than throwing good money after bad by trying to polish up the current implementation in a standalone program or otherwise, it is clear (at least to me) that more radical surgery is required, namely, using a real parser to get the job done. Unfortunately, as you may know, the C++ grammar is notoriously difficult to parse, and the Wiring language is roughly a superset of C++. This makes parsing it difficult enough to make it a waste of time for us to write up something from scratch.
In order do to the right thing, then, we'll need an existing C++ parser which we can modify in some way. Further, since we specifically advertise that we compile using CodeSourcery's arm-none-eabi-gcc, this parser must be aware of all GCC extensions to the C++ language.
I've spent some time trying to come up with an acceptable solution that satisfies these constraints, and I've concluded that although it is possible to do so, it would take up too much time that, at least for now, is better spent elsewhere.
To my knowledge, there are only two fully GCC-compatible C++ parsers in existence: that which comes with GCC itself, and (more recently) the version that comes with LLVM's clang:
http://clang.llvm.org/
While LLVM provides libclang, an excellent library for interacting with the data structures produced by its parser (see http://devimages.apple.com/llvm/videos/Libclang.mov for a very impressive talk on libclang), using it would either introduce an entire extra compilation toolchain as a dependency (gross!), or require us to switch from CodeSourcery's tools. I'm all for switching, but clang's support for the Cortex M3 architecture is not yet complete, so that isn't workable.
So, there's only one thing left to do: modify CodeSourcery's GCC C++ parser to produce a list of function declarations in the user's sketch. That would be easy enough if the sketch was valid C++; however, because Wiring doesn't require that a function is declared before it's used (unlike C++), we can't just run the program unmodified.
There is a program called protoize, part of GCC, that helped accomplish a similar goal for pure C. As I recall, protoize was useful in the early 1990s when the C standard mandated a function declaration, instead of stating that a function which was called before it was declared as implicitly assumed to return int. However, C++ is a much more complicated monster than C, and protoize seems to have fallen into disuse. So, (again, at least as far as I can tell), if we're going to do this thing right, we'll have to modify GCC itself.
While that's certainly possible, we at LeafLabs don't have enough practice implementing compilers nor understanding of GCC's internals to have confidence that we could do the job quickly and well.
I spent a few days during fall 2010 reading through the sources for arm-none-eabi-gcc, specifically, the gcc-4.4-2010q1 version we currently use (and mirror here: http://static.leaflabs.com/pub/codesourcery/originals/ ). These are my conclusions for the compiler wonks out there, in case anyone reading wants to try; as usual, patches are welcome.
First, it seems that there is some interleaving between parsing and semantic checks. Specifically, in arm-2010q1-188-arm-none-eabi/gcc-4.4-2010q1/gcc/cp/parse.c, function cp_parser_simple_declaration() (line 8153), if the parser notices an unqualified function call to an unknown function, it issues a call to unqualified_fn_lookup_error() (defined in gcc/cp/lex.c, line 473).
That's not so bad, though -- grepping through the sources, the only other place where that unqualified_fn_lookup_error() is called is in semantics.c, which (unless it's badly named) should take place after parsing, so it should be possible to instrument cp_parser_simple_declaration() to emit an appropriate prototype, or flag the function for further processing, or some other reasonable thing.
Alternatively, consider the implementation for unqualified_fn_lookup_error() (reproduced here for convenience):
tree
unqualified_fn_lookup_error (tree name)
{
if (processing_template_decl)
{
/* In a template, it is invalid to write "f()" or "f(3)" if no
declaration of "f" is available. Historically, G++ and most
other compilers accepted that usage since they deferred all name
lookup until instantiation time rather than doing unqualified
name lookup at template definition time; explain to the user what
is going wrong.
Note that we have the exact wording of the following message in
the manual (trouble.texi, node "Name lookup"), so they need to
be kept in synch. */
permerror (input_location, "there are no arguments to %qD that depend on a template "
"parameter, so a declaration of %qD must be available",
name, name);
if (!flag_permissive)
{
static bool hint;
if (!hint)
{
inform (input_location, "(if you use %<-fpermissive%>, G++ will accept your "
"code, but allowing the use of an undeclared name is "
"deprecated)");
hint = true;
}
}
return name;
}
return unqualified_name_lookup_error (name);
}
Based on this, it seems that compiling with -fpermissive might make it possible to write a custom tree dump which prints out prototypes without further modifications to GCC (which is obviously very desirable from a maintainability perspective).
As a disclaimer, I haven't presented any of these ideas to a GCC expert, so I could be totally off-base here. If anybody can offer some advice, I'm sure that the Maple (and Arduino, and Wiring!) users would be extremely grateful.
Edits: typo,clarity
