LeafLabs Garden » Topic: Maple hardware / software test configuration

LeafLabs Garden » Topic: Maple hardware / software test configuration http://forums.leaflabs.com/topic.php?id=686 A place to share, learn, and grow... en-US Fri, 22 Jan 2016 00:17:22 +0000 http://bbpress.org/?v=1.0.2 <![CDATA[Search]]> q http://forums.leaflabs.com/search.php StephenFromNYC on "Maple hardware / software test configuration" http://forums.leaflabs.com/topic.php?id=686&page=2#post-3995 Tue, 22 Mar 2011 11:02:55 +0000 StephenFromNYC 3995@http://forums.leaflabs.com/ Hello, Many thanks to mbolivar for <a href="http://forums.leaflabs.com/topic.php?id=686&page=2#post-3967">his recent words</a> and especially for the time spent composing his thoughts. If time permits, mbolivar's description can be integrated into the leaflabs documentation. My assumption is that users who are downloading the recent snapshots are not overly worried about up-to-date documentation. I assume users who download the snapshots are more worried about working code and fixes to bugs. Code examples are helpful (eg. how to use the soon-to-be-new DMA functions), but that work can wait for the regular software releases. LeafLabs is still relatively new and small, so choices need to be made about how to spend time, your company's most precious resource. My vote is to simply be clear about what you do and what you cannot do before and after each software released. Now it is clear to me that each board is tested with a C++ version of InteractiveTest, but not with the pde version. This is fine with me, because now I know what you do to test boards. However, before each major software release I would suggest a quick test compiling of the pde files (there are just a few) in the "Examples >> Maple" folder. Thanks for the work! I will try where I can to help improve the software and documentation. Thanks! <a href="http://forums.leaflabs.com/profile.php?id=843">Stephen from NYC</a> (full disclosure: I am not a member of the LeafLabs staff) [my current <a href="http://forums.leaflabs.com/topic.php?id=690#post-3949">development system</a> (old, but still working)] gbulmer on "Maple hardware / software test configuration" http://forums.leaflabs.com/topic.php?id=686&page=2#post-3986 Mon, 21 Mar 2011 13:24:11 +0000 gbulmer 3986@http://forums.leaflabs.com/ mbolivar - you answered this lot: <blockquote>StephenFromNYC: <blockquote>I do not think the automated builds (Suggestion 1; above) are a good use of storage space and network bandwidth. As long as file dependencies are clear I prefer to patch in just a few files (Suggestion 2; above) when incremental changes are made to libmaple. There is no need to download/install files which have not changed. </blockquote> gbulmer: <blockquote>I believe git does a good job of only downloading the changed parts of files. As long as LeafLabs don't do gratuitous things to files, it should be practical for git to have a 'snapshot' and only require changes to be downloaded, and not full file sets. </blockquote> gbulmer is correct in saying that git only sends the differences between files over the network, but StephenFromNYC is correct in that if we were to release full automated builds, users would have to re-download the IDE every time. </blockquote> I was making a very wild unwritten assumption. I was assumming that we could have a version of your automated build environment installed locally. So when you do a release based off an automated build, we could do a git update and run an automated build to achieve the same output as LeafLabs (an installable file). In retrospect that is a sweeping assumption which may be feasible for Linux, may be possible with a bit of bureacracy for Mac, but probably costs money for Windows. It might also be far too much work for LeafLabs to make the build environment installation and maintenance process visible. Anyway, I thought it worth explaining, as that might be something to explicitly consider (and reject:-). It is a facet of my "I'd like a 'symmetric' development community" burbling. gbulmer on "Maple hardware / software test configuration" http://forums.leaflabs.com/topic.php?id=686&page=2#post-3981 Sun, 20 Mar 2011 22:34:03 +0000 gbulmer 3981@http://forums.leaflabs.com/ mbolivar - let me try to test to see if I understand the process of turning Maple/Arduino sketches into C++. The PDE pre-processor does several tasks. The task which seems to be most difficult is finding declared functions in order to build prototypes on behalf of the user. The other tasks, specifically: - concatenating sketches - finding imports and hence identifying libraries, and - collecting the set of #includes seem to be much easier. As you explain parsing can't be done robustly with regular expressions. Edit: I have not looked at the source of the current PDE converter, but in principle I agree with you, different technology is needed to build a parser. But, with a few provisos, maybe there is a way to generate the protoypes without using the gcc/LLVM clang parsers, or equivalent, or even a parser which properly understands C++. As I understand the parsing task required for this problem, would it be acceptable to choose a solution where all that has to be found are candidate function prototypes, and to make the prototypes work also find typedef's, struct's, and enum's? Here's my first proviso, If the user knows enough to use C++ classes, is it reasonable to also leave them to write their own prototypes or header files? If no, stop here because I think this may need a better C++ parser than I want to do. Here's another proviso. If the function declaration has syntax errors, so has the prototype, and the compiler will generate two sets of error messages. Is that okay? Can a function declaration always be converted to a function prototype by nothing more than: 1. delete the function body, i.e. delete '{ ... }' 2. append a ';' in place of the deleted function body 3. optionally prepended 'extern'. (Edit: As extern is the default, and the function may be static skip this step) So, the strategy would be: 1. concatenate files as in your description (Edit: could treat one at a time) 2. Remove all imports/#includes We could try something sneaky-evil, but I think I'd parse the file(s) and throw away import/#include because their syntax shouldn't be horrible anyway. Edit: I think this might be a relatively simple parser in its own right. 3. pass file through the C preprocessor (Edit: so syntactically valid C/C++ must get created, or it contains errors which will give errors with gcc/clang anyway, so no worse) 4. parse the resulting file, replace all function bodies with ';' to create function prototypes 5. delete all top level declarations of variables, but keep struct definitions, enum definitions and typedef's so that prototypes are correct. (Edit: this will be a bit tricky, but I think the concrete C syntax would have to suffice.) 6. Edit: write an error message warning the user that they have to do classes themselves if 'class' is found. Problems? Lots. The actual prototypes won't be identical to the original source. The C preprocessor may have changed it. If it were legal before pre-processing it could become illegal by pre-processing, but that change will be made to the original code too, so the C++ compiler can't do any better. The prototype might not be a character for character match with the users source, but it will match the source that the C++ compiler sees. Line numbers on errors will be badly broken. Edit: if the current PDE converter has a solution maybe we can use its apprach, so it may be no worse. If the user has two different definitions of the same type name, say a typedef, in two different source files, it will clash and break in the prototype file. I assume this is a problem with the existing approach anyway, so it doesn't seem to be worse. The simplified parser I am thinking of would be pretty dumb, and barely understand the concrete syntax of C++, Edit: its error detection and resynchronisation will be poor. It may give up at the first error. Edit: I don't think this is "the" solution. I'm trying to understand if there is something better than the current PDE converter, but much simpler than using gcc or clang. mbolivar on "Maple hardware / software test configuration" http://forums.leaflabs.com/topic.php?id=686&page=2#post-3969 Sun, 20 Mar 2011 01:59:14 +0000 mbolivar 3969@http://forums.leaflabs.com/ In response to some other questions/comments: StephenFromNYC: <blockquote> I do not think the automated builds (Suggestion 1; above) are a good use of storage space and network bandwidth. As long as file dependencies are clear I prefer to patch in just a few files (Suggestion 2; above) when incremental changes are made to libmaple. There is no need to download/install files which have not changed. </blockquote> gbulmer: <blockquote> I believe git does a good job of only downloading the changed parts of files. As long as LeafLabs don't do gratuitous things to files, it should be practical for git to have a 'snapshot' and only require changes to be downloaded, and not full file sets. </blockquote> gbulmer is correct in saying that git only sends the differences between files over the network, but StephenFromNYC is correct in that if we were to release full automated builds, users would have to re-download the IDE every time. The tradeoff to be made here is between network bandwidth and ease of use, both for users and developers. If full releases are made, then users can just download and play with snapshots, and can report bugs with just the snapshot's file name and their local operating system information. However, every update will require a big (40-70MB) download. If we require users to point the IDE at an external libmaple, then they must use git to update their libmaple installations, which requires that they install and learn about git. Further, when reporting bugs, they'll have to report the SHA1 of their libmaple version in addition to the IDE version and their system config. Another wrinkle: since our documentation is automatically generated for each release from the ReST files in libmaple/docs/source and the Doxygen comments scattered throughout the libmaple source trees, if a user wants to read the API documentation for a snapshot, they'll have to build it themselves, a process which involves even more steps: <a href="https://github.com/leaflabs/libmaple/blob/master/docs/README" rel="nofollow">https://github.com/leaflabs/libmaple/blob/master/docs/README</a> By contrast, a full snapshot release contains the complete reference documentation already. A possible workaround for this is for LeafLabs to host the built documentation for the latest n snapshot releases on a rotating basis. Something like <a href="http://static.leaflabs.com/pub/leaflabs/maple-docs/snapshots/git" rel="nofollow">http://static.leaflabs.com/pub/leaflabs/maple-docs/snapshots/git</a> hash]/... I'm not really sure what the right thing to do is. I think that the answer depends on what you as the testers prefer, and I'm perfectly happy to accommodate the consensus opinion. StephenFromNYC: <blockquote> Does LealLabs use a thermographic camera to test for hot components? How much variation in temperature is tolerated? Of course, the temperature of the components is affected by the ambient room temperature </blockquote> The thermal test isn't a sophisticated one; we just feel them by hand and see if anything feels "too warm", at the discretion of the tester. mbolivar on "Maple hardware / software test configuration" http://forums.leaflabs.com/topic.php?id=686&page=2#post-3967 Sat, 19 Mar 2011 22:18:53 +0000 mbolivar 3967@http://forums.leaflabs.com/ Stephen, Your post brings up some important issues in both the IDE's current implementation and the way we develop it. I've been considering these issues for a while, and I'd like to respond to your concerns in detail. I hope everyone will forgive me the length of this post. These paragraphs from your previous post are particularly relevant: <blockquote> Knowing that the "Examples >> Maple >> InteractiveTest" is used to test each board is informative, but also puzzling. On a Windows XP SP3 system I am still not able to compile the InteractiveTest on Maple IDE 0.0.9 (see below). I reported this problem when 0.0.9 was released (in December, 2010). My interpretation is that when boards are evaluated the InteractiveTest QA is not performed using a Windows system, which is fine. However, in the future please be sure that the InteractiveTest sketch works on all supported platforms before major software releases. Question: I have not looked at how the Timers/PWM code changed between IDE release 0.0.7 and 0.0.9, but if Maple Rev 3/5 board runs the InteractiveTest sketch from 0.0.9 (which should evaluate PWM) onn a linux system why did crenn's code fail on a Windows system? Does crenn's code work on a linux system? Another possible way to ask the question is why does InteractiveTest work on linux, but not Windows (using IDE 0.0.9)? </blockquote> In maple-test-procedures.pdf, under the section labeled "Upload test program", we specify that we use the C++ version of the interactive test program which is included in the libmaple repository. The section goes on to say that it should be possible to use the InteractiveTest example as well, but there's a parenthetical comment "(TODO: double check that this is true)". Unfortunately, when I was putting together the 0.0.9 release, I failed to make sure that the InteractiveTest included with the IDE reflected the changes that had been made to libmaple, so, as you noticed, it failed to compile. This mistake thus does not reflect a difference between the behavior of a program when compiled on Linux vs. Windows, but rather reflects the fact that I carelessly did not ensure that all of the IDE examples were kept up to date when I released 0.0.9. These examples in general are a place where we could improve our quality significantly. It is obviously an unsatisfactory situation when the same code has to be maintained by hand in two separate places, specifically, as C++ versions in libmaple, and as .pde versions in the IDE. The "best" solution (in the sense of "most correct", "leading to least duplication of effort") is to build a standalone mechanism which enables the automatic translation of sketches written in the Maple variant of the Wiring language into C++. Presently, this is accomplished from within the Sketch#preprocess(String, PdePreprocessor) method in the IDE's Sketch.java: <a href="https://github.com/leaflabs/maple-ide/blob/master/app/src/processing/app/Sketch.java#L1261" rel="nofollow">https://github.com/leaflabs/maple-ide/blob/master/app/src/processing/app/Sketch.java#L1261</a> Instead, we could write a standalone program which performs the PDE preprocessing step, which the IDE and libmaple could both make use of. The LeafLabs developers could then write all of our examples as .pde files, and still compile them using the Unix toolchain. (It may seem as though my preference (and that of other LeafLabs developers) to continue using the Unix toolchain reflects a lack of confidence in our IDE. To respond preemptively to that: the IDE is meant to be a simple, easy-to-use tool, with minimal frills. While it definitely has its problems, I don't think it fails utterly in this regard. However, the fact remains that my tools of choice (namely, Emacs and the command line), while decidedly more difficult to use, are also significantly more powerful, and I am simply not productive enough without the benefits this extra power brings.) Unfortunately, producing such a PDE preprocessor properly turns out to be a rather subtle task, so at least for the time being, I think it's best if we continue to maintain the two separate versions of each example. We at LeafLabs will simply have to be vigilant in maintaining the two parallel versions of the test code. This is an unlovable situation to me, but I think it's the most workable solution. The remainder of this post explains my rationale. To begin, let's establish the PDE preprocessor's jobs. Glossing over some details, these are: (1) Concatenate all of the sketch's files together (2) Parse the program to find the user's imports (so we know which extra libraries to link against) and declared functions (so we can produce prototypes for them) (3) Output valid C++ which #includes wirish/ and defines prototypes for the user's functions, so they don't have to, along with a main() function and code that calls wirish's init() function. The current implementation really fails hard in step (2); I've been wanting to get rid of it and replace it with something more robust and stand-alone since I first read its implementation. For example, try compiling this sketch, which should be perfectly legal code: <pre><code>#define BB }} void setup() { if (true) { BB void loop() { }</code></pre> You'll get compile errors. This behavior isn't unique to the Maple IDE; the Arduino environment (from which we inherited the PDE preprocessor implementation) has the same problem. While the above code is certainly a gross abuse of the C preprocessor, the fact remains that the IDE's PDE preprocessor doesn't take the C preprocessor into account, which is just Bad and Wrong. For comparison, here is the equivalent C++ program: <pre><code>#include "wirish.h" #define BB }} void setup() { if (true) { BB void loop() { } __attribute__((constructor)) void premain() { init(); } int main(void) { setup(); while (1) { loop(); } return 0; }</code></pre> This compiles just fine against the current libmaple master (cdd367bdd264c9e19180 at time of writing). Now, with regards to producing function prototypes, the current implementation attempts to use regular expressions to parse function declarations: <a href="https://github.com/leaflabs/maple-ide/blob/master/app/src/processing/app/preproc/PdePreprocessor.java" rel="nofollow">https://github.com/leaflabs/maple-ide/blob/master/app/src/processing/app/preproc/PdePreprocessor.java</a> For reasons that have been perfectly well established in parsing theory for decades, regular expressions are simply inadequate to parse a context dependent grammar such as that of C++. So it's not just that the current implementation has flaws, it's that important pieces of its strategy are fundamentally invalid. For example, consider the problem of parsing out a prototype from a function declaration which takes as an argument a function pointer, like so: <pre><code>void func(void (*arg)(void)) { SerialUSB.println("calling argument function"); arg(); SerialUSB.println("done!"); }</code></pre> Again, perfectly valid Wiring and C++. However, note that the argument function pointer could also take arguments which are function pointers. We could do this an arbitrarily large number of times: <pre><code>void func1(void (*func2)(void (*func3)(...))) { ... }</code></pre> Thus, in order to detect if such a declaration is proper, we need among other things to make sure that all the nested parentheses match properly. It's an elementary example in most texts on parsing that regular expressions are unable to determine whether or not an expression with parentheses is properly balanced. Rather than throwing good money after bad by trying to polish up the current implementation in a standalone program or otherwise, it is clear (at least to me) that more radical surgery is required, namely, using a real parser to get the job done. Unfortunately, as you may know, the C++ grammar is notoriously difficult to parse, and the Wiring language is roughly a superset of C++. This makes parsing it difficult enough to make it a waste of time for us to write up something from scratch. In order do to the right thing, then, we'll need an existing C++ parser which we can modify in some way. Further, since we specifically advertise that we compile using CodeSourcery's arm-none-eabi-gcc, this parser must be aware of all GCC extensions to the C++ language. I've spent some time trying to come up with an acceptable solution that satisfies these constraints, and I've concluded that although it is possible to do so, it would take up too much time that, at least for now, is better spent elsewhere. To my knowledge, there are only two fully GCC-compatible C++ parsers in existence: that which comes with GCC itself, and (more recently) the version that comes with LLVM's clang: <a href="http://clang.llvm.org/" rel="nofollow">http://clang.llvm.org/</a> While LLVM provides libclang, an excellent library for interacting with the data structures produced by its parser (see <a href="http://devimages.apple.com/llvm/videos/Libclang.mov" rel="nofollow">http://devimages.apple.com/llvm/videos/Libclang.mov</a> for a very impressive talk on libclang), using it would either introduce an entire extra compilation toolchain as a dependency (gross!), or require us to switch from CodeSourcery's tools. I'm all for switching, but clang's support for the Cortex M3 architecture is not yet complete, so that isn't workable. So, there's only one thing left to do: modify CodeSourcery's GCC C++ parser to produce a list of function declarations in the user's sketch. That would be easy enough if the sketch was valid C++; however, because Wiring doesn't require that a function is declared before it's used (unlike C++), we can't just run the program unmodified. There is a program called protoize, part of GCC, that helped accomplish a similar goal for pure C. As I recall, protoize was useful in the early 1990s when the C standard mandated a function declaration, instead of stating that a function which was called before it was declared as implicitly assumed to return int. However, C++ is a much more complicated monster than C, and protoize seems to have fallen into disuse. So, (again, at least as far as I can tell), if we're going to do this thing right, we'll have to modify GCC itself. While that's certainly possible, we at LeafLabs don't have enough practice implementing compilers nor understanding of GCC's internals to have confidence that we could do the job quickly and well. I spent a few days during fall 2010 reading through the sources for arm-none-eabi-gcc, specifically, the gcc-4.4-2010q1 version we currently use (and mirror here: <a href="http://static.leaflabs.com/pub/codesourcery/originals/" rel="nofollow">http://static.leaflabs.com/pub/codesourcery/originals/</a> ). These are my conclusions for the compiler wonks out there, in case anyone reading wants to try; as usual, patches are welcome. First, it seems that there is some interleaving between parsing and semantic checks. Specifically, in arm-2010q1-188-arm-none-eabi/gcc-4.4-2010q1/gcc/cp/parse.c, function cp_parser_simple_declaration() (line 8153), if the parser notices an unqualified function call to an unknown function, it issues a call to unqualified_fn_lookup_error() (defined in gcc/cp/lex.c, line 473). That's not so bad, though -- grepping through the sources, the only other place where that unqualified_fn_lookup_error() is called is in semantics.c, which (unless it's badly named) should take place after parsing, so it should be possible to instrument cp_parser_simple_declaration() to emit an appropriate prototype, or flag the function for further processing, or some other reasonable thing. Alternatively, consider the implementation for unqualified_fn_lookup_error() (reproduced here for convenience): <pre><code>tree unqualified_fn_lookup_error (tree name) { if (processing_template_decl) { /* In a template, it is invalid to write "f()" or "f(3)" if no declaration of "f" is available. Historically, G++ and most other compilers accepted that usage since they deferred all name lookup until instantiation time rather than doing unqualified name lookup at template definition time; explain to the user what is going wrong. Note that we have the exact wording of the following message in the manual (trouble.texi, node "Name lookup"), so they need to be kept in synch. */ permerror (input_location, "there are no arguments to %qD that depend on a template " "parameter, so a declaration of %qD must be available", name, name); if (!flag_permissive) { static bool hint; if (!hint) { inform (input_location, "(if you use %<-fpermissive%>, G++ will accept your " "code, but allowing the use of an undeclared name is " "deprecated)"); hint = true; } } return name; } return unqualified_name_lookup_error (name); }</code></pre> Based on this, it seems that compiling with -fpermissive might make it possible to write a custom tree dump which prints out prototypes without further modifications to GCC (which is obviously very desirable from a maintainability perspective). As a disclaimer, I haven't presented any of these ideas to a GCC expert, so I could be totally off-base here. If anybody can offer some advice, I'm sure that the Maple (and Arduino, and Wiring!) users would be extremely grateful. Edits: typo,clarity gbulmer on "Maple hardware / software test configuration" http://forums.leaflabs.com/topic.php?id=686&page=2#post-3958 Fri, 18 Mar 2011 20:34:44 +0000 gbulmer 3958@http://forums.leaflabs.com/ Stephen - <blockquote>I do not think the automated builds (Suggestion 1; above) are a good use of storage space and network bandwidth. As long as file dependencies are clear I prefer to patch in just a few files (Suggestion 2; above) when incremental changes are made to libmaple. There is no need to download/install files which have not changed. </blockquote> Maybe you are confusing mechanism with meaning. I believe git does a good job of only downloading the changed parts of files. As long as LeafLabs don't do gratuitous things to files, it should be practical for git to have a 'snapshot' and only require changes to be downloaded, and not full file sets. <blockquote>I found the LeafLabs QA (Quality Assurance) PDF really interesting reading. I have to admit that I was confused when I saw the letters QA among the recent github documents. Maybe I am old-school, but I use the letters QC to mean Quality Control. </blockquote> I am old school too. In the UK customers stopped talking about quality control in the 90's. After all, we could control quality to be good or awful, Quality Control does not imply good quality, just some form of control. The view was control was a much less positive concept than assuring ('guaranteeing') quality. Of course, quality is such a fluid or nebulous concept that we also used terms like Total Quality. Having tests that run on one platform but not others is a bit worrying. I guess it might be useful to have a document page that explains that rationale. Nice detective work. Good stuff StephenFromNYC. (full disclosure: I am not a LeafLabs staffer) StephenFromNYC on "Maple hardware / software test configuration" http://forums.leaflabs.com/topic.php?id=686&page=2#post-3955 Fri, 18 Mar 2011 15:03:56 +0000 StephenFromNYC 3955@http://forums.leaflabs.com/ Hello- I have a bunch of ideas and questions. If anything is confusing please let me know. I do not think the automated builds (Suggestion 1; above) are a good use of storage space and network bandwidth. As long as <a href="">file dependencies</a> are clear I prefer to patch in just a few files (Suggestion 2; above) when incremental changes are made to <code>libmaple</code>. There is no need to download/install files which have not changed. I found the LeafLabs QA (Quality Assurance) PDF really interesting reading. I have to admit that I was confused when I saw the letters QA among the recent github documents. Maybe I am old-school, but I use the letters QC to mean Quality Control. Does LealLabs use a thermographic camera to test for hot components? How much variation in temperature is tolerated? Of course, the temperature of the components is affected by the ambient room temperature Knowing that the "Examples >> Maple >> InteractiveTest" is used to test each board is informative, but also puzzling. On a Windows XP SP3 system I am still not able to compile the InteractiveTest on Maple IDE 0.0.9 (see below). I reported <a href="http://forums.leaflabs.com/topic.php?id=575#post-3143">this problem</a> when 0.0.9 was released (in December, 2010). My interpretation is that when boards are evaluated the InteractiveTest QA is not performed using a Windows system, which is fine. However, in the future please be sure that the InteractiveTest sketch works on all supported platforms before major software releases. Question: I have not looked at how the Timers/PWM code changed between IDE release 0.0.7 and 0.0.9, but if Maple Rev 3/5 board runs the InteractiveTest sketch from 0.0.9 (which should evaluate PWM) onn a linux system why did <a href="http://forums.leaflabs.com/topic.php?id=683">crenn's code fail</a> on a Windows system? Does crenn's code work on a linux system? Another possible way to ask the question is why does InteractiveTest work on linux, but not Windows (using IDE 0.0.9)? As noted above, the good news is that today, after one successful compile but unsuccessful upload, the sketch works with the February 13, 2011 snapshot. 1) <a href="http://forums.leaflabs.com/topic.php?id=690#post-3938">Maple IDE 0.0.9 System</a>: compiling for use in Maple FLASH is successful. line 303 (possibly line337) is highlighted: timer_init(1,21); error: invalid conversion from 'int' to 'timer_dev_num' 2) <a href="http://forums.leaflabs.com/topic.php?id=690#post-3949">Patched Maple Snapshot System</a> (2011-02-13-[git-e938d1765a]): Compiling for use in Maple FLASH is successful. When uploading it stops after "Starting download: [" Recompiling for use in Maple FLASH is successful (check for size: 26898 bytes). Attempting the upload again is now successful and the sketch runs. Thanks! <a href="http://forums.leaflabs.com/profile.php?id=843">Stephen from NYC</a> (full disclosure: I am not a member of the LeafLabs staff) mbolivar on "Maple hardware / software test configuration" http://forums.leaflabs.com/topic.php?id=686&page=2#post-3950 Thu, 17 Mar 2011 18:49:54 +0000 mbolivar 3950@http://forums.leaflabs.com/ Very interesting ideas all around. We definitely welcome user testing, and look forward to working with anybody who is interested in developing more formal and distributed test procedures. Here are a few notes on our current procedures. These are the hardware tests we perform when we receive a new batch of Maples: <a href="http://static.leaflabs.com/pub/leaflabs/docs/maple-test-procedures.pdf" rel="nofollow">http://static.leaflabs.com/pub/leaflabs/docs/maple-test-procedures.pdf</a> (That document is slightly outdated in that its failure rates were given for a previous manufacturer; they're much lower now). For software testing, we use the examples/qa-slave-shield.cpp and examples/test-session.cpp files in the libmaple repository (the IDE example Maple -> InteractiveTest is based off of test-session.cpp). In particular, we connect the GPIOs of two boards, one the "master" (which does the testing) and one the "slave" (which is being tested). The master then runs the '+' command of the interactive test session (GPIO QA), which waits for the slave to toggle the pins in order, and checks if any pins that shouldn't change do, reporting any failures. The file examples/test-session.cpp is where the IDE example Maple > InteractiveTest comes from. In terms of testing features on different boards, you may want to check out pin_info.h in the InteractiveTest example in today's (17 March 2011) IDE snapshots: <a href="https://github.com/leaflabs/maple-ide/commit/32f6ee76a80f1517babb0b078613606d174b887e#diff-2" rel="nofollow">https://github.com/leaflabs/maple-ide/commit/32f6ee76a80f1517babb0b078613606d174b887e#diff-2</a> The file InteractiveTest.pde includes pin_info.h, which defines arrays that are capable of PWM and ADC conversion on a per-board basis. (BOARD_maple, BOARD_maple_mini, etc. are automatically defined by the compiler based on your build settings). This lets the test session work the same way across boards. It also defines pins_to_skip; this is an array of "special" pins that are tested separately (specifically, the pin connected to the built-in LED, which is tested by watching it blink, or the pins connected to the USB data lines on the Maple Mini, which are tested if uploading code works at all). The PWM bug referenced earlier was limited to a single pin, number 24. It was a simple error in the PIN_MAP; other PWM-capable pins were not affected. The fix went into boards.h on March 4: <a href="https://github.com/leaflabs/libmaple/commit/b4c2d4514c6d52cac8a649c5d5c24b68a3c0a416#diff-12" rel="nofollow">https://github.com/leaflabs/libmaple/commit/b4c2d4514c6d52cac8a649c5d5c24b68a3c0a416#diff-12</a> PWM is tested using the 's' interactive test command, which varies the duty cycle up and down on each PWM capable pin; hooking this up to a scope revealed the error. Other pins behaved correctly. The PIN_MAP was rechecked against the ST datasheets for other regressions at that time. Pastebin is useful for sharing code without pasting it into something like a forum or IRC channel, where you might not want to send a flood of text. It's not really appropriate to use it as a code hosting site. The libmaple directory examples/ contains various other test programs that should really be turned into .pde files, so users can test them out for themselves. To continue the thread, I'd like to offer a couple of suggestions for the community's consideration. Any feedback is very welcome. Suggestion 1: Like most other open source projects, we could offer automated builds of the Maple IDE, which interested users could download and try out. These would be like snapshots, except they'd just be whatever was in a particular branch one day. We realize that not everyone interested in doing early testing wants to set up the command-line build chain and work against the git repository. Suggestion 2: The Maple IDE already allows you to use an external editor; I think it would probably also be a useful feature for it to allow you to point it at a copy of the libmaple repository to use instead of the version that came with it. That way, you could try out different versions of libmaple without having to re-download the IDE (which has changed very little in comparison with the library). StephenFromNYC on "Maple hardware / software test configuration" http://forums.leaflabs.com/topic.php?id=686&page=2#post-3942 Thu, 17 Mar 2011 15:57:33 +0000 StephenFromNYC 3942@http://forums.leaflabs.com/ Hello- In a <a href="http://forums.leaflabs.com/topic.php?id=686#post-3899">previous post</a> gbulmer said: <blockquote>I'd prefer to have a way to test my code without a need to have many versions of the board. I would like a 'symmetric' development community; by that I mean we should all be able to work on a similar basis, without the need to buy lots of hardware.</blockquote> When I described a need for a test configuration I did not mean to suggest that users buy a lot of hardware. Of course, that is unreasonable (and expensive). Instead, I thought a simple description of how the software/hardware was tested should be made by the LeafLabs team with each IDE release. This way, if a specific pin worked as specified (according to the official documentation) without any "work around" with an older software/IDE release it should work the same way in the future (unless users are told required changes may break older code). However, instead of placing the responsibility for testing on the LeafLabs staff (which I still think should use a standard hardware configuration for "internal" testing) perhaps we can use the wiki to achieve the same result and to spread the work around. Users can contribute simple code which they believe shows that a specific pin works (or "work around" code if a certain pin does not work as advertised). I do not know if the LeafLabs team prefers that as little code as possible be stored on their wiki server. Will the <a href="http://pastebin.com">http://pastebin.com</a> site work well as an archive of working/broken code? The use of pastebin.com was described in an <a href="http://forums.leaflabs.com/topic.php?id=211#post-1551">earlier post</a> by key master mbolivar. I do not know if descriptions of a user's development system will be useful. For example, a user might have multiple different configurations. To save space, one solution is to ask each user who contributes to the "feature tested" wiki page to create a separate topic on the forum titled: "Username: System description". At the bottom of each post it says: Posted [time] ago # The hash (#) sign is a hyperlink to that specific post. In the "feature tested" wiki if a user has multiple hardware configurations then the user can use the "A" HTML markup tag to specify which hardware configuration was used for the specified test. This system can also be used in the forums. At the end of this post is a link to a description of my Maple development system. I will begin to think about different ways to present the "feature tested" information. The Maple/Maple Mini/Maple Native have a lot of pins buttons, and power options, so it will be a challenge to present the most pertinent information in a clear way. Any ideas for what works best for you? Thanks! <a href="http://forums.leaflabs.com/profile.php?id=843">Stephen from NYC</a> (full disclosure: I am not a member of the LeafLabs staff) [my current <a href="http://forums.leaflabs.com/topic.php?id=690#post-3939">development system</a> (old, but still working)] redfox74 on "Maple hardware / software test configuration" http://forums.leaflabs.com/topic.php?id=686&page=2#post-3902 Tue, 15 Mar 2011 20:51:31 +0000 redfox74 3902@http://forums.leaflabs.com/ Dear Guy, I solved the problem on vector table in maple_native , high density micro in my case STM32F103VE Multipilot32 .. This is right correction , tested on disassembly. The file to patch is in support /ld /source STM32_vector_table.s /* STM32 vector table */ .section ".cs3.interrupt_vector" .globl __cs3_stm32_vector_table .type __cs3_stm32_vector_table, %object __cs3_stm32_vector_table: /* CM3 core interrupts */ .long __cs3_stack .long __cs3_reset .long __exc_nmi .long __exc_hardfault .long __exc_memmanage .long __exc_busfault .long __exc_usagefault .long __stm32reservedexception7 .long __stm32reservedexception8 .long __stm32reservedexception9 .long __stm32reservedexception10 .long __exc_svc .long __exc_debug_monitor .long __stm32reservedexception13 .long __exc_pendsv .long __exc_systick /* Peripheral interrupts */ .long __irq_wwdg .long __irq_pvd .long __irq_tamper .long __irq_rtc .long __irq_flash .long __irq_rcc .long __irq_exti0 .long __irq_exti1 .long __irq_exti2 .long __irq_exti3 .long __irq_exti4 .long __irq_dma1_channel1 .long __irq_dma1_channel2 .long __irq_dma1_channel3 .long __irq_dma1_channel4 .long __irq_dma1_channel5 .long __irq_dma1_channel6 .long __irq_dma1_channel7 .long __irq_adc .long __irq_usb_hp_can_tx .long __irq_usb_lp_can_rx0 .long __irq_can_rx1 .long __irq_can_sce .long __irq_exti9_5 .long __irq_tim1_brk .long __irq_tim1_up .long __irq_tim1_trg_com .long __irq_tim1_cc .long __irq_tim2 .long __irq_tim3 .long __irq_tim4 .long __irq_i2c1_ev .long __irq_i2c1_er .long __irq_i2c2_ev .long __irq_i2c2_er .long __irq_spi1 .long __irq_spi2 .long __irq_usart1 .long __irq_usart2 .long __irq_usart3 .long __irq_exti15_10 .long __irq_rtcalarm .long __irq_usbwakeup .long __irq_tim8_brk .long __irq_tim8_up .long __irq_tim8_trg_com .long __irq_tim8_cc .long __irq_adc3 .long __irq_fsmc .long __irq_sdio .long __irq_tim5 .long __irq_spi3 .long __irq_uart4 .long __irq_uart5 .long __irq_tim6 .long __irq_tim7 .long __irq_dma2_channel1 .long __irq_dma2_channel2 .long __irq_dma2_channel3 .long __irq_dma2_channel4_5 .size __cs3_stm32_vector_table, . - __cs3_stm32_vector_table Best Roberto gbulmer on "Maple hardware / software test configuration" http://forums.leaflabs.com/topic.php?id=686#post-3899 Tue, 15 Mar 2011 18:08:39 +0000 gbulmer 3899@http://forums.leaflabs.com/ Stephen - I bought my Maple. I think we are on 'the same page'. When I experienced the Arduino, I was very impressed, but felt there were a few hardware/peripheral improvement I needed. I am interested in both robotics, and stand-alone applications, but also extending computing into the physical realm. For example, I designed the original board, and wrote the original firmware for the SenseBoard used on this OU coarse, TU100 <a href="http://mcs.open.ac.uk/akb235/pub_filexfer/tu100_flier.pdf" rel="nofollow">http://mcs.open.ac.uk/akb235/pub_filexfer/tu100_flier.pdf</a> which is an extension to Scratch into the physical world. I am very passionate about "the next Arduino", and Cortex-M3 (along with, maybe PIC32) is the lowest level 'sane' device that I saw as a significant improvement, while being in the same ball-park of cost and complexity (vs Arduino Mega). One of the aspects that impressed me from the start of using the Arduino was the quality of the software. Even when it is lacking in features, or a bit slow, or restricted, it works. The library ('language') is very easy to use. I much prefer simple and robust, over fast or fancy but flakey. I think you have raised a critical issue if Maple is to be as low-pain as Arduino. The Maple software must be thoroughly and consistently tested, and regressions prevented. By that I mean whenever a regression is detected, tests need to be built to prevent the bug from going unnoticed. I'd prefer to have a way to test my code without a need to have many versions of the board. I would like a 'symmetric' development community; by that I mean we should all be able to work on a similar basis, without the need to buy lots of hardware. One of the reasons I posted asking for anyone with Cortex-M3 simulator or emulator experience (<a href="http://forums.leaflabs.com/topic.php?id=675" rel="nofollow">http://forums.leaflabs.com/topic.php?id=675</a>) is I think that may be a good way to create a practical and better test environment. A simulator/emulator isn't perfect, but if we all have access to a reasonable Open Source test platform, then we can all work in a comparable way. Then we can have a high degree of confidence that we can collaborate effectively, including contributing 'unit tests'. I'm assuming with the extra observe-ability of a simulator/emulator, building automated tests should be easier and more effective. Maybe other folks will contribute their approaches to embedded system testing? StephenFromNYC on "Maple hardware / software test configuration" http://forums.leaflabs.com/topic.php?id=686#post-3890 Mon, 14 Mar 2011 10:42:05 +0000 StephenFromNYC 3890@http://forums.leaflabs.com/ Hello- I am a big fan of the LeafLab boards and I will continue to spend my time developing code which can use their 32-bit microcontroller products. The enhanced features of the STM chips (eg. DMA, 12-bit ADC, faster processor speed, wider bus, etc) are enough to make me be patient for the Maple to succeed. I am new to microcontrollers, so I cannot comment on the complexity caused by switching from 8-bit to 32-bit, but with the faster clock speed it does not sound trivial. My words were intended to be more of a request/suggestion and less of a criticism. I believe it is important to test the official major software releases with as much old code as possible. Although testing is boring it needs to be an integral and scheduled part of any software/hardware development. The LeafLabs team wants to run as soon as possible, but it is important to avoid stumbling during the first few steps. I had assumed all software/hardware companies (including LeafLabs) had an internal hardware test configuration which they were using. I am hoping they are using the "scientific method" (make a small change in software; keep the hardware configuration the same; and look for changes in the expected behavior). I do not know how many square feet the LeafLabs group occupies (or if lab space is tight), but I assumed they always had one Maple with its input pins wired up, another Maple with its output pins wired up, etc. With the different boards (currently maple rev 1/3/5 and mini) testing may become more complex. One way to reduce the amount of "daily testing" is to says which board is currently the default board for daily testing at LeafLabs (eg. Maple Revision 5). However, before every software release as many older boards as possible should be tested (and if a board is not tested before a major software is released a clear statement should be made in the software documentation). I am not asking that full testing should be done before a snapshot is released. Snapshots and alpha releases are for users who need new functionality which does not exist in an earlier version. However, snapshots and alpha releases are used with the understanding that they may have problems. For example, I am using the <a href="http://static.leaflabs.com/pub/leaflabs/maple-ide/snapshots/maple-ide-snapshot-2011-02-13-%5Bgit-e938d1765a%5D-windowsxp32.zip">git-e938d1765a snapshot</a> from February, 2011 (on a Windows XP system), because the snapshot has a version of <code>SerialUSB.print()</code> which currently works well enough for my projects. In release 0.0.9 and earlier, the version of <code>SerialUSB.print()</code> does not work well for rapid communications. The reason I said <a href="http://forums.leaflabs.com/topic.php?id=686#post-3875">I wonder if I am the only user who is interested in fast SerialUSB.print() related functions</a>, because it has been two days since I posted <a href="http://forums.leaflabs.com/topic.php?id=685">USB Observations and Questions: Part I</a> and so far no responses. Roberto (user redfox), thank you for the <a href="http://forums.leaflabs.com/topic.php?id=686#post-3877">info about the hardware test sketch which appeared with Arduino revision 022</a>. As I said in an <a href="http://forums.leaflabs.com/topic.php?id=686#post-3870">earlier post</a>, I do not think the Arduino platform with its large number of users needs an official hardware test configuration. Arduino users will quickly find the bugs. Although I hope the number of Maple/32-bit microcontroller users will increase I am surprised the PWM/timer problem described by crenn was not caught earlier. The <a href="http://forums.leaflabs.com/topic.php?id=683">PWM/timer code described by crenn</a> is very simple and works with 0.0.8), so my assumption is that the problem must be in the 0.0.9 software. Maybe this is a good place to say that I have never used PWM, but that I have played a little with interrupts (and <code>SerialUSB.print()</code>). In their development of <code>SerialUSB.print()</code> for release 0.0.9 I am guessing a change was made in the timers/interrupts which broke crenn's code. Honestly, it is not clear to me why most users find microcontroller boards such as Maple/Arduino useful. Are most users developing independent robots which do not need to be connected to a laptop/desktop? Are most users developing large programs which occupy most of the microcontroller's memory and then disconnect the USB cable after the sketch uploads (thus requiring an external power source)? I am interested in using microcontroller boards such as Maple/Arduino to connect my computer to the external world. I am interested in sensors and ADC (and eventually, in PWM). crenn, I am glad the software updates which result from my interest in fast and reliable <code>SerialUSB.print()</code> and <a href="http://forums.leaflabs.com/topic.php?id=657"><code>SerialUSB.read()</code></a> communications might help your robot project. I had assumed the non-LeafLabs members who were promoted to forum moderators were also given boards to "evaluate". The time these non-LeafLabs members (in particular <a href="http://forums.leaflabs.com/profile.php?id=756">crenn</a> and <a href="http://forums.leaflabs.com/profile.php?id=292">gbulmer</a>) have donated to the forums providing help is trivial to the cost of a board or two. Thanks! Stephen from NYC redfox74 on "Maple hardware / software test configuration" http://forums.leaflabs.com/topic.php?id=686#post-3889 Mon, 14 Mar 2011 10:10:41 +0000 redfox74 3889@http://forums.leaflabs.com/ Hi try , the impression is that all it's ok but .. on my board work fine only serial 1 serial2 don't work , serial3 ,too serial4 work in output , but not in input :( If i use chibiOS instead work fine serial1 , serial2 , serial4 , so i think that isn't an hardware problem serial3 , i think could be remapped :) Best Roberto redfox74 on "Maple hardware / software test configuration" http://forums.leaflabs.com/topic.php?id=686#post-3888 Mon, 14 Mar 2011 08:17:46 +0000 redfox74 3888@http://forums.leaflabs.com/ Hi Anton, thank you very much , yes that is right ... :) Thank you guys .. :) anton on "Maple hardware / software test configuration" http://forums.leaflabs.com/topic.php?id=686#post-3887 Mon, 14 Mar 2011 05:16:18 +0000 anton 3887@http://forums.leaflabs.com/ I see. May be <a href="https://github.com/leaflabs/libmaple/tree/refactor/support/ld" rel="nofollow">https://github.com/leaflabs/libmaple/tree/refactor/support/ld</a> is new version?